How to Zoom
Programming is a (self-) discipline to deal with complexity of a given world. To make complex systems (from simple, familiar components) we should be able to identify and maintain several levels of abstraction (not too few, not too many) and be able to zoom in and out between them.
The best examples of this approach are Lisp systems. They are written mostly in themselves (the best Scheme implementation were written in Scheme, the best Lisp systems were written in Lisp) while some crucial parts - the procedures of the OS level, were written in the C programming language (an appropriate level) and the procedures of the Hardware level were written in Assembly language, usually by embedding Assembly code into C sources.
This is worth repeating - there are languages appropriate for each level, for Hardware level, to implement micro-optimizations, we could embed Assembly code into C. We could write Lisp's so-called primitive procedures in C and call them from our high-level Lisp code. More-over, we could add new primitive procedures to our favorite Scheme system or rewrite those which interacts with an OS to make them more appropriate to our tasks.
This is where the power comes from. We could control and change almost everything we need. We can zoom in from the heights of Lisp code, through several levels, to the kernel's syscalls (which are mere software interrupts) if we need so. On the way we will see C functions and data structures, then, sometimes, memory dumps and machine code. Everything is simple and familiar.
Having such control and ability to examine and change everything is the (only) way to manage complexity of large systems.
We cannot do it with unnecessarily complicated systems such as Java - I have no interest and no resources to trace how all those bloated classes from uncountable jars being translated into machine code (how could I examine such code?) and OS calls (yeah, they're also doing it in a very inefficient way), leave alone to try to predict or control its behavior.
One finds oneself helpless with system like Flash or Java, where all you can do is mere watch how it consumes all the resources and then crashes. What could one do with closed, proprietary bloatware, written by a crowd of mediocre coders, selling their time?
Another good example is Erlang. The same principles, the same methodology. Most of Erlang's runtime system were written in Erlang, while implementation of system-level functionality and interfaces to an underlying OS were done in C.
Erlang was made to be understable, predictable and maintainable, and, because of that, fault tolerant and reliable. It is based on a proper principles, discipline and design decision which follows from those principles. It is, of course, not so elegant and consistent as Scheme, but it is fairly the best system for concurrent programming in existence.
There is a small list of things which could be done when we follow the appropriate principles and programming discipline.
We could re-write and optimize the primitive procedures without breaking or rewriting any code which uses them.
For example, we can replace default implementation of hash-tables with a hash-function optimized for our task, representation optimized for our OS and hardware we're using. (there are such things as page-sizes, cache-lines, alignment, padding, etc.)
We could add pool-based memory pre-allocator, with partitioning of the data (which gives us parallel reads). With an immutable data the bookkeeping is much simpler and effecient.
We could replace implementation of file system primitives which ones that uses AIO, or reimplemented network primitives using non-blocking I/O.
We could add new primitive procedures, which could be wrappers to specialized libraries such as memcached or libzeromq or redis.
Everything is on its place, isolated and lose-coupled, without size-effects, on an appropriate level of abstraction, providing a conventional, familiar interface for the layer above.
This is how we're making complex systems - by using high order philosophical principles - Divide et impera. Less is more. Clarity is an evidence.