Posts for the month of April 2012

Why Lisp.

Every self-respecting blogger should write about Lisp at least once. So, there is my take.

There is a nice quote, I don't remember of whom - All models are wrong, but some are useful. They are over-simplifications, but they help us think more clearly. Lets make one simple model.

What is an idea? The word "idea" is the name for some representation I hold in my mind. It is linked with a cloud of other words, with some visualizations, some memorized sentences and some non-verbal feelings. (just a different types of storage with different encoding). When I hear or see the word "idea", this is the meaning I have.

The meaning is an inner representation, encoded as a compound expression, a composite object of the mind. The word is just one of many symbolic pointers, a tag, a label, an anchor for this mental object.

There are several such meanings for each word. They can be fetched from the storage, depending on the context, the current state of the mind, the current mental environment. It is like using different dictionaries to look up the meaning of a word.

A word points to a slot in a look-up table - the context. It provides the addresses in a storage for each slot.

It is a huge over-simplification, but it is very useful one.

For example, what is recursion? It is the word of English language, which is the name for the idea of recursion. Each person on earth have his own inner representation of what it is for him. Well, some people have not. The word have no meaning for them, it is not associated with any inner representation. They have no idea.

Depending on the context it could be a mathematical definition, or an idea from computer science, or even a some feeling of being puzzled.

So, what is a word? It is a symbol which points to a chain of associations - its meaning. What is an context? It is a look-up table in between, filled with the pointers to the meaning of symbols. The storage remain unchanged, it is the contexts that change.

So, the same construction of the language, which describes a person, could have two completely different meanings, depending on the context currently present in my mind:

((The young girl) I have seen esterday).

Depending on context, it could be:

  • a cute high-school gal I saw on a street last morning.
  • my girlfriend, ten year younger, with whom I spent saturday night.

Now, what is Lisp?

Lisp is a language to represent any knowledge stored in our mind, according to this simple model - the chains (actually graphs) of named and unnamed inner representations, verbal and non-verbal.

It uses a list-structure to encode the graphs of associations as a sequence. It is the way to represent a many-dimensional hierarchical data as an one-dimensional ordered sequence.

Any kind of objects could be made out of lists. In particular there is a special kind of objects called closure which represents a piece of encoded knowledge - a procedure along with its data and its context - nested environments where the meaning of all symbols can be found.

Closures are kind of inner representations of the pieces of knowledge inside of a lisp process. They could be manipulated by procedures as data objects, or could be parts of any kind of Lisp expressions.

This provides almost unlimited power to express any idea, comparable with any Natural language. There is no distinction between code and data, between nouns and verbs. One can mix everything to produce any kind of expressions he wishes.

How it works..)

some smart-ass (A) meet some bankster (B)..

A: Look, I've heard you are looking for an opportunity to invest..

B: Forget about it!

A: Listen, I have very good idea - software for Cloud Computing! (Serious people cannot cope with this punk's Linux OS - all those text files and bloody command line! in 2010! People need some serious enterprise tools with industry-strength interface, message-passing middle-ware, database back-end) There is a huge market for it.

B: Um.. Cloud Computing (I'm recalling that John told me something about clouds on that cocktail party) But there are others doing it, aren't they? How could we sell it?

A: Easy, we will make it open source and suckers will download and install it themselves!

B: Well, do you know anything about making a software?

A: Forget about it, we will use Java and proven top-down process, like everyone else!

B: Java.. Sounds good (How do I know? Everyone speaks so. It is industry standard, you know, everyone are using it.)

A: So, we need a headquarter in a Valley, and a Coding team in East Europe, and Q/A team in Asia, and Sales and Marketing here. (I will hire remote coders instead and keep all money..)

B: Yeah, yeah. I know how to run a joint..

A: I will watch all this as CEO (and offload all that development process and all responsibility to VPs)

B: But what if is something will go wrong?

A: C'mon, I will send you a monthly reports with all the numbers and plots - team's sizes, lines of code, man hours, quality control results. You know, the more lines of Java code you have, the more money it worth, the more man-hours spent on it, the more valuable product you have, the more coders you hire, the faster it grows. You will see our growth on a daily basis!

B: Um, well, $10 millions for the first stage..

A: Yeah, 20, we should hire teams and stuff.

B: Yeah, I know!

A: When shall we start?

Something is very wrong here.

Something is very wrong when someone spends time and earns money for typing such piles of nonsense.. Get Context, Get Instance, Target, Injection? Why we need all those words (and Java! OS management from JVM! btw, it is not cool anymore! What's really cool is OS management from NodeJS written in CoffeScript! Look, you can run it on WebOS!), which is an alternative to [ -f /etc/nginx/sites-enabled/default ]. I know the answer - because some pays for it. And this is exactly what is very wrong.

This very idea of trying to manage UNIX-like operating system, which is founded on the concepts of text streams and pipelines, from JVM using all that Enterprise stuff, is so deeply wrong, that I cannot find appropriate description.. It isn't stupidity, it isn't ignorance, it is something like abandoning the reason completely.

Well, some YAML-based configuration, simple templates and some lightweight and self-evident non-OO Python3 code or even Scsh. But this..

services/service-nginx/src/main/java/org/openstack/service/nginx/ops/NginxServerBootstrap.java

package org.openstack.service.nginx.ops;

import java.io.File;

import org.platformlayer.ops.Handler;
import org.platformlayer.ops.Injection;
import org.platformlayer.ops.OpsContext;
import org.platformlayer.ops.OpsException;
import org.platformlayer.ops.OpsTarget;

public class NginxServerBootstrap {

    @Handler
    public void handler() throws OpsException {
        OpsTarget target = OpsContext.get().getInstance(OpsTarget.class);

        target.rm(new File("/etc/nginx/sites-enabled/default"));
    }

    public static NginxServerBootstrap build() {
        return Injection.getInstance(NginxServerBootstrap.class);
    }

}

Look what they do with hostname:

package org.openstack.service.nginx.model;

import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlRootElement;

import org.openstack.service.nginx.ops.NginxBackendController;
import org.platformlayer.core.model.ItemBase;
import org.platformlayer.core.model.PlatformLayerKey;
import org.platformlayer.xaas.Controller;

@XmlAccessorType(XmlAccessType.FIELD)
@XmlRootElement
@Controller(NginxBackendController.class)
public class NginxBackend extends ItemBase {
    public String hostname;
    public PlatformLayerKey backend;
}

This is what a crisis in IT looks like. The replacement of classic SYSV start-up scripts with Java-based crap in Solaris, it is the same madness..

What is Engineering

We are in the middle of the yet another bubble in IT - the bubble of mediocre coder.

We have told in high school that pointers are evil, that memory allocation is difficult, that resource management is boring, so we should buy all that JVM-based crap. (now it is Javascript or Rails - it doesn't matter).

Coders are talking about their tools and their wage labor. They are neglecting ideas, ignoring underlying principles and hardware platforms. They are sure JVM (or V8) will do everything for them.

The art of crafting simple and efficient software was lost in layers upon layers of useless abstractions coders pile up without understanding or even thinking, to get their wages based on lines of code or hour of coding.

This is where we are. In the bubble of wrong tools, bureaucratic processes and mediocre coders.

On the other side, the essence of software engineering is to apply ideas and principles exactly on the intersection between hardware and software.

It is same as body and mind - the body cannot function without the mind and mind cannot exist without the body. Neither cannot be neglected or ignored. Everybody knows what happens if it does.

Engineering is the art of managing complexity and joy of producing simple and efficient, good-enough solutions, the way Nature do.