wiki:Hybridr

A hybrid dialect of Scheme inspired by T, Arc, Erlang, Go, Plan9, nginx, redis, Clojure and Death Note.)

Hybrid (biology) - an offspring resulting from cross-breeding

Several simple ideas put together:

General principles:

  • Unity and clarity of the language and underlying data structures. It means "don't make me think why there is (a b [x y]) in code instead of usual (a b (x y)) and especially when I should say (a b [x y]) and when (a b (c d))". Keep details of representation out. Annotations is better.
  • "Mostly functional" is good enough. The compromises from Scheme and Common Lisp (real world is not ideal) - some times we need to (set! ...) something. Pure-functional languages is just another extreme. We believe in the middle way. Erlang is the result of a balanced approach.
  • A modern multi-core x86_64 CPU and OS Linux together is very good VM itself. We should use what modern operating systems (caching, asynchronous I/O) and CPUs (hardware-assisted atomic operations, 64 and 128 bit registers, vectorization) provide to us. It is more than enough.
  • Some pure-functional code could be vectorized and executed in parallel, some couldn't. Some code should be executed on CPU, some on GPU. Code should be written the way it intent to be executed.
  • We have a big namespace of functions associated with symbols. Most of them are pure, some of them aren't. We can (define ..) and (compile ..) and (ffcall ..) our own.

Language perspective:

  • All you need is Lambda (c) Brian Harvey the enlightened teacher.
  • Everything is still an atom or a list or a closure. Unity. In the language everything is an atom, a list (composed from conses) or a closure (code-as-data + environments) which accepts messages. But it must be compiled into more advanced (pool-allocated, properly aligned, cache-friendly) yet simple (pointers to pointers, everything is 64bit) data structures.
  • Prefix notation is universal. Code is a data. Clarity. Do not break the illusion (sometimes illusion is all we need) even for hash-tables (it is a closure). In code they should appear as if they are made from cons cells. Almost everything could be made out of them.

Ideas:

  • from Arc - the Core Language concept + macros and pluggable helper functions.
  • from Erlang - mostly-functional approach - isolate side-effects - 80% pure functions, 20% unsafe.
  • from Google's Go - Back to the basics - interfaces, instead of classes, channels, coroutines.
  • from Clojure - cons, car and cdr manipulate sequence abstractions, but only on the compiler level.
  • from Scheme - unity, clarity and beauty.
  • from Common Lisp - compromises.

Implementation perspective:

  • 64bit everywhere - a modern CPU with x86_64 instruction set is good hardware VM.
  • A64 will arrive shortly
  • Native code compiler - no bytecode VMs. Use your CPU.
  • A do not malloc() each cons cell. We have a lot of cheap memory.
  • Good coders code, good engineers - reuse (20/80 rule).
  • Do not re-implement the wheel, dload and call (avoid VMs).

Environment perspective:

  • Close enough to hardware, and OS (the nginx-way).
  • Not an isolated VM, but deeply integrated with OS runtime.
  • Reuse OS primitives - caching, scheduling, events, etc.

FFI:

  • no re-implementation madness of Java/NodeJS. FFI is good enough. go ahead, re-implement opencl or openmpi.
  • we do not want to re-implement cryptographic functions, we want just dlopen and call OpenSSL or nss.
  • I like to FFI and map http-level functions from nginx to Lisp, because I can't write C that good.

Assorted ideas:

  • from Common Lisp: (compile 'foo)
  • Linux/BSD kernel does scheduling and memory allocation for us. Do not even try to do it from a user-level.
  • from Erlang: we like share-nothing pre-forked processes with CPU-affinity and message-passing. No threads.
  • from Erlang: process hierarchies, fault tolerance (just restart), light-weight, isolated coroutines.
  • do not malloc/free each cons cell. Use pool-allocation, partitioning, bigger blocks. Do not interfere with kernel.
  • do not delude yourself with naive views about threads - modern OSes and CPUs are too complex. All that hyper-threading is a scam. Know how kernel manages processes and maps them to different cores (real and fake).
  • virtualization is a scam for sale to suckers - yet another level of unnecessary complexity, that mess everything up.

from varnish-cache:

  • kernel can schedule and cache I/O better that we. So, be it.
  • kernel can do sendfile(), aio_, atomic operations, semaphores.
  • remember that we are in a user level - do not switch context too often. Use bigger blocks, AIO, event-driven logic.

from nginx:

  • close-to-hardware, zero-copy, efficient, platform-optimized primitives - lists, trees, hash-tables, buffers.
  • syscalls are expensive, count each syscall.
  • modules (and plugins) are good ideas.

Why hybridr? Because it sounds like caddr, which is a small miracle itself.

When will we start coding? Who is John Galt?

Last modified 8 years ago Last modified on Oct 12, 2012, 1:47:44 PM
Note: See TracWiki for help on using the wiki.