Behind the Scenes with CHICKEN Scheme (Part 1)

For the past couple years, I’ve been playing with the Lisp family of languages, namely Common Lisp, Clojure, and Scheme. One of my favorite languages for hobby coding is CHICKEN Scheme, a mature, high-performance implementation of Scheme that compiles to portable C code. CHICKEN’s variety of built-in features and downloadable libraries, excellent FFI support, and helpful community make it a very appealing language.

Recently, I came across SPOCK, a compiler and runtime system for compiling Scheme code into JavaScript. As it turns out, SPOCK and CHICKEN have the same creator: Felix Winkelmann, a software developer in Göttingen, Germany. Intrigued, I got in touch with Felix to ask him about CHICKEN, SPOCK, how he got started, and what keeps him motivated to keep working on CHICKEN after more than a decade.

In this first part of the interview, we discuss how Felix became interested in Scheme and programming, why he created and maintains CHICKEN, and what makes it special. In part 2 (coming soon), we’ll cover SPOCK, how it relates to CHICKEN, and the origins of the names “CHICKEN” and “SPOCK”.

Felix, thanks for agreeing to an interview. Many of our readers probably haven’t heard of CHICKEN Scheme before. What is it? What kinds of software is it good for? What sets it apart from other Scheme implementations?

CHICKEN is, at its core, just another implementation of the Scheme programming language. It is R5RS-compliant and provides numerous extension libraries for all sorts of things. CHICKEN compiles Scheme into portable C code, which can subsequently be compiled into a standalone executable or a library. An interpreter is also available for interactive development.

There are a large number of extensions (what we call “eggs”) that cover a large spectrum of functionality, like bindings to C and C++ libraries and handling of many databases, protocols, networking, graphics, and user interface programming. So I’d say it is good for a lot of things. Dynamic typing combined with a compiler that can generate quite efficient code allows CHICKEN to be used for everything including scripting, application programming, and systems programming.

What I like about CHICKEN is that it makes it very easy to work with existing libraries. The foreign function interface makes integrating C code a snap. Many people have contributed extensions to our library, and installing these extensions is straightforward. CHICKEN tries to be developer-friendly and easy to use, and it puts an emphasis on making those things simple that have traditionally been neglected in dynamic languages, like generating real standalone executables. Full support for Scheme is provided, including the parts that are usually hard to implement or implemented inefficiently.

But what really makes CHICKEN special is its community. A group of helpful and faithful fanatics is actively maintaining and improving it, sometimes at a frightening pace. If you need help, ask on the mailing lists or IRC channel and you get it. Always.

What motivated you to create CHICKEN?

I was scratching my own itch: having a decent compiler for a powerful and elegant language, one that I can use for day-to-day programming instead of banging my head against the limitations of the mainstream languages that I have to use otherwise. Something that doesn’t get in the way of solving a particular programming problem.

Do you often use CHICKEN in your own programming? What kinds of software to you create with it?

I use CHICKEN as much as I can. I have done some freelancing writing Scheme, but haven’t had the chance so far to use it at work (I’d love to, though). Unfortunately I don’t have much time left between work and maintenance, even though my head is exploding with ideas. If I find the time, I usually implement other programming languages — I’m one of those programmers that always end up implementing programming languages in the hope of using them to write something interesting in the future… But I never get beyond the first stage. :-)

When did you first become interested in computers or programming? How did you learn to program?

I started around the age of 12, I think, at the start of the home computing era. I never got the computers with the cool games like the other kids, so I had to dive into BASIC programming pretty early. Later I studied mathematics and computer science but quickly realized that I’m way too dumb for maths and dropped out after just a year or so.

I’m addicted to computer books, so I was able to pick up a lot of different subjects, but I always ended up learning about programming languages.

How did you first learn about Scheme/Lisp? Did you find it challenging at first, or did it come easily? What made you like it enough to create a Scheme implementation?

I somehow came upon a small book about Lisp, which was very challenging and strange. But my fascination started early and I sucked up everything I could about Lisp, its various variants and the implementation techniques involved in making it run. Scheme, being such a clean, minimal and elegant language got me quickly hooked, as it did to so many others.

Internet access came very late, so to get access to a Lisp system I had to write one myself. I wrote countless Lisp and Scheme implementations — most of them were rubbish, and none was ever complete. But implementing Lisp is the true way of learning the language, and in the end, reading Henry Baker’s “Cheney on the M.T.A.” paper and Andrew Appel’s wonderful book “Compiling with Continuations” showed a way that was just so elegant that I had to try it out.

You first released CHICKEN in 2000 — over a decade ago! What motivates you to keep working on it after so many years? Have you ever had times of low motivation, where you didn’t want to work on it anymore? How did you cope?

Yes, I think hacking began about 15 years ago. It’s hard to believe that it has been such a long time. I wanted to stop more than once, but what made the difference was the feedback I got. Even when the system was barely usable (actually even when it wasn’t usable at all), people tried it out, sent patches, suggested improvements and, most surprisingly, they used it! For real stuff! That was both baffling and highly motivating. Being so grateful for the feedback, I couldn’t stop working on it.

Maintaining such a project, especially one that is growing very fast, can be quite a piece of work. Over the years, a core team of very capable, motivated and friendly folks has emerged that do all the hard work and additionally keep up with my moods. But before that, keeping up with the project (bug-fixing, porting, testing) turned out to be a full-time job. I was ready to walk away more than once, and not having the time to use the stuff you worked on for such a long time can be quite frustrating. Usually, taking a few weeks of vacation from all things CHICKEN related helps, until my fingers start itching, the ideas start flowing, and I throw myself back into the project.

In your initial announcement of CHICKEN, you included a disclaimer: “This is *not* a production quality/high-performance system.” A lot has changed since then. Would you say now that CHICKEN is a “production quality/high performance system”?

Yes, I think I’d say that. The compiler can generate very fast code, if you know what you’re doing and if you have an idea of how it operates. A massive amount of code has been fed to the system, which weeded out a countless number of bugs. So it is not too immodest to say that CHICKEN has become quite mature.

It will never be bug free, of course, but that is the price you pay for keeping up a fast pace of development. With maturity, the class of bugs shifts to more advanced and obscure parts of the system. Additionally, we do an awful lot of automated testing, which is of tremendous help.

Do you have future plans for CHICKEN? Where would you like it to be in 5 years?

There are many things that need to be improved. People are using it, heavily, and companies have started using it for getting real stuff done, so there is always something to fix and improve. The next Scheme standard (R7RS) is around the corner, and we plan to support it, which will be another piece of work. A lot of infrastructure has been created (testing, bug-tracking, code repositories, documentation, etc.) that needs constant attention.

I don’t know. I think in 5 years I would like it to be like it is now — just better.

The interview continues in part 2 — coming soon.