Saturday, April 1, 2017

Why I'm moving from Rust to C

Rust is a popular new language that I've been using for several years now. My project, a programming language called Lily, has been developed in Rust for the last 5 years. I chose Rust at the time, because I wanted a safer language to write Lily in.

Lily is a simple language: It's statically-typed, but interpreted. It's capable of being embedded or extended from either Rust or C. I wrote the parser, the lexer, the vm, and all that from scratch. Lily clocks in at around roughly 15K lines of Rust. The core of Lily makes use of relatively few of Rust's features. No lambdas, few generics (and those that exist are simple), and sticks to having values as simple Rust types.

After having developed Lily in Rust for several years, I've decided to transition away from Rust and into C. Changing the underlying language was moderately difficult given the size of Lily. However, after a number of considerations, I felt that it was the right choice to make, even if it is not the choice that many projects are currently making.

rustc is slow

Compilers are great at generating fast code, but it is difficult to perform many optimizations while having a fast turn-around. One of Lily's strengths is that it offers simple static typing, while parsing
as fast as you'd expect a language like Python or Lua to. I have a series of roughly 300 tests, spread out over around 2000 lines of code. Some are feature tests, several of that is method tests, and there's a few benchmark/example scripts as well. A majority of the time currently spent in Travis tests is in compiling on Rust.

I greatly enjoy that Rust has great debugging messages. However, I do wish that rustc were not so slow, because it prevents a quick full turnaround from compiling to running all tests. With the new C system, I can shut off -O2, and get a full fresh build of Lily and run all tests in under a couple minutes (down from 15 minute+ builds).

I'm tired of promises of rustc getting faster. Of "mir will make it better" or "the next version will be faster! It's time to tackle this problem now, before Rust programmers as a group get used to a slow compiler and come to accept it as a fact of life (like Haskell programmers).

rust needs unions

Rust does not, as of yet, contain C-style unions. In transitioning Lily to C, I was able to cut 11% of the memory use of an average program by squeezing structures through utilizing unions in the representation of a Lily value, and throughout Lily's symbol table/parser/ast.

I don't care that there's an issue to add unions to the language. I'm talking right now, several years into the development of Rust, that this feature is missing.

linked lists are great

I'm told I should use arrays everywhere. Arrays are a poor choice for a programming language, where I may have one class, or a thousand classes in a file. Trying to use linked lists is painful. In C, it's as natural to use them. All I need to do is fix a next/prev pointer and I'm good. No need to have a mutable Option of Cell of Refable of a Box of a simple pointer.

the circlejerk

I don't want to be associated with a community whose members constantly drive by to projects they have no investment in to inform them of a supposedly superior message. Every time there is a post on Hacker News or Reddit about a vulnerability in C, the strike team comes out to mention a rewrite in Rust. Regardless of Rust's strengths, it's ugly, and it's also insulting to other developers.

rust's safety is overrated

A majority of Lily's development is in writing the parser, the emitter, and then the tooling. A parser and an emitter do not benefit greatly from memory safety. For emitting code, I have a uint16 buffer I wrote that allows safely inserting 1-5 values at a time (it does a grow check before any insert). I don't have unsafe actions spread around my code. Transitioning from Rust to C was actually fairly boring, and I was able to eliminate a great deal of unnecessary matches on Options that used to exist. The new code is much smaller, and as someone who did a lot of C beforehand, I find the new C code to be much simpler and more pleasant to look at.

If you're interested in checking out the now-C source, you can find Lily here: