Saturday, April 1, 2017

Why I'm moving from Rust to C

Rust is a popular new language that I've been using for several years now. My project, a programming language called Lily, has been developed in Rust for the last 5 years. I chose Rust at the time, because I wanted a safer language to write Lily in.

Lily is a simple language: It's statically-typed, but interpreted. It's capable of being embedded or extended from either Rust or C. I wrote the parser, the lexer, the vm, and all that from scratch. Lily clocks in at around roughly 15K lines of Rust. The core of Lily makes use of relatively few of Rust's features. No lambdas, few generics (and those that exist are simple), and sticks to having values as simple Rust types.

After having developed Lily in Rust for several years, I've decided to transition away from Rust and into C. Changing the underlying language was moderately difficult given the size of Lily. However, after a number of considerations, I felt that it was the right choice to make, even if it is not the choice that many projects are currently making.

rustc is slow

Compilers are great at generating fast code, but it is difficult to perform many optimizations while having a fast turn-around. One of Lily's strengths is that it offers simple static typing, while parsing
as fast as you'd expect a language like Python or Lua to. I have a series of roughly 300 tests, spread out over around 2000 lines of code. Some are feature tests, several of that is method tests, and there's a few benchmark/example scripts as well. A majority of the time currently spent in Travis tests is in compiling on Rust.

I greatly enjoy that Rust has great debugging messages. However, I do wish that rustc were not so slow, because it prevents a quick full turnaround from compiling to running all tests. With the new C system, I can shut off -O2, and get a full fresh build of Lily and run all tests in under a couple minutes (down from 15 minute+ builds).

I'm tired of promises of rustc getting faster. Of "mir will make it better" or "the next version will be faster! It's time to tackle this problem now, before Rust programmers as a group get used to a slow compiler and come to accept it as a fact of life (like Haskell programmers).

rust needs unions

Rust does not, as of yet, contain C-style unions. In transitioning Lily to C, I was able to cut 11% of the memory use of an average program by squeezing structures through utilizing unions in the representation of a Lily value, and throughout Lily's symbol table/parser/ast.

I don't care that there's an issue to add unions to the language. I'm talking right now, several years into the development of Rust, that this feature is missing.

linked lists are great

I'm told I should use arrays everywhere. Arrays are a poor choice for a programming language, where I may have one class, or a thousand classes in a file. Trying to use linked lists is painful. In C, it's as natural to use them. All I need to do is fix a next/prev pointer and I'm good. No need to have a mutable Option of Cell of Refable of a Box of a simple pointer.

the circlejerk

I don't want to be associated with a community whose members constantly drive by to projects they have no investment in to inform them of a supposedly superior message. Every time there is a post on Hacker News or Reddit about a vulnerability in C, the strike team comes out to mention a rewrite in Rust. Regardless of Rust's strengths, it's ugly, and it's also insulting to other developers.

rust's safety is overrated

A majority of Lily's development is in writing the parser, the emitter, and then the tooling. A parser and an emitter do not benefit greatly from memory safety. For emitting code, I have a uint16 buffer I wrote that allows safely inserting 1-5 values at a time (it does a grow check before any insert). I don't have unsafe actions spread around my code. Transitioning from Rust to C was actually fairly boring, and I was able to eliminate a great deal of unnecessary matches on Options that used to exist. The new code is much smaller, and as someone who did a lot of C beforehand, I find the new C code to be much simpler and more pleasant to look at.

If you're interested in checking out the now-C source, you can find Lily here:


  1. Maybe you should write it in dlang. Compiling It is even faster than C. It is safer than C and have unions too

    1. Missed "the circlejerk" section huh?

  2. I don't think you are making a fair comparison, except that rustc is to slow. The main benefits of rust are in developing multithreading or parallel programs, because this is where the memory model shines. So you have program that can be well implemented in C. So you're point should be that rust is a tool like any other, in you're case C, and should be used according to the task :-).

  3. If compilation speed is really the most important aspect here, then kozzi1 suggestion is spot on. Nothing beats D in compilation speed. To give a concrete example, the reference compilers front-end was ported itself from C++ to D and the compilation times are spectacular. You can try yourself by cloning the dmd compiler from github. Building version 2.069 takes on my server at work (westmere 2GHz) 1'34". Build version 2.073 take 0'15". And most of the 15 seconds comes from compiling the backend which is not yet ported. If it was the compilation time would be in the 2 to 3 second range. That for a compiler of a language that can do everything you can do in C++ and more.

  4. I too jumped down here to suggest a look at D, and I'm happy to see others have already done so.

  5. I would love to see the results of fuzzing both versions of Lily, Rust version and C version.

  6. I’m OK with the most part of what you wrote, except on two points.

    Firstly, C-type unions are an abomination. The only legit use for them I can think of, is to interact with a C program that already uses them (damn syscalls…). Otherwise, enums are a far better replacement.

    Secondly, this is a perfectly functional linked list in Rust and it is far from being as complicated as you put it.

    enum List<A> {
    Elem(A, Box<List<A>>)

  7. Does it make sense to have Rust as target for your thingie?