Saturday, April 1, 2017

Why I'm moving from Rust to C

Rust is a popular new language that I've been using for several years now. My project, a programming language called Lily, has been developed in Rust for the last 5 years. I chose Rust at the time, because I wanted a safer language to write Lily in.

Lily is a simple language: It's statically-typed, but interpreted. It's capable of being embedded or extended from either Rust or C. I wrote the parser, the lexer, the vm, and all that from scratch. Lily clocks in at around roughly 15K lines of Rust. The core of Lily makes use of relatively few of Rust's features. No lambdas, few generics (and those that exist are simple), and sticks to having values as simple Rust types.

After having developed Lily in Rust for several years, I've decided to transition away from Rust and into C. Changing the underlying language was moderately difficult given the size of Lily. However, after a number of considerations, I felt that it was the right choice to make, even if it is not the choice that many projects are currently making.


rustc is slow

Compilers are great at generating fast code, but it is difficult to perform many optimizations while having a fast turn-around. One of Lily's strengths is that it offers simple static typing, while parsing
as fast as you'd expect a language like Python or Lua to. I have a series of roughly 300 tests, spread out over around 2000 lines of code. Some are feature tests, several of that is method tests, and there's a few benchmark/example scripts as well. A majority of the time currently spent in Travis tests is in compiling on Rust.

I greatly enjoy that Rust has great debugging messages. However, I do wish that rustc were not so slow, because it prevents a quick full turnaround from compiling to running all tests. With the new C system, I can shut off -O2, and get a full fresh build of Lily and run all tests in under a couple minutes (down from 15 minute+ builds).

I'm tired of promises of rustc getting faster. Of "mir will make it better" or "the next version will be faster! It's time to tackle this problem now, before Rust programmers as a group get used to a slow compiler and come to accept it as a fact of life (like Haskell programmers).


rust needs unions

Rust does not, as of yet, contain C-style unions. In transitioning Lily to C, I was able to cut 11% of the memory use of an average program by squeezing structures through utilizing unions in the representation of a Lily value, and throughout Lily's symbol table/parser/ast.

I don't care that there's an issue to add unions to the language. I'm talking right now, several years into the development of Rust, that this feature is missing.


linked lists are great

I'm told I should use arrays everywhere. Arrays are a poor choice for a programming language, where I may have one class, or a thousand classes in a file. Trying to use linked lists is painful. In C, it's as natural to use them. All I need to do is fix a next/prev pointer and I'm good. No need to have a mutable Option of Cell of Refable of a Box of a simple pointer.


the circlejerk

I don't want to be associated with a community whose members constantly drive by to projects they have no investment in to inform them of a supposedly superior message. Every time there is a post on Hacker News or Reddit about a vulnerability in C, the strike team comes out to mention a rewrite in Rust. Regardless of Rust's strengths, it's ugly, and it's also insulting to other developers.


rust's safety is overrated

A majority of Lily's development is in writing the parser, the emitter, and then the tooling. A parser and an emitter do not benefit greatly from memory safety. For emitting code, I have a uint16 buffer I wrote that allows safely inserting 1-5 values at a time (it does a grow check before any insert). I don't have unsafe actions spread around my code. Transitioning from Rust to C was actually fairly boring, and I was able to eliminate a great deal of unnecessary matches on Options that used to exist. The new code is much smaller, and as someone who did a lot of C beforehand, I find the new C code to be much simpler and more pleasant to look at.

If you're interested in checking out the now-C source, you can find Lily here: https://github.com/fascinatedbox/lily

12 comments:

  1. Maybe you should write it in dlang. Compiling It is even faster than C. It is safer than C and have unions too

    ReplyDelete
    Replies
    1. Missed "the circlejerk" section huh?

      Delete
  2. I don't think you are making a fair comparison, except that rustc is to slow. The main benefits of rust are in developing multithreading or parallel programs, because this is where the memory model shines. So you have program that can be well implemented in C. So you're point should be that rust is a tool like any other, in you're case C, and should be used according to the task :-).

    ReplyDelete
  3. If compilation speed is really the most important aspect here, then kozzi1 suggestion is spot on. Nothing beats D in compilation speed. To give a concrete example, the reference compilers front-end was ported itself from C++ to D and the compilation times are spectacular. You can try yourself by cloning the dmd compiler from github. Building version 2.069 takes on my server at work (westmere 2GHz) 1'34". Build version 2.073 take 0'15". And most of the 15 seconds comes from compiling the backend which is not yet ported. If it was the compilation time would be in the 2 to 3 second range. That for a compiler of a language that can do everything you can do in C++ and more.

    ReplyDelete
  4. I too jumped down here to suggest a look at D, and I'm happy to see others have already done so.

    ReplyDelete
  5. I would love to see the results of fuzzing both versions of Lily, Rust version and C version.

    ReplyDelete
  6. I’m OK with the most part of what you wrote, except on two points.

    Firstly, C-type unions are an abomination. The only legit use for them I can think of, is to interact with a C program that already uses them (damn syscalls…). Otherwise, enums are a far better replacement.

    Secondly, this is a perfectly functional linked list in Rust and it is far from being as complicated as you put it.

    enum List<A> {
    Nil,
    Elem(A, Box<List<A>>)
    }

    ReplyDelete
  7. Does it make sense to have Rust as target for your thingie?

    ReplyDelete
  8. Personally I find Rust as a language great but the community, oh boy...
    Now, that's not the fault of the language itself but it really gets annoying quickly.

    I do get it, *SOME* fans of Rust do think that Rust is your one stop solution to every Software Engineering problem on the planet but the reality is that many people will have to use a variety of languages, depending on their jobs.

    I've to use C, C++, Java and C# at my workplace, where C and C++ are used for real time applications and Jave and C# are mostly used for our desktop applications (like a GUI for one of our backends).

    So bashing on Java, C#, C++ and C doesn't help in any way, it's not like that reality will bend to their wills.
    This is especially bizzare as Rust itself is a good language, so they should just let its qualities speak for itself.

    Then again, I also do agree about the compile times.

    ReplyDelete
  9. If compilation speed is your primary criteria, then choose golang

    ReplyDelete
  10. I read your post. It was very nice. I learned something knowledgeable. Thanks for sharing.
    You can find there lot's of free stuff like free Rust skins and many more!

    ReplyDelete