Vjeux

I'm now [in July 2018] in a group full of compiler engineers at Facebook and learning a lot. Yesterday, I read a post by David Detlefs (summarizing a collaborative idea involving several members of his team) about how to efficiently encode strings for concatenation and since it's very clever I figured I would share it.

Problem

A lot of programs are taking a string as input and building a string as output. You can imagine the following code. Note: I'm going to use JavaScript as an example but it applies to almost all the languages out there.

var str = '\n';
for (var elem of elems) {
  str += ' * ';
  if (elem.isExpired) {
    str += '[expired] ';
  }
  str += elem.name + '\n';
}

An example output might be

'
 * Nutella
 * Eggs
 * [expired] Milk
'

What is being executed is

'\n' + ' * ' + 'Nutella' + '\n' + ' * ' + 'Eggs' + '\n' + ' * ' + '[expired] ' + 'Milk' + '\n'

If you implement this naively, the execution would look something like:

'\n' + ' * ' = '\n * '
'\n * ' + 'Nutella' = '\n * Nutella'
'\n * Nutella' + '\n' = '\n * Nutella\n'
'\n * Nutella\n' + ' * ' = '\n * Nutella\n * '
'\n * Nutella\n * ' + 'Eggs' = '\n * Nutella\n * Eggs'
'\n * Nutella\n * Eggs' + '\n' = '\n * Nutella\n * Eggs\n'
'\n * Nutella\n * Eggs\n' + ' * ' = '\n * Nutella\n * Eggs\n * '
'\n * Nutella\n * Eggs\n * ' + '[expired] ' = '\n * Nutella\n * Eggs\n * [expired] '
'\n * Nutella\n * Eggs\n * [expired] ' + 'Milk' = '\n * Nutella\n * Eggs\n * [expired] Milk'
'\n * Nutella\n * Eggs\n * [expired] Milk' + '\n' = '\n * Nutella\n * Eggs\n * [expired] Milk\n'

Because strings are immutable, we need to do a full copy of the string for every small concatenation. In practice this turn a O(n) algorithm into O(n²).

Solution 1: Change the code

If this becomes a bottleneck, instead of using a string all the way through, you can use an array and push all the string pieces to it. Once you are done building the result, you can join all the pieces together into the final string. Since at this point you know all the strings the operation can sum all the sizes and allocate exactly the right size.

var buffer = ['\n'];
for (var elem of elems) {
  buffer.push(' * ');
  if (elem.isExpired) {
    buffer.push('[expired] ');
  }
  buffer.push(elem.name, '\n');
}
str = buffer.join('')

This pattern works to solve the problem but requires the programmer to know about it and the performance to be bad enough that it is worth writing code in a different way. In practice, a lot of code is not written that way and it's unclear that any amount of education will change this fact.

Note that a compiler to a bytecode format could, in many cases, make the transformation of the original code to the explicit StringBuffer code. But not in all cases, since compilers have to be conservative: if the string being concatentated is passed as an argument, all bets are off.

Broken Solution 2: Mutate the original string

The solution that comes to mind is: can normal strings act as a buffer?

The idea is that you allocate a buffer of characters and whenever you do a concatenation, you keep writing at the end of the buffer. If it isn't big enough, you allocate a bigger one, do a single copy and keep going.

var str = '\n';   // size = 1, ['\n', _, _, _, _, _, _, _]
str += ' * ';     // size = 4, ['\n', ' ', '*', ' ', _, _, _, _]
 
// The next one doesn't fit so we need to alloc a new buffer and do a full copy
str += 'Nutella'; // size = 11, ['\n', ' ', '*', ' ', 'N', 'u', 't', 'e', 'l', 'l', 'a', _, _, _, _, _]
str += '\n';      // size = 12, ['\n', ' ', '*', ' ', 'N', 'u', 't', 'e', 'l', 'l', 'a', '\n', _, _, _, _]

If you are curious, this is how the Java StringBuilder class is implemented. Performance-wise, this is what we want, but there's one problem...

Aliasing

You can assign the string to a variable and assign another variable with that variable. For example:

var str = '\n';
var str2 = str; // here we make an alias

In this case, both str and str2 are pointing to the same '\n' string. In the compiler literature this is called aliasing. The big question is what happens if you try to update one of the variable:

str += ' * ';

If you look at the JavaScript specification, strings are immutable meaning that you expect str2 to be unchanged but str to be:

str2 == '\n'
str  == '\n * '

Unfortunately, if you mutate the string like in the above solution, then both of them would be '\n * because they both point to the same underlying storage.

Solution 3: Linear Types

If you've not been living under a rock, you probably have heard about Rust and linear types. This is a fancy name to say that you cannot have aliasing: there's only a single variable that can point to a value at all time.

What this means in this case is that the line var str2 = str; would be illegal. If you want to do that, you need to do a full copy of the value so it's effectively a different one.

In practice, aliasing happens all the time in normal programs, for example calling a function with a string as argument is a form of aliasing. We wouldn't want to do full copies every time aliasing is happening.

Rust is getting away with it using a concept calling "borrowing" where you can create an alias if the compiler can guarantee that the previous variable cannot be accessed during the lifetime that the alias exists.

In my understanding, you need a strong type system in order to properly enforce those guarantees and in dynamic languages like JavaScript you would have to be too pessimistic and do way more copies than necessary when you are just passing the variable around, ruining the wins you get from building the string in the first place.

Solution 4: Size in the variable

Aliasing is usually a dealbreaker because you can mutate the underlying storage that another variable could observe. But in this particular case we can exploit the fact that the only mutation we care about is appending something at the end.

So, in the variable we not only keep a pointer to the buffer but also the size we care about. If someone else appends something at the end, it will not affect us because what's in the buffer for that size didn't change.

var str = '\n';
// buffer1, size = 1, ['\n', _, _, _, _, _, _, _]
// str : size = 1, buffer = buffer1
 
var str2 = str;
// str2: size = 1, buffer = buffer1
 
str += ' * ';
// buffer1, size = 4, ['\n', ' ', '*', ' ', _, _, _, _]
// str : size = 4, buffer = buffer1

At this point, str2 points to buffer1 with '\n * ' but because it has size = 1 then we know it really is '\n' as intended.

The only edge case to consider is if you are trying to also concatenate str2. If the size of the variable is not equal to the size of the underlying buffer, this means that someone else clobbered the buffer. In this case, our only option is to do a full copy.

str2 += '|';
// buffer2, size = 8, ['\n', '|', _, _, _, _, _, _]
// str2: size = 2, buffer = buffer2

Conclusion

Before joining the team, I knew about the string builder pattern but I had no idea that there was so much theory behind this particular problem like aliasing, linear types... I hope that explaining those concepts in terms of JavaScript is helpful to get some insights into what's happening inside of compilers.

Andres Suarez pointed me to some interesting code in the Hack codebase:

let slash_escaped_string_of_path path =
  let buf = Buffer.create (String.length path) in
  String.iter (fun ch ->
    match ch with
    | '\\' -> Buffer.add_string buf "zB"
    | ':' -> Buffer.add_string buf "zC"
    | '/' -> Buffer.add_string buf "zS"
    | '\x00' -> Buffer.add_string buf "z0"
    | 'z' -> Buffer.add_string buf "zZ"
    | _ -> Buffer.add_char buf ch
  ) path;
  Buffer.contents buf

What it does is to turn all the occurrences of \, :, /, \0 and z into zB, zC, zS, z0 and zZ. This way, there won't be any of those characters in the original string which are probably invalid in the context where that string is transported. But you still have a way to get them back by transforming all the z-sequences back to their original form.

Why is it useful?

The first interesting aspect about it is that it's using z as an escape character instead of the usual \. In practice, it's less likely for a string to contain a z rather than a \ so we have to escape less often.

But the big wins are coming when escaping multiple times. In the \ escape sequence, it looks something like this:

• \ -> \\ -> \\\\ -> \\\\\\\\ -> \\\\\\\\\\\\\\\\

whereas with the z escape sequence:

• z -> zZ -> zZZ -> zZZZ -> zZZZZ

The fact that escaping a second time doubles the number of escape characters is problematic in practice. I was working on a project once where we found out that the \ character represented 70% of the payload!

Conclusion

It's way too late to change all the existing programming languages to use a different way to escape characters but if you have the opportunity to design an escape sequence, know that \ escape sequence is not always the best 🙂

It's very trendy to bash at Computer Science degrees saying that it costs a lot of time and money and at the end, you haven't learned much useful things for your day job. My experience going to EPITA, a French school in Paris, has been the complete opposite!

I started programming when I was around 10. By 13, I was already being contracted for real money by someone in the US (Hi Thott!). So, when I started EPITA at 18, I already had a ton of experience as a self-taught programmer. My biggest fear was: "Would I learn something new?" and, I did learn a ton! I know wouldn't have done all the impactful things I have at Facebook without it.

What struck me was that I already knew a lot and I knew there was a lot left to learn, but I didn't really know what nor how to get started learning it on my own.

Big Themes

Here are some themes where I learned entirely new domains of knowledge at school, many of which I would likely not have learned, or at least not as in depth if I didn't do a CS degree.

• C: I've written a ton of C at school and it was super annoying but learning how to do manual memory management, use pointers and have raw access to memory was a huge breakthrough in how I understood how code was actually running.
• Data Structures and Algorithms. I had 6 hours of courses per day for 2 years. There's so much to learn! What's interesting is that you don't need any of it to build what you want, but once you hit scale and you want to optimize and maintain it, knowing about all of them become critical!
• OCaml: The whole idea of manipulating immutable lists via recursion to actually do something useful at the end was very weird. Even though I still despise this style of programming, there are good ideas behind it and React is a good example of a practical system that makes use of them.
• Source Control: I used to change index2.php on the production machine and cp it to index.php using sftp when I was happy with my changes... for a website with 1 million unique visitors a day... It was fun and scary, but I wouldn't go back to that in a million years!
• Assembly: Learning how a computer works from nand gates all the way to assembly was really mind blowing. There are so many levels of abstractions happening between the hardware and some piece of JavaScript that executes in the browser that it's hard to believe it works and isn't crazy slow.
• Machine Learning (it wasn't hype at the time) is really not that magical. It's a bunch of heuristics in order to massage the data in a way that it can be separated by a line. It also made me happy that finally all the linear algebra I've learned for years was useful to something. (Math was also useful for all the signal processing stuff with fourrier transform!)
• Academia. My theory is that nothing is new, everything has already been explored but because people there insist on using latex with maths symbols everywhere and publish to closed platforms, it doesn't get the reach it deserves.
• Tradeoffs. If I were to summarize all the learnings, it's probably that there is not one perfect solution. There's usually a bunch of solutions with different sets of tradeoffs and you need to find the least shitty one. It was a sad realization but gave me a much better framework to work with.

Specific School Projects

What was awesome about EPITA is that not only we had classes like any normal school but also a ton of actual non-trivial projects to implement. Here are some that had a profound impact on me:

• Reimplementing Warcraft 3 from scratch. I learned so much about parsing (and reverse engineering) data structures, the whole 3D rendering space... But also having a project with a team of 4 people for an entire year and how to sell. It was a pretty awesome first year project 🙂
• Reimplementing malloc from scratch (btw, printf uses malloc which makes debugging interesting...). Learned so much about memory management and running firefox using my malloc was so awesome!
• Reimplementing bash from scratch. I learned a ton about how to parse and execute programming languages and how crazy the unix low level apis are. I'm still using this knowledge today when writing scripts thanks to this project.
• Implementing a regex execution using OCaml. OCaml is --really-- good for working with well defined data structures and algorithms (I have big reservations for anything else outside of this narrow scope...). Since I'm using regexes almost on a daily basis, it's been super helpful.
• We had 4-hour machine exams every couple of weeks for a year where you basically have to solve as many interview-style problems as possible. This was so much fun and helped to land a job at Facebook.
• Implement the fastest possible fuzzy finder. I learned so much about orders of magnitude in perf: just opening the file in JavaScript took way too long, C is faster... malloc is really slow, custom bump allocator is better. Smaller structs is better, bit packing ftw. Pre-computing data for heuristics can dramatically help.

And many more... but that's a good enough list for now!

Actual Use Cases

Now, the big question is: but did you use any of that at work? And the answer is yes, so many times!

• Small thing but when building our open source websites, I wanted to only write next: page in the markdown and not have to have to write prev nor have a global index and yet have an table of content. It took me a second to realize that I had a list of edges of a graph and I needed to rebuild a graph from it. Because I've written so many graph traversals at school, it took me under an hour to build it correctly.
• EPITA enforced strict lint rules where for each violation you'd lose 2 points (out of 20) for all your assignments. The trick is that you were only given a pdf with the rules and no program to verify them, so you had to learn them the hard way. This motivated me years later to work on prettier so that we can have the benefits of a consistent codebase, the less hardcore way...
• When I was pitched React for the first time, it sounded to me crazy that re-rendering everything could be fast, but once I was proved it was not the case, I already knew it would work. I already made the mental journey of all thinking about the tree traversals, functional programming paradigms... It didn't sound foreign to me as it did for so many other people.
• I've spent a lot of time building lint rules, codemods, extended syntax, pretty printers... because I've had excellent courses on how to parse and execute programming languages and had to build many at school, the challenge was not about learning how it works but more how to actually implement it in the context of JavaScript.
• While on the photos team at Facebook, one of the issue was that we cropped photos in a terrible way. Building an algorithm to find the best crop turned out to be manageable because I knew how to read the literature around existing approaches. Once I settled on one, I already wrote a ton of similar algorithms so it was just a matter of writing code down with little challenge.

Conclusion

I could have gotten a well paying programming job right out of high school as a web developer, but going to EPITA made me such a better programmer because I've been exposed in depth to a lot of areas that I would likely have never gone on my own.

Now, if I see a problem on a web developer task, I also know that changing the programming language, using machine learning, writing code in a more C-like way, using other data structures and algorithms... are possible tools to solve that problem. More importantly, if I need to use those, I can implement them because I've already done similar things in the past.

Your experience may vary, but for me, getting a CS degree at EPITA was a really good decision and I would do it again if I had to.

During the past few weeks, I've been working on prettier, which is a JavaScript pretty printer. We are approaching the phase where we can actually use it so this is a good time to explain how it works.

We're going to go through an example

if (!pretty) { makePretty() }

String -> AST

The first step is to take this string that represents some JavaScript and to parse it in order to get an AST out of it. An AST is a tree that represents the program. Using either Babylon or Flow we can parse this example and we get the following tree.

Program
  IfStatement
    UnaryExpression(!)
      Identifier(pretty)
    BlockStatement({})
      ExpressionStatement
        CallExpression
          Identifier(makePretty)

You can explore the full AST using astexplorer.net.

AST -> IR

Now that we have this tree, we want to print it. For each type of node like IfStatement, UnaryExpression... we're going to output something. In the case of prettier, this something is an intermediate representation called a document as described by the paper a prettier printer by Philip Wadler.

[
  group([
    "if (",
    group([ indent(2, [ softline, "!", "pretty" ]), softline ]),
    ")",
    " ",
    "{",
    indent(2, [ hardline, "makePretty", "()", ";" ]),
    hardline,
    "}"
  ]),
  hardline
];

You can play around with this representation on the prettier explorer.

IR -> String

The interesting thing about this representation is that it is the same no matter what the line-length is. The basic idea is that the primitives such as group, indent, softline encode the way they should look if they fit in the line or if they don't.

The most important primitive is group. The algorithm will first try to recursively print a group on a single line. If it doesn't fit the desired width, then it's going to break the outer group and keep going.

Then, we have primitives that behave differently if they are in a group that fits a single line or not: softline that does not print anything if the group it is contained in fits and a line otherwise. indent adds a level of indentation if it doesn't fit. If you are curious, you can look at the short list of available commands.

So, we just need to take this IR, send it through a solver along with the desired line width and we get the result!

Conclusion

Hopefully this gives you a better idea of how a pretty printer that takes into account the desired width work.

I had the opportunity to do a live fireside chat with Lee Byron to chat about React Native and GraphQL. I outlined the boring future of React Native 😉

It was pretty surreal as I got to have makeup done by a professional!

I also tested the live video feature of Facebook on the open source booth at f8. I showcased React Native running on Xbox, Windows Phone and Desktop and the F8 app being open sourced.