Original post

I lead a PL/VMs/compilers research group and the number of organisations interested in hiring people in this area keeps growing and growing. Now, that’s not to say that there are ever going to be hundreds of thousands people working in this field, but it turns out that lots of organisations have realised they have pressing language needs. It varies all the way from those wanting to make mainstream language implementations perform better on their codebases to those wanting to reimplement a major internal language in a modern fashion.

At the moment — and I expect for the foreseeable future — demand quite significantly outstrips supply. Because of that, organisations are often quite flexible in how they view potential hires in this area. Simplifying a bit, there are two main ways to get in to the field: obtain specific training in the field (e.g. a PhD); start / contribute to a project (not necessarily OSS) that shows expertise (e.g. if you start filing high quality PRs against Rust or LLVM, people will fairly quickly realise you know what you’re doing).

Best of luck — it’s a really fun field, with lots of interesting challenges, and a good group of people!

Any comment on the salary in these positions? Are they higher than for the average software developer, and if so, by how much? I am wondering since to my experience, the application and development of logics and verification procedures to be used in language development requires vastly different and more rigorous personality and working methods than in your average web developer gig.

I worked as a compiler engineer at a big tech company in SV and lived very comfortably, was able to put a lot of money aside. The demand outstrips the supply, it’s hard to hire qualified people, so the salaries have to be high to attract those qualified.

Tangentially, I think the supply is small because compiler engineering is not a fashionable field of study. Everyone wants to do “AI” these days.

I hope am not being pushy but could you please say how much you made and in which company? Thanks

I don’t pretend to have a total picture of all the possibilities, and there’s quite a bit of not-always-explainable variation in what I do know, but I think it would be fair to say that people in this field are quite some way from going hungry. It tends to be better remunerated than a typical software gig but it’s certainly possible to earn more in AI/ML. A lot also depends on whether you want to be a “normal” engineer or are willing/capable of taking on managerial responsibility.

I wonder if it’s similar to embedded software development (which also requires “rigorous working methods”). Salaries for embedded are generally lower than for web development because there’s much less demand for embedded skills.

I’ve worked in compilers (optimizations) at Microsoft (vc++) and NVIDIA(CUDA) – register allocators, schedulers and loop optimizers were super fun.

Then I worked on database (query) compilers. I managed Apache Hive at Hortonworks, and that has a compiler. These are very different beasts.

Now, my startup is working on code analysis and cross compilers for data engineering, including Spark.

Some of my colleagues are working in AI chip startups.

It’s hard to find high quality engineers in both kinds of compilers. Just like database internals engineers there is high demand.

Plug – I’m hiring at Prophecy.io for compiler engineers.

Arm is another company that is frequently and currently hiring compiler engineers (mostly for UK Manchester/Cambridge; looking at the current listings it’s largely looking for people with some experience at the moment but there are also some intern/student placements.)

I second this: My alma mater (in Germany) has a compiler group, and, as far as I’m aware, at least some of its alumni went to the companies mentioned by parent.

Since there are people in the know here:

How would you in the adjacent field of language design? I would be super interested in doing research on the mental load of different programming languages (and different representations of information). I.e. put a hacker in a fMRI and let her solve tasks in different languages.

This is an interesting idea.

I wonder how many people you’d need to measure to produce a usable stereotype model. (Hmm, I wonder if studies of other types of problem-solving challenges have asked the same question and published any useful data?)

I fear you’d need to parametrically plot developers, experience, and language together to get a useful result.

And then there’s the “I get it” threshold point at which the brain suddenly finds a particular subject much easier to reason about, ie, fundamental learning. Everyone’s on a spectrum there – one that seems to be very hard to self-assess.

IMO, you should forget about compiler engineering as working on the backend of, say, gcc, or llvm. Yes, there are such positions (e.g., intel needs people adapting compilers to new processor instruction sets) but they are few and very specialized.

Instead, think about languages in general. There is a myriad of opportunities between the js eco system, JVM and .NET, things like Rust and even low-level languages like LLVM. It should be straightforward to pick features from one eco system and port them to another. Say linear types from Rust to the JVM. Pick one or two such features, port them to your favorite eco system and provide a prototype compiler/macro library on github. With that kind of experience under your belt, you should be the goto guy for any software company that does tooling in that ecosystem. From that vantage point you could start looking for ever more interesting work.

I wonder if the growth of “low-code” will lead to many more well designed domain specific languages.

LookML in Looker is one example I can think of – imo it has enabled Looker to solve the self-service data issue much better via a low-code approach than Tableau and Qlikview (both “no-code”).

minor nit: please use “goto person” if you’d like to keep all readers connected to your writing.

(or use “goto woman” to subvert so-called typical practitioner expectations)

thank you for considering

I work at Oracle Labs where GraalVM (graalvm.org) is being developed. I think this is one of the most exciting places to be right now for compiler engineers, and people with the right skills are certainly in demand. Some things that came out of this: partial escape analysis, the Graal JIT compiler, SubstrateVM, native-image, several Truffle language implementations (JS, Ruby, Python, LLVM), IGV for visualizing different layers of IRs, integrating GraalVM in the Oracle Database (MLE).

Basic/naive/stupid question that I’ve been genuinely curious about for a little while now:

What are the real-world motivations behind reimplementing the noted languages on top of Truffle? These projects are frankly incredible – hundreds (thousands) of total man-years of work – and all to _re_build something that already exists. I’m missing context that I don’t know where to look for.

Is there anywhere I might look online to immerse myself in the relevant discussion of the business use cases that drive these types of huge projects?

Communicating or interfacing massive legacy code bases together. You can write python code and call it from Ruby code in a slightly, but livable, way.

Also eventually I’m assuming there’s some plans to compile these other languages with native-image. In my eyes native-image is Java’s response to : reasonably sized small binaries that are perfect for containerization and quick to elastically scale up/down (startup time).

I work at an embedded OS company on the entire command-line toolchain (as opposed to the GUI-based IDE). That includes maintaining existing and doing new compiler ports.

We always have a great deal of trouble finding qualified candidates to fill open reqs. We find few are interested in this sort of technology, everyone want to either work on cool new applications (like self-driving automobiles) or high-profile web-based stuff. Our entire department is either waiting for retirement or a heart attack (whichever strikes first, and retirement is not a realistic expectation).

tl;dr compiler engineers are in demand and they work on compilers.

There’s some work happening in the blockchain space because people are starting to create their own smart contract languages. I don’t know of any other space seeing so many ‘new’ languages being created in such a short space.

I’ve seen jobs advertised by teams developing Cardano, Tezos, and Ethereum. It’s worth looking at their project pages, seeing which teams work on it, and then checking their careers pages. I know IOHK have been hiring in the space although that’s specifically functional compilers – however nothings up on their careers pages right now.

There has been a resurgence due to the proliferation of more special-purpose processors, like Google TPU or Tesla FSD. I anticipate this trend to continue due to Moore’s Law slowdown. However, if you’re not really into this, there’s no reason to not pursue some other subset as on a relative basis, higher levels of the stack likely will have many more opportunities than infrastructure.

Is there a way I can integrate Machine Learning to Compiler stuff? I’ve been meddling with ml for a while and it’s pretty interesting too.

Can you elaborate on that please? Like how do they merge together? I am really interested in those two fields

There’s been a resurgence due to interest in specialized processors and new computing architectures and systems in general, mostly for ML.

There’s a lot of compiler work in query processors for analytical systems. High-level languages compiling down to machines at different levels of abstraction, LLVM being the lowest and database operator libraries a bit higher. I’d like to find someone interested in pursuing that direction.

Check out http://www.compilerworks.com

I worked there for awhile but compilers aren’t really my thing. They’re building cool stuff, though. Their tech converts database queries between dialects, and can even run as a translating proxy (for example, run Oracle code on Postgres) and provide performance insights into your code. Their clients are big name companies who need to maintain millions of lines of DB code.

What’s a good paper (or book) reviewing current trends in compilers?

I’m interested in writing parsers, but my experience is only with small scale web editors for specific type of documents full of business rules

e.g user write documents with something like markdown and then editor renders document with things like auto numeration, displaying other referenced elements by syntax e.g ‘@element2’

Does there exist job in writing parsers?

Definitely in demand! For example, there’s a bunch of really interesting commercial work happening building or working on compiler toolchains for WebAssembly.

Definitely in demand! We are hiring compiler developers at www.edument.se (remote or onsite in Sweden or Prauge), contact me at tore (at) edument . se

Obfuscators, I work for a company that builds mobile obfuscation at a compiler level (LLVM ir and Java bytecode). There are also gaming companies that use similar techniques and technology for anti cheat engines.

I also often see job postings for compiler developers in the embedded field.

What qualifies someone to be a Compiler Engineer?

Compsci degree? Working compilers? Opensource commits? Github repo full of parsers?

The ability to help deliver better compilers.

How you demonstrate that can be in a variety of ways. Any of those would probably be helpful if you can get the right person to notice them.

Having written one or more compilers, or large parts thereof. Obvious bootstrap problem notwithstanding..

A lot of work with compilers in blockchain/Ethereum.

But if you’re considering an education choice, don’t overspecialise. You will most likely change specialisation in 5-10 years (different market, different technology), so if all you know is just compiler engineering, it will be way tougher for you.

Also, make sure you learn a lot about algorithms / computer science in general. If you know that, switching engineering fields will be easy.

Codeplay Software develop compilers for new architectures and new high performance programming models. They are always looking to hire talented engineers.

mostly i see people working in probabilistic programming languages and formal methods