The AI Talent Shortage

It’s early 2018 and a Bloomberg piece is making the rounds — Just How Shallow is the Artificial Intelligence Talent Pool?

It’s a surprisingly good piece. Rather than gratuitous estimates pulled out of a hat, the authors cite numbers with some objective criteria.

there are about 22,000 PhD-educated researchers working on AI, of which about 3,000 are currently seeking work.
…
PhDs since 2015 and whose profiles also mentioned technical terms such as deep learning, artificial neural networks, computer vision, natural language processing or robotics. In addition, to make the cut, people needed coding skills in programming languages such as Python, TensorFlow or Theano.

That’s a pretty good heuristic. Of course my VP jokes that neither him nor I meet this requirement. But it’s a pretty good proxy for whom we’re looking to hire on our team, besides industry experienced Deep Learning researchers and research engineers.

People with 3+ years of DL experience outside of PhD are even more rare, so including them wouldn’t do much to goose the 22,000 number. The article goes on to say.

There’s another subset of about 5,000 people at the cutting edge of AI research who are publishing papers and presenting at academic conferences.

This I think is also broadly true. By papers published, the Pareto Principle works its magic and any field can shrink quickly. But if you’re looking for a nominally qualified researcher who’s published in AI/DL — not a lot of people who have done this already. And many of those who have, want to work with the few others who have done so as well. So they are all kinds of not available.

None of this is news. Cade Metz’s October 2017 NYTimes piece about AI talent and big companies giving them huge salaries has aged well and is still broadly true today. Companies have expanded locations, ramped up training — but that costs money too, and time.

I agree with the Bloomberg piece that we should encourage more people to go from undergrad to AI. There’s nothing in our field that can’t be learned in four years of undergrad — at least not to get started. The big universities can do more to set up their undergrad CS programs to make that possible [separate piece about the 4–5 skills necessary to be a decent AI researcher]. But you’re still talking about junior colleagues. You can’t start an AI team with one expert and seven undergrads.

It’s hard to think about things this way when it’s your friends and colleagues — most of them talent but somewhat normal people. We are not professional athletes, although those comps keep being made, and they aren’t entirely incorrect.

But when I look for a designer, a Java developer, a real estate agent, etc — some are way better than others and deserve to get paid more than an AI researcher — but you’re fundamentally talking about pulling from a large well-balanced pool. It’s mostly an information game, and a matter of getting a little better than you need, but not much more than you can afford or should be paying.

In AI, it’s different. There just aren’t enough people to go around. And there aren’t enough people for every good project that can be attempted. Either academic, or something that if it works, can save the company $1M.

Most importantly, we need to be mindful of how we spend our time. And also the time of the people we are interviewing. The world is abundant. Everyone has options.

Disclaimer: I am a senior DL researcher at NVIDIA in Santa Clara. I do research in deep learning NLP, game AI, and am getting into DL for genomics. These views are my own, not those of my employer. Yes, we are hiring!