(When) will an AI win the World Series of Poker?

Not my best talk.

too much material presented too quickly, but stripes on stripes

You know when something that you wanted, happen when you least have the time for it? I’ve been a fan of the Sloan Sports Analytics conference for a long time, attended it a couple of times as a fully paid participant — a pleb behind the velvet ropes, and listened jealously as podcast hosts like Behind the Bets’ Chad Millman talked about the luminaries for sports analytics they’d meet at the conference.

So this year when I got an invitation from an MIT MBA student to speak at Sloan, my thought were:

• F — , I don’t have time for this.
• I have to do it.

She asked if I wanted to talk about poker or baseball — I guess she read my blog. Of course I chose poker. It’s the MIT business school after all, and I hear they have a poker class. Not to be confused with the MIT poker AI undergrad CS class. Strangely I’ve not been invited to guest teach at either. 🤔

In mytalk I tried to cover too many things — starting with my journey into underground “Rounders” style poker in NYC as a Columbia grad student, to the boom and subsequent transition of online poker, to poker bots that people tried to write for those sites, to the academic poker AI community. Which started before Moneymaker and staying mostly in its own world until the last couple of years, when the U-Alberta and CMU poker research groups first “solved” limit heads-up Hold’em, then matched and then subsequently beat the pros at heads-up NLH.

CMU’s Libratus Bluffs its way to Victory in #BrainsVsAI Poker Match 🤖

Nassim Nicholas Taleb likes to say that “surgeons should not look like surgeons,” and that one should

Hire the successful trader, conditional on a satisfactory track record, whose details you can understand the least.
…
One [trader] looked the part of the investment manager, down to a T: tailored clothes, expensive watch, shiny shoes, and clarity of exposition. … The second [trader] looked closer to our butcher-surgeon and was totally incomprehensible; he even gave the impression that he was confused. … The first, not unexpectedly, was in the equivalent of the soup kitchen for that business; the second was at least a centimillionaire.

Taleb may not think highly of the suit, but hey, I deadlift. And I wouldn’t turn down a squid ink pasta, neither.

I did hear from a couple of friends (who know the online poker or the poker AI world) that they found the talk enjoyable. Although of course a bit too basic, and I didn’t get into the interesting details of poker AI in the short time slot.

I didn’t even answer the question that was in the title:

The “All In” Bet: How Soon Can an AI Bot Become the Next World Series of Poker Champion?

I’ll answer it now.

Poker bots will be highly competitive with tournament poker player five years from now. Maybe sooner, if people really care. The truth is, this is not a terribly valuable problem.

You’d think that a poker AI that can play like a pro would make millions like a pro (assuming poker pros do make millions). But it’s not that simple. The most straightforward value to a strong poker AI is as a teaching/analysis tool. I’ve heard that some of those systems do make money. The PLO tool I’ve seen is quite good. But this isn’t a billion dollar company. Scanning government forms is a billion dollar company. Drone surveillance with AI… these are just two of many more valuable applications for AI practitioners.

Academically of course poker AI is very interesting. It’s the closest we have to a real world game with asymmetric information that matters. I’m rambling, and will leave it at that.

I do think that others will come up with more valuable uses for poker AI than as a training tool, but I don’t know what those might be. And I’d by no means be against playing real-feeling poker against eight tough robots, with different human-like characteristics. Especially if the skills against those robots transferred to playing against the nits and the drunkards of Borgata on a Friday night.

The Borgata — I spent many a weekend here (when I should have been coding)

Channeling Elon Musk’s fireside chat at NIPS [where Noam Brown won best paper for his work on the Libratus poker AI], here’s how strong tournament poker AI development could play out:

1. Year one — CFR/equilibrium solving (which beat the pros heads-up) works well enough for 6-max poker to be competitive and not easily exploitable.
2. Year two — a deep learning based system learns the CFR parameters well enough to play the equilibrium machines to a draw, without the need for (much) online computation.
3. Year three — the deep learning system plays itself, and is used to develop a suite of strategies, which succeed against different types of players (players which can be summarized by summary statistics like % hands played, aggression level, how often they call on the river, etc).
4. Year four — these strategies go into the wild, and successfully compete against real tournament players.
5. Year five — lots of details and fixes. Dealing with tournament dynamics, anti-bot play, implicit collusion, etc.

Why can’t we get there right away? The history of this space, or any AI, has been one or two breakthroughs at a time. As Nvidia’s esteemed CEO Jensen Huang likes to say “no new product should rely on [too many] miracles.”

“five miracles”

The academic world moves in annual cycles. New ideas start in the fall, get worked on over the winter, and submitted in a mad scramble for NIPS deadlines (which is why my own Spring is so busy). Work slows down for the summer then accelerates into the fall with a mad scramble for the next conference deadlines.

My friends in the poker AI world tell me equilibrium solving will work for 6-max no limit Hold’em. But I’m from Missouri.

Nobody has tried teaching a deep learning system to play poker directly from training on simulation/equilibrium solving outputs, as far as I know. Although U-Alberta’s DeepStack sort of does a version of that.

I’ve played DeepStack, and it’s pretty good. It plays alright with shallow stacks — adjusting to looser allin requirements, dealing with the bigger blinds, etc, albeit still heads-up only.

We don’t necessarily need bots that will adapt to other players’ styles and exploit them a bit, for a bot to be competitive at tournament poker. But it would help a lot. An average tourney player can certainly win any tournament. Heck, a below average player wins every once in a while. But edges accumulate throughout a multi-day event. Especially early in the tournament, against very predictable competition.

Sloan is a two day conference. I spent the first day at my hotel making the slides. On day two, I had an our to kill before my noon talk, and ran into a fascinating fellow presenter in the speakers’ room.

Dr Meeta Singh on athletes, sleep, and jet lag

Dr Meeta Singh is a sleep doctor, with a practice in Detroit, and consults for several NFL and MLB teams on travel and jetlag. Her talk, unlike mine, was excellent. I heard it twice as she demoed the talk to me in the green room — though she’d given a version of it several times before, and you could tell she had it down pat.

The TL;DR on jet lag, from a professional:

• The only things that consistently work are light exposure (or non-exposure) and melatonin — and coffee does help to wake you up too
• A meditation practice also helps, especially in the long run

It took me week to starts, but Dr. Singh has gotten me on the Headspace app— somehow Naval’s recommendation was not enough.

I can’t say I use it every day, but I should, and I do like it. [Headspace is just guided meditation on your phone.]

After my talk I had to catch a flight to DC for… a story for another time. I try to keep culture and politics off this blog. Until my Jordan B. Peterson book review.

I did say hi to Abe and TJ

Before my flight I saw Bill James on a bench, with a line of students waiting to talk to him. He’s my favorite modern writer, not just in baseball. So I cut the line and took a picture.

I doubt I’ll be on the short list to get invited back, but I’d do Sloan again. My talk was not a miracle, but I am on the MIT Sloan Sports Analytics Conference speakers’ list, along with Chad Millman, Bill James, Dr. Singh, Nate Silver, and many others besides.

Speaking of personal stuff and mentions in the media, I’ll write separate posts but a few have come out this Spring.

My friend Michael Kaplan wrote this piece about me in the New York Post. Specifically about my recovery from traumatic brain injury (TBI) that led to a full right-side paralysis above the waist. It will have been six years since my injury this April.

How I healed my own brain injury

I don’t like the title, and there’s more to the story — Michael’s a friend, and I understand space is limited and they need to take an angle on any story. But there’s some truth to it. Once I could reasonably take care of myself (this took a while) nothing has helped me get back to full strength more than lifting, eating good food (a lot of meat), and sun exposure. Talking to neurologist (and other doctors, with one exception) did less than zero. And my recovery has been pretty amazing. Many thanks to God for that.

One of my work projects at Nvidia got a mention in last week’s Economist.

Nvidia, a chipmaker, also gets more résumés than it can comfortably cope with, so it spent a year building its own system to predict which candidates are worth interviewing. It has recognised patterns that recruiters might not: for example, candidates who submit especially long résumés turn out to do less well than others, so those extra words will count against them.

Managing human resources is about to become easier

I’m proud of our work, and we should have an official, public writeup about this soon. It’s long overdue. Perhaps after the NIPS deadline…