From Orphanage to Google Brain: David Sussillo on Heroin, Neural Networks and the Mysteries of the Heart

“I can point to things. But is that a systemic explanation? I think there the answer is a little less clear. I mean, surely people need love and all of that, but then there’s this risk of just devolving into platitude.” — David Sussillo

David Sussillo is a big time neural reverse engineer. The Stanford brain scientist worked at Google Brain with Geoffrey Hinton, and now is at Meta Reality Labs. What distinguishes Sussillo, however, is not his Silicon Valley good luck, but the bad luck of his origins. In his memoir, Emergent: A Memoir of Boyhood, Computation, and the Mysteries of the Mind, Sussillo begins at the Albuquerque Christian Children’s Home — a modern-day orphanage — and the Milton Hershey School, the boarding school endowed by the chocolate magnate for kids with nowhere else to go. Both his parents were addicts. His mom died young. His dad spent his life as an untrained preacher ministering to homeless people on the streets of Albuquerque while managing a lifelong heroin habit.

The book’s thesis borrows from the science he studies: “emergence” — simple things interacting to produce complex behaviour that none of them could produce alone. His life is both proof of and a challenge to this concept. He made it out. Most of the kids he grew up with didn’t. He can point to moments — a gifted-and-talented test in third grade, an aunt and uncle’s intervention at nine, a first love in college — but he can’t build an explanatory system from these haphazard events. The Sussillo quilt doesn’t have an innate pattern. It just has patches.

What makes Sussillo unusual as a memoirist is his refusal to sentimentalise. Twenty years of psychotherapy, he confesses, has taught him something most authors never learn: that understanding your own story doesn’t mean you’ve explained it. His science can’t explain his childhood either. “The big dirty secret of neuroscience,” he says, “is that we don’t really understand much in the ways that people would love us to understand.” The man who reverse-engineers neural networks can’t reverse-engineer himself.

I asked him whether having children would have been harder than writing the book. Yes, he said. With the book, you can take a break. With kids, you relive things through a very specific way of relating. He and his wife chose not to. His mentors all told him he’d have been great at it. He’s not so sure. That honesty — the willingness to say “I don’t know” and mean it — runs through everything Sussillo does. He says he’s happy, claiming to have found peace with his past. But he still carries the baggage. Who wouldn’t? He’s just learned to manage it. Emergent, not emerged.

Five Takeaways

•       From Orphanage to Google Brain: Both parents were heroin addicts. Sussillo grew up in a modern-day orphanage in Albuquerque and then the Milton Hershey School. He went on to work at Google Brain with Geoffrey Hinton, now works at Meta Reality Labs, teaches at Stanford. Most of the kids he grew up with didn’t make it.

•       Emergence as Autobiography: The book’s thesis borrows from the science he studies: simple pieces combining into complicated outcomes. His life is the proof of concept and the counter-example simultaneously. The quilt doesn’t have a pattern. It just has patches.

•       The Dirty Secret of Neuroscience: The man who reverse-engineers neural networks can’t reverse-engineer himself. “We don’t really understand much in the ways that people would love us to understand.” Twenty years of therapy taught him more than the science.

•       Would Kids Have Been Harder Than the Book? Yes. With the book, you can take a break. With kids, you relive trauma through a very specific way of