Daniel D. Johnson

Hi! I am a PhD student at the University of Toronto working with David Duvenaud and Chris Maddison, and a research scientist at Google Brain on the Machine Learning for Code team. I'm interested in building systems that "know what they don't know" and that act in predictable, interpretable, and safe ways in the presence of uncertainty. I'm also interested in understanding the connections between computation, intelligent behavior, and probabilistic reasoning, especially under capacity or memory constraints.

One specific direction I'm excited about is extracting information from the full distribution of outputs under a generative model, and exposing this information to the user to help them interpret the output. An example of this is my recent paper on the R-U-SURE system.

Another exciting direction is interpreting reasoning and information-seeking as sequential decision making processes. Language models are surprisingly good at imitating human reasoning, but frequently make errors due to having less information than the humans they are imitating. How do we build systems that seek out new information when they do not already know the answer? I discuss this a bit in my blog post, "Uncertain Simulators Don't Always Simulate Uncertain Agents".

More broadly, I'm also interested in many areas of probabilistic machine learning, especially those involving discrete objects or automatic differentiation. In the past, I have worked on generative models of discrete data structures (such as trees, sets, and graphs), theoretical analyses of self-supervised learning, a strongly-typed language (Dex) for building unconventional machine learning models, generative models for music, and many others. See my research page for more information.

I was an AI Resident at Google from 2019 to 2021, and I worked on applied machine learning at Cruise from 2018 to 2019. Before that, I was an undergraduate CS/Math joint major at Harvey Mudd College, where I did research on applying deep learning to music generation, and worked as a math tutor in the Academic Excellence tutoring program at HMC.

In my free time, I enjoy playing board games and indie video games (current recommendations: Outer Wilds, Baba is You, Tunic), reading about math and programming languages, and telling myself that someday I'll get back into making music.