The Jim Rutt Show
Currents 033: Connor Leahy on Deep Learning
This episode isn’t summarized yet
Create a free account and summarize it. In a few minutes you’ll have the key takeaways and a section-by-section summary.
Show notes
Connor Leahy has a wide-ranging chat with Jim about the state & future of Deep Learning. They cover the history of EleutherAI, how GPT-3 works, the dynamics & power of scaling laws, ideal sampling rates & sizes for models, data sets, EleutherAI's opensource GTP-Neo & GTP-NeoX, PyTorch vs TensorFlow, TPU's vs GPU's, the challenge of benchmarking & evaluations, quadradic bottlenecks, broad GTP-3 applications, Connors thoughts on Jim's proposed GPT-3 research project, untapped GPT-3 potential, OpenAI's move away from opensource, alignment, AI safety, the unknown future, and much more.