Ilya Sutskever / John Carmack Reading List

2026-05-30 · reading-list, deep-learning, carmack

---

This is the reported Sutskever-to-Carmack deep-learning list that circulates online. I am treating it as a useful study map, not as a canonical historical artifact.

Sources checked while adding it:

• dzyim/ilya-sutskever-recommended-reading
• Tensorlabbet summary
• Justmalhar/ilya-sutskever-reading-list

Foundations and Compression

• Keeping Neural Networks Simple by Minimizing the Description Length of the Weights
• A Tutorial Introduction to the Minimum Description Length Principle
• Kolmogorov Complexity and Algorithmic Randomness
• The First Law of Complexodynamics
• Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton

CNNs and Vision

• CS231n: Convolutional Neural Networks for Visual Recognition
• ImageNet Classification with Deep Convolutional Neural Networks
• Deep Residual Learning for Image Recognition
• Identity Mappings in Deep Residual Networks
• Multi-Scale Context Aggregation by Dilated Convolutions

Sequence Models

• The Unreasonable Effectiveness of Recurrent Neural Networks
• Understanding LSTM Networks
• Recurrent Neural Network Regularization
• Order Matters: Sequence to Sequence for Sets
• Pointer Networks
• Neural Machine Translation by Jointly Learning to Align and Translate

Attention, Transformers, and Scaling

• Attention Is All You Need
• The Annotated Transformer
• GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Scaling Laws for Neural Language Models

Reasoning, Memory, and Structure

• Neural Turing Machines
• A Simple Neural Network Module for Relational Reasoning
• Relational Recurrent Neural Networks
• Variational Lossy Autoencoder
• Neural Message Passing for Quantum Chemistry

Long-Horizon Theory

• Machine Super Intelligence