A curated shelf of the books, papers and lecture notes I keep coming back to.

Foundational AI/ML papers, GPU and parallel programming, the mathematics underneath it all, and the systems books that explain how the machine actually works.

The shelves  ·  Where to start  ·  Full catalogue  ·  Topics  ·  Contributing

---

📚 The shelves

Every folder is a shelf. Click one to browse it, or open the full catalogue to see every title at once.

Shelf	Count	What's inside
ai-and-machine-learning	16	The canonical papers, split into foundations, language models, probabilistic ML and surveys
mathematics	8	Linear algebra, probability & statistics, analysis, matrix calculus
cuda-and-gpu	6	CUDA from first kernels to massively parallel processor design
parallel-and-distributed	3	MPI and the theory of distributed systems
computer-architecture	3	How computers work, from NAND gates upward
cryptography-and-privacy	2	Cryptography and differential privacy lecture notes
programming	2	Competitive programming and language references
quantum-computing	2	Quantum information and Ising solvers
notes	2	My own write-ups
physics	1	Relativity lecture notes
coursework	1	Course assignments

Files are named Title - Author (Year).pdf so every shelf sorts and reads cleanly.

---

🌟 Where to start

Six papers, 1951 to 2023, that cover the most ground between them.

Paper	Why it matters
The Perceptron — Rosenblatt (1958)	Where learning machines begin
Prediction and Entropy of Printed English — Shannon (1951)	Language modelling, decades before language models
A Neural Probabilistic Language Model — Bengio et al. (2003)	Word embeddings, from first principles
Deep Learning — LeCun, Bengio & Hinton (2015)	The field's own summary of itself
BERT — Devlin et al. (2019)	Pretraining as the default
FlashAttention-2 — Dao (2023)	Why attention got fast

Three reading paths

Path	Order
GPU programming	Programming Massively Parallel Processors → CUDA by Example → CUDA C++ Programming Guide
The machine underneath	Code (Petzold) → The Elements of Computing Systems
The maths	The Mathematics of AI → Matrix Calculus → Probability & Statistics for Data Science

---

📖 Full catalogue

All 46 titles, grouped by shelf. Click any shelf to expand it.

AI and machine learning  ·  16 titles

Foundations

• A Neural Probabilistic Language Model — Bengio, Ducharme, Vincent & Jauvin (2003)
• Backpropagation Through Time for Networks with Long-Term Dependencies — Bird (2021)
• Deep Learning — LeCun, Bengio & Hinton (2015)
• Prediction and Entropy of Printed English — Shannon (1951)
• The Perceptron — Rosenblatt (1958)

Language models

• BERT — Devlin, Chang, Lee & Toutanova (2019)
• FlashAttention-2 — Dao (2023)
• Large Language Models in Machine Translation — Brants, Popat, Xu, Och & Dean (2007)

Probabilistic ML

• Hidden Markov Models, from Speech and Language Processing — Jurafsky & Martin (2026 draft)
• Hidden Markov Neural Networks — Rimella & Whiteley (2025)
• Practical Bayesian Optimization of Machine Learning Algorithms — Snoek, Larochelle & Adams (2012)
• Probabilistic Artificial Intelligence — Krause & Hübotter (2025)

Surveys and applications

• A Comprehensive Guide to Explainable AI — Hsieh et al. (2024)
• Artificial Intelligence in Service — Huang & Rust (2018)
• Curvature Tuning — Hu & Balestriero (2025)
• Is Complexity an Illusion? — Bennett (2024)

Mathematics  ·  8 titles

• Applied Linear Algebra and Differential Equations — Chasnov (2019)
• Calculus 1 for Honours Mathematics — Forrest & Forrest (2023)
• Lecture Notes on Randomized Linear Algebra — Mahoney (2016)
• Matrix Calculus for Machine Learning and Beyond — Edelman & Johnson (2025)
• Probability and Statistics for Data Science — Fernandez-Granda
• Problems in Mathematical Analysis — Demidovich (1970)
• The Mathematics of Artificial Intelligence — Peyré (2025)
• The Problems Submitted by Ramanujan to the JIMS — Berndt, Choi & Kang (1999)

CUDA and GPU  ·  6 titles

• CUDA by Example — Sanders & Kandrot (2010)
• CUDA C++ Programming Guide v12.8 — NVIDIA (2025)
• CUDA for Engineers — Storti & Yurtoglu (2015)
• Learn CUDA Programming — Han & Sharma (2019)
• Professional CUDA C Programming — Cheng, Grossman & McKercher (2014)
• Programming Massively Parallel Processors — Kirk & Hwu (2010)

Parallel and distributed  ·  3 titles

• An MPI-based Algorithm for Mapping Complex Networks onto Hierarchical Architectures — Predari et al. (2021)
• Introduction to MPI: The Message Passing Interface — Nielsen (2016)
• Notes on Theory of Distributed Systems — Aspnes (2025)

Computer architecture  ·  3 titles

• Code: The Hidden Language of Computer Hardware and Software, 1st Edition — Petzold (2000)
• Code: The Hidden Language of Computer Hardware and Software, 2nd Edition — Petzold (2022)
• The Elements of Computing Systems, 2nd Edition — Nisan & Schocken (2021)

Cryptography and privacy  ·  2 titles

• Introduction to Differential Privacy and the Laplace Mechanism
• Lecture Notes on Introduction to Cryptography — Goyal, CMU (2020)

Programming  ·  2 titles

• Competitive Programmer's Handbook — Laaksonen (2018)
• The Julia Language Documentation v1.11.2 — The Julia Project (2024)

Quantum computing  ·  2 titles

• Multi Digit Ising Mapping for Low Precision Ising Solvers — Singh & Jamieson (2024)
• Quantum Information and Computation — Bub (2006)

Notes, physics and coursework  ·  4 titles

My own write-ups

• Data Structures and Algorithms in Java — Sinha
• Learning Probability and Statistics for Machine Learning and Data Science — Sinha

Physics

• PHY405 Relativity, Lecture Notes — Lim

Coursework

• CS336 Assignment 1: Building a Transformer LM (Spring 2025)

---

🔗 External links

Good material that lives elsewhere:

• Awesome Low Latency — low-latency programming resources
• UC Berkeley — Introduction to Mathematical Thinking
• University of Waterloo — Data Analytics — videos and slides

---

🏷️ Topics

Repository topics, so this shelf is findable:

awesome  awesome-list  books  ebooks  pdf  learning-resources  computer-science
machine-learning  deep-learning  artificial-intelligence  neural-networks
research-papers  cuda  gpu-programming  parallel-computing  mpi  hpc
mathematics  linear-algebra  probability  statistics
computer-architecture  distributed-systems  cryptography  quantum-computing

Apply them in one command with the GitHub CLI:

gh repo edit --add-topic awesome,awesome-list,books,ebooks,pdf,learning-resources,computer-science,machine-learning,deep-learning,artificial-intelligence,neural-networks,research-papers,cuda,gpu-programming,parallel-computing,mpi,hpc,mathematics,linear-algebra

GitHub caps topics at 20 per repository — the command above uses that budget; swap in probability, statistics, computer-architecture, distributed-systems, cryptography or quantum-computing if they suit you better.

---

🤝 Contributing

To add a book or paper:

1. Drop the PDF on the shelf it fits, or propose a new one.
2. Name it Title - Author (Year).pdf — matching the rest of the shelf.
3. Add it to the full catalogue above, keeping each list alphabetical.
4. Open a PR with a line on why it earned its place.

Only material that is freely and legally distributable, please.

---

⚖️ License

The MIT License covers this repository's own content — the structure, my notes, the logo.

It does not cover the PDFs. Every paper and book here remains under its own copyright, held by its authors and publishers. They are collected for personal study and reference. If you hold rights to something here and would like it removed, open an issue and it will be.

_{Built and maintained by Parth Sinha}