Implement Qwen3.8‑27B from scratch in PyTorch.
Finally understand how frontier LLMs work. Write every layer yourself, load the official weights, then chat with Qwen3.8‑27B running on your own code.
Under 1,000 lines. The same building blocks run DeepSeek 7B and 67B.
- 64 layers
- official 27B weights
- + DeepSeek 7B & 67B
Not a toy model. The full 27B.
Most LLM-from-scratch courses stop at a small model of limited intelligence. This LLM course goes all the way to the released Qwen3.8‑27B checkpoint.
A 27B open model that competes with Opus 4.6 Max.
Small enough to implement and run yourself. Strong enough to matter.
| Benchmark | Qwen3.8-27B | Opus 4.6 Max |
|---|---|---|
| Agentic codingSWE-bench Pro | 61.7 | 53.4 |
| Computer useOSWorld-Verified | 84.3 | 72.7 |
| Instruction followingIFBench | 79.5 | 62.5 |
| Agentic terminalTerminal Bench 2.1 | 73.0 | 78.2 |
| Frontier reasoningHLE | 30.8 | 40.0 |
Curriculum: Qwen3.8‑27B from scratch, then DeepSeek 7B & 67B.
- 01
LLM fundamentals and the BPE tokenizer
Next-token prediction, tokenization, the training objective.
Watch free → - 02
Attention, RoPE, RMSNorm, SwiGLU
The building blocks of Qwen and today's open LLMs.
- 03
Gated DeltaNet: recurrent linear attention
Linear attention with a fixed-size state instead of a growing cache.
- 04
Assemble Qwen3.8‑27B, load the weights
The complete 64-layer model, with every tensor of the official checkpoint loaded and accounted for.
- 05
Inference, KV cache, then DeepSeek
Tokenizer, hybrid KV cache, generation — then rewire the same building blocks into DeepSeek 7B and 67B.
+ 2 bonus models
For engineers who know PyTorch and want to deeply understand how modern LLMs work.
For you
- ML engineers who use LLMs as black boxes
- Software engineers building LLM products
- Researchers and students moving into LLM work
Prerequisites: Python and PyTorch basics. Everything else is taught and implemented from scratch.
Not for you if you want
- To train an LLM from scratch — a different problem
- To fine-tune for your use case — a different course
From the instructor behind the NeRF, 3D Gaussian Splatting and Diffusion Models from-scratch courses. Same method: first principles, pure PyTorch, nothing hidden.
Based on Udemy student reviews as of May 2026.- Qwen3.8-27B, every layer, in pure PyTorch
- Runs the official 27B weights
- Bonus: DeepSeek 7B & 67B from the same building blocks
- All the code — yours to keep and modify
Questions before you buy.
How is the course structured?
14 chapters, from LLM fundamentals to text generation. Each concept is first explained in theory, then implemented in code.
What do I need to know before starting?
Python and PyTorch basics. The course starts from the fundamentals and teaches everything else from scratch.
Will this help me understand other LLMs?
Yes. RMSNorm, SwiGLU, RoPE and grouped-query attention are shared by most open LLMs today, including Llama, Mistral and DeepSeek. The bonus shows it: DeepSeek 7B and 67B run on the same building blocks, configured differently.
Do you use any LLM library?
No. Everything is written in plain PyTorch.