[COURSE] Add Deep Generative Model Roadmap (#744)

* add DGM section

* add roadmap

docs/机器学习进阶/roadmap.en.md

@@ -1,4 +1,4 @@
-# Advanced Machine Learning
+# Advanced Machine Learning Roadmap
 
 This learning path is suitable for students who have already learned the basics of machine learning (ML, NLP, CV, RL), such as senior undergraduates or junior graduate students, and have published at least one paper in top conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, NAACL, CVPR, ICCV) and are interested in pursuing a research path in machine learning.
 

docs/机器学习进阶/roadmap.md

@@ -1,4 +1,4 @@
-# 机器学习进阶
+# 机器学习进阶学习路线
 
 此路线图适用于已经学过了基础机器学习 (ML, NLP, CV, RL) 的同学 (高年级本科生或低年级研究生)，已经发表过至少一篇顶会论文 (NeurIPS, ICML, ICLR, ACL, EMNLP, NAACL, CVPR, ICCV) 想要走机器学习科研路线的选手。
 
@@ -53,11 +53,11 @@
 - 读完 PRML 第 13 章之后，再去读 PRML 第 8 章 (Graphical Models) -- 此时这部分应该会读得很轻松
 - 以上的内容可以进一步对照 CMU 10-708 PGM 课程材料
 
-到目前为止，应该能够掌握
+到目前为止，应该能够掌握:
 
 - 概率模型的基础定义
 - 精准推断 - Sum-Product
 - 近似推断 - MCMC
 - 近似推断 - VI
 
-然后就可以去做更进阶的内容
+然后就可以去做更进阶的内容。

docs/深度学习/CS224n.en.md

@@ -8,7 +8,7 @@
 - Course Difficulty：🌟🌟🌟🌟
 - Estimated Hours：80 hours
 
-CS224n is an introductory course in Natural Language Processing (NLP) offered by Stanford and led by renowned NLP expert Chris Manning, the creator of the word2vec algorithm. The course covers core concepts in the field of NLP, including word embeddings, RNNs, LSTMs, Seq2Seq models, machine translation, attention mechanisms, Transformers, and more.
+CS224n is an introductory course in Natural Language Processing (NLP) offered by Stanford and led by renowned NLP expert Chris Manning. The course covers core concepts in the field of NLP, including word embeddings, RNNs, LSTMs, Seq2Seq models, machine translation, attention mechanisms, Transformers, and more.
 
 The course consists of 5 progressively challenging programming assignments covering word vectors, the word2vec algorithm, dependency parsing, machine translation, and fine-tuning a Transformer.
 

docs/深度学习/CS224n.md

@@ -8,7 +8,7 @@
 - 课程难度：🌟🌟🌟🌟
 - 预计学时：80 小时
 
-Stanford 的 NLP 入门课程，由自然语言处理领域的巨佬 Chris Manning 领衔教授（word2vec 算法的开创者）。内容覆盖了词向量、RNN、LSTM、Seq2Seq 模型、机器翻译、注意力机制、Transformer 等等 NLP 领域的核心知识点。
+Stanford 的 NLP 入门课程，由自然语言处理领域的巨佬 Chris Manning 领衔教授。内容覆盖了词向量、RNN、LSTM、Seq2Seq 模型、机器翻译、注意力机制、Transformer 等等 NLP 领域的核心知识点。
 
 5 个编程作业难度循序渐进，分别是词向量、word2vec 算法、Dependency parsing、机器翻译以及 Transformer 的 fine-tune。
 

docs/深度生成模型/roadmap.en.md

@@ -0,0 +1,27 @@
+# Deep Generative Models Roadmap
+
+In recent years, large language models (LLMs) have become a hot topic, which is also highly relevant to the author’s PhD research. This roadmap aims to share various course materials and references that the author found useful while getting familiar with and delving into the field of deep generative models. It’s intended to help practitioners in related areas or anyone interested in the underlying principles of generative models. Due to limited time outside of research, the author has not completed all the course assignments; updates will be added gradually when time permits.
+
+In fact, LLMs are just one branch of deep generative models. Other types such as VAEs, GANs, Diffusion Models, and Flows still play an important role in the broader domain of "generation." The term AIGC (AI-generated content) broadly refers to these technologies.
+
+Recommended courses for learning:
+
+- [MIT 6.S978: Deep Generative Models](https://mit-6s978.github.io/schedule.html): Taught by MIT’s rising star Prof. Kaiming He, this course covers fundamental theories and cutting-edge papers related to various generative models. The assignments include well-prepared scaffold code. While not overly difficult, they help deepen understanding and provide a quick, comprehensive view of the field.
+
+- [UCB CS294-158-SP24: Deep Unsupervised Learning](https://sites.google.com/view/berkeley-cs294-158-sp24/home): Taught by reinforcement learning giant Pieter Abbeel. Compared to the MIT course, this one is more comprehensive and includes lecture videos and slides. The homework only provides test code, so students must implement model architecture and training code themselves. Though demanding, it’s ideal for those who want hands-on experience in training models. As is well known, there are many practical tricks in deep learning, and the devil is often in the details. Nothing teaches those details better than training a model yourself.
+
+- [CMU 10423: Generative AI](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html): CMU’s GenAI course focuses more on large language models compared to the previous two, but shares much of the same content otherwise. The assignments are quite engaging and worth trying out in your spare time.
+
+The GPT series by OpenAI has demonstrated remarkable performance under the guidance of scaling laws, especially in mathematics and coding. If you are primarily interested in LLMs, the following courses are recommended:
+
+- [Stanford CS336: Language Modeling from Scratch](https://stanford-cs336.github.io/spring2025/index.html): As the title suggests, this course teaches you to build all the core components of an LLM from scratch, such as the tokenizer, model architecture, training optimizer, low-level operators, data cleaning, and post-training algorithms. Each assignment has a 40-50 page PDF handout—very rigorous. Highly recommended if you want to fully understand every low-level detail of LLMs.
+
+- [CMU 11868: Large Language Model Systems](https://llmsystem.github.io/llmsystem2025spring/): This CMU course focuses on system-level optimization of LLMs, including GPU acceleration, distributed training/inference, and cutting-edge techniques. Great for students in systems research to gain a holistic understanding of the field. (Disclosure: One of my papers on PD decoupling is included in the syllabus, hence the personal recommendation.) Assignments involve implementing a mini-PyTorch framework and then building system-level LLM optimizations on top of it.
+
+- [CMU 11667: Large Language Models: Methods and Applications](https://cmu-llms.org/) and [CMU 11711: Advanced NLP](https://www.phontron.com/class/anlp-fall2024/): Compared to the previous two, these courses focus more on higher-level algorithms and applications. Each lecture includes many recommended readings, making them suitable for gaining a broad understanding of LLM research frontiers. You can then dive deeper into any subfield that interests you based on the reading materials.
+
+In addition to courses, the following resources are also highly recommended:
+
+- [Awesome-LLM](https://github.com/Hannibal046/Awesome-LLM): A curated list of LLM-related resources.
+- [LLMSys-PaperList](https://github.com/AmberLJC/LLMSys-PaperList): A collection of system-related papers on LLMs.
+- [MLsys-Guide](https://github.com/PKU-DAIR/Starter-Guide/blob/main/docs/systems/Readme.md): A beginner’s guide to deep learning systems.

docs/深度生成模型/roadmap.md

@@ -0,0 +1,27 @@
+# 深度生成模型学习路线
+
+近几年大语言模型成为大热的方向，也和笔者博士期间的课题非常相关。这篇路线图旨在分享笔者在熟悉和深入深度生成模型这一领域过程中学习和参考的各类课程资料，方便相关领域的从业者或者对生成模型的底层原理感兴趣的朋友共同学习。由于笔者科研之余时间有限，很多课程的实验并未完成，等后续有时间完成之后会在该目录下一一添加。
+
+其实，大语言模型只是深度生成模型的一个分支，而其他生成模型例如 VAE，GAN，Diffusion Model，Flow 等等，都还在“生成”这一领域占有重要地位，所谓的 AIGC，就是泛指这一类技术。
+推荐学习下列课程：
+
+- [MIT 6.S978: Deep Generative Models](https://mit-6s978.github.io/schedule.html): MIT 新晋明星教授何恺明亲授，涵盖了各种生成模型的基础理论和相关前沿论文，几次作业都有丰富的脚手架代码，难度不高但能加深理解，能对这个领域有个快速全貌了解。
+
+- [UCB CS294-158-SP24: Deep Unsupervised Learning](https://sites.google.com/view/berkeley-cs294-158-sp24/home): 强化学习领域的顶级巨佬 Pieter Abbeel 主讲，相比 MIT 的课程内容更加丰富全面，并且有配套课程视频和 Slides。此外课后作业只有测试代码，需要学生自主编写模型架构定义和训练代码，虽然硬核但很适合有志于炼丹的同学练手。众所周知，深度学习理论实践中存在着很多经验技巧，魔鬼往往存在于细节里。没有什么比自己上手训一个模型更能掌握这些细节了。
+
+- [CMU 10423: Generative AI](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html): CMU 的 GenAI 课程，相比前两门课更侧重于大语言模型一些，其他内容和前两门课重合较多。不过课程作业都挺有意思，推荐闲暇时间练练手。
+
+
+OpenAI 的 GPT 系列让大语言模型在 Scaling Law 的指引下展现出惊人的效果，在数学和代码领域取得了很大进展。如果你主要关注大语言模型这个方向，那么推荐如下课程：
+
+- [Stanford CS336: Language Modeling from Scratch](https://stanford-cs336.github.io/spring2025/index.html): 正如课程标题写的，在这门课程中你将从头编写大语言模型的所有核心组件，例如 Tokenizer，模型架构，训练优化器，底层算子，训练数据清洗，后训练算法等等。每次作业的 handout 都有四五十页 pdf，相当硬核。如果你想充分吃透大语言模型的所有底层细节，那么非常推荐学习这门课程。
+
+- [CMU 11868: Large Language Model Systems](https://llmsystem.github.io/llmsystem2025spring/): CMU 的大语言模型系统课程，侧重底层系统优化，例如 GPU 加速，分布式训练和推理，以及各种前沿技术。非常适合从事系统领域的同学对这个方向有个全貌性的了解。课表里还包含了一篇我发表的 PD 分离相关的文章，因此私心推荐一下。课程作业的话会让你先实现一个迷你 Pytorch，然后在上面实现各种大语言模型的系统级优化。
+
+- [CMU 11667: Large Language Models: Methods and Applications](https://cmu-llms.org/) 和 [CMU 11711: Advanced NLP](https://www.phontron.com/class/anlp-fall2024/): 和前两门课相比，这两门课更偏重上层算法和应用，而且每节课都列举了很多相关阅读材料，适合对大语言模型发展前沿的各个方向都有个粗糙的认识，如果对某个子领域感兴趣的话再寻着参考资料深入学习。
+
+除了课程以外，还有很多不错的资料作为参考：
+
+- [Awesome-LLM](https://github.com/Hannibal046/Awesome-LLM): 大语言模型相关资料汇总
+- [LLMSys-PaperList](https://github.com/AmberLJC/LLMSys-PaperList): 大语言模型系统相关论文汇总
+- [MLsys-Guide](https://github.com/PKU-DAIR/Starter-Guide/blob/main/docs/systems/Readme.md): 深度学习系统入门指南

mkdocs.yml

@@ -112,8 +112,10 @@ plugins:
             智能计算系统: Intelligent Computing Systems
             深度学习: Deep Learning
             "国立台湾大学: 李宏毅机器学习": NTU Machine Learning
+            深度生成模型: Deep Generative Models
+            学习路线图: Roadmap
             机器学习进阶: Advanced Machine Learning
-            进阶路线图: Roadmap
+            学习路线图: Roadmap
             后记: Postscript
   - search:
       lang:
@@ -278,8 +280,10 @@ nav:
       - "Stanford CS224n: Natural Language Processing": "深度学习/CS224n.md"
       - "Stanford CS224w: Machine Learning with Graphs": "深度学习/CS224w.md"
       - "UCB CS285: Deep Reinforcement Learning": "深度学习/CS285.md"
+  - 深度生成模型:
+      - "学习路线图": "深度生成模型/roadmap.md"
   - 机器学习进阶:
-      - "进阶路线图": "机器学习进阶/roadmap.md"
+      - "学习路线图": "机器学习进阶/roadmap.md"
       - "CMU 10-708: Probabilistic Graphical Models": "机器学习进阶/CMU10-708.md"
       - "Columbia STAT 8201: Deep Generative Models": "机器学习进阶/STAT8201.md"
       - "U Toronto STA 4273 Winter 2021: Minimizing Expectations": "机器学习进阶/STA4273.md"