From 6069cc86a55da96e5e96d26def2b2a4aece321d5 Mon Sep 17 00:00:00 2001 From: Yinmin Zhong Date: Sat, 7 Jun 2025 23:57:28 +0800 Subject: [PATCH] add cmu11711 --- docs/深度生成模型/大语言模型/CMU11-667.md | 32 ++++++++++++++++++++ docs/深度生成模型/大语言模型/CMU11-711.en.md | 27 +++++++++++++++++ docs/深度生成模型/大语言模型/CMU11-711.md | 27 +++++++++++++++++ mkdocs.yml | 5 +-- 4 files changed, 89 insertions(+), 2 deletions(-) create mode 100644 docs/深度生成模型/大语言模型/CMU11-711.en.md create mode 100644 docs/深度生成模型/大语言模型/CMU11-711.md diff --git a/docs/深度生成模型/大语言模型/CMU11-667.md b/docs/深度生成模型/大语言模型/CMU11-667.md index e69de29b..b448b0cd 100644 --- a/docs/深度生成模型/大语言模型/CMU11-667.md +++ b/docs/深度生成模型/大语言模型/CMU11-667.md @@ -0,0 +1,32 @@ +# Carnegie Mellon University CS 11-667: Large Language Models Methods and Applications + +## 课程简介 + +- **所属大学**: 卡内基梅隆大学 (Carnegie Mellon University) +- **先修要求**: + - 机器学习的基本知识（相当于课程 10-301/10-601） + - 自然语言处理的相关概念（相当于课程 11-411/11-611） + - 熟练掌握 Python 编程，了解 PyTorch 或类似的深度学习框架。 +- **课程难度**: 🌟🌟🌟🌟🌟🌟 +- **课程网站**: [大型语言模型的方法与应用](https://cmu-llms.org/) + +《大型语言模型的方法与应用 (11-667)》是一门研究生课程，提供关于大型语言模型（LLMs）最新进展的全面概述。 + +课程涵盖了语言模型的基础知识、网络架构、训练、推理与评估，并深入探讨了解释性、对齐、涌现能力、扩展定律，以及高效训练和部署的最新技术。学生还将学习 LLM 的部署风险及挑战，并探索其在新应用中的潜力。 + +### 课程内容： +- 语言模型的基础 +- 网络架构、训练与评估 +- 涌现能力与扩展定律 +- 在自然语言处理及其他领域的应用 +- 隐私保护、对齐与鲁棒性 +- 部署中的挑战与伦理考虑 + +### 学习目标： +完成课程后，学生将能够： +- 比较并分析不同的 LLM 模型及其应用场景。 +- 使用 PyTorch 实现和训练语言模型。 +- 使用开源工具微调并进行预训练模型的推理。 +- 理解 LLM 在下游任务中的应用，并评估训练时的决策对这些任务的影响。 +- 阅读并理解 LLM 领域的学术论文，掌握相关术语（如扩展定律、RLHF、提示工程等）。 +- 设计创新方法，将现有大型语言模型应用于新场景。 \ No newline at end of file diff --git a/docs/深度生成模型/大语言模型/CMU11-711.en.md b/docs/深度生成模型/大语言模型/CMU11-711.en.md new file mode 100644 index 00000000..3de7e84a --- /dev/null +++ b/docs/深度生成模型/大语言模型/CMU11-711.en.md @@ -0,0 +1,27 @@ +# CMU 11-711: Advanced Natural Language Processing (ANLP) + +## Course Overview + +* University: Carnegie Mellon University +* Prerequisites: No strict prerequisites, but students should have experience with Python programming, as well as a background in probability and linear algebra. Prior experience with neural networks is recommended. +* Programming Language: Python +* Course Difficulty: 🌟🌟🌟🌟 +* Estimated Workload: 100 hours + +This is a graduate-level course covering both foundational and advanced topics in Natural Language Processing (NLP). The syllabus spans word representations, sequence modeling, attention mechanisms, Transformer architectures, and cutting-edge topics such as large language model pretraining, instruction tuning, complex reasoning, multimodality, and model safety. Compared to similar courses, this course stands out for the following reasons: + +1. **Comprehensive and research-driven content**: In addition to classical NLP methods, it offers in-depth discussions of recent trends and state-of-the-art techniques such as LLaMa and GPT-4. +2. **Strong practical component**: Each lecture includes code demonstrations and online quizzes, and the final project requires reproducing and improving upon a recent research paper. +3. **Highly interactive**: Active engagement is encouraged through Piazza discussions, Canvas quizzes, and in-class Q&A, resulting in an immersive and well-paced learning experience. + +Self-study tips: + +* Read the recommended papers before class and follow the reading sequence step-by-step. +* Set up a Python environment and become familiar with PyTorch and Hugging Face, as many hands-on examples are based on these frameworks. + +## Course Resources + +* Course Website: [https://www.phontron.com/class/anlp-fall2024/](https://www.phontron.com/class/anlp-fall2024/) +* Course Videos: Lecture recordings are available on Canvas (CMU login required) +* Course Texts: Selected classical and cutting-edge research papers + chapters from *A Primer on Neural Network Models for Natural Language Processing* by Yoav Goldberg +* Course Assignments: [https://www.phontron.com/class/anlp-fall2024/assignments/](https://www.phontron.com/class/anlp-fall2024/assignments/) diff --git a/docs/深度生成模型/大语言模型/CMU11-711.md b/docs/深度生成模型/大语言模型/CMU11-711.md new file mode 100644 index 00000000..636ae215 --- /dev/null +++ b/docs/深度生成模型/大语言模型/CMU11-711.md @@ -0,0 +1,27 @@ +# CMU 11-711: Advanced Natural Language Processing (ANLP) + +## 课程简介 + +* 所属大学：Carnegie Mellon University +* 先修要求：无硬性先修要求，但需具备 Python 编程经验，以及概率论和线性代数基础；有神经网络使用经验者更佳。 +* 编程语言：Python +* 课程难度：🌟🌟🌟🌟 +* 预计学时：100 学时 + +该课程为研究生级别的 NLP 入门与进阶课程，覆盖从词表征、序列建模，到注意力机制、Transformer 架构，再到大规模语言模型预训练、指令微调与复杂推理、多模态和安全性等前沿主题。与其他同类课程相比，本课程： + +1. **内容全面且紧跟最新研究**：除经典算法外，深入讲解近年热门的大模型方法（如 LLaMa、GPT-4 等）。 +2. **实践性强**：每次课配套代码演示与在线小测，学期末项目需复现并改进一篇前沿论文。 +3. **互动良好**：Piazza 讨论、Canvas 测验及现场答疑，学习体验沉浸而有节奏。 + +自学建议： + +* 提前阅读课前推荐文献，跟着阅读顺序循序渐进。 +* 准备好 Python 环境并熟悉 PyTorch/Hugging Face，因为大量实战代码示例基于此。 + +## 课程资源 + +* 课程网站：[https://www.phontron.com/class/anlp-fall2024/](https://www.phontron.com/class/anlp-fall2024/) +* 课程视频：课堂讲座录制并上传至 Canvas（需 CMU 帐号登录） +* 课程教材：各类经典与前沿论文＋Goldberg《A Primer on Neural Network Models for Natural Language Processing》章节阅读 +* 课程作业：[https://www.phontron.com/class/anlp-fall2024/assignments/](https://www.phontron.com/class/anlp-fall2024/assignments/) diff --git a/mkdocs.yml b/mkdocs.yml index e8a348f3..aef35d18 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -284,8 +284,9 @@ nav: - 深度生成模型: - "学习路线图": "深度生成模型/roadmap.md" - "大语言模型": - - "CMU 11868: Large Language Model System": "深度生成模型/大语言模型/CMU11-868.md" - - "CMU 11667: Large Language Models: Methods and Applications": "深度生成模型/大语言模型/CMU11-667.md" + - "CMU 11-868: Large Language Model System": "深度生成模型/大语言模型/CMU11-868.md" + - "CMU 11-667: Large Language Models: Methods and Applications": "深度生成模型/大语言模型/CMU11-667.md" + - "CMU 11-711: Advanced Natural Language Processing": "深度生成模型/大语言模型/CMU11-711.md" - 机器学习进阶: - "学习路线图": "机器学习进阶/roadmap.md" - "CMU 10-708: Probabilistic Graphical Models": "机器学习进阶/CMU10-708.md"