11-411/611: Natural Language Processing (Fall 2025)

Place and Time: HOA 160, TR 2:00-3:20P

Instructor: Eric Nyberg

Course Description: This course is designed to be accessible to Masters and advanced undergraduate students who seek the basic skills necessary to implement practical Natural Language Processing (NLP) applications using Language Models (LMs) in specific information domains. The syllabus includes learning materials on the core concepts of NLP and LMs, and how they are applied in closed commercial systems (e.g. ChatGPT) as well as open systems (e.g. Llama, T5). Students complete a set of hands-on exercises in Python that develop skills in applying NLP for various practical problems.

Textbook: Jurafsky and Martin, "Speech and Language Processing"

Prerequisite Knowledge: Strong programming skills (in Python); A course in data structures and algorithms (or equivalent experience); A basic knowledge of probability theory and linear algebra

Course Goals: Students acquire basic knowledge of NLP approaches, including language representations, probability theory and language modeling, logistic and softmax regression, word embeddings, neural networks and large language models; and NLP tasks, such as document classification, parsing, knowledge representation and reasoning, translation, and question answering.

Grading (S'25, subject to revision):

Syllabus (S'25, subject to revision):

  1. NLP Landscape and History, Course Objectives
  2. Representation in NLP
  3. Designing, Evaluating, and Incrementally Improving NLP Systems
  4. Probability Theory and Language Modeling
  5. Naive Bayes and Document Classification
  6. Logistic Regression
  7. Softmax Regression
  8. Feed-Forward Neural Networks
  9. Word Embeddings & Distributional Semantics
  10. Modeling Sequences: RNNs and NER
  11. Encoder-Decoder Models, Beam Search
  12. Self-Attention and Transformers
  13. LLMs I: Pretraining, Encoder-only (BERT), Finetuning
  14. LLMs II: Encoder-Decoder (T5) and Decoder-Only (GPT), ICL
  15. LLMs III: RLHF, DPO, Guardrails
  16. Ethics and NLP
  17. Syntax and Parsing
  18. Semantics and Reasoning over Knowledge Representations
  19. Natural Language Inference
  20. Machine Translation
  21. Multilingual NLP
  22. Information Extraction & Coreference
  23. Question Answering I (information retrieval, information extraction)
  24. Question Answering II (LLMs, prompting, RAG)

Last Updated March 25, 2025