10-315, Spring 2025

Introduction to Machine Learning (SCS majors)

Overview

Key Information

Monday + Wednesday, 11:00 am - 12:20 pm, Wean Hall 7500

Section A, Friday 10:00 am - 10:50 am, GHC 4102, see Recitation

Section B/C, Friday 11:00 am - 11:50 am, GHC 4301

Section D, Friday 12:00 pm - 12:50 pm, GHC 4301

Section E, Friday 1:00 pm - 1:50 pm, PH 126A

Section F, Friday 2:00 pm - 2:50 pm, GHC 4102

Margaret He, Derek Yuan, Shreya Sridhar, Gaurika Sawhney, Jerick Shi, Ethan Wang, AJ Seo, see the 315 Staff page

Grades will be collected in Canvas.
Midterm 1 20%, Midterm 2 20%, Written/Programming Homework 35%, Pre-Lecture Reading Checkpoints 5%, Online Homework 5%, Participation 5%, Mini-project 10%

There is no required textbook for this course. Any recommended readings will come from sources freely available online.

We will use Ed for questions and any course announcements.

Students will turn in their homework electronically using Gradescope.

Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize human faces, recommend music and movies, and drive autonomous robots). This course covers the core concepts, theory, algorithms and applications of machine learning.

Learning Objectives

After completing the course, students should be able to:

  • Select and apply an appropriate supervised learning algorithm for classification and regression problems (e.g., linear regression, logistic regression, ridge regression, nonparametric kernel regression, neural networks, naive Bayes).
  • Recognize different types of unsupervised learning problems, and select and apply appropriate algorithms (e.g., k-means clustering, Gaussian mixture models, linear and nonlinear dimensionality reduction).
  • Work with probability (Bayes rule, conditioning, expectations, independence), linear algebra (vector and matrix operations, eigenvectors, SVD), and calculus (gradients, Jacobians) to derive machine learning methods such as linear regression, naive Bayes, and principal component analysis.
  • Understand machine learning principles such as model selection, overfitting, and underfitting, and techniques such as cross-validation,regularization, feature learning, fine-tuning, and transfer learning.
  • Implement machine learning algorithms such as logistic regression via stochastic gradient descent, linear regression, or k-means clustering.
  • Work with machine learning toolkits, such as PyTorch, to implement, train, and analyze various deep learning networks, including convolutional neural networks and transformer architectures.
  • Run appropriate supervised and unsupervised learning algorithms on real and synthetic data sets and interpret the results.

Levels

This course is designed for SCS undergraduate majors. It covers many similar topics to other introductory machine learning course, such as 10-301/10-601 and 10-701. This 10-315 course and 15-281 AI Representation and Problem Solving are designed to complement each other and provide both breadth and depth across AI and ML topics. Contact the instructor if you are concerned about which machine learning course is appropriate for you.

Prerequisites

The prequisites for this course are:

  • 15-122: Principles of Imperative Computation
  • 15-151 or 21-127 or 21-128: Mathematical Foundations of Computer Science / Concepts of Mathematics.
  • 36-225 or 36-218 or 36-217 or 15-259 or 15-359 or 21-325 or 36-219: Probability
  • 21-241 or 21-240 or 21-242: Linear Algebra
  • [Implied from prereqs above] Calc II Integration and Approximation

While not explicitly a prerequisite, we will be programming exclusively in Python. Please see the instructor if you are unsure whether your background is suitable for the course.

Office Hours

OHQueue: Link

Pat's Office Hours

In addition to Pat's standing office hourse, he often have "OH" (or "Open") appointment slots on his office hours appointment calendar. If no there are no available OH or appointments that meet your needs, please contact Pat via a private post on Ed with a list of times that work for you to meet.

Schedule

Subject to change

Textbooks:

Bishop, Christopher. Pattern Recognition and Machine Learning, available online

Daumé III, Hal. A Course in Machine Learning, available online

(DL) Goodfellow, Ian, Yoshua Bengio, Aaron Courville. Deep Learning, available online

(MML) Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for Machine Learning, available online

Mitchell, Tom. Machine Learning, available online

Murphy, Kevin P. Machine Learning: A Probabilistic Perspective, available online

(KMPA) Shaw-Taylor, John, Nello Cristianini. Kernel Methods for Pattern Analysis, available online

Dates Topic Lecture Materials Pre-Reading Reading (optional)
1/13 Mon 1. Introduction Notation_Guide.pdf
Math_Background.pdf

pptx (inked) pdf (inked)
MML 2.1-3, 2.5, 2.6 and 3.1, 3.2.1, 3.3
1/15 Wed 2. ML Problem Formulation pptx (inked) pdf (inked) Mitchell 1.1-1.2
Daumé 1
1/20 Mon No class: MLK Day
1/22 Wed 3. Decision Trees pptx (inked) pdf (inked) Decision Trees.pdf
Checkpoint due 1/21 Tue, 11:59 pm
Daumé 2, Entropy, Cross-Entropy video, A. Géron
Paper: ID3
1/27 Mon 4. K-Nearest Neighbor and
Model Selection
pptx (inked) pdf (inked)
pptx (inked) pdf (inked)
Opt and LinReg.pdf
Checkpoint due 1/26 Sun, 11:59 pm
Daumé 3
MML 8.3.3
1/29 Wed 5. Optimization and Linear Regression pptx (inked) pdf (inked)

regression interactive.ipynb
regression blind interactive.ipynb
MML 8.2-8.2.2, 8.2.4
MML 5.2-5.5
2/3 Mon 6. Optimization and Linear Regression (cont.) See previous lecture slides
FeatEng and LogReg.pdf
Checkpoint due 2/2 Sun, 11:59 pm
2/5 Wed 7. Feature Engineering
Logistic Regression
pptx (inked) pdf (inked)
Demos:
Bishop 4.1.3, 4.3.2, 4.3.4
2/10 Mon 8. Logistic Regression (cont.) pptx (inked) pdf (inked)

Convex functions Desmos
Neural Networks.pdf
Checkpoint due 2/9 Sun, 11:59 pm
2/12 Wed 9. Neural Networks pptx (inked) pdf (inked)

three neuron interactive.ipynb
MML 5.6
DL 6
The Matrix Cookbook
2/17 Mon 10. Neural Networks (cont.) pptx (inked) pdf (inked)

Universal network Desmos
Perceptron neuron Desmos
2/19 Wed 11. Regularization
regression regularization.ipynb
Regularization interpolation Desmos (3D)
L1_sparsity.ipynb
DL 7.1,7.8
Bishop 3.1.4
2/24 Mon 12. MLE and Probabilistic Modeling MLE.pdf
Checkpoint due 2/23 Sun, 11:59 pm
MML 9-9.2.2
Bishop 1.2.4-5, 3.1.1-2
2/26 Wed EXAM 1
In-class
Learning objectives: pdf
Practice exam: pdf (sol)
3/3 Mon No class: Spring Break
3/5 Wed No class: Spring Break
3/10 Mon 13. MLE (cont.) & MAP MAP.pdf
Checkpoint due 3/9 Sun, 11:59 pm
MML 9.2.3-4
Mitchell MLE and MAP
3/12 Wed 14. MAP (cont.) Model Cards For Model Reporting. Margaret Mitchell, et al (2019)
PyTorch Basics Tutorial
3/17 Mon 15. Practial & Responsible ML
PyTorch, ML Model Cards
Demos: CNNs.pdf
Checkpoint due 3/17 Mon, 10:30 am
DL 9
3/19 Wed 16. Deep Learning for Computer Vision
3/24 Mon 17. Natrual Language Processing: Features, N-grams Demo: N-grams Word Embeddings
No checkpoint
3/26 Wed 18. NLP: Word Embeddings, Attention
Demos:
The Illustrated Word2vec. Jay Alammar
The Illustrated {AttentionTransformerGPT-2}. Jay Alammar
Video (and code): Let's build GPT. Andrej Karpathy
3/31 Mon 19. NLP (cont.): Transformers, LLMs Demos: PCA, Recommender Systems, Clustering
Checkpoint due 3/30 Sun, 11:59 pm
4/2 Wed 20. Dimensionality Reduction: PCA, Autoencoders, Feature Learning
Demos: PCA: Bishop 12.1, Murphy 12.2
Murphy 25.5
4/7 Mon 21. PCA (cont.) & Clustering Generative.pdf
Checkpoint due 4/6 Sun, 11:59 pm
K-means: Bishop 9.1,
4/9 Wed 22. Recommender Systems &
Probabilistic Generative Models
Matrix Factorization Techniques for Recommender Systems (pdf). Koren, Bell, and Volinsky (2009)
Generative models: Mitchell Generative and Discriminative Classifiers
Murphy 3.5, 4.2, 8.6
4/14 Mon 23. Variational Autoencoders Pre-reading
Checkpoint due 4/13 Sun, 11:59 pm
4/16 Wed 24. Diffusion
4/21 Mon 25. Learning Theory A Few Useful Things to Know about Machine Learning. Pedro Domingos (2012).
Generalization Abilities: Sample Complexity Results. Nina Balcan (2015). Lecture notes.
4/23 Wed EXAM 2
In-class

Recitation

Recitation starts the first week of class, Friday, Jan. 17th. Recitation attendance is recommended to help solidify weekly course topics. That being said, the recitation materials published below are required content and are in scope for midterms 1 and 2. Students frequently say that recitations are one of the most important aspects of the course.

Recitation section assignments will be locked down after the third week. Until then, you may try attending different recitation sections to find the best fit for you. In the case of any over-crowded recitation sections, priority goes to students that are officially registered for that section in SIO. The process to select your final recitation assignment will be announced on Ed as we get closer to Recitation 4.

Recitations will be on Fridays in the following individual recitation sections:


Section Time Location TAs
A Friday 10:00 am - 10:50 am GHC 4102 AJ and Gaurika
B Friday 11:00 am - 11:50 am N/A Merged with Section C
C Friday 11:00 am - 11:50 am GHC 4301 Shreya and Margaret
D Friday 12:00 pm - 12:50 pm GHC 4301 Ethan and Margaret
E Friday 1:00 pm - 1:50 pm PH 126A Jerick and Ethan
F Friday 2:00 pm - 2:50 pm GHC 4102 Derek and Jerick


Dates Recitation Handout/Code
1/17 Fri Recitation 1: NumPy hello_notebooks.ipynb

1_Intro_to NumPy.ipynb (solution)
2_Loading_and_Visualizing_Data.ipynb (solution)
3_Messing_with_MNIST.ipynb

Additional reference:
NumPy_Tutorial_from_11-785.ipynb
1/24 Fri Recitation 2: Decision Trees Worksheet: pdf (solution)
kNN.ipynb
DT.ipynb
1/31 Fri Recitation 3: Matrix Calculus and Linear Regression pdf (solution)
2/7 Fri Recitation 4: Logistic Regression pdf (solution)
2/14 Fri Recitation 5: Neural Networks pdf (solution)
2/21 Fri Recitation 6: Regularization, Prob/Stat Review Worksheet: pdf (solution)
Notebook: Gaussian Contour Plots
2/28 Fri Recitation 7: MLE
No in-person recitation (worksheet only)
pdf (solution)
3/7 Fri No recitation -- Spring Break
3/14 Fri Recitation 8: PyTorch, MAP Worksheet: pdf (solution)
PyTorch Overview Slides
PyTorch Tutorial Notebook.ipynb
3/21 Fri Recitation 9: Computer Vision Worksheet: pdf (solution)
3/28 Fri Recitation 10: NLP Worksheet: pdf (solution)
word_embeddings.ipynb (solution)
minGPT_pico.ipynb
minGPT_femto.ipynb
4/4 Fri Recitation 11: PCA, Recommender Systems, K-means
Carnival: No in-person recitation (worksheet only)
Worksheet: pdf (solution)
4/11 Fri Recitation 12: Generative & Discriminative Models + MAP
4/18 Fri Recitation 13: Variational Autoencoders
4/25 Fri Recitation 14: Last Recitation! Mini-project Help Session

Exams

The course includes two midterm exams. The midterms will be on Feb. 26 and Apr. 23. Both will take place in class, 11:00 am-12:20 pm. Plan any travel around exams, as exams cannot be rescheduled. There is no final exam.

Mini-project

A mini-project due during the final exam period. This will be an opportunity to work with a team and apply machine learning concepts from class to a project that is more customized to your interests. More details about the mini-project details and deadlines will be announce later in the semester.

Assignments

There will be approximately five homework assignments that will have written and programming components and approximately six online assignments (subject to change). Written and online components will involve working through algorithms presented in the class, deriving and proving mathematical results, and critically analyzing material presented in class. Programming assignments will involve writing code in Python to implement various algorithms.

For any assignments that aren't released yet, the dates below are tentative and subject to change.

Assignment due dates (Tentative)

Assignment Link (if released) Due Date
HW 0 (online) Gradescope 1/23 Thu, 11:59 pm
HW 1 (programming) hw1.ipynb 1/23 Thu, 11:59 pm
HW 2 (online) Gradescope 1/30 Thu, 11:59 pm
HW 3 (written/programming) hw3_blank.pdf, hw3_tex.zip, hw3.ipynb 2/6 Thu, 11:59 pm
HW 4 (online) Gradescope 2/13 Thu, 11:59 pm
HW 5 (written/programming) hw5_blank.pdf, hw5_tex.zip, hw5.ipynb 2/21 Fri, 11:59 pm
HW 6 (online) Gradescope 3/20 Thu, 11:59 pm
HW 7 (written/programming) hw7_blank.pdf, hw7_tex.zip, hw7.ipynb 3/27 Thu, 11:59 pm
HW 8 (online) Gradescope 4/2 Wed, 11:59 pm
HW 9 (online) Gradescope 4/12 Sat, 11:59 pm
HW 10 (written/programming) hw10_blank.pdf, hw10_tex.zip, Programming notebook coming soon! 4/18 Fri, 11:59 pm

Pre-reading due dates (Tentative)

Assignment Link (if released) Due Date
Decision Trees Checkpoint 1/21 Tue, 11:59 pm
Opt and LinReg Checkpoint 1/26 Sun, 11:59 pm
Feature Eng., Logistic Reg. Checkpoint 2/2 Sun, 11:59 pm
Neural Networks Checkpoint 2/9 Sun, 11:59 pm
Regularization CANCELLED N/A
MLE Checkpoint 2/23 Sun, 11:59 pm
MAP Checkpoint 3/9 Sun, 11:59 pm
CNNs Checkpoint 3/17 Mon, 10:30 am
Word Embeddings Released: 3/19 Thu 3/23 Sun, 11:59 pm
PCA, Recommender Sys., Clustering Checkpoint 3/30 Sun, 11:59 pm
Probabilistic Generative Models Checkpoint 4/6 Sun, 11:59 pm
Variational Autoencoders Released: 4/9 Wed 4/13 Sun, 11:59 pm

Project due dates

Assignment Link (if released) Due Date
Prjoject Group Selected Google Form 4/2 Thur, 11:59 pm
Prjoject Proposal with Data Google Form 4/9 Wed, 11:59 pm
Final Project Submission Google Form 5/2 Fri, 11:59 pm

Policies

Grading

Grades will ultimately be collected and reported in Canvas.

Final scores will be composed of:

Final Grade

This class is not curved. However, we convert final course scores to letter grades based on grade boundaries that are determined at the end of the semester. What follows is a rough guide to how course grades will be established, not a precise formula — we will fine-tune cutoffs and other details as we see fit after the end of the course. This is meant to help you set expectations and take action if your trajectory in the class does not take you to the grade you are hoping for. So, here's a rough heuristics about the correlation between final grades and total scores:

This heuristic assumes that the makeup of a student's grade is not wildly anomalous: exceptionally low overall scores on exams, programming assignments, or written assignments will be treated on a case-by-case basis and, while rare, could potentially drop a students grade.

Precise grade cutoffs will not be discussed at any point during or after the semester. For students very close to grade boundaries, instructors may, at their discretion, consider participation in lecture and recitation, exam performance, and overall grade trends when assigning the final grade.

Participation

In class, we will use a series of polls as part of an active learning technique called Peer Instruction. Your participation grade will be based on the percentage of these in-class poll questions answered:

It is against the course academic integrity policy to answer in-class polls when you are not present in lecture. Violations of this policy will be reported as an academic integrity violation. Information about academic integrity at CMU may be found at https://www.cmu.edu/academic-integrity.

Late Policies, and Extensions, and Exceptions

Participation

Pre-reading checkpoints

Pre-reading checkpoints don't have any extensions or late days. However, the lowest two checkpoints will be dropped when computing your semester score. Reasoning: We want to make sure that everyone is able to complete the pre-reading prior to lecture, so we can build on that knowledge in class; minor illness and other minor disruptive events outside of your control happen occasionally and thus dropping the lowest two scores. See below for information on rare exceptions.

Written/programming homework and online homework

You have a pool of 6 late days across all written/programming and online assignment types

Exceptions and extensions

Aside from late days, dropping the lowest checkpoints, and the 80% threshold for participation, there will be no extensions on assignments in general. If you think you really really need an extension on a particular assignment, e-mail Nichelle, nichellp@andrew.cmu.edu, as soon as possible and before the deadline. Please be aware that extensions are entirely discretionary and will be granted only in exceptional circumstances outside of your control (e.g., due to severe illness or major personal/family emergencies, but not for competitions, club-related events, or interviews). The instructors will require confirmation from your academic advisor, as appropriate.

We certainly understand that unfortunate things happen in life. However, not all unfortunate circumstances are valid reasons for an extension. Nearly all situations that make you run late on an assignment homework can be avoided with proper planning - often just starting early. Here are some examples:

Collaboration and Academic Integrity Policies

Collaboration

AI Assistance

To best support your own learning, you should complete all graded assignments in this course yourself, without any use of generative artificial intelligence (AI), such as ChatGPT. Please refrain from using AI tools to generate any content (text, video, audio, images, code, etc.) for an assessment. Passing off any AI generated content as your own (e.g., cutting and pasting content into written assignments, or paraphrasing AI content) constitutes a violation of CMU's academic integrity policy.

Policy Regarding “Found Code”

You are encouraged to read books and other instructional materials, both online and offline, to help you understand the concepts and algorithms taught in class. These materials may contain example code or pseudo code, which may help you better understand an algorithm or an implementation detail. However, when you implement your own solution to an assignment, you must put all materials aside, and write your code completely on your own, starting “from scratch”. Specifically, you may not use any code you found or came across. If you find or come across code that implements any part of your assignment, you must disclose this fact in your collaboration statement.

Duty to Protect One's Work

Students are responsible for pro-actively protecting their work from copying and misuse by other students. If a student's work is copied by another student, the original author is also considered to be at fault and in gross violation of the course policies. It does not matter whether the author allowed the work to be copied or was merely negligent in preventing it from being copied. When overlapping work is submitted by different students, both students will be punished.

Do not post your solutions publicly, neither during the course nor afterwards.

Penalties for Violations of Course Policies

Violations of these policies will be reported as an academic integrity violation and will also result in a -100% score on the associated assignment/exam. Information about academic integrity at CMU may be found at https://www.cmu.edu/academic-integrity. Please contact the instructor if you ever have any questions regarding academic integrity or these collaboration policies.

(The above policies are adapted from 10-601 Fall 2018 and 10-301/601 Fall 2023 course policies.)

Accommodations for Students with Disabilities

If you have a disability and have an accommodations letter from the Disability Resources office, we encourage you to discuss your accommodations and needs with us as early in the semester as possible. We will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, we encourage you to visit their website.

Statement of Support for Students' Health & Well-being

Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, getting enough sleep, and taking some time to relax. This will help you achieve your goals and cope with stress.
All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is almost always helpful.
If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at http://www.cmu.edu/counseling/. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.
If you have questions about this or your coursework, please let us know. Thank you, and have a great semester.