| Instructors |
Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Aarti Singh (aarti at cs dot cmu dot edu)
|
| Teaching Assistants |
Hao Gu (hg1 at andrew dot cmu dot edu)
Devendra Sachan (dsachan at andrew dot cmu dot edu)
Yifeng Tao (yifengt at andrew dot cmu dot edu)
Yichong Xu (yichongx at andrew dot cmu dot edu)
Hongyang Zhang (hongyanz at andrew dot cmu dot edu)
|
| Office Hours |
Pradeep Ravikumar: GHC 8111, Mondays 3:00-4:00 PM
Aarti Singh: GHC 8207, Wednesdays 3:00-4:00 PM
Hao Gu: Citadel Teaching commons, GHC 5th floor, Tuesdays 4:00-5:00 PM
Devendra Sachan: LTI Open Space, 5th floor, Fridays 3:00-4:00 PM
Yifeng Tao: GHC 7405, Mondays 10:00-11:00 AM
Yichong Xu: GHC 8215, Tuesdays, 10:00-11:00 AM
Hongyang Zhang: GHC 8008, Wednesdays 9:00-10:00 AM
|
| Grading |
45% Homeworks, 5% Quizzes, 25% Little Tests, 25% Project
|
| Textbooks |
Lectures are intended to be self-contained. For supplementary readings, with each lecture, we will have pointers to chapters from the following books:
- BV: Convex Optimization, Stephen Boyd and Lieven Vandenberghe,
(available online for free).
- DB: Nonlinear Programming, Dimitri P. Bertsekas.
- NW: Numerical Optimization, Jorge Nocedal and Stephen Wright.
- YN: Introductory lectures on convex optimization: a basic course, Yurii Nesterov.
|
| Course Details |
Syllabus. Piazza Discussion Board. Homeworks. Project.
|
| Tentative Schedule |
| Date |
Topic |
Inst. |
Quiz |
Readings |
Notes |
| Module: Basics |
| Aug 28 |
Intro (slides) |
AS |
quiz |
|
|
| Aug 30 |
Convex Sets, Functions (slides, notes) |
AS |
quiz |
BV Chap. 2,3 |
HW 1a out (Sept 1) |
| Sep 06 |
Convex Opt. (slides, notes) |
AS |
quiz |
BV Chap. 4 |
HW 1b out (Sept 6) |
| Sep 11 |
Canonical Forms (slides) |
PR |
quiz |
BV Chap. 6,7,8 |
HW1a due (Sept 11) |
| Sep 13 |
Applications (slides ) |
PR |
quiz |
BV Chap. 6,7,8 |
HW1b due, HW 2 out (Sept 15) |
| Module: Unconstrained |
| Sep 18 |
Descent Alg., Line Search (slides ) |
PR |
quiz |
NW Chap. 3 |
|
| Sep 20 |
Gradient Descent (slides ) |
PR |
quiz |
NW Chap. 3 |
|
| Sep 25 |
Conjugate Grad. Desc.(slides ) |
AS |
quiz |
NW Chap. 5
CGThmProof (slides24-38) |
|
| Sep 27 |
Accel. Grad. Desc. (slides ) |
PR |
quiz |
YN Chap. 2 |
HW2 due, HW3 out(Sept 29) |
| Oct 02 |
Newton Methods (slides ) |
AS |
quiz |
NW Chap. 3
Convergence |
Project proposal due (Oct 6) |
| Oct 04 |
Quasi Newton (slides ) |
AS |
quiz |
NW Chap. 3 |
|
| Oct 11 |
Trust Region (slides ) |
PR |
quiz |
NW Chap. 4 |
HW3 due (Oct 13) |
| Module: Constrained: Simple Convex Sets |
| Oct 09 |
Proximal, Projected Grad. Desc. (slides, notes) |
AS |
quiz |
DB Chap. 2 |
|
| Oct 16 |
Little Test I |
|
|
|
|
| Oct 18 |
Cond. Gradient/Frank Wolfe (slides, notes) |
AS |
quiz |
DB Chap. 2 |
|
| Module: Constrained: Lagrange Methods |
| Oct 23 |
Lagrange Mult. Theory (slides) |
PR |
quiz |
DB Chap. 3 |
HW4 out (Oct 20) |
| Oct 25 |
Barrier Methods (slides) |
PR |
quiz |
DB Chap. 4 |
|
| Oct 30 |
Augmented Lagrangian (slides) |
AS |
quiz |
DB Chap. 4 |
|
| Nov 01 |
ADMM, Primal Dual Interior Point Methods (slides) (slides) |
AS |
quiz |
DB Chap. 4 |
|
| Module: Constrained: Dual Methods |
| Nov 06 |
Duality Theory (slides) |
AS |
quiz |
DB Chap. 5 |
HW4 due, HW5 out (Nov 4) |
| Nov 08 |
Duality Theory (slides) |
AS |
quiz |
DB Chap. 5 |
|
| Nov 13 |
Dual Ascent (slides) |
AS |
quiz |
DB Chap. 6 |
|
| Nov 15 |
Coordinate Descent (slides) |
PR |
quiz |
NW Chap. 9 |
|
| Nov 20 |
Subgradients, Cutting Plane Methods (slides) |
PR |
quiz |
DB Chap. 6 |
HW5 due (Nov 18) |
| Module: Misc. Frameworks |
| Nov 22 |
No Class, Thanksgiving Break |
|
|
|
Project midterm due (Nov 22) |
| Nov 27 |
Discrete Optimization: Relaxations, Branch and Bound (slides) |
PR |
quiz |
DB Chap. 5 |
|
| Nov 29 |
Stochastic Optimization (slides) |
PR |
quiz |
Papers |
|
| Dec 04 |
Non-smooth Optimization (slides) |
PR |
|
YN Chap. 3 |
|
| Dec 06 |
Little Test II |
|
|
|
|
| Dec 13 |
Project Poster Pres. |
|
|
|
Project final report due (Dec 15) |
|
|
| Homeworks |
- HW 1 out. Prob 1-3 Due Sept 11, 5:00 pm, Prob 4 Due Sept 15 5:00 pm (for data files, please see Resources section in course Piazza page).
- HW 2 out. Due Sept 29, 5:00 pm
- HW 3 out. Due Oct 13, 5:00 pm
- HW 4 out. Due Nov 4, 11:59 pm
- HW 5 out. Due Nov 18, 11:59 pm
|
| Project |
|