Foundations of Autonomous Decision Making under Uncertainty

10-734, Fall 2024

Aarti Singh

Home Lectures Homeworks Project

Note: this is a tentative lecture schedule that is subject to change.

Date Topic Notes Useful links
Aug 27 Tuesday Intro to Decision Making Lecture1.pdf
Aug 29 Thursday Experimental Design Lecture2.pdf On Computationally Tractable Selection of Experiments
Near optimal Design of experiments via Regret Minimization
Combinatorial Algorithms for Optimal Design
Sept 3 Tuesday Active Learning Lecture3.pdf Active Learning Survey-Burr Settles
Noise-adaptive margin based active learning linear classifiers
Active Learning Decision Trees1,2
Active Learning NN: Coresets
Sept 5 Thursday Multi Armed Bandits Lecture4.pdf Intro to Multi-Armed Bandits, Ch1
Sept 10 Tuesday UCB Sampling Lecture5.pdf Intro to Multi-Armed Bandits, Ch1,2
Sept 12 Thursday Nonparametric Bandits Lecture6.pdf Intro to Multi-Armed Bandits, Ch4
Sept 17 Tuesday Linear Bandits Lecture7.pdf Reinforcement Learning: Theory & Algorithms, Ch6, Dani et al, Abbasi et al
Sept 19 Thursday Thompson Sampling Lecture8.pdf Intro to Multi-Armed Bandits, Ch3
Optimize via posterior sampling, Russo_VanRoy
Sept 24 Tuesday GP bandits Lecture9.pdf GP optimization in Bandit Setting
Sept 26 Thursday Model selection in Bandits Lecture10.pdf CORRAL,SmoothCORRAL,RBBE,ModCB,
AdaptLipConstant,AdaptKernel
Oct 1 Tuesday Online learning with experts Lecture11.pdf Intro to Multi-Armed Bandits, Ch5
Oct 3 Thursday Adversarial Bandits Lecture12.pdf Intro to Multi-Armed Bandits, Ch6
Oct 8 Tuesday Class canceled, TA office hours
Oct 10 Thursday Contextual & Generalized Bandits Intro to Multi-Armed Bandits, Ch8, Square-CB paper
Oct 15 Tuesday No Class - FALL BREAK
Oct 17 Thursday No Class - FALL BREAK
Oct 22 Tuesday RCT and Bandits Lecture14.pdf ATE estimation_Kato et al
Oct 24 Thursday Reinforcement Learning Lecture15.pdf RL_theory book AJKS, Ch 1.1,1.2
Oct 29 Tuesday Value and Policy Iteration Lecture16.pdf RL_theory book AJKS, Ch 1.3
Oct 31 Thursday Tabular MDP Lecture17.pdf RL_theory book AJKS, Ch 7
Nov 5 Tuesday No Class - ELECTION DAY
Nov 7 Thursday Linear MDP Lecture18.pdf LSVI-UCB paper
Nov 12 Tuesday Linear MDP contd.. Lecture19.pdf LSVI-UCB paper
Nov 14 Thursday Nonlinear RL Lecture20.pdf General_func_Akshaynotes
Nov 19 Tuesday Offline & Hybrid RL OfflineRL_FQI, HybridRL
Nov 21 Thursday Policy Gradient Lecture22.pdf RL_theory book AJKS, Ch 11
Nov 26 Tuesday Human-AI decision making Lecture23.pdf
Nov 28 Thursday No Class - THANKSGIVING
Dec 3 Tuesday Project Presentations
Dec 5 Thursday Project Presentations