Schedule and Reading List

Date	Topic
Aug 30	Introduction and Course Overview [pptx]
Sep 2	ML Systems Research SysML: The New Frontier of Machine Learning Systems A Few Useful Things to Know About Machine Learning A Berkeley View of Systems Challenges for AI Optional How to Read a Paper ML Lifecycle [pptx]
Sep 6	Background Review The second week will be dedicated to review the developments occurred in the most recent wave of AI successes. We will focus specifically on machine learning with graphs and use Stanford’s CS224W: Machine Learning with Graphs as the source of lecture materials. Below is the list of lectures we will cover. Lecture 1.1 - Why Graphs Lecture 6.1 - Introduction to Graph Neural Networks Lecture 6.2 - Basics of Deep Learning Lecture 6.3 - Deep Learning for Graphs Lecture 7.1 - A general Perspective on GNNs Lecture 7.2 - A Single Layer of a GNN Lecture 7.3 - Stacking layers of a GNN Lecture 8.1 - Graph Augmentation for GNNs Lecture 8.2 - Training Graph Neural Networks Lecture 8.3 - Setting up GNN Prediction Tasks Also see A Gentle Introduction to Graph Neural Networks This is not a introductory class to ML. Students taking the class should already be familiar with most of the concepts reviewed. Below is a list of additional resources that would be helpful to brush up some concepts. Kevin Murphy’s Textbook Introduction to Machine Learning. This provides a very high-level overview of machine learning. You should probably know all of this. Stanford CS231n Tutorial on Neural Networks. Module 1 is especially recommended for a quick crash course in machine learning and some of the techniques encountered in this class. Google’s Machine Learning Crash Course. This includes several short video lectures giving a practical introduction to machine learning with TensorFlow API. 3Blue1Brown’s Neural Network Basics For a more complete array of resources, see The ML Hub @ KAUST’s page on the topic.
Sep 9	Background Review We continue with a background review and initiate the discussion of ML Systems as Software 2.0. Software 2.0 Blog Post Software 2.0 and the Paradigm Shift in Programming ML Systems
	GNNs
Sep 13	Design Space for Graph Neural Networks Watch: How Powerful are Graph Neural Networks?
Sep 16	No class
Sep 20	DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs Improving the Accuracy, Scalability, and Performance of Graph Neural Networks with Roc
Sep 23	No class (Holiday)
Sep 27	Global Neighbor Sampling Training Graph Neural Networks with 1000 Layers
Sep 30	P3: Distributed Deep Graph Learning at Scale
Oct 4	GCN meets GPU: Decoupling “When to Sample” from “How to Sample” Accelerating Graph Sampling for Graph Machine Learning using GPUs
Oct 7	Marius: Learning Massive Graph Embeddings on a Single Machine
Oct 11	No class
Oct 14	Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
Oct 18	No class (Mid-semester break)
Oct 21	GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs
Oct 25	Seastar: Vertex-Centric Programming for Graph Neural Networks
Oct 28	GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings
Nov 1	Mid-semester Project Presentations
Nov 4	On peer review
Nov 8	Efficient Sparse Collective Communication and its application to Accelerate Distributed Deep Learning
Nov 11	Elastic Hyperparameter Tuning on the Cloud
Nov 15	INFaaS: Automated Model-less Inference Serving
Nov 18	No class (project time)
Nov 22	No class (project time)
Nov 25	DistAI: Data-Driven Automated Invariant Learning for Distributed Protocols
Nov 29	A graph placement methodology for fast chip design
Dec 2	Evaluating Large Language Models Trained on Code
Dec 6	GRAF: A Graph Neural Network based Proactive Resource Allocation Framework for SLO-Oriented Microservices
Dec 9	Final Project Presentations