General Information¶
Administrivia¶
Instructor: Marco Canini
Time: M 4:45PM-6:15PM; Th 1:15PM - 2:45PM at Bldg. 9, Room 3221
Credits: 3
Office Hours: By appointment in 1-4418.
The class is open to CS MS and Ph.D. students. Others may be admitted by permission of the instructor.
Communication¶
ALL communication regarding this course must be via Campuswire. This includes questions, discussions, announcements, as well as private messages.
Paper reviews must be submitted via HotCRP.
Course Description¶
This class is an advanced course in systems and AI.
The recent successes of AI/ML owe a great debt to the rapid innovations in hardware and software systems and to the broad accessibility to this technology. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the ML community. On the one hand, innovations in AI/ML have the potential to improve the systems infrastructure itself by appling AI to improve system design, job scheduling, video streaming or simplify network management. This class covers a broad array of research topics and latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems
The course has one semester-long project that is aimed at producing a conference/workshop-quality research paper and must involve writing some software. The project must address an open research problem. Suggested project ideas will be provided by the instructor. To get ideas for projects, please DO be courageous and read ahead in the course schedule! It is highly recommended that you work in groups. Teams are suggested to be 2-3 students. All projects will have to submit initial proposals and final reports; these final reports should read like research papers. Up to five “best projects” at the end of the semester will be earmarked for expedited submission to a renowned conference, with the help of the instructor’s involvement even after the semester is over.
Sign up on the Campuswire forum for this class.
Objectives¶
The goal of this class is to cover a broad array of research topics and latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems. The format of this course will be a mix of lectures, seminar-style discussions, and student presentations. Students will be responsible for paper readings, and completing a hands-on project. Readings will be selected from recent conference proceedings and journals. Suggested project ideas will be provided by the instructor, but students are strongly encouraged to come up with their own project ideas.
On completion of this course students should have a broad understanding of some key papers and concepts in research at the intersection of computer systems and AI/ML, as well as an appreciation of how to argue for or against any particular idea. By the end of the course, students should be able to:
Comprehend and critique relevant research papers in the area of “systems for AI” and “AI for systems”.
Present their research both orally in a concise way and within the allotted time as well as in writing.
Defend the research approach, design decisions, and the evaluation methods in a discussion.
Moderate a discussion after a research presentation.
About the course¶
Prerequisites¶
Students are expected to have good programming skills and must have taken at least one systems-related course (e.g., from operating systems/CS140, distributed systems/CS240, networking/CS244, or databases/CS245), and one AI/ML-related course (e.g., machine learning/CS229) or instructor consent, is required. Students must have experience using a machine learning framework such as TensorFlow, PyTorch, Ray, etc.
Textbook¶
There is no textbook for the course. You will be expected to read 2-4 papers per week. The papers are available electronically through the course website.
Organization and workload¶
The class consists of two main activities:
Papers and discussions: We will read 2-4 papers per week, and discuss them in class. You will need to spend about 2 hours reading each paper, and making notes, to prepare you to discuss them in class. This only works if you come to class ready to discuss the papers in detail, which is why 15% of your grade is for in-class participation. There will be as much as 10-15 hours of reading per week. Do not take this course unless you are willing and able to do a lot of reading.
Project: You will do a semester-long open-ended project.
Grading¶
15% Attendance and Participation
20% Scientific review article presentation
15% Presentation
50% Research Project
Groups¶
It is highly recommended that you work in groups. Teams are suggested to be 2-3 students.
Declare your group’s membership as well as preferences for papers by September 28, 2021. After this date, we will form groups from the remaining students.
Note that paper preferences are only hints.
Paper discussions¶
The course is organized around topics in ML/AI systems, each of which generally comprises of an overview lecture by the instructor and a discussion section. During the discussion, we will dissect the papers’ key ideas, approaches, contributions, positioning against related work, and evaluation methods and results. It is expected that students would have read and submitted their paper reviews in advance. To organize the discussion, students will play different roles and rotate through these.
Reviewers: every student will review each paper (see below)
Detailed Reviewers: three students will write detailed reviews; these are longer reviews that carry a substantially greater level of detail.
Area Chair: one student per topic serves as the area chair; this student collects the reviews by the day before discussion at 1:00PM and summarizes them for everyone’s benefit. They posts the summary publicly on Piazza by 8:00PM the day before discussion.
Champion: the role of the Champion is to advocate in favor of the paper; the Champion is chosen by the Area Chair in agreement with the instructor based on the reviews; there will be one Champion for each paper.
Detractor: the role of the Detractor is to argument against the paper; the Detractor is also chosen by the Area Chair as the Champion; there will be one Detractor for each paper.
During the discussion, the purpose of the Area Chair’s summary will be to capture the paper’s key technical idea and summarize the reviews.
The Area Chair ideally should create a written summary using the following structure:
This paper solves problem X using method Y, demonstrated with {proofs, models, simulations, experiments, bold assertions, etc.}.
Three reasons to accept it are x, y and z. (Should be from reviews)
Three concerns that came up about it are x, y, and z. (Should be from reviews)
My personal recommendation is {accept, reject, not sure}.
In class, the Area Chair will lead the paper discussion. They will start by presenting their summary of the paper (taking up just a few minutes; it’s not a formal paper presentation), then calling up to the Champion who will champion the paper, and then calling up to the Detractor to hear the concerns. Then they will enlarge the discussion to involve other Reviewers and moderate the discussion.
Paper reviews¶
Read all the papers of each day carefully and write your own paper review. Being able to critically judge others’ work is crucial for your understanding. Each critique should be about a half a page and should roughly cover the following questions:
What is the problem addressed by the paper? Is the problem real? Why is this problem important?
What is the hypothesis of the work?
What is the proposed solution’s main idea, and what key insight guides their solution?
Why is the solution different from previous work?
Is problem new?
Are assumptions different?
Is workload different?
Does the paper (or do you) identify any fundamental/hard trade-offs?
What is one (or more) drawback or limitation of the proposal, and how will you improve it?
Do you think the work will be influential in 10 years?
Why or why not?
Detailed Reviewers
When a student is assigned to do a detailed review, the critique will follow the format of an actual review and is expected to enter the paper is much more depth. Every student will do 2 or 3 detailed reviews.
Submission
The paper reviews of each day must be submitted electronically at least 24 hours prior to the corresponding class. Late reviews will not be counted. You can miss up to 4 paper regular reviews during the term without penalty. Each missing review beyond the fourth one will result in 25% decrease in grade for paper reviews. Meaning, failing to turn in 8 or more reviews on time will result in a zero for all paper reviews.
Submit reviews via HotCRP.
Grading by peers
The Area Chairs, Champions and Detractors identify the top 3 regular reviews on each paper and submit this information privately to the instructors.
Grading
At the end of term, the grade for this part will be assigned as follows:
two of your reviews (not including detailed) will be randomly selected and graded. The higher grade of the two will be used for grading.
one of your detailed reviews chosen at random will be graded.
The final grade will be the average between the two parts above.
Additional, the final grade will be influenced by the grading by peers of your reviews during the semester. Specifically, being graded as a top 3 reviewer increases this part of the grade as follows: * by 2 peers awards 1% * by 4 peers awards 2% * … * by 10 peers or more awards 5%
Participation¶
Is class participation based solely on attendance? No. Attendance is a necessary but not sufficient condition for good class participation. You are expected to attend all lectures (you may skip up to 2 lectures due to legitimate reasons), and more importantly, participate in class discussions. If you have any concerns about not being able to regularly attend class (e.g., you will have to miss several classes during the quarter) please discuss this as soon as possible with the instructor. Beyond attendance, we evaluate class participation by observing how prepared students are to discuss the covered paper when they come to class. This is not a trivial requirement because we expect papers to have been read thoroughly prior to lecture.
Project¶
Research projects are the critical aspect of the course. Your goal is to do publishable quality ML systems research on an instructor-approved problem; that is, to add to our understanding of how to build ML systems. Research projects must be written up in a term paper. See details in the Project page.
Exam¶
This class has no exam.
Late work¶
There is no policy on late work. If you cannot submit your work by the deadline, it will not be accepted.
Exceptional circumstances¶
If you have a medical emergency, then email the instructor to request an extension. At times, there may be system problems with the computer infrastructure which may prevent you from submitting your reviews on time. In these cases an extension will be granted to the entire class.
Misconduct policy¶
We will have zero tolerance for academic misconduct. Cheating, plagiarism, and any form of dishonesty will be handled with maximum severity, according to university regulations. If you are ever in doubt about whether an action on your part may constitute unacceptable behavior, please ask the instructor before proceeding—doing so afterward is too late.
Apart from the term project, any work you turn in must be your own and is to be done individually, and the usual code of conduct applies. You must acknowledge any sources of your words, ideas, and software when they are not your own, and you must disclose in advance, without any specific request, any sources you used. Do not use code from a student who took the course in a previous semester.