Assignments & project

Implementation assignments

Three short, hands-on assignments, 5% of the final grade each (15% in total). They are about getting real systems running end to end, not about squeezing out the last point of accuracy.

Each assignment is released on a Tuesday and is due at 23:59 AST. All three deadlines were extended by one week on 2026-10-06, because of limitations with Google Colab; the original dates are shown struck through. The handout and the starter files are posted here on release day; you submit on Blackboard.

Late policy: 10% penalty per 24 hours (or fraction thereof) up to 72 hours, after which a zero is recorded without an approved extension.

AI tools are not forbidden on the assignments, but they are discouraged: the point is to build these things yourself. If you use them, disclosure is mandatory — name the tools and what you used them for.

Assignment 1: Parameter-efficient fine-tuning

Released 2026-09-29 · due 2026-10-06 2026-10-13. Fine-tune a small LLM (Phi-2) to classify news articles with QLoRA, then measure what changed: task accuracy, and how often the model still refuses a small set of harmful requests. Four short training runs vary the training-set size and the LoRA rank. Needs a CUDA GPU with 16 GB; Colab’s free T4 is enough. Builds on the week 4 post-training material; the safety half is inspired by Safety-Tuned LLaMAs (Bianchi et al., ICLR 2024). Handout (PDF) · starter notebook

Assignment 2: An agent loop from scratch

Released 2026-10-06 · due 2026-10-13 2026-10-20. Write, in about 20 lines of Python, the loop that every agent framework hides: call the LLM, execute the tool it asks for, return the observation, repeat until it answers. No framework allowed. Runs on a laptop, against either the free Mistral API or the course’s LLM server on Ibex. Builds on week 4’s tool use and ReAct. Handout (PDF) · starter notebook

Assignment 3: Retrieval-augmented generation

Released 2026-10-13 · due 2026-10-20 2026-10-27. Build a RAG system over the KAUST Student Handbook — embeddings, a FAISS index, a retriever and an LLM — put a router in front of it, and run a parameter study on chunking and retrieval settings. Half the test questions have no answer in the handbook, so the study is about abstaining correctly as much as answering correctly. Uses the same two LLM providers as assignment 2. Builds on week 7. Handout (PDF) · starter notebook · handbook (PDF) · test set (CSV)

Research project

The project is 50% of the final grade and runs the length of the semester. It is the main deliverable of the course, and it should read like a workshop or conference submission by the end.

Teams

Teams of up to three students. Solo projects are permitted but held to the same standard of scope, so a team is usually the better choice. Conceptual discussion across teams is encouraged; code sharing across teams is not permitted.

Agree your team between yourselves, then each member writes the same team number next to their own name on the team sign-up sheet — sign in with your KAUST account. One person per team adds a one-line topic direction. Due Thursday 17 September, 23:59.

What makes a good project

Pick something you could plausibly finish and measure in a semester. The strongest projects share a shape:

1. A specific claim about LLM or agentic systems that is not yet settled in the literature.
2. An experiment that could falsify it.
3. A baseline you can actually run, and a metric you can defend.

Projects may be systems-flavored (serving, retrieval, caching, orchestration), evaluation-flavored (benchmarks, judge reliability, contamination), or application-flavored (a domain deployment with real constraints). Reproduction studies of a recent paper are welcome when paired with an extension of your own.

Deliverables

There are three graded milestones. Together they make up the 50%:

Due dates for every milestone are in the key dates list on the home page.

Proposal — 10%

A short document (2–3 pages) covering:

• The problem and why it matters now.
• Closely related work, and precisely what you would add to it.
• Planned method and the experiments you intend to run.
• How you will evaluate success — metrics, baselines, datasets.
• A week-by-week plan and a division of labor across the team.

Midterm report — 15%

Progress against the proposal — what works, what you have measured so far, what turned out to be harder than expected, and a revised plan for the remainder of the term. Negative results reported honestly are worth more than optimistic projections.

Write it as the first draft of your final report: 4–6 pages in the two-column template below, references and appendices not counted. Submit it through HotCRP as a new submission titled Midterm: <project title>. LaTeX template (zip)

There are no midterm presentations this term: week 10 is needed for the lecture displaced by the mid-semester break (week 8), so the report carries the full 15%.

Final report & presentation — 25%

• Report (15%): a conference-style write-up — motivation, method, experimental setup, results, limitations, related work — together with your code and instructions sufficient for someone else to reproduce the headline result.
• Presentation (10%): in class during week 15 (2026-12-07).
• AI-use disclosure: if you used AI coding assistants, include a short appendix listing the models and tools, representative prompts, and where in the work they were applied. This is required, and it is not held against you — undisclosed use is.

Grading emphasis

Projects are judged on the quality of the question, the rigor of the evaluation, and the honesty of the analysis — not on whether the result was positive. A carefully executed project that refutes its own hypothesis scores well. A project with impressive numbers and an unfalsifiable setup does not.

Submission

Assignments are submitted on Blackboard, and the project proposal and midterm report through HotCRP, with deadlines at 23:59 Saudi time (AST) on the stated date.

Paper summaries are the exception: they go through HotCRP and are due 12:30 AST on the day of class. See the summary policy.