CS8803: Systems for LLMs (Spring 2024)

Where: College of Computing 53
When: MW 3:30 pm - 4:45 pm

Course Description

Large Language Models (LLMs) have transformed the machine learning landscape in a very short period of time and arguably become the fastest growing subfield in machine learning. Triggered by the unprecedented popularity of ChatGPT which managed to sign-up over a 100 million users in less than two months of its release, LLMs have enabled many breakthroughs in the field of generative AI.

The success of LLMs can be attributed to the systems that underpin them; the massive scale and complexity of these models are made possible by the advancements in the area of systems. In this course, we will focus on the systems that enable LLMs. In particular, we will study the latest advances in system design that make generative AI/LLMs possible and how the breakthroughs in generative AI/LLMs have influenced how we design systems.

(Tentative) Format & Pre-requisites

This course will be offered in a graduate seminar format, where students will lead paper discussions. Being a graduate seminar, the majority of the grade will be determined by a research project. Students will be expected to work on a substantial, original research project that combines systems and LLMs. The ultimate goal of this project is to produce high quality work that can potentially be published in a top-tier systems conference such as SOSP, OSDI and NSDI.

Students should have completed distributed systems (CS7210) and graduate level operating systems (CS4210/6210) courses. Strong system building skills and proficiency with C/C++/Java/Scala/Python programming is required. Experience with machine learning frameworks and tools (e.g., PyTorch or TensorFlow) is highly recommended. Preference will be given to PhD students. MS students with research experience may be admitted if space permits.