Efficient Model Inference and Tuning

This unit teaches techniques for producing concise outputs, scaling inference efficiently, and improving model performance with minimal parameter updates. Students will explore summarization, batch prompting, optimization and parameter-efficient fine-tuning methods, and assess cost/performance trade-offs while preparing to select appropriate models for deployment.

Learning Objectives

• Analyze how summarization techniques and prompt design affect output quality, token usage, and downstream task performance using provided examples
• Apply batch prompting and inference batching strategies to increase throughput and reduce latency across multiple inputs in simulated workloads
• Demonstrate parameter-efficient fine-tuning methods (for example, adapters and LoRA) on a pretrained model and measure resulting changes in accuracy, size, and resource requirements
• Evaluate cost and performance implications of optimization and fine-tuning choices and recommend model-selection criteria for given operational constraints