
LLM Production Engineering: Ship AI Apps That Scale
Udemy · English
About this course
- Language
- English
- Category
- Provider
- Udemy
- Data updated
Your LLM demo works. Your production system is where it goes to die — in the latency tail your users feel, in a bill nobody measured, in an eval set that drifted away from real traffic, in an injection attack a user pastes in on a Friday. This course is the engineering between the demo and the system that survives: serving and inference, prompts as code, latency and cost engineering, reliability ladders, context budgets, evaluation with golden sets and CI gates, observability, and guardrails — every topic taught as a measurement or a build, never as a slideware claim. The core ten lectures map the whole discipline: the four forces that kill LLM products (latency, cost, reliability, drift); hosted APIs versus self-hosting, decided with numbers; prompts under version control with structured outputs; latency engineering — streaming, caching, and the tails that matter more than the average; token economics and cascades; retries, fallbacks and circuit breakers; context budgets, memory and fresh RAG; evaluation with golden sets, judges and CI gates; traces and injection defence; and the deployment architecture that assembles it all. Then you build three production systems. Project 1: AnswerDesk, a RAG service over a real corpus — designed by contract and census, built with citation-bound answers and abstention guards that took three measured calibration rounds to get honest, and shipped behind an evals gate that can block its own deploy. Project 2: the Optimization Lab — a 240-request workload measured to the token with a real tokenizer, then compressed, cached and cascaded from $479.66 to $25.91 a month (94.6% down) behind a quality gate that had to pass before the savings counted. Project 3: ShipGate — structured traces for every request, a 12-attack injection battery refused 12/12 with zero false refusals, PII masked before it ever reaches a log, and a five-gate ship board you will watch fail three differen…











