Advanced
LLMOps
Building and operating LLM and RAG systems that survive contact with real users.
What you'll be able to do
- Build a RAG pipeline that answers from your own documents
- Evaluate output quality with something better than reading samples
- Control token cost and latency under real traffic
- Add guardrails against prompt injection and data leakage
Modules
- 01LLMs for engineersTokens, context windows, temperature, streaming, model selection and the cost of each
- 02Prompting and evaluationStructured prompts, tool calling, eval sets, LLM-as-judge and where it misleads
- 03Retrieval-augmented generationChunking strategy, embeddings, hybrid search, reranking, citation and grounding
- 04Vector storespgvector, Qdrant, Milvus, index tuning, refresh strategy for changing documents
- 05Serving and scalingInference servers, GPU capacity, caching, batching, fallback between providers
- 06Safety, cost and observabilityPrompt injection, PII handling, output filtering, token accounting, tracing a bad answer
Capstone
Ship a production RAG assistant over a real document set, with evaluation, cost tracking, guardrails and an observable failure path.