LLMOps Cheat Sheet

Updated 2026-04-20

Next Topic: LoRA and Parameter-Efficient Fine-Tuning Cheat Sheet

🧠Study flashcards on this topic176 cards · spaced repetition→

LLMOps (Large Language Model Operations) is the specialized discipline of deploying, managing, monitoring, and maintaining large language models in production environments. It extends traditional MLOps practices while addressing unique challenges of LLMs, including prompt engineering, inference optimization, hallucination detection, and massive computational requirements. Unlike classic ML pipelines, LLMOps must handle non-deterministic outputs, context window constraints, costly API calls, and the rapid evolution of model capabilities—making observability, cost control, and iterative experimentation central to every deployment. In 2026, the field has matured around agentic workflows, standardized protocols like MCP and A2A, and a growing emphasis on governance as regulations such as the EU AI Act take full effect.

What This Cheat Sheet Covers

This topic spans 17 focused tables and 204 indexed concepts, 176 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core LLMOps ConceptsTable 2: Prompt Management & EngineeringTable 3: Model Fine-Tuning & AdaptationTable 4: Deployment & ServingTable 5: Retrieval-Augmented Generation (RAG)Table 6: Monitoring & ObservabilityTable 7: Evaluation & TestingTable 8: Hallucination Detection & MitigationTable 9: Security & SafetyTable 10: Cost OptimizationTable 11: Data Management & PreprocessingTable 12: CI/CD & AutomationTable 13: Error Handling & ResilienceTable 14: Agent Orchestration & Multi-Agent SystemsTable 15: Governance & ComplianceTable 16: Advanced Optimization TechniquesTable 17: LLMOps Platforms & Tools

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Core LLMOps Concepts

The vocabulary every LLMOps practitioner reaches for daily — from prompt and context engineering to RAG, guardrails, and gateways. Get these mental models straight first and the rest of the cheat sheet clicks into place, because nearly every later table is just a deeper dive into one of these terms.

Concept	Example	Description
LLMOps	Deploying ChatGPT-like apps in production	Practices for developing, deploying, and managing LLMs throughout their lifecycle
Prompt Engineering	Crafting instructions for specific output format	• Designing input prompts to guide LLM behavior • prompts directly impact quality and cost
Context Engineering	Dynamic retrieval + compression + memory	Systematically controlling what information the LLM sees at inference time to maximize output quality
Fine-Tuning	Adapting GPT for legal document analysis	Training a pre-trained LLM on domain-specific data to improve performance on specialized tasks
Inference Optimization	Using quantization + batching to reduce latency	Techniques to speed up LLM predictions and reduce computational cost during serving
RAG (Retrieval-Augmented Generation)	Answering questions using external docs	Combining retrieval from a knowledge base with generation to ground responses in facts
Observability	Tracing every LLM call with metadata	Monitoring LLM applications via logging, tracing, and metrics to detect issues and optimize performance
Guardrails	Blocking toxic or PII-containing responses	Runtime controls that validate inputs/outputs against security, safety, and compliance policies
LLM Gateway	Portkey, LiteLLM, Kong AI Gateway	Reverse proxy between app and LLM providers providing unified API, fallbacks, and cost controls
Model Versioning	Tracking model checkpoints with DVC	Managing different versions of models, prompts, and configurations to enable rollback and experimentation
Data Flywheel	User feedback improving model over time	Continuous feedback loop where production interactions generate data used to refine models iteratively
Context Window	GPT-4 supports up to 128k tokens	Maximum number of tokens (input + output) an LLM can process in a single request
Model Context Protocol (MCP)	Standardized tool integration for LLM apps	Open protocol that lets LLMs discover and invoke external tools through a unified interface

Table 2: Prompt Management & Engineering

Prompts are the primary interface to a model, and in production they need the same discipline you'd give to code — templating, versioning, and a central registry. These techniques span both the craft of shaping a single prompt (chain-of-thought, few-shot, role prompting) and the operational side of testing and rolling out prompt changes without breaking what already works.

Technique	Example	Description
Prompt Templates	`Answer in JSON: {user_query}`	Reusable prompt structures with placeholders for dynamic content

The rest of this section is part of the full cheat sheet

Table 3: Model Fine-Tuning & Adaptation

When prompting alone can't get a model to behave the way you need, you adapt its weights. The methods here trade off cost against control — parameter-efficient approaches like LoRA and QLoRA touch a tiny slice of the model on a single GPU, while full fine-tuning and alignment techniques such as RLHF, DPO, and GRPO reshape behavior more deeply. Knowing which lever to pull saves enormous compute.

Method	Example	Description
LoRA (Low-Rank Adaptation)	`lora_alpha=16, rank=8`	• Parameter-efficient method that trains small adapter matrices instead of full model • significantly reduces memory

The rest of this section is part of the full cheat sheet

Table 4: Deployment & Serving

Getting a model to answer fast and cheap under real traffic is its own engineering problem. These cover the serving engines — vLLM, SGLang, TensorRT-LLM — and the throughput tricks they rely on like continuous batching, KV cache reuse, and speculative decoding, alongside the rollout strategies (blue-green, canary, routing) that let you ship a new version without a 3 a.m. incident.

Strategy	Example	Description
vLLM	PagedAttention for KV cache management	High-throughput serving engine with advanced memory optimization and continuous batching

The rest of this section is part of the full cheat sheet

Table 1: Core LLMOps Concepts

Concept	Example	Description
LLMOps	Deploying ChatGPT-like apps in production	Practices for developing, deploying, and managing LLMs throughout their lifecycle
Prompt Engineering	Crafting instructions for specific output format	• Designing input prompts to guide LLM behavior • prompts directly impact quality and cost
Context Engineering	Dynamic retrieval + compression + memory	Systematically controlling what information the LLM sees at inference time to maximize output quality
Fine-Tuning	Adapting GPT for legal document analysis	Training a pre-trained LLM on domain-specific data to improve performance on specialized tasks
Inference Optimization	Using quantization + batching to reduce latency	Techniques to speed up LLM predictions and reduce computational cost during serving
RAG (Retrieval-Augmented Generation)	Answering questions using external docs	Combining retrieval from a knowledge base with generation to ground responses in facts
Observability	Tracing every LLM call with metadata	Monitoring LLM applications via logging, tracing, and metrics to detect issues and optimize performance
Guardrails	Blocking toxic or PII-containing responses	Runtime controls that validate inputs/outputs against security, safety, and compliance policies
LLM Gateway	Portkey, LiteLLM, Kong AI Gateway	Reverse proxy between app and LLM providers providing unified API, fallbacks, and cost controls
Model Versioning	Tracking model checkpoints with DVC	Managing different versions of models, prompts, and configurations to enable rollback and experimentation
Data Flywheel	User feedback improving model over time	Continuous feedback loop where production interactions generate data used to refine models iteratively
Context Window	GPT-4 supports up to 128k tokens	Maximum number of tokens (input + output) an LLM can process in a single request
Model Context Protocol (MCP)	Standardized tool integration for LLM apps	Open protocol that lets LLMs discover and invoke external tools through a unified interface

Table 2: Prompt Management & Engineering

Technique	Example	Description
Prompt Templates	`Answer in JSON: {user_query}`	Reusable prompt structures with placeholders for dynamic content

Table 3: Model Fine-Tuning & Adaptation

Method	Example	Description
LoRA (Low-Rank Adaptation)	`lora_alpha=16, rank=8`	• Parameter-efficient method that trains small adapter matrices instead of full model • significantly reduces memory

Table 4: Deployment & Serving

Strategy	Example	Description
vLLM	PagedAttention for KV cache management	High-throughput serving engine with advanced memory optimization and continuous batching