Ray for Distributed AI and ML Cheat Sheet

Updated 2026-05-21

Next Topic: Real-Time Machine Learning Pipelines Cheat Sheet

🧠Study flashcards on this topic109 cards · spaced repetition→

Ray is an open-source Python framework for distributed computing that provides a unified runtime for building and scaling AI and ML workloads across laptops, clusters, and cloud platforms. It solves the core pain point of distributed systems complexity by exposing three simple primitives — tasks, actors, and objects — that map naturally to any Python program. The key mental model is that Ray is not a framework you adopt wholesale but a layer you add incrementally: a single decorator turns a regular function into a distributed task, and the rest of your code stays unchanged.

What This Cheat Sheet Covers

This topic spans 13 focused tables and 128 indexed concepts, 109 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Ray Core — TasksTable 2: Ray Core — ActorsTable 3: Ray Core — Object Store and PatternsTable 4: Ray Core — Scheduling and Placement GroupsTable 5: Ray Cluster Setup and Runtime EnvironmentsTable 6: Ray Train — Distributed TrainingTable 7: Ray Tune — Hyperparameter SearchTable 8: Ray Serve — Model Deployment and CompositionTable 9: Ray Data — Distributed Data ProcessingTable 10: KubeRay — Ray on KubernetesTable 11: Ray Dashboard and ObservabilityTable 12: Fault ToleranceTable 13: Design Patterns and Anti-patterns

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Ray Core — Tasks

Tasks are the fundamental unit of stateless parallelism in Ray. Decorating a Python function with @ray.remote registers it as a remote function that executes asynchronously on any available worker in the cluster; results are retrieved lazily via ray.get(), enabling you to fire off thousands of tasks without blocking.

Technique	Example	Description
@ray.remote (task)	`@ray.remote` `def add(x, y):` `return x + y` `ref = add.remote(1, 2)`	• Decorates a Python function to run as a distributed remote task • `.remote()` returns an `ObjectRef` immediately without blocking
ray.get	`result = ray.get(ref)` `results = ray.get([r1, r2, r3])`	• Blocks the caller and retrieves the concrete value(s) from one or more `ObjectRef`s • accepts a single ref or a list
ray.wait	`ready, remaining = ray.wait(` `refs, num_returns=2, timeout=5.0)`	• Returns two lists — refs that are ready and those still pending — without blocking indefinitely • use to process results as they finish rather than waiting for all
ray.init	`ray.init()` `ray.init(address="ray://head:10001")` `ray.init(num_cpus=4)`	• Starts a local Ray runtime or connects to an existing cluster • call once at the start of your program
Resource specification (tasks)	`@ray.remote(num_cpus=2, num_gpus=1,` `memory=2_000_000_000)` `def train(): ...`	• Declares logical resources reserved for the task • Ray uses these for scheduling, not for enforcing physical limits
Custom resources	`@ray.remote(resources={"TPU": 1})` `def infer(): ...`	• Reserves user-defined resource labels set at cluster startup • useful for special hardware like TPUs or FPGAs
Fractional resources	`@ray.remote(num_gpus=0.5)` `def light_infer(): ...`	• Allows multiple tasks to share a single GPU • Ray supports fractions down to 0.0001 precision
Runtime options override	`add.options(num_cpus=4).remote(x, y)`	Overrides resource or metadata settings per invocation without redefining the function.
Nested tasks	`@ray.remote` `def outer():` `refs = [inner.remote(i) for i in range(4)]` `return ray.get(refs)`	Tasks can themselves spawn further remote tasks, enabling hierarchical parallelism where work fans out across the cluster dynamically.
Task generators (streaming)	`@ray.remote` `def gen():` `for i in range(5):` `yield i` `for ref in gen.remote(): ...`	A remote function can `yield` values instead of returning all at once, reducing peak heap memory for large outputs.

Table 2: Ray Core — Actors

Actors are stateful, long-lived workers. A class decorated with @ray.remote becomes an actor: each instance runs as a separate Ray process that executes method calls serially in submission order, making actors the right choice whenever you need shared mutable state across tasks.

Technique	Example	Description
@ray.remote (actor)	`@ray.remote` `class Counter:` `def __init__(self): self.n = 0` `def inc(self): self.n += 1` `def get(self): return self.n` `c = Counter.remote()`	• Turns a Python class into a distributed stateful actor • each `.remote()` instantiation creates an independent actor process

The rest of this section is part of the full cheat sheet

Table 3: Ray Core — Object Store and Patterns

The distributed object store (backed by Plasma / Apache Arrow shared memory) is the data plane of Ray. Numpy arrays stored in the object store are shared between workers on the same node via zero-copy deserialization, eliminating expensive data copies for large tensors and arrays.

Technique	Example	Description
ray.put	`ref = ray.put(large_array)` `results = [task.remote(ref) for _ in range(100)]`	• Explicitly places an object in the distributed store and returns a reference • pass the reference to many tasks to avoid re-serializing the same data repeatedly

The rest of this section is part of the full cheat sheet

Table 4: Ray Core — Scheduling and Placement Groups

Placement groups reserve resources across the cluster as an atomic unit, ensuring that co-located tasks and actors can fit together on the same node or spread across specific nodes. They are the primary tool for gang-scheduling multi-GPU training jobs and tensor-parallel inference.

Technique	Example	Description
Scheduling strategies	`task.options(scheduling_strategy="SPREAD").remote()` `task.options(scheduling_strategy="DEFAULT").remote()`	• `DEFAULT` packs by locality then load • `SPREAD` distributes evenly across nodes • set per task/actor via `.options()`.

The rest of this section is part of the full cheat sheet

Table 1: Ray Core — Tasks

Technique	Example	Description
@ray.remote (task)	`@ray.remote` `def add(x, y):` `return x + y` `ref = add.remote(1, 2)`	• Decorates a Python function to run as a distributed remote task • `.remote()` returns an `ObjectRef` immediately without blocking
ray.get	`result = ray.get(ref)` `results = ray.get([r1, r2, r3])`	• Blocks the caller and retrieves the concrete value(s) from one or more `ObjectRef`s • accepts a single ref or a list
ray.wait	`ready, remaining = ray.wait(` `refs, num_returns=2, timeout=5.0)`	• Returns two lists — refs that are ready and those still pending — without blocking indefinitely • use to process results as they finish rather than waiting for all
ray.init	`ray.init()` `ray.init(address="ray://head:10001")` `ray.init(num_cpus=4)`	• Starts a local Ray runtime or connects to an existing cluster • call once at the start of your program
Resource specification (tasks)	`@ray.remote(num_cpus=2, num_gpus=1,` `memory=2_000_000_000)` `def train(): ...`	• Declares logical resources reserved for the task • Ray uses these for scheduling, not for enforcing physical limits
Custom resources	`@ray.remote(resources={"TPU": 1})` `def infer(): ...`	• Reserves user-defined resource labels set at cluster startup • useful for special hardware like TPUs or FPGAs
Fractional resources	`@ray.remote(num_gpus=0.5)` `def light_infer(): ...`	• Allows multiple tasks to share a single GPU • Ray supports fractions down to 0.0001 precision
Runtime options override	`add.options(num_cpus=4).remote(x, y)`	Overrides resource or metadata settings per invocation without redefining the function.
Nested tasks	`@ray.remote` `def outer():` `refs = [inner.remote(i) for i in range(4)]` `return ray.get(refs)`	Tasks can themselves spawn further remote tasks, enabling hierarchical parallelism where work fans out across the cluster dynamically.
Task generators (streaming)	`@ray.remote` `def gen():` `for i in range(5):` `yield i` `for ref in gen.remote(): ...`	A remote function can `yield` values instead of returning all at once, reducing peak heap memory for large outputs.

Table 2: Ray Core — Actors

Technique	Example	Description
@ray.remote (actor)	`@ray.remote` `class Counter:` `def __init__(self): self.n = 0` `def inc(self): self.n += 1` `def get(self): return self.n` `c = Counter.remote()`	• Turns a Python class into a distributed stateful actor • each `.remote()` instantiation creates an independent actor process

Table 3: Ray Core — Object Store and Patterns

Technique	Example	Description
ray.put	`ref = ray.put(large_array)` `results = [task.remote(ref) for _ in range(100)]`	• Explicitly places an object in the distributed store and returns a reference • pass the reference to many tasks to avoid re-serializing the same data repeatedly

Table 4: Ray Core — Scheduling and Placement Groups

Technique	Example	Description
Scheduling strategies	`task.options(scheduling_strategy="SPREAD").remote()` `task.options(scheduling_strategy="DEFAULT").remote()`	• `DEFAULT` packs by locality then load • `SPREAD` distributes evenly across nodes • set per task/actor via `.options()`.