AI Cloud Infrastructure and Neocloud Providers Cheat Sheet

Updated 2026-05-21

Next Topic: Amazon Web Services (AWS) - Core Cheat Sheet

🧠Study flashcards on this topic112 cards · spaced repetition→

AI cloud infrastructure has split into two distinct tiers: the traditional hyperscalers (AWS, Azure, GCP) offering broad general-purpose clouds, and a newer breed of neocloud providers (CoreWeave, Lambda Labs, Nebius, Crusoe, Nscale) purpose-built around GPU-first architecture for AI workloads. The defining difference is specialization — neoclouds strip away the overhead of legacy services to deliver 40–66% lower GPU costs and faster access to the latest accelerators. The single most important mental model is that AI infrastructure is no longer measured in FLOPS but in tokens per second per dollar, and every architectural choice — from interconnect topology to quantization format — flows from that optimization target.

What This Cheat Sheet Covers

This topic spans 16 focused tables and 107 indexed concepts, 112 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Neocloud Providers — Key Players and DifferentiatorsTable 2: Hyperscalers vs. Neoclouds — Strategic ComparisonTable 3: GPU Accelerator Families — Specifications and Cloud AvailabilityTable 4: GPU Cluster Interconnects and Networking FabricsTable 5: AI-Optimized Storage and Parallel File SystemsTable 6: Capacity Pricing Models — Reserved, On-Demand, and SpotTable 7: AI Workload Scheduling and Quota ManagementTable 8: Distributed Training Parallelism StrategiesTable 9: Fine-Tuning Infrastructure PatternsTable 10: Inference Optimization TechniquesTable 11: LLM Serving Engines — vLLM, TensorRT-LLM, SGLangTable 12: NVIDIA NIM and NeMo EcosystemTable 13: Inference-Optimized Accelerators Beyond GPUsTable 14: Sovereign AI Clouds and Data ResidencyTable 15: Total Cost of Ownership and Capacity PlanningTable 16: Vendor Lock-In and Portability Strategies

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Neocloud Providers — Key Players and Differentiators

The neocloud market is no longer a fringe alternative to hyperscalers — Forrester projected neocloud revenue hitting $20 billion in 2026, with Microsoft alone committing $60+ billion across multi-year neocloud contracts. Understanding each provider's focus, scale, and differentiation is the first step in selecting infrastructure for AI workloads.

Provider	Example	Description
CoreWeave	Kubernetes Namespace Capacity; 8× H100 nodes with InfiniBand; $21B Meta contract	• The largest neocloud • sells GPU capacity as Kubernetes namespace compute rather than traditional VMs • Q1 2026 revenue ~$1B • $99.4B backlog • NVIDIA-backed
Lambda Labs	1-click H100/B200 clusters with InfiniBand; bare metal instances; on-prem private clusters	• Positions as the "AI developer cloud" with one-click cluster provisioning • launch partner for NVIDIA Vera CPU and Quantum-X800 InfiniBand • offers bare metal and on-prem private clusters
Nebius	H100/H200 cloud; $3.50/hr H200; API + Terraform control; Finland data center	• Amsterdam-based full-stack AI cloud • projected $750M–$1B ARR • InfiniBand networking with hourly and reserved billing • building gigawatt-scale AI factories with NVIDIA
Crusoe	Energy-first H100/H200 clusters; Stargate OpenAI Abilene campus (200MW Phase 1)	• Energy-first neocloud leveraging stranded natural gas and renewables at 30–50% lower energy costs than hyperscalers • revenues growing from $276M (2024) toward $2B (2026 est.).
Nscale	Stargate Norway (100,000 GPUs target); $2B Series C; $23B Microsoft commitment	• NVIDIA-backed European sovereign AI cloud builder • valued at $14.6B in March 2026 • focused on large-scale GB300 cluster deployments for sovereign and enterprise clients
RunPod	18 GPU model families; on-demand, spot, and reserved billing; community cloud pods	• Marketplace-style GPU cloud exposing three billing tiers • broad GPU catalog including AMD MI300X • popular for cost-sensitive research and startup workloads
Vast.ai	Distributed marketplace; community GPU rental; spot pricing from ~$0.30/hr	• Peer-to-peer GPU marketplace aggregating consumer and data-center hardware • lowest entry price but variable reliability • suited for fault-tolerant batch workloads

Table 2: Hyperscalers vs. Neoclouds — Strategic Comparison

Hyperscalers and neoclouds are not interchangeable; they serve different organizational needs and risk profiles. The core trade-off is cost and GPU access speed (neocloud advantage) versus ecosystem depth, compliance, and enterprise trust (hyperscaler advantage).

Dimension	Example	Description
GPU cost advantage (neocloud)	8× H100 node: $34/hr neocloud vs. $98/hr hyperscaler on-demand (66% saving)	Neoclouds eliminate legacy infrastructure overhead and maintain narrower product scope, translating directly to lower per-GPU prices.

The rest of this section is part of the full cheat sheet

Table 3: GPU Accelerator Families — Specifications and Cloud Availability

Selecting the right GPU generation is the single largest variable in AI infrastructure cost and capability. The key insight is that memory capacity (not just FLOPS) often determines what models can run: MI300X's 192GB HBM3 and H200's 141GB HBM3e both serve memory-bound inference workloads that the H100's 80GB cannot serve without multi-GPU sharding.

Accelerator	Example	Description
NVIDIA H100 SXM5	80GB HBM3; 3.35 TB/s BW; NVLink 4.0 (900 GB/s); H100 on-demand from $2.49/hr (neocloud)	• Hopper-generation flagship • industry standard for training and inference in 2024–2026 • NVSwitch enables all-to-all 7.2 TB/s intra-node fabric in 8-GPU HGX configurations

The rest of this section is part of the full cheat sheet

Table 4: GPU Cluster Interconnects and Networking Fabrics

The interconnect fabric is what separates a collection of GPUs from a training cluster. Collective communication primitives (AllReduce, AllGather, ReduceScatter) saturate the network — choosing the wrong fabric makes the network, not compute, the bottleneck.

Technology	Example	Description
NVLink (intra-node)	NVLink 4.0: 900 GB/s per GPU (H100); NVLink 5.0: 1,800 GB/s (B200); NVLink 6.0: 3,600 GB/s (Rubin)	• NVIDIA's direct GPU-to-GPU interconnect within a server node • combined with NVSwitch provides non-blocking all-to-all fabric • 14× faster than PCIe Gen6.

The rest of this section is part of the full cheat sheet

Table 1: Neocloud Providers — Key Players and Differentiators

Provider	Example	Description
CoreWeave	Kubernetes Namespace Capacity; 8× H100 nodes with InfiniBand; $21B Meta contract	• The largest neocloud • sells GPU capacity as Kubernetes namespace compute rather than traditional VMs • Q1 2026 revenue ~$1B • $99.4B backlog • NVIDIA-backed
Lambda Labs	1-click H100/B200 clusters with InfiniBand; bare metal instances; on-prem private clusters	• Positions as the "AI developer cloud" with one-click cluster provisioning • launch partner for NVIDIA Vera CPU and Quantum-X800 InfiniBand • offers bare metal and on-prem private clusters
Nebius	H100/H200 cloud; $3.50/hr H200; API + Terraform control; Finland data center	• Amsterdam-based full-stack AI cloud • projected $750M–$1B ARR • InfiniBand networking with hourly and reserved billing • building gigawatt-scale AI factories with NVIDIA
Crusoe	Energy-first H100/H200 clusters; Stargate OpenAI Abilene campus (200MW Phase 1)	• Energy-first neocloud leveraging stranded natural gas and renewables at 30–50% lower energy costs than hyperscalers • revenues growing from $276M (2024) toward $2B (2026 est.).
Nscale	Stargate Norway (100,000 GPUs target); $2B Series C; $23B Microsoft commitment	• NVIDIA-backed European sovereign AI cloud builder • valued at $14.6B in March 2026 • focused on large-scale GB300 cluster deployments for sovereign and enterprise clients
RunPod	18 GPU model families; on-demand, spot, and reserved billing; community cloud pods	• Marketplace-style GPU cloud exposing three billing tiers • broad GPU catalog including AMD MI300X • popular for cost-sensitive research and startup workloads
Vast.ai	Distributed marketplace; community GPU rental; spot pricing from ~$0.30/hr	• Peer-to-peer GPU marketplace aggregating consumer and data-center hardware • lowest entry price but variable reliability • suited for fault-tolerant batch workloads

Table 2: Hyperscalers vs. Neoclouds — Strategic Comparison

Dimension	Example	Description
GPU cost advantage (neocloud)	8× H100 node: $34/hr neocloud vs. $98/hr hyperscaler on-demand (66% saving)	Neoclouds eliminate legacy infrastructure overhead and maintain narrower product scope, translating directly to lower per-GPU prices.

Table 3: GPU Accelerator Families — Specifications and Cloud Availability

Accelerator	Example	Description
NVIDIA H100 SXM5	80GB HBM3; 3.35 TB/s BW; NVLink 4.0 (900 GB/s); H100 on-demand from $2.49/hr (neocloud)	• Hopper-generation flagship • industry standard for training and inference in 2024–2026 • NVSwitch enables all-to-all 7.2 TB/s intra-node fabric in 8-GPU HGX configurations

Table 4: GPU Cluster Interconnects and Networking Fabrics

Technology	Example	Description
NVLink (intra-node)	NVLink 4.0: 900 GB/s per GPU (H100); NVLink 5.0: 1,800 GB/s (B200); NVLink 6.0: 3,600 GB/s (Rubin)	• NVIDIA's direct GPU-to-GPU interconnect within a server node • combined with NVSwitch provides non-blocking all-to-all fabric • 14× faster than PCIe Gen6.