PyTorch Cheat Sheet

Updated 2026-04-28

Next Topic: PyTorch Lightning Cheat Sheet

PyTorch is an open-source deep learning framework originally developed by Facebook AI Research and now stewarded by the PyTorch Foundation, powering research and production at Meta, Tesla, OpenAI, and beyond. Its dynamic computational graph (define-by-run) builds the graph on the fly during execution, making debugging and experimentation far more intuitive than static-graph frameworks. As of PyTorch 2.11 (March 2026), the framework spans a full stack from research to edge deployment: torch.compile brings compiler-level speedups without code changes, torch.export captures clean static graphs for production, and ExecuTorch deploys models to mobile and embedded devices. Understanding how tensors flow through models, how autograd tracks gradients, and how to structure training loops is fundamental to leveraging PyTorch effectively.

What This Cheat Sheet Covers

This topic spans 24 focused tables and 266 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Tensor CreationTable 2: Tensor Properties & Device ManagementTable 3: Tensor Reshaping & RearrangingTable 4: Tensor Joining & SplittingTable 5: Tensor Indexing & SlicingTable 6: Tensor Mathematical OperationsTable 7: Automatic Differentiation (Autograd)Table 8: Neural Network Layers (nn.Module)Table 9: Activation FunctionsTable 10: Loss FunctionsTable 11: nn.functional OperationsTable 12: OptimizersTable 13: Learning Rate SchedulersTable 14: Data LoadingTable 15: Model Building (nn.Module)Table 16: Training Loop ComponentsTable 17: Model Saving & LoadingTable 18: GPU & Device ManagementTable 19: Regularization TechniquesTable 20: Mixed Precision Training (AMP)Table 21: Transfer Learning & Fine-TuningTable 22: Advanced Training TechniquesTable 23: Model Export & DeploymentTable 24: Common Patterns & Gotchas

Table 1: Tensor Creation

Method	Example	Description
torch.tensor()	`x = torch.tensor([[1, 2], [3, 4]])`	• Creates tensor from existing data (list, ndarray) • always copies data and infers dtype automatically.
torch.zeros()	`z = torch.zeros(3, 4)`	• Creates tensor filled with zeros • shape specified as individual args or tuple.
torch.ones()	`o = torch.ones(2, 3, dtype=torch.float32)`	• Creates tensor filled with ones • useful for initializations and masks.
torch.rand()	`r = torch.rand(3, 3)`	• Creates tensor with uniform distribution [0, 1) • common for random initialization.
torch.randn()	`n = torch.randn(2, 5)`	• Creates tensor with standard normal distribution (mean=0, std=1) • preferred for weight initialization.
torch.randint()	`x = torch.randint(0, 10, (3, 4))`	Creates integer tensor with values drawn uniformly from [low, high).
torch.zeros_like() / ones_like()	`z = torch.zeros_like(x)` `o = torch.ones_like(x)`	• Creates tensor of same shape and dtype as input, filled with zeros or ones • device-aware.
torch.randn_like()	`n = torch.randn_like(x)`	Creates normal-distributed tensor matching shape, dtype, and device of input.
torch.arange()	`a = torch.arange(0, 10, 2)`	Creates 1D tensor with evenly spaced values from start to end (exclusive) with step.

Table 1: Tensor Creation

Method	Example	Description
torch.tensor()	`x = torch.tensor([[1, 2], [3, 4]])`	• Creates tensor from existing data (list, ndarray) • always copies data and infers dtype automatically.
torch.zeros()	`z = torch.zeros(3, 4)`	• Creates tensor filled with zeros • shape specified as individual args or tuple.
torch.ones()	`o = torch.ones(2, 3, dtype=torch.float32)`	• Creates tensor filled with ones • useful for initializations and masks.
torch.rand()	`r = torch.rand(3, 3)`	• Creates tensor with uniform distribution [0, 1) • common for random initialization.
torch.randn()	`n = torch.randn(2, 5)`	• Creates tensor with standard normal distribution (mean=0, std=1) • preferred for weight initialization.
torch.randint()	`x = torch.randint(0, 10, (3, 4))`	Creates integer tensor with values drawn uniformly from [low, high).
torch.zeros_like() / ones_like()	`z = torch.zeros_like(x)` `o = torch.ones_like(x)`	• Creates tensor of same shape and dtype as input, filled with zeros or ones • device-aware.
torch.randn_like()	`n = torch.randn_like(x)`	Creates normal-distributed tensor matching shape, dtype, and device of input.
torch.arange()	`a = torch.arange(0, 10, 2)`	Creates 1D tensor with evenly spaced values from start to end (exclusive) with step.