Numerical Methods for Computing Cheat Sheet

Updated 2026-05-19

Next Topic: Probability in Statistics Cheat Sheet

Numerical methods are algorithms for approximating solutions to mathematical problems — root finding, integration, linear systems, eigenvalues, and differential equations — that either have no closed-form answer or are impractical to solve analytically at scale. They sit at the intersection of mathematics, computer science, and engineering, powering everything from climate simulations to GPS positioning. The critical insight every practitioner needs: accuracy (closeness to the true answer) and stability (whether rounding errors grow or decay through the computation) are independent properties — an algorithm can be one without the other, and only the combination of a well-conditioned problem with a backward-stable algorithm guarantees trustworthy results.

What This Cheat Sheet Covers

This topic spans 14 focused tables and 110 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Error Analysis FundamentalsTable 2: IEEE 754 Floating-Point Representation and PitfallsTable 3: Root-Finding MethodsTable 4: Numerical DifferentiationTable 5: Numerical IntegrationTable 6: Polynomial InterpolationTable 7: Linear Systems — Direct MethodsTable 8: Linear Systems — Iterative MethodsTable 9: Matrix Analysis — Norms, Condition Number, and SVDTable 10: Eigenvalue MethodsTable 11: ODE Solvers — Explicit MethodsTable 12: ODE Solvers — Implicit and Stiff SystemsTable 13: FFT and Spectral MethodsTable 14: Convergence Rates and Algorithm Stability

Table 1: Error Analysis Fundamentals

Every numerical computation introduces error. Understanding the sources — rounding (finite precision) versus truncation (finite approximation) — and how to measure them is the foundation for assessing whether a result can be trusted.

Concept	Example	Description
Machine epsilon	$\varepsilon_{\text{mach}} = 2^{-52} \approx 2.22 \times 10^{-16}$ (double)	Smallest positive value such that $\text{fl}(1 + \varepsilon) > 1$ ; upper bound on relative rounding error for any single floating-point operation.
Absolute error	$\lvert \hat{x} - x \rvert = \lvert 3.14 - \pi \rvert \approx 0.00159$	Raw magnitude of the deviation between computed value $\hat{x}$ and true value $x$ ; misleading without context of the scale of $x$ .
Relative error	$\frac{\lvert \hat{x} - x \rvert}{\lvert x \rvert} \approx \frac{0.00159}{3.14159} \approx 5.07 \times 10^{-4}$	Absolute error scaled by $\lvert x \rvert$ ; directly reflects the number of correct significant digits ( $-\log_{10}(\text{rel. err.})$ digits).
Roundoff error	$\text{fl}(0.1) \neq 0.1$ exactly in binary	Error from representing a real number in finite-precision floating-point; each arithmetic op introduces at most $\varepsilon_{\text{mach}}$ of relative error.

Table 1: Error Analysis Fundamentals

Concept	Example	Description
Machine epsilon	$\varepsilon_{\text{mach}} = 2^{-52} \approx 2.22 \times 10^{-16}$ (double)	Smallest positive value such that $\text{fl}(1 + \varepsilon) > 1$ ; upper bound on relative rounding error for any single floating-point operation.
Absolute error	$\lvert \hat{x} - x \rvert = \lvert 3.14 - \pi \rvert \approx 0.00159$	Raw magnitude of the deviation between computed value $\hat{x}$ and true value $x$ ; misleading without context of the scale of $x$ .
Relative error	$\frac{\lvert \hat{x} - x \rvert}{\lvert x \rvert} \approx \frac{0.00159}{3.14159} \approx 5.07 \times 10^{-4}$	Absolute error scaled by $\lvert x \rvert$ ; directly reflects the number of correct significant digits ( $-\log_{10}(\text{rel. err.})$ digits).
Roundoff error	$\text{fl}(0.1) \neq 0.1$ exactly in binary	Error from representing a real number in finite-precision floating-point; each arithmetic op introduces at most $\varepsilon_{\text{mach}}$ of relative error.