Edge AI and TinyML Cheat Sheet

Updated 2026-05-02

Next Topic: Ensemble Methods Cheat Sheet

Edge AI and TinyML (Tiny Machine Learning) bring machine learning inference directly to resource-constrained devices like microcontrollers, embedded systems, and IoT endpoints. Edge AI runs on moderately powerful edge devices (~100mW to several watts), while TinyML pushes ML capabilities onto ultra-low-power microcontrollers operating at milliwatt-level consumption (often <1mW idle). The key innovation is deploying optimized neural networks directly on-device rather than relying on cloud servers, enabling real-time inference with enhanced privacy, reduced latency, and minimal connectivity dependence. Successful Edge AI deployment hinges on aggressive model optimization (quantization, pruning, knowledge distillation), understanding hardware accelerator capabilities (NPU, DSP, GPU delegation), and navigating the tradeoff triangle of accuracy, latency, and power consumption.

What This Cheat Sheet Covers

This topic spans 17 focused tables and 108 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core Concepts and DefinitionsTable 2: Model Quantization TechniquesTable 3: Model Compression and OptimizationTable 4: TensorFlow Lite and LiteRTTable 5: TensorFlow Lite Micro (TFLM)Table 6: ONNX Runtime for EdgeTable 7: Hardware AcceleratorsTable 8: Model Conversion WorkflowsTable 9: Deployment Platforms and ToolsTable 10: Power and Performance OptimizationTable 11: Benchmarking and EvaluationTable 12: Debugging and MonitoringTable 13: Real-World Application PatternsTable 14: Advanced Optimization TechniquesTable 15: Over-the-Air (OTA) UpdatesTable 16: Specialized Libraries and FrameworksTable 17: Mobile and Cross-Platform Deployment

Table 1: Core Concepts and Definitions

Before optimizing anything, it helps to fix the vocabulary that everyone in this field throws around — the difference between TinyML and the broader Edge AI umbrella, why latency and memory footprint dominate every decision, and where the line between running on-device and falling back to the cloud actually sits.

Concept	Example	Description
TinyML	ML inference on Arduino Nano 33 BLE Sense (256KB RAM)	• Machine learning running on microcontrollers with <1MB memory and milliwatt-scale power • focuses on ultra-constrained environments where every kilobyte matters
Edge AI	Object detection on NVIDIA Jetson Nano	• Broader category encompassing ML inference on edge devices from powerful SBCs to smartphones • typically 100mW-10W power range with megabytes to gigabytes of memory
On-device inference	Real-time face recognition on iPhone Neural Engine	• Executing trained ML models locally on end-user devices without cloud connectivity • model weights and computations stay on the device
Model deployment	Converting TensorFlow model to TFLite and flashing to ESP32	• Process of converting, optimizing, and embedding a trained model into firmware on target hardware • includes format conversion and integration with application code

Table 1: Core Concepts and Definitions

Concept	Example	Description
TinyML	ML inference on Arduino Nano 33 BLE Sense (256KB RAM)	• Machine learning running on microcontrollers with <1MB memory and milliwatt-scale power • focuses on ultra-constrained environments where every kilobyte matters
Edge AI	Object detection on NVIDIA Jetson Nano	• Broader category encompassing ML inference on edge devices from powerful SBCs to smartphones • typically 100mW-10W power range with megabytes to gigabytes of memory
On-device inference	Real-time face recognition on iPhone Neural Engine	• Executing trained ML models locally on end-user devices without cloud connectivity • model weights and computations stay on the device
Model deployment	Converting TensorFlow model to TFLite and flashing to ESP32	• Process of converting, optimizing, and embedding a trained model into firmware on target hardware • includes format conversion and integration with application code