CatBoost Cheat Sheet

Updated 2026-05-21

🧠Study flashcards on this topic97 cards · spaced repetition→

CatBoost is a gradient boosting library developed by Yandex that handles categorical features natively without manual encoding, making it a strong default choice for tabular data with mixed feature types. Its core innovations — ordered boosting and symmetric oblivious trees — address target leakage and deliver fast, regularized training that often requires minimal hyperparameter tuning. Unlike XGBoost and LightGBM, CatBoost computes target statistics on previous-row permutations to prevent prediction shift, and its symmetric tree structure enables efficient vectorized evaluation on both CPU and GPU. The key mental model: CatBoost trades some flexibility (symmetric splits only by default) for strong out-of-the-box generalization and native handling of high-cardinality categoricals.

What This Cheat Sheet Covers

This topic spans 20 focused tables and 123 indexed concepts, 97 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core Classes and InstallationTable 2: Pool — the Core Data StructureTable 3: Categorical Feature HandlingTable 4: Ordered Boosting and Prediction Shift PreventionTable 5: Symmetric Oblivious TreesTable 6: Key Hyperparameters for TuningTable 7: Overfitting Detection and Early StoppingTable 8: GPU TrainingTable 9: Loss Functions and ObjectivesTable 10: Ranking ObjectivesTable 11: Text and Embedding FeaturesTable 12: Feature Importance and InterpretabilityTable 13: Cross-Validation and Hyperparameter SearchTable 14: Model Saving, Loading, and ExportTable 15: Missing Value HandlingTable 16: Regularization TechniquesTable 17: Uncertainty Estimation with Virtual EnsemblesTable 18: CLI (Command-Line) TrainingTable 19: Custom Loss Functions and MetricsTable 20: Advanced Features — select_features, Snapshots, and Shrink

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Core Classes and Installation

CatBoost exposes four main Python classes — one per task type — each with a scikit-learn-compatible API. Knowing which class to use and how to install the library correctly is the starting point for any CatBoost workflow.

Class / Command	Example	Description
CatBoostClassifier	`from catboost import CatBoostClassifier` `model = CatBoostClassifier(iterations=500, depth=6)`	• Estimator for binary and multi-class classification • default loss is `Logloss` (binary) or `MultiClass` (>2 classes).
CatBoostRegressor	`from catboost import CatBoostRegressor` `model = CatBoostRegressor(loss_function='RMSE')`	• Estimator for regression tasks • default loss is `RMSE`.
CatBoostRanker	`from catboost import CatBoostRanker` `model = CatBoostRanker(loss_function='YetiRank')`	• Estimator for learning-to-rank tasks • default loss is `YetiRank`.
CatBoost	`from catboost import CatBoost` `model = CatBoost({'loss_function': 'CrossEntropy'})`	• Universal estimator • `loss_function` must be set explicitly to determine task type
install CatBoost	`pip install catboost`	• Installs the CatBoost Python package • GPU support is included in the wheel

Table 2: Pool — the Core Data Structure

Pool is CatBoost's native dataset container that attaches feature metadata (categorical indices, weights, group IDs) directly to the data so the model receives a fully self-described input at every call.

Method	Example	Description
Pool (basic)	`from catboost import Pool` `pool = Pool(X, y, cat_features=[0, 2])`	• Creates a dataset object from array-like data • `cat_features` attaches column indices of categorical features once and reuses across calls

The rest of this section is part of the full cheat sheet

Table 3: Categorical Feature Handling

CatBoost's native categorical handling is its most distinctive feature. Understanding when it uses one-hot encoding versus ordered target statistics determines both model quality and training speed.

Concept	Example	Description
cat_features parameter	`model = CatBoostClassifier(` `cat_features=[1, 3, 'city'])`	• Specifies columns that are categorical by index or name • CatBoost encodes them internally — do not manually encode before passing to CatBoost.

The rest of this section is part of the full cheat sheet

Table 4: Ordered Boosting and Prediction Shift Prevention

Ordered boosting is CatBoost's answer to prediction shift — a subtle form of leakage in standard gradient boosting where the residuals used for training depend on the same targets used to produce the model's predictions.

Concept	Example	Description
boosting_type='Ordered'	`model = CatBoostClassifier(` `boosting_type='Ordered')`	Uses ordered boosting: builds a separate model for each sample using only data that appeared earlier in the permutation for gradient computation, preventing prediction shift. Default on small datasets.

The rest of this section is part of the full cheat sheet

Table 1: Core Classes and Installation

Class / Command	Example	Description
CatBoostClassifier	`from catboost import CatBoostClassifier` `model = CatBoostClassifier(iterations=500, depth=6)`	• Estimator for binary and multi-class classification • default loss is `Logloss` (binary) or `MultiClass` (>2 classes).
CatBoostRegressor	`from catboost import CatBoostRegressor` `model = CatBoostRegressor(loss_function='RMSE')`	• Estimator for regression tasks • default loss is `RMSE`.
CatBoostRanker	`from catboost import CatBoostRanker` `model = CatBoostRanker(loss_function='YetiRank')`	• Estimator for learning-to-rank tasks • default loss is `YetiRank`.
CatBoost	`from catboost import CatBoost` `model = CatBoost({'loss_function': 'CrossEntropy'})`	• Universal estimator • `loss_function` must be set explicitly to determine task type
install CatBoost	`pip install catboost`	• Installs the CatBoost Python package • GPU support is included in the wheel

Table 2: Pool — the Core Data Structure

Method	Example	Description
Pool (basic)	`from catboost import Pool` `pool = Pool(X, y, cat_features=[0, 2])`	• Creates a dataset object from array-like data • `cat_features` attaches column indices of categorical features once and reuses across calls

Table 3: Categorical Feature Handling

CatBoost's native categorical handling is its most distinctive feature. Understanding when it uses one-hot encoding versus ordered target statistics determines both model quality and training speed.

Concept	Example	Description
cat_features parameter	`model = CatBoostClassifier(` `cat_features=[1, 3, 'city'])`	• Specifies columns that are categorical by index or name • CatBoost encodes them internally — do not manually encode before passing to CatBoost.

Table 4: Ordered Boosting and Prediction Shift Prevention

Concept	Example	Description
boosting_type='Ordered'	`model = CatBoostClassifier(` `boosting_type='Ordered')`	Uses ordered boosting: builds a separate model for each sample using only data that appeared earlier in the permutation for gradient computation, preventing prediction shift. Default on small datasets.