CatBoostRegressor Target Transform: Log, Scale, and Inverse

Quick answer: A target transform changes the regression target before CatBoostRegressor learns it, often to reduce skew or stabilize variance. Treat the transform as part of the modeling pipeline: fit learned transforms only on training data, apply the same transform to validation data, inverse-transform predictions before reporting original units, and choose a metric that matches the business question.

Python Pool infographic showing CatBoostRegressor target transformation, training split, inverse transform, and evaluation — Transform the training target inside the training workflow, fit only on training data when needed, and inverse-transform predictions before reporting business metrics.

CatBoostRegressor is CatBoost’s regression estimator for gradient boosting on decision trees. A target transform changes the value being predicted before training, then converts predictions back to the original scale. This is useful when the target is strongly skewed, has a long tail, or grows multiplicatively.

The most common example is a log transform for positive targets such as prices, counts, revenue, or durations. The model learns on log1p(y), then predictions are converted with expm1(). Evaluation should still happen on the original target scale so the metric reflects real-world error.

Target transformation is most helpful when the original target has a few very large values that dominate the loss. A log-style transform compresses those large values and can help the model learn relative differences. It is less useful when the target is already symmetric, includes negative values, or when errors on the original scale must be treated linearly.

Be careful with validation. The transform must be learned and applied inside the training pipeline, not fitted on information from the test set. That is why wrappers such as TransformedTargetRegressor are useful: they keep the transformation tied to the estimator workflow.

Primary references include the CatBoostRegressor documentation, CatBoost Python examples, CatBoost loss functions guide, TransformedTargetRegressor documentation, and mean squared error documentation.

Contents

Create A Regression Dataset

Start with a clear feature matrix and a positive target. The example below uses a small housing-style dataset so the transform is easy to inspect.

import pandas as pd

frame = pd.DataFrame({
    "rooms": [2, 3, 3, 4, 5],
    "age": [30, 18, 12, 8, 4],
    "city": ["A", "A", "B", "B", "C"],
    "price": [180000, 260000, 310000, 420000, 700000],
})

features = frame[["rooms", "age", "city"]]
target = frame["price"]

Keep categorical columns explicit. CatBoost can handle categorical features when you tell it which columns should be treated that way.

In a real dataset, split train and validation data before comparing models. The tiny example is only for showing the API shape.

Fit A Raw-Target Model

A baseline model on the original target is useful before trying a transform. It tells you whether the transform actually improves results.

from catboost import CatBoostRegressor

raw_model = CatBoostRegressor(
    iterations=50,
    learning_rate=0.1,
    depth=4,
    loss_function="RMSE",
    verbose=False,
)

raw_model.fit(features, target, cat_features=["city"])

Use this baseline with the same train and test split as the transformed model. Otherwise the comparison is not meaningful.

Apply A Log Target Transform

Use scikit-learn’s TransformedTargetRegressor to apply the transform during training and inverse transform during prediction.

import numpy as np
from sklearn.compose import TransformedTargetRegressor

model = CatBoostRegressor(iterations=50, learning_rate=0.1, depth=4, verbose=False)

wrapped_model = TransformedTargetRegressor(
    regressor=model,
    func=np.log1p,
    inverse_func=np.expm1,
)

wrapped_model.fit(features, target)

Use log1p instead of plain log when zero can appear. For negative targets, choose a different transform.

Predict On The Original Scale

The wrapped model returns predictions on the original scale because it applies the inverse function automatically.

predictions = wrapped_model.predict(features)

for actual, predicted in zip(target.tolist(), predictions.tolist()):
    print(round(actual), round(predicted))

This is the scale stakeholders understand. Do not report log-scale predictions as if they were original target values.

Evaluate With RMSE

Compare transformed and raw models using the same metric on the same scale. RMSE is common for regression, but choose the metric that matches the project.

from sklearn.metrics import mean_squared_error

raw_predictions = raw_model.predict(features)
log_predictions = wrapped_model.predict(features)

raw_rmse = mean_squared_error(target, raw_predictions) ** 0.5
log_rmse = mean_squared_error(target, log_predictions) ** 0.5

print(round(raw_rmse, 2), round(log_rmse, 2))

If the transformed model improves only on training data, check validation data before trusting the result.

Also inspect residuals. A transformed model may reduce large errors but increase small errors. Whether that tradeoff is acceptable depends on the application.

For reporting, show both the metric and a simple plot of actual values against predicted values. That makes scale problems easier to see than a single score.

Store Transform Metadata

Record the transform along with the model settings. Future maintainers need to know how predictions were produced.

model_notes = {
    "estimator": "CatBoostRegressor",
    "target_transform": "log1p",
    "inverse_transform": "expm1",
    "metric": "RMSE on original target scale",
    "categorical_features": ["city"],
}

print(model_notes)

Metadata prevents accidental double transforms and makes experiment results easier to compare.

Practical Checklist

Try target transformation when the target is positive, skewed, and difficult to model on its original scale. Keep a raw-target baseline so you can prove the transform helped.

Always evaluate predictions after applying the inverse transform. Use validation data, inspect residuals, and plot actual values against predicted values before choosing a model.

Do not transform blindly. If the target is already well behaved, a transform can add complexity without improving the model. The goal is better validated predictions, not a more complicated pipeline.

When a transform is used, document it next to the model artifact. Future prediction code must know whether the model already applies the inverse function or whether the caller must do it manually.

Choose A Transform From The Target

A log-like transform can help a positive, right-skewed target, while a scale transform changes magnitude without changing ordering. Inspect zeros, negative values, outliers, and the error distribution before choosing log1p, a power transform, or no transform.

import numpy as np

values = np.array([0.0, 2.0, 5.0, 20.0])
transformed = np.log1p(values)
restored = np.expm1(transformed)
print(transformed)
print(restored)

Split Before Fitting Learned Transforms

A transform with estimated parameters must not learn from validation or test targets. Split first, fit the transform on y_train, then use that fitted object for every later target conversion.

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

y = np.array([10.0, 12.0, 14.0, 40.0, 45.0, 50.0])
y_train, y_test = train_test_split(y, test_size=0.33, random_state=7)
scaler = StandardScaler().fit(y_train.reshape(-1, 1))
y_train_scaled = scaler.transform(y_train.reshape(-1, 1)).ravel()
y_test_scaled = scaler.transform(y_test.reshape(-1, 1)).ravel()
print(y_train_scaled, y_test_scaled)

Train CatBoost And Restore Units

CatBoostRegressor receives the transformed target, but the reported prediction should normally be in the original target units. Keep the inverse operation beside the prediction code so plots and metrics do not silently mix scales.

import numpy as np
from catboost import CatBoostRegressor

model = CatBoostRegressor(iterations=50, depth=4, verbose=False, random_seed=7)
X = np.array([[1], [2], [3], [4]])
y = np.array([2.0, 4.0, 8.0, 16.0])
model.fit(X, np.log1p(y))
prediction_original_scale = np.expm1(model.predict([[3.5]]))
print(prediction_original_scale)

Evaluate The Question You Actually Have

Compute metrics on the scale stakeholders interpret unless the transformed-scale metric is intentionally the objective. Log transforms can make large absolute errors look smaller, so compare residuals and inspect predictions across the target range.

import numpy as np
from sklearn.metrics import mean_absolute_error

y_true = np.array([10.0, 20.0, 100.0])
y_pred = np.array([12.0, 16.0, 80.0])
print(mean_absolute_error(y_true, y_pred))
print(np.abs(y_true - y_pred))

Use the current CatBoostRegressor reference for estimator parameters and the scikit-learn composition documentation for keeping transformations with a modeling workflow. Related references include regression metrics, modeling APIs, and algorithm selection.

For related model evaluation, compare regression metrics, modeling APIs, and algorithm selection before choosing a target transformation.