Skip to content

Decision Tree Regression

DecisionTreeRegressor is the built-in model alternative in Scikit-learn that’s  created for Decision Tree Regression. In this tutorial we will cover the basics of implementing DecisionTreeRegressor.

DecisionTreeClassifier is capable of high performance training and it will handle up to million rows and 100 features in a few minutes. Furthermore, DecisionTreeClassifier can make very fast predictions thanks to its Linear Time Complexity.

In this tutorial we will demonstrate basics of a DecisionTreeClassifier implementation with Scikit-learn in Python.

How to Construct?


You can import DecisionTreeClassifier from sklearn.tree module as below and use it to create a Decision Tree model object.

Creating DecisionTreeRegressor Model:

from sklearn.tree import DecisionTreeRegressor
DT = DecisionTreeRegressor()

Training DecisionTreeRegressor Model:

Once the model is created next steps will be to fit the model and it will be ready for prediction., y_train)

Predicting with DecisionTreeRegressor Model:

After training model will be ready for inference. You can call .predict() method with suitable data to start making predictions. 

yhat = DT.predict(X_test)

DecisionTreeClassifier has plenty of hyperparameters that can be tuned. Tuning the model can be used to:

  • increase accuracy
  • increase performance
  • decrease overfitting
  • avoid bias

What's the difference between DecisionTreeRegressor and DecisionTreeClassifier?

Splitting Criteria

DecisionTreeRegressor uses different splitting criteria than DecisionTreeClassifier. This is because DecisionTreeRegressor attempts to predict continuous values and quality of splitting criteria can’t be evaluated based on gini or entropy values.

DecisionTreeRegressor uses: mse, mae, poisson and friedman_mse for its splitting criteria.

  • mse: mean squared error calculation
  • mae: mean absolute error calculation
  • poisson: reduction in poisson deviance
  • friedman_mse: mean squared error calculation with Friedman improvement score.

Scikit-learn uses mse by default for its built-in DecisionTreeRegressor model.

You can change the splitting criteria while constructing the Decision Tree Regressor model as below:

DT = DecisionTreeRegressor(criterion="poisson")
DecisionTreeRegressor Summary

In summary, we have seen the basics of DecisionTreeRegressor, a built-in Scikit-learn decision tree regression model. You can use it to implement decision tree machine learning algorithm for predicting continuous values.

If you’d like to learn more about decision trees you can visit our Decision Tree Tutorial page.