Skip to content

Support Vector Machine Regression

SVR is the class that’s most commonly used for Support Vector Regression in Scikit-learn.

SVC is a general SVM classifier while LinearSVC can only handle linear data and make linear predictions. Additionally, while SVC can be limited in terms of performance especially when data is large (think ~50K row as potential upper limits), LinearSVC will be much more performant and be able to handle much larger datasets without any problem.

How to Construct?

SVR

LinearSVR is explained very well in Scikit-learn official documentation. It uses linear kernel as expected. But it uses “liblinear” estimator instead of “libsvm” which makes it more optimized and performant especially when working with large datasets. Liblinear is also more ideal for linear data in terms of loss function and penalties.

from sklearn import svm
regressor = svm.SVR(kernel="linear", C=1)

NuSVR

LinearSVR is explained very well in Scikit-learn official documentation. It uses linear kernel as expected. But it uses “liblinear” estimator instead of “libsvm” which makes it more optimized and performant especially when working with large datasets. Liblinear is also more ideal for linear data in terms of loss function and penalties.

from sklearn import svm
regressor = svm.NuSVR(nu=0.5)

LinearSVR

LinearSVR is explained very well in Scikit-learn official documentation. It uses linear kernel as expected. But it uses “liblinear” estimator instead of “libsvm” which makes it more optimized and performant especially when working with large datasets. Liblinear is also more ideal for linear data in terms of loss function and penalties.

from sklearn import svm
regressor = svm.LinearSVR()

Difference Between "LinearSVC" and "SVC with linear kernel

LinearSVR vs SVR(kernel="linear")

So, if SVC has an option to choose linear kernel why the need for a separate class as LinearSVC?

Depending on the size of the data and different parameters LinearSVC can easily be 500X faster than SVC class even with a linear kernel. So, obviously somethings are different.

The answer mostly boils down to the LinearSVC using parameters that are much more optimized for linear predictions by default.

SVC
  • Estimator is libsvm by default
  • Uses One vs One multiclass reduction


LinearSVC
  • Estimator is liblinear by default
  • Uses One vs All multiclass reduction

Original research paper publication of Liblinear solver, one of the most popular solver algorithms in Machine Learning.