Support Vector Machine Regression
SVR is the class that’s most commonly used for Support Vector Regression in Scikit-learn.
SVC is a general SVM classifier while LinearSVC can only handle linear data and make linear predictions. Additionally, while SVC can be limited in terms of performance especially when data is large (think ~50K row as potential upper limits), LinearSVC will be much more performant and be able to handle much larger datasets without any problem.
How to Construct?
SVR
LinearSVR is explained very well in Scikit-learn official documentation. It uses linear kernel as expected. But it uses “liblinear” estimator instead of “libsvm” which makes it more optimized and performant especially when working with large datasets. Liblinear is also more ideal for linear data in terms of loss function and penalties.
from sklearn import svm
regressor = svm.SVR(kernel="linear", C=1)
NuSVR
LinearSVR is explained very well in Scikit-learn official documentation. It uses linear kernel as expected. But it uses “liblinear” estimator instead of “libsvm” which makes it more optimized and performant especially when working with large datasets. Liblinear is also more ideal for linear data in terms of loss function and penalties.
from sklearn import svm
regressor = svm.NuSVR(nu=0.5)
LinearSVR
LinearSVR is explained very well in Scikit-learn official documentation. It uses linear kernel as expected. But it uses “liblinear” estimator instead of “libsvm” which makes it more optimized and performant especially when working with large datasets. Liblinear is also more ideal for linear data in terms of loss function and penalties.
from sklearn import svm
regressor = svm.LinearSVR()
Difference Between "LinearSVC" and "SVC with linear kernel
LinearSVR vs SVR(kernel="linear")
So, if SVC has an option to choose linear kernel why the need for a separate class as LinearSVC?
Depending on the size of the data and different parameters LinearSVC can easily be 500X faster than SVC class even with a linear kernel. So, obviously somethings are different.
The answer mostly boils down to the LinearSVC using parameters that are much more optimized for linear predictions by default.
SVC
- Estimator is libsvm by default
- Uses One vs One multiclass reduction
LinearSVC
- Estimator is liblinear by default
- Uses One vs All multiclass reduction
Original research paper publication of Liblinear solver, one of the most popular solver algorithms in Machine Learning.
