RegressionSVM Class
Superclasses: CompactRegressionSVM
Support vector machine regression model
Description
RegressionSVM
is a support vector machine (SVM) regression model. Train a RegressionSVM
model using fitrsvm
and the sample data.
RegressionSVM
models store data, parameter values, support vectors, and algorithmic implementation information. You can use these models to:
Estimate resubstitution predictions. For details, see
resubPredict
.Predict values for new data. For details, see
predict
.Compute resubstitution loss. For details, see
resubLoss
.Compute the mean square error or epsilon-insensitive loss. For details, see
loss
.
Construction
Create a RegressionSVM
object by using fitrsvm
.
Properties
Alpha
— Dual problem coefficients
vector of numeric values
Dual problem coefficients, specified as a vector of numeric values. Alpha
contains m elements, where m is the number of
support vectors in the trained SVM regression model. The dual problem introduces two
Lagrange multipliers for each support vector. The values of Alpha
are
the differences between the two estimated Lagrange multipliers for the support vectors.
For more details, see Understanding Support Vector Machine Regression.
If you specified to remove duplicates using RemoveDuplicates
,
then, for a particular set of duplicate observations that are support vectors,
Alpha
contains one coefficient corresponding to the entire set.
That is, MATLAB® attributes a nonzero coefficient to one observation from the set of
duplicates and a coefficient of 0
to all other duplicate observations
in the set.
Data Types: single
| double
Beta
— Primal linear problem coefficients
vector of numeric values | '[]'
Primal linear problem coefficients, stored as a numeric vector of length p, where p is the number of predictors in the SVM regression model.
The values in Beta
are the linear coefficients for the primal optimization problem.
If the model is obtained using a kernel function other than 'linear'
, this property is empty ('[]'
).
The predict
method computes predicted response values for the model as YFIT = (X/S)×Beta + Bias
, where S
is the value of the kernel scale stored in the KernelParameters.Scale
property.
Data Types: single
| double
Bias
— Bias term
scalar value
Bias term in the SVM regression model, stored as a scalar value.
Data Types: single
| double
BoxConstraints
— Box constraints for dual problem coefficients
vector of numeric values
Box constraints for dual problem alpha coefficients, stored as a numeric vector containing n elements, where n is the number of observations in X
(Mdl.NumObservations
).
The absolute value of the dual coefficient Alpha
for observation i cannot exceed BoxConstraints(i)
.
If you specify removing duplicates using 'RemoveDuplicates'
, then for a given set of duplicate observations, MATLAB sums the box constraints, and then attributes the sum to one observation and box constraints of 0
to all other observations in the set.
Data Types: single
| double
CacheInfo
— Caching information
structure
Caching information, stored as a structure with the following fields.
Field | Description |
---|---|
'Size' | Positive scalar value indicating the cache size (in MB) that the software reserves to store entries of the Gram matrix. Set the cache size by using the 'CacheSize' name-value pair argument in fitrsvm . |
'Algorithm' | Character vector containing the name of the algorithm used to remove entries from the cache when its capacity is exceeded. Currently, the only available caching algorithm is 'Queue' . You cannot set the caching algorithm. |
Data Types: struct
CategoricalPredictors
— Indices of categorical predictors
vector of positive integers | []
This property is read-only.
Categorical predictor
indices, specified as a vector of positive integers. CategoricalPredictors
contains index values indicating that the corresponding predictors are categorical. The index
values are between 1 and p
, where p
is the number of
predictors used to train the model. If none of the predictors are categorical, then this
property is empty ([]
).
Data Types: single
| double
ConvergenceInfo
— Convergence information
structure
Convergence information, stored as a structure with the following fields.
Field | Description |
---|---|
Converged | Logical flag indicating whether the algorithm converged. A value of 1 indicates convergence. |
ReasonForConvergence | Character vector indicating the criterion the software used to detect convergence. |
Gap | Scalar feasibility gap between the dual and primal objective functions. |
GapTolerance | Scalar tolerance for the feasibility gap. You can set this tolerance using the 'GapTolerance' name-value pair argument in fitrsvm . |
DeltaGradient | Scalar gradient difference between upper and lower violators. |
DeltaGradientTolerance | Scalar tolerance for the gradient difference. You can set this tolerance using the DeltaGradientTolerance name-value pair argument in fitrsvm . |
LargestKKTViolation | Maximal scalar Karush-Kuhn-Tucker (KKT) violation value. |
KKTTolerance | Scalar tolerance for the largest KKT violation. You can set this tolerance using the 'KKTTolerance' name-value pair argument in fitrsvm . |
History | Structure containing convergence information recorded at periodic intervals during the model training process. This structure contains the following fields:
|
Objective | Numeric value of the dual objective. |
Data Types: struct
Epsilon
— Half the width of the epsilon-insensitive band
nonnegative scalar value
Half the width of the epsilon-insensitive band, stored as a nonnegative scalar value.
Data Types: single
| double
ExpandedPredictorNames
— Expanded predictor names
cell array of character vectors
Expanded predictor names, stored as a cell array of character vectors.
If the model uses encoding for categorical variables, then ExpandedPredictorNames
includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames
is the same as PredictorNames
.
Data Types: cell
Gradient
— Gradient values in training data
vector of numeric values
Gradient values in training data, stored as a numeric vector containing 2n elements, where n is the number of observations in the training data.
Element i of Gradient
contains the gradient value for the Alpha
coefficient that corresponds to the upper boundary of the epsilon-insensitive band at observation i at the end of the optimization.
Element i + NumObservations of Gradient
contains the gradient value for the Alpha
coefficient that corresponds to the lower boundary of the epsilon-insensitive band at observation i at the end of the optimization.
Data Types: single
| double
HyperparameterOptimizationResults
— Cross-validation optimization of hyperparameters
BayesianOptimization
object | table
This property is read-only.
Cross-validation optimization of hyperparameters, specified as a BayesianOptimization
object or a table of hyperparameters and associated
values. This property is nonempty if the 'OptimizeHyperparameters'
name-value pair argument is nonempty when you create the model. The value of
HyperparameterOptimizationResults
depends on the setting of the
Optimizer
field in the
HyperparameterOptimizationOptions
structure when you create the
model.
Value of Optimizer Option | Value of HyperparameterOptimizationResults |
---|---|
"bayesopt" (default) | Object of class BayesianOptimization |
"gridsearch" or "randomsearch" | Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst) |
IsSupportVector
— Flag indicating whether observation is support vector
logical vector
Flag indicating whether an observation is a support vector, stored as an n-by-1 logical vector. n is the number of observations in X
(see NumObservations
). A value of 1
indicates that the corresponding observation in the training data is a support vector.
If you specify removing duplicates using RemoveDuplicates
, then for a given set of duplicate observations that are support vectors, IsSupportVector
flags only one as a support vector.
Data Types: logical
KernelParameters
— Kernel function parameters
structure
Kernel function parameters, stored as a structure with the following fields.
Field | Description |
---|---|
Function
| Kernel function name (a character vector). |
Scale | Numeric scale factor used to divide predictor values. |
You can specify values for KernelParameters.Function
and KernelParameters.Scale
by using the KernelFunction
and KernelScale
name-value pair arguments in fitrsvm
, respectively.
Data Types: struct
ModelParameters
— Parameter values
model parameter object
Parameter values used to train the SVM regression model, stored as a model parameter object. Access the properties of ModelParameters
using dot notation. For example, access the value of Epsilon
used to train the model as Mdl.ModelParameters.Epsilon
.
Mu
— Predictor means
vector of numeric values | '[]'
Predictor means, stored as a vector of numeric values.
If the predictors are standardized, then Mu
is a numeric vector of length p, where p is the number of predictors used to train the model. In this case, the predict
method centers predictor matrix X
by subtracting the corresponding element of Mu
from each column.
If the predictors are not standardized, then Mu
is empty ('[]'
).
If the data contains categorical predictors, then Mu
includes elements for the dummy variables for those predictors. The corresponding entries in Mu
are 0 because dummy variables are not centered or scaled.
Data Types: single
| double
NumIterations
— Number of iterations required for convergence
positive integer value
Number of iterations required for the optimization routine to reach convergence, stored as a positive integer value.
To set a limit on the number of iterations, use the 'IterationLimit'
name-value pair argument of fitrsvm
.
Data Types: single
| double
NumObservations
— Number of observations
positive integer value
Number of observations in the training data, stored as a positive integer value.
Data Types: single
| double
PredictorNames
— Predictor names
cell array of character vectors
Predictor names, stored as a cell array of character vectors containing the name of each predictor in the order they appear in X
. PredictorNames
has a length equal to the number of columns in X
.
Data Types: cell
OutlierFraction
— Expected fraction of outliers
scalar value in the range [0,1]
Expected fraction of outliers in the training set, stored as a scalar value in the range [0,1]. You can specify the expected fraction of outliers using the 'OutlierFraction'
name-value pair argument in fitrsvm
.
Data Types: double
ResponseName
— Response variable name
character vector
Response variable name, stored as a character vector.
Data Types: char
ResponseTransform
— Response transformation function
'none'
| function handle
Response transformation function, specified as 'none'
or a function handle.
ResponseTransform
describes how the software transforms raw
response values.
For a MATLAB function or a function that you define, enter its function handle. For
example, you can enter Mdl.ResponseTransform =
@function
, where
function
accepts a numeric vector of the
original responses and returns a numeric vector of the same size containing the
transformed responses.
Data Types: char
| function_handle
RowsUsed
— Rows of original training data stored
logical vector | []
Rows of the original training data stored in the model, specified as a
logical vector. This property is empty if all rows are stored in
X
and Y
.
Data Types: logical
ShrinkagePeriod
— Number of iterations between reductions of active set
nonnegative integer value
Number of iterations between reductions of the active set during optimization, stored as a nonnegative integer value.
You can set the shrinkage period by using the 'ShrinkagePeriod'
name-value pair argument in fitrsvm
.
Data Types: single
| double
Sigma
— Predictor standard deviations
vector of numeric values | '[]'
Predictor standard deviations, stored as a vector of numeric values.
If the predictors are standardized, then Sigma
is a numeric vector of length p, where p is the number of predictors used to train the model. In this case, the predict
method scales the predictor matrix X
by dividing each column by the corresponding element of Sigma
, after centering each element using Mu
.
If the predictors are not standardized, then Sigma
is empty ('[]'
).
If the data contains categorical predictors, Sigma
includes elements for the dummy variables for those predictors. The corresponding entries in Sigma
are 1, because dummy variables are not centered or scaled.
Data Types: single
| double
Solver
— Name of solver algorithm
character vector
Name of the solver algorithm used to solve the optimization problem, stored as a value in this table.
Value | Description |
---|---|
'SMO' | Sequential Minimal Optimization |
'ISDA' | Iterative Single Data Algorithm |
'L1QP' | L1 soft-margin minimization by quadratic programming (requires an Optimization Toolbox™ license). |
SupportVectors
— Support vectors
matrix of numeric values
Support vectors, stored as an m-by-p matrix
of numeric values. m is the number of support vectors
(sum(Mdl.IsSupportVector)
), and p is
the number of predictors in X
.
If you specified to remove duplicates using RemoveDuplicates
,
then for a given set of duplicate observations that are support vectors,
SupportVectors
contains one unique support vector.
Data Types: single
| double
W
— Observation weights
vector of numeric values
Observation weights used to train the model, stored as a numeric vector containing NumObservation
number of elements. fitrsvm
normalizes the weights used for training so that they sum to 1.
Data Types: single
| double
X
— Predictor values
matrix of numeric values | table of numeric values
Predictor values used to train the model, stored as a matrix of numeric values if the model is trained on a matrix, or a table if the model is trained on a table. X
has size n-by-p, where n is the number of rows and p is the number of predictor variables or columns in the training data.
Data Types: single
| double
| table
Y
— Observed response values
vector of numeric values
Observed response values, stored as a numeric vector containing NumObservations
number of elements.
Data Types: single
| double
Object Functions
compact | Compact support vector machine regression model |
crossval | Cross-validate machine learning model |
discardSupportVectors | Discard support vectors for linear support vector machine (SVM) regression model |
gather | Gather properties of Statistics and Machine Learning Toolbox object from GPU |
incrementalLearner | Convert support vector machine (SVM) regression model to incremental learner |
lime | Local interpretable model-agnostic explanations (LIME) |
loss | Regression error for support vector machine regression model |
partialDependence | Compute partial dependence |
plotPartialDependence | Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots |
predict | Predict responses using support vector machine regression model |
resubLoss | Resubstitution loss for support vector machine regression model |
resubPredict | Predict resubstitution response of support vector machine regression model |
resume | Resume training support vector machine regression model |
shapley | Shapley values |
Copy Semantics
Value. To learn how value classes affect copy operations, see Copying Objects.
Examples
Train Linear Support Vector Machine Regression Model
This example shows how to train a linear support vector machine (SVM) regression model using sample data stored in matrices.
Load the carsmall
data set.
load carsmall rng default % for reproducibility
Specify Horsepower
and Weight
as the predictor variables (X
) and MPG
as the response variable (Y
).
X = [Horsepower,Weight]; Y = MPG;
Train a linear SVM regression model.
Mdl = fitrsvm(X,Y)
Mdl = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [75x1 double] Bias: 57.3800 KernelParameters: [1x1 struct] NumObservations: 94 BoxConstraints: [94x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [94x1 logical] Solver: 'SMO'
The Command Window shows that Mdl
is a trained RegressionSVM
model and a list of its properties.
Check the model for convergence.
Mdl.ConvergenceInfo.Converged
ans = logical
0
0
indicates that the model did not converge.
MdlStd = fitrsvm(X,Y,'Standardize',true)
MdlStd = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [77x1 double] Bias: 22.9131 KernelParameters: [1x1 struct] Mu: [109.3441 2.9625e+03] Sigma: [45.3545 805.9668] NumObservations: 94 BoxConstraints: [94x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [94x1 logical] Solver: 'SMO'
Check the model for convergence.
MdlStd.ConvergenceInfo.Converged
ans = logical
1
1
indicates that the model did converge.
Compute the resubstitution mean squared error for the new model.
lStd = resubLoss(MdlStd)
lStd = 16.8551
Train Support Vector Machine Regression Model
Train a support vector machine regression model using the abalone data from the UCI Machine Learning Repository.
Download the data and save it in your current folder with the name 'abalone.csv'
.
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data'; websave('abalone.csv',url);
Read the data into a table. Specify the variable names.
varnames = {'Sex'; 'Length'; 'Diameter'; 'Height'; 'Whole_weight';... 'Shucked_weight'; 'Viscera_weight'; 'Shell_weight'; 'Rings'}; Tbl = readtable('abalone.csv','Filetype','text','ReadVariableNames',false); Tbl.Properties.VariableNames = varnames;
The sample data contains 4177 observations. All the predictor variables are continuous except for Sex
, which is a categorical variable with possible values 'M'
(for males), 'F'
(for females), and 'I'
(for infants). The goal is to predict the number of rings (stored in Rings
) on the abalone and determine its age using physical measurements.
Train an SVM regression model, using a Gaussian kernel function with an automatic kernel scale. Standardize the data.
rng default % For reproducibility Mdl = fitrsvm(Tbl,'Rings','KernelFunction','gaussian','KernelScale','auto',... 'Standardize',true)
Mdl = RegressionSVM PredictorNames: {'Sex' 'Length' 'Diameter' 'Height' 'Whole_weight' 'Shucked_weight' 'Viscera_weight' 'Shell_weight'} ResponseName: 'Rings' CategoricalPredictors: 1 ResponseTransform: 'none' Alpha: [3635×1 double] Bias: 10.8144 KernelParameters: [1×1 struct] Mu: [0 0 0 0.5240 0.4079 0.1395 0.8287 0.3594 0.1806 0.2388] Sigma: [1 1 1 0.1201 0.0992 0.0418 0.4904 0.2220 0.1096 0.1392] NumObservations: 4177 BoxConstraints: [4177×1 double] ConvergenceInfo: [1×1 struct] IsSupportVector: [4177×1 logical] Solver: 'SMO' Properties, Methods
The Command Window shows that Mdl
is a trained RegressionSVM
model and displays a property list.
Display the properties of Mdl
using dot notation. For example, check to confirm whether the model converged and how many iterations it completed.
conv = Mdl.ConvergenceInfo.Converged
conv = logical
1
iter = Mdl.NumIterations
iter = 2759
The returned results indicate that the model converged after 2759 iterations.
References
[1] Nash, W.J., T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford. "The Population Biology of Abalone (Haliotis species) in Tasmania. I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait." Sea Fisheries Division, Technical Report No. 48, 1994.
[2] Waugh, S. "Extending and Benchmarking Cascade-Correlation: Extensions to the Cascade-Correlation Architecture and Benchmarking of Feed-forward Supervised Artificial Neural Networks." University of Tasmania Department of Computer Science thesis, 1995.
[3] Clark, D., Z. Schreter, A. Adams. "A Quantitative Comparison of Dystal and Backpropagation." submitted to the Australian Conference on Neural Networks, 1996.
[4] Lichman, M. UCI Machine Learning Repository, [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
Extended Capabilities
C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.
Usage notes and limitations:
To integrate the prediction of an SVM regression model into Simulink®, you can use the RegressionSVM Predict block in the Statistics and Machine Learning Toolbox™ library or a MATLAB Function block with the
predict
function.When you train an SVM regression model by using
fitrsvm
, the following restrictions apply.The value of the
ResponseTransform
name-value argument cannot be an anonymous function. For fixed-point code generation, the value must be'none'
(default).For fixed-point code generation, the value of the
KernelFunction
name-value argument must be'gaussian'
,'linear'
, or'polynomial'
.Fixed-point code generation and code generation with a coder configurer do not support categorical predictors (
logical
,categorical
,char
,string
, orcell
). You cannot use theCategoricalPredictors
name-value argument. To include categorical predictors in a model, preprocess them by usingdummyvar
before fitting the model.
For more information, see Introduction to Code Generation.
GPU Arrays
Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™. (since R2023a)
Usage notes and limitations:
The following object functions fully support GPU arrays:
The object functions execute on a GPU if at least one of the following applies:
The model was fitted with GPU arrays.
The predictor data that you pass to the object function is a GPU array.
The response data that you pass to the object function is a GPU array.
For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
Version History
Introduced in R2015bR2023b: Model stores observations with missing predictor values
Starting in R2023b, training observations with missing predictor values are included in the X
, Y
, and W
data properties. The RowsUsed
property indicates the training observations stored in the model, rather than those used for training. Observations with missing predictor values continue to be omitted from the model training process.
In previous releases, the software omitted training observations that contained missing predictor values from the data properties of the model.
R2023a: GPU array support for object functions (requires Parallel Computing Toolbox)
Starting in R2023a, you can fit a RegressionSVM
object on a GPU by
using fitrsvm
. Most
RegressionSVM
object functions now support GPU array input
arguments so that they can execute on a GPU. The object functions that do not
support GPU array inputs are incrementalLearner
, lime
, and
shapley
.
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)