discardSupportVectors

Discard support vectors

Syntax

mdlOut = discardSupportVectors(mdl)


Description

mdlOut = discardSupportVectors(mdl) returns the trained, linear support vector machine (SVM) regression model mdlOut, which is similar to the trained, linear SVM regression model mdl, except:

• The Alpha and SupportVectors properties are empty ([]).

• If you display mdlOut, the software lists the Beta property instead of the Alpha property.

Input Arguments

mdl — Trained, linear SVM regression model

RegressionSVM model | CompactRegressionSVM model

Trained, linear SVM regression model, specified as a RegressionSVM or CompactRegressionSVM model.

If you train the model using a kernel function that is not linear (i.e., if the field mdl.KernelFunction is something other than 'linear'), the software returns an error. You can only discard support vectors for linear models.

Output Arguments

mdlOut — Trained, linear SVM regression model

RegressionSVM model | CompactRegressionSVM model

Trained, linear SVM regression model, returned as a RegressionSVM or CompactRegressionSVM model. mdlOut is the same type as mdl.

After discarding the support vectors, the properties Alpha and SupportVectors are empty ([]). The software lists the property Beta in its display, and does not list the property Alpha. The predict and resubPredict methods compute predicted responses using the coefficients stored in the Beta property.

Examples

Discard Support Vectors for SVM Regression Model

This model shows how to reduce the disk space used by a trained, linear SVM regression model by discarding the support vectors and other related parameters.

Load the carsmall data set. Specify Horsepower and Weight as the predictor variables (X), and MPG as the response variable (Y).

load carsmall
X = [Horsepower,Weight];
Y = MPG;

Train a linear SVM regression model, standardizing the data. Display the number of support vectors.

mdl = fitrsvm(X,Y,'Standardize',true)
numSV = size(mdl.SupportVectors,1)

mdl = 

  RegressionSVM
           PredictorNames: {'x1'  'x2'}
             ResponseName: 'Y'
    CategoricalPredictors: []
        ResponseTransform: 'none'
                    Alpha: [77x1 double]
                     Bias: 22.9131
         KernelParameters: [1x1 struct]
                       Mu: [109.3441 2.9625e+03]
                    Sigma: [45.3545 805.9668]
          NumObservations: 93
           BoxConstraints: [93x1 double]
          ConvergenceInfo: [1x1 struct]
          IsSupportVector: [93x1 logical]
                   Solver: 'SMO'


  Properties, Methods


numSV =

    77

By default, fitrsvm trains a linear SVM regression model. The software lists Alpha in the display. The model has 77 support vectors.

Note that the predictor and response variables contain several NaN values. When training a model, fitrsvm will remove rows that contain NaN values from both the predictor and response data. As a result, the trained model uses only 93 of the 100 total observations contained in the sample data.

Discard the support vectors and other related parameters.

mdlOut = discardSupportVectors(mdl)
mdlOut.Alpha
mdlOut.SupportVectors

mdlOut = 

  RegressionSVM
           PredictorNames: {'x1'  'x2'}
             ResponseName: 'Y'
    CategoricalPredictors: []
        ResponseTransform: 'none'
                     Beta: [2x1 double]
                     Bias: 22.9131
         KernelParameters: [1x1 struct]
                       Mu: [109.3441 2.9625e+03]
                    Sigma: [45.3545 805.9668]
          NumObservations: 93
           BoxConstraints: [93x1 double]
          ConvergenceInfo: [1x1 struct]
          IsSupportVector: [93x1 logical]
                   Solver: 'SMO'


  Properties, Methods


ans =

     []


ans =

     []

The software lists Beta in the display instead of Alpha. The Alpha and SupportVectors properties are empty.

Compare the sizes of the models.

vars = whos('mdl','mdlOut');
[vars(1).bytes,vars(2).bytes]

ans =

       15004       13156

mdlOut consumes less memory than mdl because it does not store the support vectors.

Reduce Memory Consumption of SVM Regression Model

This example shows how to reduce the memory consumption of a full, trained SVM regression model by compacting the model and discarding the support vectors.

Load the carsmall sample data.

load carsmall
rng default  % for reproducibility

Train a linear SVM regression model using Weight as the predictor variable and MPG as the response variable. Standardize the data.

mdl = fitrsvm(Weight,MPG,'Standardize',true);

Note that MPG contains several NaN values. When training a model, fitrsvm will remove rows that contain NaN values from both the predictor and response data. As a result, the trained model uses only 94 of the 100 total observations contained in the sample data.

Compact the regression model to discard the training data and some information related to the training process.

compactMdl = compact(mdl);

compactMdl is a CompactRegressionSVM model that has the same parameters, support vectors, and related estimates as mdl, but no longer stores the training data.

Discard the support vectors and related estimates for the compacted model.

mdlOut = discardSupportVectors(compactMdl);

mdlOut is a CompactRegressionSVM model that has the same parameters as mdl and compactMdl, but no longer stores the support vectors and related estimates.

Compare the sizes of the three SVM regression models, compactMdl, mdl, and mdlOut.

vars = whos('compactMdl','mdl','mdlOut');
[vars(1).bytes,vars(2).bytes,vars(3).bytes]

ans =

        3601       13727        2305

The compacted model compactMdl consumes 3601 bytes of memory, while the full model mdl consumes 13727 bytes of memory. The model mdlOut, which also discards the support vectors, consumes 2305 bytes of memory.

Tips

For a trained, linear SVM regression model, the SupportVectors property is an n_sv-by-p matrix. n_sv is the number of support vectors (at most the training sample size) and p is the number of predictor variables. If any of the predictors are categorical, then p includes the number of dummy variables necessary to account for all of the categorical predictor levels. The Alpha property is a vector with n_sv elements.

The SupportVectors and Alpha properties can be large for complex data sets that contain many observations or examples. However, the Beta property is a vector with p elements, which may be considerably smaller. You can use a trained SVM regression model to predict response values even if you discard the support vectors because the predict and resubPredict methods use Beta to compute the predicted responses.

If the trained, linear SVM regression model has many support vectors, use discardSupportVectors to reduce the amount of disk space that the trained, linear SVM regression model consumes. You can display the size of the support vector matrix by entering size(mdlIn.SupportVectors).

Algorithms

The predict and resubPredict estimate response values using the formula

$$f\left(x\right)=\left(\frac{X}{S}\right)\beta +{\beta }_0,$$

where:

• β is the Beta value, stored as mdl.Beta.

• β₀ is the bias value, stored as mdl.Bias.

• X is the training data.

• S is the kernel scale value, stored as mdl.KernelParameters.Scale.

In this way, the software can use the value of mdl.Beta to make predictions even after discarding the support vectors.