Support vector machine template
returns
a support vector machine (SVM) learner template suitable for training
errorcorrecting output code (ECOC) multiclass models.t
= templateSVM()
If you specify a default template, then the software uses default values for all input arguments during training.
Specify t
as a binary learner, or one in a set of binary learners, in
fitcecoc
to train an ECOC multiclass
classifier.
returns
a template with additional options specified by one or more namevalue
pair arguments.t
= templateSVM(Name,Value
)
For example, you can specify the box constraint, the kernel function, or whether to standardize the predictors.
If you display t
in the Command Window, then
all options appear empty ([]
), except those that
you specify using namevalue pair arguments. During training, the
software uses default values for empty options.
Use templateSVM
to specify a default SVM template.
t = templateSVM()
t = Fit template for classification SVM. Alpha: [0x1 double] BoxConstraint: [] CacheSize: [] CachingMethod: '' ClipAlphas: [] DeltaGradientTolerance: [] Epsilon: [] GapTolerance: [] KKTTolerance: [] IterationLimit: [] KernelFunction: '' KernelScale: [] KernelOffset: [] KernelPolynomialOrder: [] NumPrint: [] Nu: [] OutlierFraction: [] RemoveDuplicates: [] ShrinkagePeriod: [] Solver: '' StandardizeData: [] SaveSupportVectors: [] VerbosityLevel: [] Version: 2 Method: 'SVM' Type: 'classification'
All properties of the template object are empty except for Method
and Type
. When you pass t
to the training function, the software fills in the empty properties with their respective default values. For example, the software fills the KernelFunction
property with 'linear'
. For details on other default values, see fitcsvm
.
t
is a plan for an SVM learner, and no computation occurs when you specify it. You can pass t
to fitcecoc
to specify SVM binary learners for ECOC multiclass learning. However, by default, fitcecoc
uses default SVM binary learners.
Create a nondefault SVM template for use in fitcecoc
.
Load Fisher's iris data set.
load fisheriris
Create a template for SVM binary classifiers, and specify to use a Gaussian kernel function.
t = templateSVM('KernelFunction','gaussian')
t = Fit template for classification SVM. Alpha: [0x1 double] BoxConstraint: [] CacheSize: [] CachingMethod: '' ClipAlphas: [] DeltaGradientTolerance: [] Epsilon: [] GapTolerance: [] KKTTolerance: [] IterationLimit: [] KernelFunction: 'gaussian' KernelScale: [] KernelOffset: [] KernelPolynomialOrder: [] NumPrint: [] Nu: [] OutlierFraction: [] RemoveDuplicates: [] ShrinkagePeriod: [] Solver: '' StandardizeData: [] SaveSupportVectors: [] VerbosityLevel: [] Version: 2 Method: 'SVM' Type: 'classification'
All properties of the template object are empty except for DistributionNames
, Method
, and Type
. When trained on, the software fills in the empty properties with their respective default values.
Specify t
as a binary learner for an ECOC multiclass model.
Mdl = fitcecoc(meas,species,'Learners',t);
Mdl
is a ClassificationECOC
multiclass classifier. By default, the software trains Mdl
using the oneversusone coding design.
Display the insample (resubstitution) misclassification error.
L = resubLoss(Mdl,'LossFun','classiferror')
L = 0.0200
When you train an ECOC model with linear SVM binary learners, fitcecoc
empties the Alpha
, SupportVectorLabels
, and SupportVectors
properties of the binary learners by default. You can choose instead to retain the support vectors and related values, and then discard them from the model later.
Load Fisher's iris data set.
load fisheriris rng(1); % For reproducibility
Train an ECOC model using the entire data set. Specify retaining the support vectors by passing in the appropriate SVM template.
t = templateSVM('SaveSupportVectors',true); MdlSV = fitcecoc(meas,species,'Learners',t);
MdlSV
is a trained ClassificationECOC
model with linear SVM binary learners. By default, fitcecoc
implements a oneversusone coding design, which requires three binary learners for threeclass learning.
Access the estimated $$\alpha $$ (alpha) values using dot notation.
alpha = cell(3,1); alpha{1} = MdlSV.BinaryLearners{1}.Alpha; alpha{2} = MdlSV.BinaryLearners{2}.Alpha; alpha{3} = MdlSV.BinaryLearners{3}.Alpha; alpha
alpha=3×1 cell
{ 3x1 double}
{ 3x1 double}
{23x1 double}
alpha
is a 3by1 cell array that stores the estimated values of $$\alpha $$.
Discard the support vectors and related values from the ECOC model.
Mdl = discardSupportVectors(MdlSV);
Mdl
is similar to MdlSV
, except that the Alpha
, SupportVectorLabels
, and SupportVectors
properties of all the linear SVM binary learners are empty ([]
).
areAllEmpty = @(x)isempty([x.Alpha x.SupportVectors x.SupportVectorLabels]); cellfun(areAllEmpty,Mdl.BinaryLearners)
ans = 3x1 logical array
1
1
1
Compare the sizes of the two ECOC models.
vars = whos('Mdl','MdlSV'); 100*(1  vars(1).bytes/vars(2).bytes)
ans = 4.6743
Mdl
is about 5% smaller than MdlSV
.
Reduce your memory usage by compacting Mdl
and then clearing Mdl
and MdlSV
from the workspace.
CompactMdl = compact(Mdl); clear Mdl MdlSV;
Predict the label for a random row of the training data using the more efficient SVM model.
idx = randsample(size(meas,1),1)
idx = 63
predictedLabel = predict(CompactMdl,meas(idx,:))
predictedLabel = 1x1 cell array
{'versicolor'}
trueLabel = species(idx)
trueLabel = 1x1 cell array
{'versicolor'}
Specify optional
commaseparated pairs of Name,Value
arguments. Name
is
the argument name and Value
is the corresponding value.
Name
must appear inside quotes. You can specify several name and value
pair arguments in any order as
Name1,Value1,...,NameN,ValueN
.
'BoxConstraint',0.1,'KernelFunction','gaussian','Standardize',1
specifies
a box constraint of 0.1
, to use the Gaussian (RBF)
kernel, and to standardize the predictors.'BoxConstraint'
— Box constraintBox constraint,
specified as the commaseparated pair consisting of 'BoxConstraint'
and
a positive scalar.
For oneclass learning, the software always sets the box constraint
to 1
.
For more details on the relationships and algorithmic behavior
of BoxConstraint
, Cost
, Prior
, Standardize
,
and Weights
, see Algorithms.
Example: 'BoxConstraint',100
Data Types: double
 single
'CacheSize'
— Cache size1000
(default)  'maximal'
 positive scalarCache size, specified as the commaseparated pair consisting
of 'CacheSize'
and 'maximal'
or
a positive scalar.
If CacheSize
is 'maximal'
,
then the software reserves enough memory to hold the entire nbyn Gram matrix.
If CacheSize
is a positive scalar, then the
software reserves CacheSize
megabytes of memory
for training the model.
Example: 'CacheSize','maximal'
Data Types: double
 single
 char
 string
'ClipAlphas'
— Flag to clip alpha coefficientstrue
(default)  false
Flag to clip alpha coefficients, specified as the commaseparated
pair consisting of 'ClipAlphas'
and either true
or false
.
Suppose that the alpha coefficient for observation j is α_{j} and the box constraint of observation j is C_{j}, j = 1,...,n, where n is the training sample size.
Value  Description 

true  At each iteration, if α_{j} is near 0 or near C_{j}, then MATLAB^{®} sets α_{j} to 0 or to C_{j}, respectively. 
false  MATLAB does not change the alpha coefficients during optimization. 
MATLAB stores the final values of α in
the Alpha
property of the trained SVM model object.
ClipAlphas
can affect SMO and ISDA convergence.
Example: 'ClipAlphas',false
Data Types: logical
'DeltaGradientTolerance'
— Tolerance for gradient differenceTolerance for the gradient difference between upper and lower
violators obtained by Sequential Minimal Optimization (SMO) or Iterative
Single Data Algorithm (ISDA), specified as the commaseparated pair
consisting of 'DeltaGradientTolerance'
and a nonnegative
scalar.
If DeltaGradientTolerance
is 0
,
then the software does not use the tolerance for the gradient difference
to check for optimization convergence.
The default values are:
1e3
if the solver is SMO (for example, you set
'Solver','SMO'
)
0
if the solver is ISDA (for example, you set
'Solver','ISDA'
)
Example: 'DeltaGradientTolerance',1e2
Data Types: double
 single
'GapTolerance'
— Feasibility gap tolerance0
(default)  nonnegative scalarFeasibility gap tolerance obtained by SMO or ISDA, specified
as the commaseparated pair consisting of 'GapTolerance'
and
a nonnegative scalar.
If GapTolerance
is 0
,
then the software does not use the feasibility gap tolerance to check
for optimization convergence.
Example: 'GapTolerance',1e2
Data Types: double
 single
'IterationLimit'
— Maximal number of numerical optimization iterations1e6
(default)  positive integerMaximal number of numerical optimization iterations, specified
as the commaseparated pair consisting of 'IterationLimit'
and
a positive integer.
The software returns a trained model regardless of whether the
optimization routine successfully converges. Mdl.ConvergenceInfo
contains
convergence information.
Example: 'IterationLimit',1e8
Data Types: double
 single
'KernelFunction'
— Kernel function'linear'
 'gaussian'
 'rbf'
 'polynomial'
 function nameKernel function used to compute the elements of the Gram
matrix, specified as the commaseparated pair consisting of
'KernelFunction'
and a kernel function name. Suppose
G(x_{j},x_{k})
is element (j,k) of the Gram matrix, where
x_{j} and
x_{k} are
pdimensional vectors representing observations j
and k in X
. This table describes supported
kernel function names and their functional forms.
Kernel Function Name  Description  Formula 

'gaussian' or 'rbf'  Gaussian or Radial Basis Function (RBF) kernel, default for oneclass learning 
$$G\left({x}_{j},{x}_{k}\right)=\mathrm{exp}\left({\Vert {x}_{j}{x}_{k}\Vert}^{2}\right)$$

'linear'  Linear kernel, default for twoclass learning 
$$G({x}_{j},{x}_{k})={x}_{j}\prime {x}_{k}$$

'polynomial'  Polynomial kernel. Use
'PolynomialOrder',
to specify a polynomial kernel of order
q . 
$$G({x}_{j},{x}_{k})={(1+{x}_{j}\prime {x}_{k})}^{q}$$

You can set your own kernel function, for example,
kernel
, by setting 'KernelFunction','kernel'
.
The value kernel
must have this form.
function G = kernel(U,V)
U
is an mbyp
matrix. Columns correspond to predictor variables, and rows correspond to
observations.
V
is an nbyp
matrix. Columns correspond to predictor variables, and rows correspond to
observations.
G
is an mbyn
Gram matrix of
the rows of U
and V
.
kernel.m
must be on the MATLAB path.
It is a good practice to avoid using generic names for kernel functions. For example, call a
sigmoid kernel function 'mysigmoid'
rather than
'sigmoid'
.
Example: 'KernelFunction','gaussian'
Data Types: char
 string
'KernelOffset'
— Kernel offset parameterKernel offset parameter, specified as the commaseparated pair
consisting of 'KernelOffset'
and a nonnegative
scalar.
The software adds KernelOffset
to each element
of the Gram matrix.
The defaults are:
0
if the solver is SMO (that is,
you set 'Solver','SMO'
)
0.1
if the solver is ISDA (that
is, you set 'Solver','ISDA'
)
Example: 'KernelOffset',0
Data Types: double
 single
'KernelScale'
— Kernel scale parameter1
(default)  'auto'
 positive scalarKernel scale parameter, specified as the commaseparated pair
consisting of 'KernelScale'
and 'auto'
or
a positive scalar. The software divides all elements of the predictor
matrix X
by the value of KernelScale
.
Then, the software applies the appropriate kernel norm to compute
the Gram matrix.
If you specify 'auto'
, then the
software selects an appropriate scale factor using a heuristic procedure.
This heuristic procedure uses subsampling, so estimates can vary from
one call to another. Therefore, to reproduce results, set a random
number seed using rng
before
training.
If you specify KernelScale
and your own kernel
function, for example, 'KernelFunction','kernel'
, then
the software throws an error. You must apply scaling within
kernel
.
Example: 'KernelScale','auto'
Data Types: double
 single
 char
 string
'KKTTolerance'
— KarushKuhnTucker complementarity conditions violation toleranceKarushKuhnTucker
(KKT) complementarity conditions violation tolerance, specified
as the commaseparated pair consisting of 'KKTTolerance'
and
a nonnegative scalar.
If KKTTolerance
is 0
,
then the software does not use the KKT complementarity conditions
violation tolerance to check for optimization convergence.
The default values are:
0
if the solver is SMO (for example, you set
'Solver','SMO'
)
1e3
if the solver is ISDA (for example, you set
'Solver','ISDA'
)
Example: 'KKTTolerance',1e2
Data Types: double
 single
'NumPrint'
— Number of iterations between optimization diagnostic message output1000
(default)  nonnegative integerNumber of iterations between optimization diagnostic message
output, specified as the commaseparated pair consisting of 'NumPrint'
and
a nonnegative integer.
If you specify 'Verbose',1
and 'NumPrint',numprint
, then
the software displays all optimization diagnostic messages from SMO and ISDA every
numprint
iterations in the Command Window.
Example: 'NumPrint',500
Data Types: double
 single
'OutlierFraction'
— Expected proportion of outliers in training data0
(default)  numeric scalar in the interval [0,1)Expected proportion of outliers in the training data, specified
as the commaseparated pair consisting of 'OutlierFraction'
and
a numeric scalar in the interval [0,1).
Suppose that you set 'OutlierFraction',outlierfraction
, where
outlierfraction
is a value greater than 0.
For twoclass learning, the software implements robust
learning. In other words, the software attempts to remove
100*outlierfraction
% of the observations when the
optimization algorithm converges. The removed observations correspond to
gradients that are large in magnitude.
For oneclass learning, the software finds an appropriate bias term such
that outlierfraction
of the observations in the training
set have negative scores.
Example: 'OutlierFraction',0.01
Data Types: double
 single
'PolynomialOrder'
— Polynomial kernel function order3
(default)  positive integerPolynomial kernel function order, specified as the commaseparated
pair consisting of 'PolynomialOrder'
and a positive
integer.
If you set 'PolynomialOrder'
and KernelFunction
is
not 'polynomial'
, then the software throws an error.
Example: 'PolynomialOrder',2
Data Types: double
 single
'SaveSupportVectors'
— Store support vectors, their labels, and the estimated α coefficientstrue
 false
Store support vectors, their labels, and the estimated α coefficients
as properties of the resulting model, specified as the commaseparated
pair consisting of 'SaveSupportVectors'
and true
or false
.
If SaveSupportVectors
is true
, the resulting model
stores the support vectors in the SupportVectors
property, their labels in the SupportVectorLabels
property, and the estimated
α coefficients in the Alpha
property
of the compact, SVM learners.
If SaveSupportVectors
is false
and KernelFunction
is 'linear'
,
the resulting model does not store the support vectors and the related
estimates.
To reduce memory consumption by compact SVM models, specify SaveSupportVectors
.
For linear, SVM binary learners in an ECOC model, the default
value is false
. Otherwise, the default value is true
.
Example: 'SaveSupportVectors',true
Data Types: logical
'ShrinkagePeriod'
— Number of iterations between reductions of active set0
(default)  nonnegative integerNumber of iterations between reductions of the active set, specified as the
commaseparated pair consisting of 'ShrinkagePeriod'
and a
nonnegative integer.
If you set 'ShrinkagePeriod',0
, then the software does not shrink
the active set.
Example: 'ShrinkagePeriod',1000
Data Types: double
 single
'Solver'
— Optimization routine'ISDA'
 'L1QP'
 'SMO'
Optimization routine, specified as the commaseparated pair
consisting of 'Solver'
and a value in this table.
Value  Description 

'ISDA'  Iterative Single Data Algorithm (see [30]) 
'L1QP'  Uses quadprog to implement
L1 softmargin minimization by quadratic
programming. This option requires an Optimization
Toolbox™ license. For more details, see Quadratic Programming Definition (Optimization Toolbox). 
'SMO'  Sequential Minimal Optimization (see [17]) 
The default value is 'ISDA'
for twoclass learning or
if you set 'OutlierFraction'
to a positive value, and
'SMO'
otherwise.
Example: 'Solver','ISDA'
'Standardize'
— Flag to standardize predictor datafalse
(default)  true
Flag to standardize the predictor data, specified as the commaseparated
pair consisting of 'Standardize'
and true
(1
)
or false
(0)
.
If you set 'Standardize',true
:
The software centers and scales each column of the
predictor data (X
) by the weighted column mean
and standard deviation, respectively (for details on weighted standardizing,
see Algorithms). MATLAB does
not standardize the data contained in the dummy variable columns generated
for categorical predictors.
The software trains the classifier using the standardized
predictor matrix, but stores the unstandardized data in the classifier
property X
.
Example: 'Standardize',true
Data Types: logical
'Verbose'
— Verbosity level0
(default)  1
 2
Verbosity level, specified as the commaseparated pair consisting of
'Verbose'
and 0
, 1
, or
2
. The value of Verbose
controls the amount of
optimization information that the software displays in the Command Window and saves the
information as a structure to Mdl.ConvergenceInfo.History
.
This table summarizes the available verbosity level options.
Value  Description 

0  The software does not display or save convergence information. 
1  The software displays diagnostic messages and saves convergence
criteria every numprint iterations, where
numprint is the value of the namevalue pair
argument 'NumPrint' . 
2  The software displays diagnostic messages and saves convergence criteria at every iteration. 
Example: 'Verbose',1
Data Types: double
 single
t
— SVM classification templateSVM classification template suitable for training errorcorrecting
output code (ECOC) multiclass models, returned as a template object.
Pass t
to fitcecoc
to
specify how to create the SVM classifier for the ECOC model.
If you display t
to the Command Window, then
all, unspecified options appear empty ([]
). However,
the software replaces empty options with their corresponding default
values during training.
By default and for efficiency, fitcecoc
empties the Alpha
, SupportVectorLabels
,
and SupportVectors
properties
for all linear SVM binary learners. fitcecoc
lists Beta
, rather than
Alpha
, in the model display.
To store Alpha
, SupportVectorLabels
, and
SupportVectors
, pass a linear SVM template that specifies storing
support vectors to fitcecoc
. For example,
enter:
t = templateSVM('SaveSupportVectors',true) Mdl = fitcecoc(X,Y,'Learners',t);
You can remove the support vectors and related values by passing the resulting
ClassificationECOC
model to
discardSupportVectors
.
[1] Christianini, N., and J. C. ShaweTaylor. An Introduction to Support Vector Machines and Other KernelBased Learning Methods. Cambridge, UK: Cambridge University Press, 2000.
[2] Fan, R.E., P.H. Chen, and C.J. Lin. “Working set selection using second order information for training support vector machines.” Journal of Machine Learning Research, Vol 6, 2005, pp. 1889–1918.
[3] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, Second Edition. NY: Springer, 2008.
[4] Kecman V., T. M. Huang, and M. Vogt. “Iterative Single Data Algorithm for Training Kernel Machines from Huge Data Sets: Theory and Performance.” In Support Vector Machines: Theory and Applications. Edited by Lipo Wang, 255–274. Berlin: SpringerVerlag, 2005.
[5] Scholkopf, B., J. C. Platt, J. C. ShaweTaylor, A. J. Smola, and R. C. Williamson. “Estimating the Support of a HighDimensional Distribution.” Neural Comput., Vol. 13, Number 7, 2001, pp. 1443–1471.
[6] Scholkopf, B., and A. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond, Adaptive Computation and Machine Learning. Cambridge, MA: The MIT Press, 2002.
A modified version of this example exists on your system. Do you want to open this version instead?
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
Select web siteYou can also select a web site from the following list:
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.