Create mini-batches for deep learning
Use a minibatchqueue
object to create, preprocess, and manage
mini-batches of data for training using custom training loops.
A minibatchqueue
object iterates over a datastore to provide data in a
suitable format for training using custom training loops. The object prepares a queue of
mini-batches that are preprocessed on demand. Use a minibatchqueue
object to
automatically convert your data to dlarray
or gpuArray
,
convert data to a different precision, or apply a custom function to preprocess your data. You
can prepare your data in parallel in the background.
During training, you can manage your data using the minibatchqueue
object. You
can shuffle the data at the start of each training epoch using the shuffle
function
and collect data from the queue for each training iteration using the next
function. You
can check if any data is left in the queue using the hasdata
function,
and reset
the queue
when it is empty.
creates a mbq
= minibatchqueue(ds
)minibatchqueue
object from the input datastore
ds
. The mini-batches in mbq
have the same number
of variables as the results of read
on the input datastore.
creates a mbq
= minibatchqueue(ds
,numOutputs
)minibatchqueue
object from the input datastore
ds
and sets the number of variables in each mini-batch. Use this
syntax when you use MiniBatchFcn
to specify a mini-batch
preprocessing function that has a different number of outputs than the number of variables
of the input datastore ds
.
sets one or more properties using name-value options. For example,
mbq
= minibatchqueue(___,Name,Value)minibatchqueue(ds, "MiniBatchSize",64,"PartialMiniBatches","discard")
sets the size of the returned mini-batches to 64 and discards any mini-batches with fewer
than 64 observations.