This example shows how to generate images using deepDreamImage
with the pretrained convolutional neural network AlexNet.
Deep Dream is a feature visualization technique in deep learning that synthesizes images that strongly activate network layers. By visualizing these images, you can highlight the image features learned by a network. These images are useful for understanding and diagnosing network behavior.
You can generate interesting images by visualizing the features of the layers towards the end of the network.
The example uses Deep Learning Toolbox™ and Deep Learning Toolbox Model for AlexNet Network to generate the images.
Load a pretrained AlexNet Network. If the Deep Learning Toolbox Model for AlexNet Network support package is not installed, then the software provides a download link.
net = alexnet;
To produce images that resemble a given class the most closely, select the final fully connected layer. First, locate the layer index of this layer by viewing the network architecture in the Layers
property of the network net
.
net.Layers
ans = 25x1 Layer array with layers: 1 'data' Image Input 227x227x3 images with 'zerocenter' normalization 2 'conv1' Convolution 96 11x11x3 convolutions with stride [4 4] and padding [0 0 0 0] 3 'relu1' ReLU ReLU 4 'norm1' Cross Channel Normalization cross channel normalization with 5 channels per element 5 'pool1' Max Pooling 3x3 max pooling with stride [2 2] and padding [0 0 0 0] 6 'conv2' Grouped Convolution 2 groups of 128 5x5x48 convolutions with stride [1 1] and padding [2 2 2 2] 7 'relu2' ReLU ReLU 8 'norm2' Cross Channel Normalization cross channel normalization with 5 channels per element 9 'pool2' Max Pooling 3x3 max pooling with stride [2 2] and padding [0 0 0 0] 10 'conv3' Convolution 384 3x3x256 convolutions with stride [1 1] and padding [1 1 1 1] 11 'relu3' ReLU ReLU 12 'conv4' Grouped Convolution 2 groups of 192 3x3x192 convolutions with stride [1 1] and padding [1 1 1 1] 13 'relu4' ReLU ReLU 14 'conv5' Grouped Convolution 2 groups of 128 3x3x192 convolutions with stride [1 1] and padding [1 1 1 1] 15 'relu5' ReLU ReLU 16 'pool5' Max Pooling 3x3 max pooling with stride [2 2] and padding [0 0 0 0] 17 'fc6' Fully Connected 4096 fully connected layer 18 'relu6' ReLU ReLU 19 'drop6' Dropout 50% dropout 20 'fc7' Fully Connected 4096 fully connected layer 21 'relu7' ReLU ReLU 22 'drop7' Dropout 50% dropout 23 'fc8' Fully Connected 1000 fully connected layer 24 'prob' Softmax softmax 25 'output' Classification Output crossentropyex with 'tench' and 999 other classes
Then, select the final fully connected layer. The final fully connected layer is layer 23.
layer = 23;
You can generate multiple images at once by selecting multiple classes. Select the classes you want to visualize by setting channels
to be the indices of those class names.
channels = [9 188 231 563 855 975];
The classes are stored in the Classes
property of the output layer (the last layer). You can view the names of the selected classes by selecting the entries in channels
.
net.Layers(end).Classes(channels)
ans = 6×1 categorical array
hen
Yorkshire terrier
Shetland sheepdog
fountain
theater curtain
geyser
Generate the images using deepDreamImage
. This command uses a compatible GPU, if available. Otherwise it uses the CPU. A CUDA® enabled NVIDIA® GPU with compute capability 3.0 or higher is required for training on a GPU.
I = deepDreamImage(net,layer,channels);
|==============================================| | Iteration | Activation | Pyramid Level | | | Strength | | |==============================================| | 1 | 1.82 | 1 | | 2 | 4.62 | 1 | | 3 | 8.05 | 1 | | 4 | 12.29 | 1 | | 5 | 18.16 | 1 | | 6 | 21.09 | 1 | | 7 | 30.07 | 1 | | 8 | 34.69 | 1 | | 9 | 40.84 | 1 | | 10 | 47.12 | 1 | | 1 | 15.60 | 2 | | 2 | 14.05 | 2 | | 3 | 23.96 | 2 | | 4 | 21.54 | 2 | | 5 | 31.55 | 2 | | 6 | 25.97 | 2 | | 7 | 34.30 | 2 | | 8 | 33.81 | 2 | | 9 | 38.06 | 2 | | 10 | 33.47 | 2 | | 1 | 39.11 | 3 | | 2 | 47.44 | 3 | | 3 | 52.80 | 3 | | 4 | 58.48 | 3 | | 5 | 52.16 | 3 | | 6 | 63.92 | 3 | | 7 | 65.21 | 3 | | 8 | 67.41 | 3 | | 9 | 76.46 | 3 | | 10 | 71.61 | 3 | |==============================================|
Display all the images together using imtile
.
figure I = imtile(I); imshow(I)
Increasing the number of pyramid levels and iterations per pyramid level can produce more detailed images at the expense of additional computation.
You can increase the number of iterations using the 'NumIterations'
option. Set the number of iterations to 100.
iterations = 100;
Generate a detailed image that strongly activates the 'hen' class (channel 9). Set 'Verbose'
to false to suppress detailed information on the optimization process.
channels = 9; I = deepDreamImage(net,layer,channels, ... 'Verbose',false, ... 'NumIterations',iterations); figure imshow(I)
To produce larger and more detailed output images, you can increase both the number of pyramid levels and iterations per pyramid level.
Set the number of pyramid levels to 4.
levels = 4;
Generate a detailed image that strongly activates the 'pot' class (channel 739).
channels = 739; I = deepDreamImage(net,layer,channels, ... 'Verbose',false, ... 'NumIterations',iterations, ... 'PyramidLevels',levels); figure imshow(I)