Convolutional Neural Network (CNN)

Definition

• Type of feed-forward neural network;

• Extract attributes by from grid-like data (images, 2D data, etc.) via filters (aka kernels).

Network Structure

Base

• Extracts features from the input.

• Bases are generally chosen from existing and knowns one, such as VGG16 or InceptionV1. Using pretrained models or parts of models is called "transfer learning". Their weights are generally frozen (the base is set to "not trainable") so the head weights don't alter the pretrained ones. If the bases weights trainable, and are planned to be improved, we call that "fine-tuning".

Steps:

1. Filter (using kernels);

2. Detect (using ReLU);

3. Condense (using pooling).

Head

Classifies the input using the extracted features.

Layers

Convolutional Layers

• Most of the computation is done here;

• Extracts attributes from the input data;

• The number of neurons is equal to the number of filters (kernels);

• Each neuron has a different kernel;

• Each neuron processes a part of the input data (receptive field) at the time. When it is done with a receptive field, it moves to the next one;

• After a neuron has processed the whole input data, it results in a feature map.

Convolution Operation

Pooling Layers

• Reduces the spatial dimension of the input data;

• Helps reduce the number of parameters and computational complexity in the network;

• Methods are average, global average and max pooling;

• Global average pooling can replace some or even all hidden dense layers and the flatten one in the head of the network;

• Creates invariance (recognize an object even after its appearance varies), which destroys small translations.

Example of Average and Max Pooling Methods

Fully Connected Layers

• Used to classify the extracted features;

• Each neuron is connected to all the neurons in the previous layer;

• Needs a flattened input (1D array).

Fully Connected Layer | Source: Diego Unzueta

References

• DataCamp—An Introduction to Convolutional Neural Networks (CNNs)

• DeepAI—Convolutional Neural Network

• FreeCodeCamp—What Is a Convolutional Neural Network? A Beginner's Tutorial for Machine Learning and Deep Learning

• MATLAB & Simulink—What Is a Convolutional Neural Network?

• Nvidia—Convolutional Neural Networks

• Shervine Amidi—Convolutional Neural Networks cheatsheet

• TechTarget—convolutional neural network (CNN)

• TowardsDataScience—A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

• TowardsDataScience—Convolutional Neural Networks, Explained

• Wikipedia—Convolutional neural network