Convolutional Neural Networks(Part-1)

 

Convolutional Neural Networks

Convolutional Neural networks are designed to process data through multiple layers of arrays. This type of neural networks is used in applications like image recognition or face recognition. The primary difference between CNN and any other ordinary neural network is that CNN takes input as a two-dimensional array and operates directly on the images rather than focusing on feature extraction which other neural networks focus on.

The classic, and arguably most popular, use case of these networks is for image processing.
Image classification is the task of taking an input image and outputting a class (a cat, dog, etc) or a probability of classes that best describes the image.

When a computer sees an image (takes an image as input), it will see an array of pixel values. Depending on the resolution and size of the image, it will see a 32 x 32 x 3 array of numbers (The 3 refers to RGB values).

What we want the computer to do is to be able to differentiate between all the images it’s given and figure out the unique features that make a dog a dog or that make a cat a cat.

When we look at a picture of a dog, we can classify it as such if the picture has identifiable features such as paws or 4 legs. In a similar way, the computer is able perform image classification by looking for low level features such as edges and curves, and then building up to more abstract concepts through a series of convolutional layers. This is a general overview of what a CNN does.

History of Convolutional Neural Networks

The history of Convolutional Neural Networks began with a famous experiment "Receptive Fields of Single Neurons in the Cat's Striate Cortex" conducted by Hubel and Wiesel (1950).
Neuroscientists made the first real strides in understanding how vision works inside the brain by studying cats.

The outcomes of the experiment were as given below:-

1. Neurons in the visual cortex are orientation detectors.
2. Simple cells respond to flashing and moving edges.
3. Complex cells respond to moving edges and are often direction specific.
4. Hyper complex cells are selective for the length of the moving edge.

A convolutional neural network uses three basic ideas −

• Local respective fields
• Convolution
• Pooling

CNN utilizes spatial correlations that exist within the input data. Each concurrent layer of a neural network connects some input neurons. This specific region is called local receptive field. Local receptive field focusses on the hidden neurons. The hidden neurons process the input data inside the mentioned field not realizing the changes outside the specific boundary.

Here, individual neurons perform a shift from time to time. This process is called “convolution”.

The mapping of connections from the input layer to the hidden feature map is defined as “shared weights” and bias included is called “shared bias”.

CNN or convolutional neural networks use pooling layers, which are the layers, positioned immediately after CNN declaration. It takes the input from the user as a feature map that comes out of convolutional networks and prepares a condensed feature map. Pooling layers helps in creating layers with neurons of previous layers.

LeNet

It is a Convolutional Network developed by Yann LeCun in 1990's.

LeNet architecture was used to read zip codes, digits, etc.

Structure of LeNet

It consists of seven layers. In addition to input, every other layer can train parameters. In the figure, Cx represents convolution layer, Sx represents sub-sampling layer, Fx represents complete connection layer, and x represents layer index.

• Layer C1 is a convolution layer with six convolution kernels of 5x5 and the size of feature mapping is 28x28, which can prevent the information of the input image from falling out of the boundary of convolution kernel.

• Layer S2 is the subsampling/pooling layer that outputs 6 feature graphs of size 14x14. Each cell in each feature map is connected to 2x2 neighborhoods in the corresponding feature map in C1.

• Layer C3 is a convolution layer with 16 5-5 convolution kernels. The input of the first six C3 feature maps is each continuous subset of the three feature maps in S2, the input of the next six feature maps comes from the input of the four continuous subsets, and the input of the next three feature maps comes from the four discontinuous subsets. Finally, the input for the last feature graph comes from all feature graphs of S2.

• Layer S4 is similar to S2, with size of 2x2 and output of 16 5x5 feature graphs.

• Layer C5 is a convolution layer with 120 convolution kernels of size 5x5. Each cell is connected to the 5*5 neighborhood on all 16 feature graphs of S4. Here, since the feature graph size of S4 is also 5x5, the output size of C5 is 1*1. So S4 and C5 are completely connected. C5 is labeled as a convolutional layer instead of a fully connected layer, because if lenet-5 input becomes larger and its structure remains unchanged, its output size will be greater than 1x1, i.e. not a fully connected layer.

• F6 layer is fully connected to C5, and 84 feature graphs are output.

Features of LeNet

• Every convolutional layer includes three parts: convolution, pooling, and nonlinear activation functions.
  
• Using convolution to extract spatial features (Convolution was called receptive fields originally).
• Subsampling average pooling layer.
• tanh activation function.
  
• Using MLP as the last classifier.
• Sparse connection between layers to reduce the complexity of computational.