Simple Understanding of INCEPTION_V3 and it's Architecture (part 3)

INCEPTION_V3

Prerequisites: 

VGGNet OR VGG16: VGG16 Architecture 

By looking it's name, everybody think's that it is a complicated story just like the movie INCEPTION. But trust me, I will prove you that it is wrong by explaining in the most detailed way.

Till now, If we take a layer in any neural network we only applied single operation like convolution or maxpooling and also with fixed kernel size for the whole layer. 

But Now, The idea is, why can't we use all the operations in a single layer at a time. There comes INCEPTION_V3.

Lets zoom a single layer in the inception_v3,

source: It's a screenshot from AndrewNg class

If you observe the above figure, convolution operation with kernel sizes 1x1,3x3,5x5 and the max-pool operation, all have applied at a time.

Here comes a problem, COMPUTATION. only from single layer we are getting billions of computations. For example,

lets do a simple mathematical calculation here,

Note: To understand this you need to know how convolution operation works.

From the above fig, our input size is 28x28x192, Now we are going to convolve it with 5x5 kernel and remember that the number of channels must be same with our input. so our kernel size is 5x5x192 and our output size is 28x28x32.

Inorder to get a single pixel value in our output matrix, we need to do 5x5x192 numerical operations.

                   single pixel -> 5x5x192

                  (28x28x32) pixels -> ?

So, To get whole output we need to do (5x5x192) x (28x28x32)  operations. [120 billion operations approx] . 

OH MY GOD, WHAT SHALL WE DO NOW?

Inorder to decrease the number of operations, Google has came out with a hack. lets look into it.

The hack is simple, Instead of convolving directly with 5x5 kernel, First do with 1x1 kernel followed by 5x5 kernel.

Now lets see number of mathematical operations required to get our output.

Number of operations with 1x1 kernel : (1x1x192) x (28x28x16)

Number of operations with 5x5 kernel : (5x5x16) x (28x28x32)

                                                   Total   :  12 billion approx

WOW GREAT, our operations has decreased from 120 billion to 12 billion operations.

FINALLY, let's put together what we learnt till now.

INCEPTION IDEA + OPTIMIZATION WITH 1X1 KERNEL

Atlast, we able to do both convolution and maxpooling operations at a time with optimal number of numerical operations by including 1x1 kernel.

As we will have many layers like this in the whole INCEPTION architecture, lets look into the final figure.

                                         source: https://cdn.analyticsvidhya.com/wp-content/uploads/2018/10/googlenet.png

In order to get the clear picture please go to the source link below the image.

Advantages:

1. We are not compromising on which kernel size should we use in a layer.

2. We also not compromising at convolution and maxpool operations.

3. 1x1 kernel is using for getting optimal results.

Disadvantages:

As of now, I didn't find any disadvantages. If you have please comment below. 

THANKYOU ALL.

Following Blogs might be useful to you.

ALEXNET:                ALEXNET ARCHITECTURE

RESNET:                   RESNET ARCHITECTURE

If you have any queries, please comment below...

For more content please follow the blogpost...