Further Understanding of Convolutional Neural Networks

Recently, I learned some convolution neural networks again and got some deep understanding.

1. Tool Installation

Installing tensorflow and keras took a bit more effort and found a good installation blog with the following links:
Installation tutorial for installing Tensorflow and Keras under Anaconda
Use more now
It is worth noting that
We used anaconda for the installation, because Keas also needs support for these libraries before installing the blog installation linked above.
pip install numpy
pip install matplotlib
pip install scipy
pip install tensorflow
pip install keras

2. Understanding and insights

(1) Optimizer

The optimizer has these
Deep Learning - Optimizer algorithm Optimizer detail (BGD, SGD, MBGD, Momentum, NAG, Adagrad, Adadelta, RMSprop, Adam)
Of course, SGD, the random gradient descent algorithm, is commonly used.
Random Gradient Decrease b Station Tutorial

(2)Alex classical cnn network

AlexNet's entire network structure consists of five convolution layers and three fully connected layers, with a total depth of eight layers

# Guide Pack
import keras
from keras.models import Sequential
from keras.layers import Dense,Activation,Dropout,Flatten,Conv2D,MaxPool2D
from keras.layers.normalization import BatchNormalization
import numpy as np
np.random.seed(1000)
# get data
import tflearn.datasets.oxflower17 as oxflower17
x,y = oxflower17.load_data(one_hot=True)
# Define the AlexNet model
model = Sequential()
# 1block
model.add(Conv2D(filters = 97,
                 kernel_size = (11,11),
                 strides = (4,4),
                 padding = "valid",
                 input_shape = (224,224,3)))
model.add(Activation("relu"))
model.add(MaxPool2D(pool_size = (2,2),
                    strides = (2,2),
                    padding = "valid"))
model.add(BatchNormalization())
# 2block
model.add(Conv2D(filters = 256,
                 kernel_size = (11,11),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(MaxPool2D(pool_size = (2,2),
                    strides = (2,2),
                    padding = "valid"))
model.add(BatchNormalization())
# 3 block
model.add(Conv2D(filters = 384,
                 kernel_size = (3,3),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(BatchNormalization())
# 4 block
model.add(Conv2D(filters = 384,
                 kernel_size = (3,3),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(BatchNormalization())
# 5 block
model.add(Conv2D(filters = 256,
                 kernel_size = (3,3),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(MaxPool2D(pool_size = (2,2),
                    strides = (2,2),
                    padding = "valid"))
model.add(BatchNormalization())
# 6 dense
model.add(Flatten())
model.add(Dense(4096, input_shape=(224*224*3,)))
model.add(Activation("relu"))
model.add(Dropout(0.4))
model.add(BatchNormalization())
# 7 dense
model.add(Dense(4096))
model.add(Activation("relu"))
model.add(Dropout(0.4))
model.add(BatchNormalization())
# 8 dense
model.add(Dense(17))
model.add(Activation("softmax"))


model.summary()
# compile
model.compile(loss = "categorical_crossentropy",
              optimizer = "adam",
              metrics = ["accuracy"])
# train
model.fit(x,
          y,
          batch_size = 32,
          epochs = 8,
          verbose = 1,
          validation_split = 0.3,
          shuffle = True)

(3) Residual network

Residual network is the core structure to solve the problem of deep network gradient disappearance in ResNet

The residual network adds an identity mapping, which transfers the current output directly to the next layer of the network (all 1:1 transfers with no additional parameters), which is equivalent to taking a shortcut and skipping the layer's operation, a direct connection named skip connection.At the same time, in the process of reverse propagation, the gradient of the next network is directly transferred to the previous network, which solves the problem of the disappearance of the gradient of the deep network.

import keras
from keras.layers import Conv2D,Input

x = Input(shape=(224,224,3))
y = Conv2D(3,(3,3),padding="same")(x)

z = keras.layers.add([x,y])

from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np

model = ResNet50(weights="imagenet")

img_path = "elephant.jpg"
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

preds = model.predict(x)
# decode the results into a list of tuples (class, description, probability)
# (one such list for each sample in the batch)
print("Predicted:", decode_predictions(preds, top=3)[0])


b Station Learn Video Point Here

70 original articles published, 12 praised, 9653 visited
Private letter follow

Tags: network pip Anaconda

Posted on Wed, 15 Jan 2020 19:31:19 -0800 by markanite