Recursive Neural Network for Python Deep Learning Experiment

Recursive Neural Network

Experimental environment

keras 2.1.5
tensorflow 1.4.0

Experimental tools

Jupyter Notebook

Experiment 1: Use of word embedding

Word Embedding

Another popular and powerful way to associate vectors with words wraps more information in fewer dimensions.

Experimental purpose

Implement word embedding.

data set

IMDB Dataset

Implementation

1. Learn word embedding with the main tasks you care about.
2. Load into model text embedding.

Experimental process

1. Learn to embed words in the embedded layer
The easiest way to associate dense vectors with words is to randomly select vectors whose geometric relationships should reflect the semantic relationships between these words.
Embedding Layer:
A dictionary that maps integer indexes (representing specific words) to dense vectors.

from keras.layers import Embedding
#Two parameters: number of tags and embedded dimension.
embedding_layer = Embedding(1000, 64) 

Emotion prediction tasks for IMDB movie reviews:
The layer returns a floating-point tensor of the shape of 3D that can be processed by RNN or 1D convolution layers.
Embedding layer weights are initially random, and during training, these word vectors will be gradually adjusted by reverse propagation to construct space that can be used by downstream models.Once trained, your embedded space will show many structures - a structure that is specific to the problem you are trying to solve.

from keras.datasets import imdb
from keras import preprocessing

max_features = 10000
maxlen = 20
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
x_train = preprocessing.sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = preprocessing.sequence.pad_sequences(x_test, maxlen=maxlen)

from keras.models import Sequential
from keras.layers import Flatten, Dense

model = Sequential()
model.add(Embedding(10000, 8, input_length=maxlen))
model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])
model.summary()
history = model.fit(x_train, y_train,
                    epochs=10,
                    batch_size=32,
                    validation_split=0.2)
'''
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_5 (Embedding)      (None, 20, 8)             80000     
_________________________________________________________________
flatten_2 (Flatten)          (None, 160)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 161       
=================================================================
Total params: 80,161
Trainable params: 80,161
Non-trainable params: 0
_________________________________________________________________
Train on 20000 samples, validate on 5000 samples
Epoch 1/10
20000/20000 [==============================] - 2s 76us/step - loss: 0.6759 - acc: 0.6044 - val_loss: 0.6398 - val_acc: 0.6810
Epoch 2/10
20000/20000 [==============================] - 1s 47us/step - loss: 0.5657 - acc: 0.7428 - val_loss: 0.5467 - val_acc: 0.7206
Epoch 3/10
20000/20000 [==============================] - 1s 44us/step - loss: 0.4752 - acc: 0.7808 - val_loss: 0.5113 - val_acc: 0.7384
Epoch 4/10
20000/20000 [==============================] - 1s 52us/step - loss: 0.4263 - acc: 0.8079 - val_loss: 0.5008 - val_acc: 0.7454
Epoch 5/10
20000/20000 [==============================] - 1s 60us/step - loss: 0.3930 - acc: 0.8257 - val_loss: 0.4981 - val_acc: 0.7540
Epoch 6/10
20000/20000 [==============================] - 1s 63us/step - loss: 0.3668 - acc: 0.8394 - val_loss: 0.5013 - val_acc: 0.7532
Epoch 7/10
20000/20000 [==============================] - 1s 62us/step - loss: 0.3435 - acc: 0.8534 - val_loss: 0.5051 - val_acc: 0.7518
Epoch 8/10
20000/20000 [==============================] - 1s 49us/step - loss: 0.3223 - acc: 0.8658 - val_loss: 0.5132 - val_acc: 0.7486
Epoch 9/10
20000/20000 [==============================] - 1s 49us/step - loss: 0.3022 - acc: 0.8765 - val_loss: 0.5213 - val_acc: 0.7494
Epoch 10/10
20000/20000 [==============================] - 1s 50us/step - loss: 0.2839 - acc: 0.8860 - val_loss: 0.5302 - val_acc: 0.7466
'''
#Since you only look at the first 20 words of each comment, the accuracy is good.

2. Use pre-trained word embedding
Load the embedding vectors from a highly structured, pre-computed embedding space and remove the useful variables so that the natural parts of the language structure can be obtained.

To sum up: from original text to text embedding

Embedding sentences into vector sequences flattens them and trains a layer of full connection at the top.
1. Use IMDB data as raw text for download:

http://ai.stanford.edu/~amaas/data/sentiment/

2. Pretreatment:
Collect each training comment in a string list, one string for each comment, and these comment tags (positive/negative) in the labels list.
Note: Modify the path and character encoding.

import os
from keras.datasets import imdb
imdb_dir = 'C:/Users/Administrator/Desktop/deeplearning/03.Recursive Neural Network/aclImdb'
train_dir = os.path.join(imdb_dir, 'train')

labels = []
texts = []

for label_type in ['neg', 'pos']:
    dir_name = os.path.join(train_dir, label_type)
    for fname in os.listdir(dir_name):
        if fname[-4:] == '.txt':
            f = open(os.path.join(dir_name,fname),encoding='utf-8')
            texts.append(f.read())
            f.close()
            if label_type == 'neg':
                labels.append(0)
            else:
                labels.append(1)

3. Tag data:
The collected text is vectorized and divided into training and test samples.
Because pre-trained word embedding is particularly useful for problems where only a small amount of training data is available, add the following turning point: We limited the training data to the first 200 samples.

from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
import numpy as np

maxlen = 100  
training_samples = 200  
validation_samples = 10000  
max_words = 10000  

tokenizer = Tokenizer(num_words=max_words)
tokenizer.fit_on_texts(texts)
sequences = tokenizer.texts_to_sequences(texts)

word_index = tokenizer.word_index
print('Found %s unique tokens.' % len(word_index))

data = pad_sequences(sequences, maxlen=maxlen)

labels = np.asarray(labels)
print('Shape of data tensor:', data.shape)
print('Shape of label tensor:', labels.shape)

indices = np.arange(data.shape[0])
np.random.shuffle(indices)
data = data[indices]
labels = labels[indices]

x_train = data[:training_samples]
y_train = labels[:training_samples]
x_val = data[training_samples: training_samples + validation_samples]
y_val = labels[training_samples: training_samples + validation_samples]
'''
Found 88582 unique tokens.
Shape of data tensor: (25000, 100)
Shape of label tensor: (25000,)
'''

4. Download GloVe's word embedding:

https://nlp.stanford.edu/projects/glove/

5. Preprocessing Embedding:
We parse the decompressed files to construct a vector representation of index mapping terms to them.
Note: Path modification and character encoding.

glove_dir = 'C:/Users/Administrator/Desktop/deeplearning/03.Recursive Neural Network/glove.6B'

embeddings_index = {}
f = open(os.path.join(glove_dir, 'glove.6B.100d.txt'),encoding='utf-8')
for line in f:
    values = line.split()
    word = values[0]
    coefs = np.asarray(values[1:], dtype='float32')
    embeddings_index[word] = coefs
f.close()

print('Found %s word vectors.' % len(embeddings_index))
'''
Found 400000 word vectors.
'''

Build an embedded matrix:

embedding_dim = 100

embedding_matrix = np.zeros((max_words, embedding_dim))
for word, i in word_index.items():
    embedding_vector = embeddings_index.get(word)
    if i < max_words:
        if embedding_vector is not None:
            embedding_matrix[i] = embedding_vector

6. Define the model:

from keras.models import Sequential
from keras.layers import Embedding, Flatten, Dense

model = Sequential()
model.add(Embedding(max_words, embedding_dim, input_length=maxlen))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.summary()
'''
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_6 (Embedding)      (None, 100, 100)          1000000   
_________________________________________________________________
flatten_3 (Flatten)          (None, 10000)             0         
_________________________________________________________________
dense_3 (Dense)              (None, 32)                320032    
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 33        
=================================================================
Total params: 1,320,065
Trainable params: 1,320,065
Non-trainable params: 0
_________________________________________________________________
'''

7. Load GloVe embeddings into the model:
The embedded layer has a single weight matrix: a 2D floating-point matrix.
Load our prepared GloVe matrix into the embedded layer and freeze the embedded layer.

model.layers[0].set_weights([embedding_matrix])
model.layers[0].trainable = False

8. Training:

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['acc'])
history = model.fit(x_train, y_train,
                    epochs=10,
                    batch_size=32,
                    validation_data=(x_val, y_val))
model.save_weights('pre_trained_glove_model.h5')
'''
Train on 200 samples, validate on 10000 samples
Epoch 1/10
200/200 [==============================] - 1s 6ms/step - loss: 1.6337 - acc: 0.5250 - val_loss: 0.7130 - val_acc: 0.5100
Epoch 2/10
200/200 [==============================] - 1s 5ms/step - loss: 0.7565 - acc: 0.5800 - val_loss: 0.6910 - val_acc: 0.5418
Epoch 3/10
200/200 [==============================] - 1s 5ms/step - loss: 0.5956 - acc: 0.6950 - val_loss: 1.1205 - val_acc: 0.4936
Epoch 4/10
200/200 [==============================] - 1s 7ms/step - loss: 0.5335 - acc: 0.7350 - val_loss: 0.7134 - val_acc: 0.5362
Epoch 5/10
200/200 [==============================] - 1s 5ms/step - loss: 0.4713 - acc: 0.8100 - val_loss: 0.7177 - val_acc: 0.5589
Epoch 6/10
200/200 [==============================] - 1s 5ms/step - loss: 0.1448 - acc: 0.9800 - val_loss: 1.3373 - val_acc: 0.4952
Epoch 7/10
200/200 [==============================] - 1s 5ms/step - loss: 0.2545 - acc: 0.8800 - val_loss: 1.3110 - val_acc: 0.4960
Epoch 8/10
200/200 [==============================] - 1s 7ms/step - loss: 0.1102 - acc: 0.9800 - val_loss: 0.8168 - val_acc: 0.5558
Epoch 9/10
200/200 [==============================] - 1s 5ms/step - loss: 0.0760 - acc: 0.9800 - val_loss: 1.5204 - val_acc: 0.5115
Epoch 10/10
200/200 [==============================] - 1s 5ms/step - loss: 0.0680 - acc: 0.9850 - val_loss: 0.7458 - val_acc: 0.5759
'''

Mapping:

import matplotlib.pyplot as plt

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(acc) + 1)

plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()



9. Adjustment:
The model has been fitted and the accuracy of the verification set has a high variance due to the small number of training samples.
Training the same model without loading the pre-trained word embedding and without freezing the embedding layer.

from keras.models import Sequential
from keras.layers import Embedding, Flatten, Dense

model = Sequential()
model.add(Embedding(max_words, embedding_dim, input_length=maxlen))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.summary()

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['acc'])
history = model.fit(x_train, y_train,
                    epochs=10,
                    batch_size=32,
                    validation_data=(x_val, y_val))
'''
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_7 (Embedding)      (None, 100, 100)          1000000   
_________________________________________________________________
flatten_4 (Flatten)          (None, 10000)             0         
_________________________________________________________________
dense_5 (Dense)              (None, 32)                320032    
_________________________________________________________________
dense_6 (Dense)              (None, 1)                 33        
=================================================================
Total params: 1,320,065
Trainable params: 1,320,065
Non-trainable params: 0
_________________________________________________________________
Train on 200 samples, validate on 10000 samples
Epoch 1/10
200/200 [==============================] - 2s 8ms/step - loss: 0.6951 - acc: 0.4350 - val_loss: 0.6950 - val_acc: 0.5167
Epoch 2/10
200/200 [==============================] - 1s 6ms/step - loss: 0.5028 - acc: 0.9800 - val_loss: 0.7054 - val_acc: 0.5069
Epoch 3/10
200/200 [==============================] - 1s 7ms/step - loss: 0.2897 - acc: 0.9850 - val_loss: 0.7012 - val_acc: 0.5189
Epoch 4/10
200/200 [==============================] - 1s 6ms/step - loss: 0.1182 - acc: 1.0000 - val_loss: 0.7165 - val_acc: 0.5156
Epoch 5/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0523 - acc: 1.0000 - val_loss: 0.7150 - val_acc: 0.5288
Epoch 6/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0261 - acc: 1.0000 - val_loss: 0.7253 - val_acc: 0.5262
Epoch 7/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0141 - acc: 1.0000 - val_loss: 0.7211 - val_acc: 0.5384
Epoch 8/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0082 - acc: 1.0000 - val_loss: 0.7393 - val_acc: 0.5268
Epoch 9/10
200/200 [==============================] - 1s 7ms/step - loss: 0.0049 - acc: 1.0000 - val_loss: 0.7283 - val_acc: 0.5393
Epoch 10/10
200/200 [==============================] - 1s 7ms/step - loss: 0.0030 - acc: 1.0000 - val_loss: 0.7474 - val_acc: 0.5316
'''

Mapping:

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(acc) + 1)

plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()



10. Evaluation model:
Mark test data:

test_dir = os.path.join(imdb_dir, 'test')

labels = []
texts = []

for label_type in ['neg', 'pos']:
    dir_name = os.path.join(test_dir, label_type)
    for fname in sorted(os.listdir(dir_name)):
        if fname[-4:] == '.txt':
            f = open(os.path.join(dir_name, fname),encoding='utf-8')
            texts.append(f.read())
            f.close()
            if label_type == 'neg':
                labels.append(0)
            else:
                labels.append(1)

sequences = tokenizer.texts_to_sequences(texts)
x_test = pad_sequences(sequences, maxlen=maxlen)
y_test = np.asarray(labels)

Load and evaluate:
Due to the small number of training samples, the accuracy is already high.

model.load_weights('pre_trained_glove_model.h5')
model.evaluate(x_test, y_test)
'''
25000/25000 [==============================] - 3s 120us/step
[0.74487344819068912, 0.57604]
'''

Experiment 2: First-order recursive layer in Keras

Experimental purpose

Learn first-order recursion in Keras.
Is the network layer that replaces Numpy in the process with Keras: the SimpleRNN network layer.
There is only one small difference: the simpleRNN process batch sequence.

thinking

Train two models, one to generate a given random noise as input output example G, and one to identify the generated model example A from the actual example.Then, by training A as an effective discriminator, we can overlay G and A into our GAN, freeze the weights of the antagonistic part of the network, and train the generated network weights to push the random noise input to the "real" class output of the antagonistic half class.

Work process:

Experimental process

1.return_sequences constructor:

from keras.layers import SimpleRNN

from keras.models import Sequential
from keras.layers import Embedding, SimpleRNN

model = Sequential()
model.add(Embedding(10000, 32))
model.add(SimpleRNN(32))
model.summary()
'''
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, None, 32)          320000    
_________________________________________________________________
simple_rnn_1 (SimpleRNN)     (None, 32)                2080      
=================================================================
Total params: 322,080
Trainable params: 322,080
Non-trainable params: 0
_________________________________________________________________
'''
model = Sequential()
model.add(Embedding(10000, 32))
model.add(SimpleRNN(32, return_sequences=True))
model.summary()
'''
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_2 (Embedding)      (None, None, 32)          320000    
_________________________________________________________________
simple_rnn_2 (SimpleRNN)     (None, None, 32)          2080      
=================================================================
Total params: 322,080
Trainable params: 322,080
Non-trainable params: 0
_________________________________________________________________
'''

To increase the representativeness of a network, it is useful to overlay several recursive layers one by one.

model = Sequential()
model.add(Embedding(10000, 32))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32))
model.summary()
'''
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_3 (Embedding)      (None, None, 32)          320000    
_________________________________________________________________
simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      
_________________________________________________________________
simple_rnn_4 (SimpleRNN)     (None, None, 32)          2080      
_________________________________________________________________
simple_rnn_5 (SimpleRNN)     (None, None, 32)          2080      
_________________________________________________________________
simple_rnn_6 (SimpleRNN)     (None, 32)                2080      
=================================================================
Total params: 328,320
Trainable params: 328,320
Non-trainable params: 0
_________________________________________________________________
'''

2. Preprocessing data:

from keras.datasets import imdb
from keras.preprocessing import sequence

max_features = 10000 
maxlen = 500 
batch_size = 32

print('Loading data...')
(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)
print(len(input_train), 'train sequences')
print(len(input_test), 'test sequences')

print('Pad sequences (samples x time)')
input_train = sequence.pad_sequences(input_train, maxlen=maxlen)
input_test = sequence.pad_sequences(input_test, maxlen=maxlen)
print('input_train shape:', input_train.shape)
print('input_test shape:', input_test.shape)
'''
Loading data...
25000 train sequences
25000 test sequences
Pad sequences (samples x time)
input_train shape: (25000, 500)
input_test shape: (25000, 500)
'''

3. Train a simple recursive neural network consisting of an embedded layer and a simple RNN:

from keras.layers import Dense

model = Sequential()
model.add(Embedding(max_features, 32))
model.add(SimpleRNN(32))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])
history = model.fit(input_train, y_train,
                    epochs=10,
                    batch_size=128,
                    validation_split=0.2)
'''
Train on 20000 samples, validate on 5000 samples
Epoch 1/10
20000/20000 [==============================] - 24s 1ms/step - loss: 0.6418 - acc: 0.6144 - val_loss: 0.4725 - val_acc: 0.7934
Epoch 2/10
20000/20000 [==============================] - 17s 829us/step - loss: 0.4259 - acc: 0.8150 - val_loss: 0.4076 - val_acc: 0.8274
Epoch 3/10
20000/20000 [==============================] - 20s 990us/step - loss: 0.3028 - acc: 0.8801 - val_loss: 0.3612 - val_acc: 0.8458
Epoch 4/10
20000/20000 [==============================] - 19s 942us/step - loss: 0.2329 - acc: 0.9083 - val_loss: 0.3842 - val_acc: 0.8438
Epoch 5/10
20000/20000 [==============================] - 19s 937us/step - loss: 0.1717 - acc: 0.9366 - val_loss: 0.3924 - val_acc: 0.8556
Epoch 6/10
20000/20000 [==============================] - 20s 1ms/step - loss: 0.1288 - acc: 0.9547 - val_loss: 0.3990 - val_acc: 0.8502
Epoch 7/10
20000/20000 [==============================] - 19s 954us/step - loss: 0.0805 - acc: 0.9739 - val_loss: 0.4570 - val_acc: 0.8556
Epoch 8/10
20000/20000 [==============================] - 19s 968us/step - loss: 0.0520 - acc: 0.9838 - val_loss: 0.6217 - val_acc: 0.7936
Epoch 9/10
20000/20000 [==============================] - 20s 981us/step - loss: 0.0314 - acc: 0.9908 - val_loss: 0.6018 - val_acc: 0.8234
Epoch 10/10
20000/20000 [==============================] - 22s 1ms/step - loss: 0.0226 - acc: 0.9931 - val_loss: 0.6232 - val_acc: 0.8248
'''

4. Result:
The performance is not good, mainly because simpleRNN is not very good at handling long sequences, such as text.

import matplotlib.pyplot as plt

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(len(acc))

plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()



5. Examples of mixed LSTM in Keras:
Only the dimension of the LSTM network layer is specified, while the other parameters are set to default values in Keras.Keras has a better default value and will still perform better without spending a lot of time tuning parameters.

from keras.layers import LSTM

model = Sequential()
model.add(Embedding(max_features, 32))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['acc'])
history = model.fit(input_train, y_train,
                    epochs=10,
                    batch_size=128,
                    validation_split=0.2)
'''
Train on 20000 samples, validate on 5000 samples
Epoch 1/10
20000/20000 [==============================] - 51s 3ms/step - loss: 0.5097 - acc: 0.7618 - val_loss: 0.5537 - val_acc: 0.7454
Epoch 2/10
20000/20000 [==============================] - 47s 2ms/step - loss: 0.2905 - acc: 0.8856 - val_loss: 0.3063 - val_acc: 0.8714
Epoch 3/10
20000/20000 [==============================] - 48s 2ms/step - loss: 0.2315 - acc: 0.9106 - val_loss: 0.2942 - val_acc: 0.8906
Epoch 4/10
20000/20000 [==============================] - 46s 2ms/step - loss: 0.1951 - acc: 0.9268 - val_loss: 0.5121 - val_acc: 0.8334
Epoch 5/10
20000/20000 [==============================] - 48s 2ms/step - loss: 0.1724 - acc: 0.9370 - val_loss: 0.3116 - val_acc: 0.8832
Epoch 6/10
20000/20000 [==============================] - 50s 3ms/step - loss: 0.1525 - acc: 0.9433 - val_loss: 0.3921 - val_acc: 0.8792
Epoch 7/10
20000/20000 [==============================] - 47s 2ms/step - loss: 0.1380 - acc: 0.9500 - val_loss: 0.5595 - val_acc: 0.8574
Epoch 8/10
20000/20000 [==============================] - 44s 2ms/step - loss: 0.1310 - acc: 0.9526 - val_loss: 0.3344 - val_acc: 0.8768
Epoch 9/10
20000/20000 [==============================] - 45s 2ms/step - loss: 0.1174 - acc: 0.9580 - val_loss: 0.3414 - val_acc: 0.8834
Epoch 10/10
20000/20000 [==============================] - 46s 2ms/step - loss: 0.1077 - acc: 0.9627 - val_loss: 0.3649 - val_acc: 0.8814
'''

Result:

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(len(acc))

plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()


173 original articles published. 6. 40,000 visits+
Private letter follow

Tags: network encoding jupyter

Posted on Sat, 11 Jan 2020 17:03:20 -0800 by trampolinejoe