Cainvas
Model Files
traffic_signal.h5
keras
Model
deepSea Compiled Models
traffic_signal.exe
deepSea
Ubuntu

Traffic Sign Classification

Credit: AITS Cainvas Community

Photo by Rwds on Dribbble

Traffic Sign Classification is an essential part of Autonomous Vehicles so that it can understand the traffic signs properly and take actions accordingly.

Import Dataset

In [1]:
# This will load the dataset.You will see a folder called ALL in your workspace.
!wget -N "https://cainvas-static.s3.amazonaws.com/media/user_data/cainvas-admin/traffic.zip"
!unzip -qo traffic.zip 
!rm traffic.zip

Importing Libraries

In [2]:
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
sns.set_style('whitegrid')
%matplotlib inline
import os
import time
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, classification_report
from collections import Counter

import cv2
from tensorflow.keras.utils import plot_model
from keras.layers import Dense, Flatten, AveragePooling2D, Dropout
from keras.models import Model
from keras.applications.vgg16 import VGG16
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import Adam

Reading Data of Class Labels

In [3]:
path = 'traffic'
lab = pd.read_csv('traffic/labels.csv')
In [4]:
d = dict()
class_labels = dict()
for dirs in os.listdir(path + '/myData'):
    count = len(os.listdir(path+'/myData/'+dirs))
    d[dirs+' => '+lab[lab.ClassId == int(dirs)].values[0][1]] = count
    class_labels[int(dirs)] = lab[lab.ClassId == int(dirs)].values[0][1]

Reading Image Data

In [5]:
# input image dimensions
img_rows, img_cols = 224, 224
# The images are RGB.
img_channels = 3
nb_classes = len(class_labels.keys())

datagen = ImageDataGenerator()
data = datagen.flow_from_directory('traffic/myData',
                                    target_size=(224, 224),
                                    batch_size=2000,
                                    class_mode='categorical',
                                    shuffle=True )
Found 2000 images belonging to 20 classes.
In [6]:
X , y = data.next()
In [7]:
# Labels are one hot encoded
print(f"Data Shape   :{X.shape}\nLabels shape :{y.shape}")
Data Shape   :(2000, 224, 224, 3)
Labels shape :(2000, 20)

Sample Images of Dataset

In [8]:
fig, axes = plt.subplots(10,10, figsize=(18,18))
for i,ax in enumerate(axes.flat):
    r = np.random.randint(X.shape[0])
    ax.imshow(X[r].astype('uint8'))
    ax.grid(False)
    ax.axis('off')
    ax.set_title('Label: '+str(np.argmax(y[r])))
    

Test-Train Split

In [9]:
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.10, random_state=11)
In [10]:
print("Train Shape: {}\nTest Shape : {}".format(X_train.shape, X_test.shape))
Train Shape: (1800, 224, 224, 3)
Test Shape : (200, 224, 224, 3)

Model Archutecture

In [11]:
# Model creation with changes

model = VGG16(input_shape=(224,224,3),include_top=False)

for layer in model.layers:
    layer.trainable = False

newModel = model.output
newModel = AveragePooling2D()(newModel)
newModel = Flatten()(newModel)
newModel = Dense(128, activation="relu")(newModel)
newModel = Dropout(0.5)(newModel)
newModel = Dense(20, activation='softmax')(newModel)

model = Model(inputs=model.input, outputs=newModel)
In [12]:
model.summary()
Model: "functional_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 224, 224, 3)]     0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 224, 224, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 224, 224, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 112, 112, 64)      0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 112, 112, 128)     73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 112, 112, 128)     147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 56, 56, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 56, 56, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 28, 28, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 28, 28, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 14, 14, 512)       0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 7, 7, 512)         0         
_________________________________________________________________
average_pooling2d (AveragePo (None, 3, 3, 512)         0         
_________________________________________________________________
flatten (Flatten)            (None, 4608)              0         
_________________________________________________________________
dense (Dense)                (None, 128)               589952    
_________________________________________________________________
dropout (Dropout)            (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 20)                2580      
=================================================================
Total params: 15,307,220
Trainable params: 592,532
Non-trainable params: 14,714,688
_________________________________________________________________

Compiling the Model

In [13]:
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

Model Training

In [14]:
n_epochs = 10
history =  model.fit(X_train, y_train,  batch_size = 32, epochs = n_epochs, verbose = 1, 
              validation_data = (X_test, y_test))
Epoch 1/10
57/57 [==============================] - 7s 128ms/step - loss: 2.9081 - accuracy: 0.3411 - val_loss: 1.1677 - val_accuracy: 0.7500
Epoch 2/10
57/57 [==============================] - 7s 115ms/step - loss: 1.2052 - accuracy: 0.6506 - val_loss: 0.6896 - val_accuracy: 0.8400
Epoch 3/10
57/57 [==============================] - 7s 116ms/step - loss: 0.7333 - accuracy: 0.7794 - val_loss: 0.4004 - val_accuracy: 0.9000
Epoch 4/10
57/57 [==============================] - 7s 116ms/step - loss: 0.5760 - accuracy: 0.8167 - val_loss: 0.2477 - val_accuracy: 0.9500
Epoch 5/10
57/57 [==============================] - 7s 117ms/step - loss: 0.3953 - accuracy: 0.8744 - val_loss: 0.2276 - val_accuracy: 0.9500
Epoch 6/10
57/57 [==============================] - 7s 117ms/step - loss: 0.2970 - accuracy: 0.9056 - val_loss: 0.1847 - val_accuracy: 0.9600
Epoch 7/10
57/57 [==============================] - 7s 118ms/step - loss: 0.2553 - accuracy: 0.9244 - val_loss: 0.1660 - val_accuracy: 0.9650
Epoch 8/10
57/57 [==============================] - 7s 119ms/step - loss: 0.2418 - accuracy: 0.9222 - val_loss: 0.1779 - val_accuracy: 0.9700
Epoch 9/10
57/57 [==============================] - 7s 120ms/step - loss: 0.2198 - accuracy: 0.9328 - val_loss: 0.0879 - val_accuracy: 0.9850
Epoch 10/10
57/57 [==============================] - 7s 120ms/step - loss: 0.1905 - accuracy: 0.9389 - val_loss: 0.0603 - val_accuracy: 0.9950

Training Plot

In [15]:
model.save("traffic_signal.h5")
pd.DataFrame(history.history).plot(figsize=(8, 5))
plt.grid(True)
plt.show()
In [16]:
loss, acc = model.evaluate(X_test, y_test)
print('Accuracy: ', acc, '\nLoss    : ', loss)
7/7 [==============================] - 1s 84ms/step - loss: 0.0603 - accuracy: 0.9950
Accuracy:  0.9950000047683716 
Loss    :  0.0603308230638504

Obtaining Predictions

In [17]:
%%time
pred = np.argmax(model.predict(X_test), axis = 1)
CPU times: user 878 ms, sys: 93.8 ms, total: 972 ms
Wall time: 915 ms

Classification Report

In [18]:
labels = [class_labels[i] for i in range(20)]
print(classification_report(np.argmax(y_test, axis = 1), pred, target_names = labels))
                                              precision    recall  f1-score   support

                        Speed limit (20km/h)       1.00      1.00      1.00         7
                        Speed limit (30km/h)       1.00      0.92      0.96        13
                        Speed limit (50km/h)       1.00      1.00      1.00         7
                        Speed limit (60km/h)       1.00      1.00      1.00         8
                        Speed limit (70km/h)       1.00      1.00      1.00         9
                        Speed limit (80km/h)       1.00      1.00      1.00         7
                 End of speed limit (80km/h)       0.86      1.00      0.92         6
                       Speed limit (100km/h)       1.00      1.00      1.00        14
                       Speed limit (120km/h)       1.00      1.00      1.00        10
                                  No passing       1.00      1.00      1.00         8
No passing for vechiles over 3.5 metric tons       1.00      1.00      1.00        11
       Right-of-way at the next intersection       1.00      1.00      1.00        16
                               Priority road       1.00      1.00      1.00        12
                                       Yield       1.00      1.00      1.00        14
                                        Stop       1.00      1.00      1.00         5
                                 No vechiles       1.00      1.00      1.00         8
    Vechiles over 3.5 metric tons prohibited       1.00      1.00      1.00        12
                                    No entry       1.00      1.00      1.00        10
                             General caution       1.00      1.00      1.00        11
                 Dangerous curve to the left       1.00      1.00      1.00        12

                                    accuracy                           0.99       200
                                   macro avg       0.99      1.00      0.99       200
                                weighted avg       1.00      0.99      1.00       200

Accessing the performance of the Model

In [19]:
fig, axes = plt.subplots(5,5, figsize=(18,18))
for i,ax in enumerate(axes.flat):
    r = np.random.randint(X_test.shape[0])
    ax.imshow(X_test[r].astype('uint8'))
    ax.grid(False)
    ax.axis('off')
    ax.set_title('Original: {} Predicted: {}'.format(np.argmax(y_test[r]), np.argmax(model.predict(X_test[r].reshape(1, 224, 224, 3)))))

Compiling Model with DeepC

In [20]:
!deepCC traffic_signal.h5
[INFO]
Reading [keras model] 'traffic_signal.h5'
[SUCCESS]
Saved 'traffic_signal.onnx'
[INFO]
Reading [onnx model] 'traffic_signal.onnx'
[INFO]
Model info:
  ir_vesion : 5
  doc       : 
[WARNING]
[ONNX]: graph-node block1_conv1's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block1_conv2's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block2_conv1's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block2_conv2's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block3_conv1's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block3_conv2's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block3_conv3's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block4_conv1's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block4_conv2's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block4_conv3's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block5_conv1's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block5_conv2's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node block5_conv3's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: terminal (input/output) input_1's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) dense_1's shape is less than 1. Changing it to 1.
WARN (GRAPH): found operator node with the same name (dense_1) as io node.
[INFO]
Running DNNC graph sanity check ...
[SUCCESS]
Passed sanity check.
[INFO]
Writing C++ file 'traffic_signal_deepC/traffic_signal.cpp'
[INFO]
deepSea model files are ready in 'traffic_signal_deepC/' 
[RUNNING COMMAND]
g++ -std=c++11 -O3 -fno-rtti -fno-exceptions -I. -I/opt/tljh/user/lib/python3.7/site-packages/deepC-0.13-py3.7-linux-x86_64.egg/deepC/include -isystem /opt/tljh/user/lib/python3.7/site-packages/deepC-0.13-py3.7-linux-x86_64.egg/deepC/packages/eigen-eigen-323c052e1731 traffic_signal_deepC/traffic_signal.cpp -o traffic_signal_deepC/traffic_signal.exe
[RUNNING COMMAND]
size "traffic_signal_deepC/traffic_signal.exe"
   text	   data	    bss	    dec	    hex	filename
 213972	61232944	    760	61447676	3a99dfc	traffic_signal_deepC/traffic_signal.exe
[SUCCESS]
Saved model as executable "traffic_signal_deepC/traffic_signal.exe"