Cainvas

Object Detection using YOLO v3

Credit: AITS Cainvas Community

Photo by Gleb Kuznetsov on Dribbble

Object Detection can be defined as the task of object classification and localization.Object Detection plays an important role in many Deep Learning projects and IoT Applications such as we need object detection for Autonomous Vehicles, we need it for security camera surveillance, and even for applications used by us for generating PDFs nowadays uses Object Detection for detecting documents.In short, Object Detection has a wide range of appliactions.

Importing necessary libraries

In [1]:
import cv2
import numpy as np
import time
import matplotlib.pyplot as plt
from tensorflow import keras

Importing the Yolov3 pretrained model and some other necessary files to our workspace

In [2]:
!wget -N "https://cainvas-static.s3.amazonaws.com/media/user_data/cainvas-admin/Yolo_data.zip"
!unzip -o Yolo_data.zip 
!rm Yolo_data.zip
--2020-12-15 04:27:02--  https://cainvas-static.s3.amazonaws.com/media/user_data/cainvas-admin/Yolo_data.zip
Resolving cainvas-static.s3.amazonaws.com (cainvas-static.s3.amazonaws.com)... 52.219.64.24
Connecting to cainvas-static.s3.amazonaws.com (cainvas-static.s3.amazonaws.com)|52.219.64.24|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 462457774 (441M) [application/zip]
Saving to: ‘Yolo_data.zip’

Yolo_data.zip       100%[===================>] 441.03M  77.0MB/s    in 5.7s    

2020-12-15 04:27:08 (77.1 MB/s) - ‘Yolo_data.zip’ saved [462457774/462457774]

Archive:  Yolo_data.zip
  inflating: yolov3.weights          
  inflating: images3.jpg             
  inflating: images1.jpg             
   creating: cfg/
  inflating: cfg/yolov3.cfg          
  inflating: coco.names              
  inflating: model.h5                
  inflating: test.jpg                

Loading the pretrained Yolov3 Model

Since Yolov3 model has been trained on the MS COCO dataset for thousands of epochs and hours of training, so there is no need for explicit training rather we can just use the pretrained model and its weights provided by Yolo.

In [3]:
model = keras.models.load_model('model.h5')
model.summary()
WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
Model: "functional_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, None, None,  0                                            
__________________________________________________________________________________________________
conv_0 (Conv2D)                 (None, None, None, 3 864         input_1[0][0]                    
__________________________________________________________________________________________________
bnorm_0 (BatchNormalization)    (None, None, None, 3 128         conv_0[0][0]                     
__________________________________________________________________________________________________
leaky_0 (LeakyReLU)             (None, None, None, 3 0           bnorm_0[0][0]                    
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           leaky_0[0][0]                    
__________________________________________________________________________________________________
conv_1 (Conv2D)                 (None, None, None, 6 18432       zero_padding2d[0][0]             
__________________________________________________________________________________________________
bnorm_1 (BatchNormalization)    (None, None, None, 6 256         conv_1[0][0]                     
__________________________________________________________________________________________________
leaky_1 (LeakyReLU)             (None, None, None, 6 0           bnorm_1[0][0]                    
__________________________________________________________________________________________________
conv_2 (Conv2D)                 (None, None, None, 3 2048        leaky_1[0][0]                    
__________________________________________________________________________________________________
bnorm_2 (BatchNormalization)    (None, None, None, 3 128         conv_2[0][0]                     
__________________________________________________________________________________________________
leaky_2 (LeakyReLU)             (None, None, None, 3 0           bnorm_2[0][0]                    
__________________________________________________________________________________________________
conv_3 (Conv2D)                 (None, None, None, 6 18432       leaky_2[0][0]                    
__________________________________________________________________________________________________
bnorm_3 (BatchNormalization)    (None, None, None, 6 256         conv_3[0][0]                     
__________________________________________________________________________________________________
leaky_3 (LeakyReLU)             (None, None, None, 6 0           bnorm_3[0][0]                    
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 6 0           leaky_1[0][0]                    
                                                                 leaky_3[0][0]                    
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, None, None, 6 0           add[0][0]                        
__________________________________________________________________________________________________
conv_5 (Conv2D)                 (None, None, None, 1 73728       zero_padding2d_1[0][0]           
__________________________________________________________________________________________________
bnorm_5 (BatchNormalization)    (None, None, None, 1 512         conv_5[0][0]                     
__________________________________________________________________________________________________
leaky_5 (LeakyReLU)             (None, None, None, 1 0           bnorm_5[0][0]                    
__________________________________________________________________________________________________
conv_6 (Conv2D)                 (None, None, None, 6 8192        leaky_5[0][0]                    
__________________________________________________________________________________________________
bnorm_6 (BatchNormalization)    (None, None, None, 6 256         conv_6[0][0]                     
__________________________________________________________________________________________________
leaky_6 (LeakyReLU)             (None, None, None, 6 0           bnorm_6[0][0]                    
__________________________________________________________________________________________________
conv_7 (Conv2D)                 (None, None, None, 1 73728       leaky_6[0][0]                    
__________________________________________________________________________________________________
bnorm_7 (BatchNormalization)    (None, None, None, 1 512         conv_7[0][0]                     
__________________________________________________________________________________________________
leaky_7 (LeakyReLU)             (None, None, None, 1 0           bnorm_7[0][0]                    
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 1 0           leaky_5[0][0]                    
                                                                 leaky_7[0][0]                    
__________________________________________________________________________________________________
conv_9 (Conv2D)                 (None, None, None, 6 8192        add_1[0][0]                      
__________________________________________________________________________________________________
bnorm_9 (BatchNormalization)    (None, None, None, 6 256         conv_9[0][0]                     
__________________________________________________________________________________________________
leaky_9 (LeakyReLU)             (None, None, None, 6 0           bnorm_9[0][0]                    
__________________________________________________________________________________________________
conv_10 (Conv2D)                (None, None, None, 1 73728       leaky_9[0][0]                    
__________________________________________________________________________________________________
bnorm_10 (BatchNormalization)   (None, None, None, 1 512         conv_10[0][0]                    
__________________________________________________________________________________________________
leaky_10 (LeakyReLU)            (None, None, None, 1 0           bnorm_10[0][0]                   
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 1 0           add_1[0][0]                      
                                                                 leaky_10[0][0]                   
__________________________________________________________________________________________________
zero_padding2d_2 (ZeroPadding2D (None, None, None, 1 0           add_2[0][0]                      
__________________________________________________________________________________________________
conv_12 (Conv2D)                (None, None, None, 2 294912      zero_padding2d_2[0][0]           
__________________________________________________________________________________________________
bnorm_12 (BatchNormalization)   (None, None, None, 2 1024        conv_12[0][0]                    
__________________________________________________________________________________________________
leaky_12 (LeakyReLU)            (None, None, None, 2 0           bnorm_12[0][0]                   
__________________________________________________________________________________________________
conv_13 (Conv2D)                (None, None, None, 1 32768       leaky_12[0][0]                   
__________________________________________________________________________________________________
bnorm_13 (BatchNormalization)   (None, None, None, 1 512         conv_13[0][0]                    
__________________________________________________________________________________________________
leaky_13 (LeakyReLU)            (None, None, None, 1 0           bnorm_13[0][0]                   
__________________________________________________________________________________________________
conv_14 (Conv2D)                (None, None, None, 2 294912      leaky_13[0][0]                   
__________________________________________________________________________________________________
bnorm_14 (BatchNormalization)   (None, None, None, 2 1024        conv_14[0][0]                    
__________________________________________________________________________________________________
leaky_14 (LeakyReLU)            (None, None, None, 2 0           bnorm_14[0][0]                   
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 2 0           leaky_12[0][0]                   
                                                                 leaky_14[0][0]                   
__________________________________________________________________________________________________
conv_16 (Conv2D)                (None, None, None, 1 32768       add_3[0][0]                      
__________________________________________________________________________________________________
bnorm_16 (BatchNormalization)   (None, None, None, 1 512         conv_16[0][0]                    
__________________________________________________________________________________________________
leaky_16 (LeakyReLU)            (None, None, None, 1 0           bnorm_16[0][0]                   
__________________________________________________________________________________________________
conv_17 (Conv2D)                (None, None, None, 2 294912      leaky_16[0][0]                   
__________________________________________________________________________________________________
bnorm_17 (BatchNormalization)   (None, None, None, 2 1024        conv_17[0][0]                    
__________________________________________________________________________________________________
leaky_17 (LeakyReLU)            (None, None, None, 2 0           bnorm_17[0][0]                   
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 2 0           add_3[0][0]                      
                                                                 leaky_17[0][0]                   
__________________________________________________________________________________________________
conv_19 (Conv2D)                (None, None, None, 1 32768       add_4[0][0]                      
__________________________________________________________________________________________________
bnorm_19 (BatchNormalization)   (None, None, None, 1 512         conv_19[0][0]                    
__________________________________________________________________________________________________
leaky_19 (LeakyReLU)            (None, None, None, 1 0           bnorm_19[0][0]                   
__________________________________________________________________________________________________
conv_20 (Conv2D)                (None, None, None, 2 294912      leaky_19[0][0]                   
__________________________________________________________________________________________________
bnorm_20 (BatchNormalization)   (None, None, None, 2 1024        conv_20[0][0]                    
__________________________________________________________________________________________________
leaky_20 (LeakyReLU)            (None, None, None, 2 0           bnorm_20[0][0]                   
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 2 0           add_4[0][0]                      
                                                                 leaky_20[0][0]                   
__________________________________________________________________________________________________
conv_22 (Conv2D)                (None, None, None, 1 32768       add_5[0][0]                      
__________________________________________________________________________________________________
bnorm_22 (BatchNormalization)   (None, None, None, 1 512         conv_22[0][0]                    
__________________________________________________________________________________________________
leaky_22 (LeakyReLU)            (None, None, None, 1 0           bnorm_22[0][0]                   
__________________________________________________________________________________________________
conv_23 (Conv2D)                (None, None, None, 2 294912      leaky_22[0][0]                   
__________________________________________________________________________________________________
bnorm_23 (BatchNormalization)   (None, None, None, 2 1024        conv_23[0][0]                    
__________________________________________________________________________________________________
leaky_23 (LeakyReLU)            (None, None, None, 2 0           bnorm_23[0][0]                   
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 2 0           add_5[0][0]                      
                                                                 leaky_23[0][0]                   
__________________________________________________________________________________________________
conv_25 (Conv2D)                (None, None, None, 1 32768       add_6[0][0]                      
__________________________________________________________________________________________________
bnorm_25 (BatchNormalization)   (None, None, None, 1 512         conv_25[0][0]                    
__________________________________________________________________________________________________
leaky_25 (LeakyReLU)            (None, None, None, 1 0           bnorm_25[0][0]                   
__________________________________________________________________________________________________
conv_26 (Conv2D)                (None, None, None, 2 294912      leaky_25[0][0]                   
__________________________________________________________________________________________________
bnorm_26 (BatchNormalization)   (None, None, None, 2 1024        conv_26[0][0]                    
__________________________________________________________________________________________________
leaky_26 (LeakyReLU)            (None, None, None, 2 0           bnorm_26[0][0]                   
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 2 0           add_6[0][0]                      
                                                                 leaky_26[0][0]                   
__________________________________________________________________________________________________
conv_28 (Conv2D)                (None, None, None, 1 32768       add_7[0][0]                      
__________________________________________________________________________________________________
bnorm_28 (BatchNormalization)   (None, None, None, 1 512         conv_28[0][0]                    
__________________________________________________________________________________________________
leaky_28 (LeakyReLU)            (None, None, None, 1 0           bnorm_28[0][0]                   
__________________________________________________________________________________________________
conv_29 (Conv2D)                (None, None, None, 2 294912      leaky_28[0][0]                   
__________________________________________________________________________________________________
bnorm_29 (BatchNormalization)   (None, None, None, 2 1024        conv_29[0][0]                    
__________________________________________________________________________________________________
leaky_29 (LeakyReLU)            (None, None, None, 2 0           bnorm_29[0][0]                   
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 2 0           add_7[0][0]                      
                                                                 leaky_29[0][0]                   
__________________________________________________________________________________________________
conv_31 (Conv2D)                (None, None, None, 1 32768       add_8[0][0]                      
__________________________________________________________________________________________________
bnorm_31 (BatchNormalization)   (None, None, None, 1 512         conv_31[0][0]                    
__________________________________________________________________________________________________
leaky_31 (LeakyReLU)            (None, None, None, 1 0           bnorm_31[0][0]                   
__________________________________________________________________________________________________
conv_32 (Conv2D)                (None, None, None, 2 294912      leaky_31[0][0]                   
__________________________________________________________________________________________________
bnorm_32 (BatchNormalization)   (None, None, None, 2 1024        conv_32[0][0]                    
__________________________________________________________________________________________________
leaky_32 (LeakyReLU)            (None, None, None, 2 0           bnorm_32[0][0]                   
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 2 0           add_8[0][0]                      
                                                                 leaky_32[0][0]                   
__________________________________________________________________________________________________
conv_34 (Conv2D)                (None, None, None, 1 32768       add_9[0][0]                      
__________________________________________________________________________________________________
bnorm_34 (BatchNormalization)   (None, None, None, 1 512         conv_34[0][0]                    
__________________________________________________________________________________________________
leaky_34 (LeakyReLU)            (None, None, None, 1 0           bnorm_34[0][0]                   
__________________________________________________________________________________________________
conv_35 (Conv2D)                (None, None, None, 2 294912      leaky_34[0][0]                   
__________________________________________________________________________________________________
bnorm_35 (BatchNormalization)   (None, None, None, 2 1024        conv_35[0][0]                    
__________________________________________________________________________________________________
leaky_35 (LeakyReLU)            (None, None, None, 2 0           bnorm_35[0][0]                   
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 2 0           add_9[0][0]                      
                                                                 leaky_35[0][0]                   
__________________________________________________________________________________________________
zero_padding2d_3 (ZeroPadding2D (None, None, None, 2 0           add_10[0][0]                     
__________________________________________________________________________________________________
conv_37 (Conv2D)                (None, None, None, 5 1179648     zero_padding2d_3[0][0]           
__________________________________________________________________________________________________
bnorm_37 (BatchNormalization)   (None, None, None, 5 2048        conv_37[0][0]                    
__________________________________________________________________________________________________
leaky_37 (LeakyReLU)            (None, None, None, 5 0           bnorm_37[0][0]                   
__________________________________________________________________________________________________
conv_38 (Conv2D)                (None, None, None, 2 131072      leaky_37[0][0]                   
__________________________________________________________________________________________________
bnorm_38 (BatchNormalization)   (None, None, None, 2 1024        conv_38[0][0]                    
__________________________________________________________________________________________________
leaky_38 (LeakyReLU)            (None, None, None, 2 0           bnorm_38[0][0]                   
__________________________________________________________________________________________________
conv_39 (Conv2D)                (None, None, None, 5 1179648     leaky_38[0][0]                   
__________________________________________________________________________________________________
bnorm_39 (BatchNormalization)   (None, None, None, 5 2048        conv_39[0][0]                    
__________________________________________________________________________________________________
leaky_39 (LeakyReLU)            (None, None, None, 5 0           bnorm_39[0][0]                   
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 5 0           leaky_37[0][0]                   
                                                                 leaky_39[0][0]                   
__________________________________________________________________________________________________
conv_41 (Conv2D)                (None, None, None, 2 131072      add_11[0][0]                     
__________________________________________________________________________________________________
bnorm_41 (BatchNormalization)   (None, None, None, 2 1024        conv_41[0][0]                    
__________________________________________________________________________________________________
leaky_41 (LeakyReLU)            (None, None, None, 2 0           bnorm_41[0][0]                   
__________________________________________________________________________________________________
conv_42 (Conv2D)                (None, None, None, 5 1179648     leaky_41[0][0]                   
__________________________________________________________________________________________________
bnorm_42 (BatchNormalization)   (None, None, None, 5 2048        conv_42[0][0]                    
__________________________________________________________________________________________________
leaky_42 (LeakyReLU)            (None, None, None, 5 0           bnorm_42[0][0]                   
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 5 0           add_11[0][0]                     
                                                                 leaky_42[0][0]                   
__________________________________________________________________________________________________
conv_44 (Conv2D)                (None, None, None, 2 131072      add_12[0][0]                     
__________________________________________________________________________________________________
bnorm_44 (BatchNormalization)   (None, None, None, 2 1024        conv_44[0][0]                    
__________________________________________________________________________________________________
leaky_44 (LeakyReLU)            (None, None, None, 2 0           bnorm_44[0][0]                   
__________________________________________________________________________________________________
conv_45 (Conv2D)                (None, None, None, 5 1179648     leaky_44[0][0]                   
__________________________________________________________________________________________________
bnorm_45 (BatchNormalization)   (None, None, None, 5 2048        conv_45[0][0]                    
__________________________________________________________________________________________________
leaky_45 (LeakyReLU)            (None, None, None, 5 0           bnorm_45[0][0]                   
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 5 0           add_12[0][0]                     
                                                                 leaky_45[0][0]                   
__________________________________________________________________________________________________
conv_47 (Conv2D)                (None, None, None, 2 131072      add_13[0][0]                     
__________________________________________________________________________________________________
bnorm_47 (BatchNormalization)   (None, None, None, 2 1024        conv_47[0][0]                    
__________________________________________________________________________________________________
leaky_47 (LeakyReLU)            (None, None, None, 2 0           bnorm_47[0][0]                   
__________________________________________________________________________________________________
conv_48 (Conv2D)                (None, None, None, 5 1179648     leaky_47[0][0]                   
__________________________________________________________________________________________________
bnorm_48 (BatchNormalization)   (None, None, None, 5 2048        conv_48[0][0]                    
__________________________________________________________________________________________________
leaky_48 (LeakyReLU)            (None, None, None, 5 0           bnorm_48[0][0]                   
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 5 0           add_13[0][0]                     
                                                                 leaky_48[0][0]                   
__________________________________________________________________________________________________
conv_50 (Conv2D)                (None, None, None, 2 131072      add_14[0][0]                     
__________________________________________________________________________________________________
bnorm_50 (BatchNormalization)   (None, None, None, 2 1024        conv_50[0][0]                    
__________________________________________________________________________________________________
leaky_50 (LeakyReLU)            (None, None, None, 2 0           bnorm_50[0][0]                   
__________________________________________________________________________________________________
conv_51 (Conv2D)                (None, None, None, 5 1179648     leaky_50[0][0]                   
__________________________________________________________________________________________________
bnorm_51 (BatchNormalization)   (None, None, None, 5 2048        conv_51[0][0]                    
__________________________________________________________________________________________________
leaky_51 (LeakyReLU)            (None, None, None, 5 0           bnorm_51[0][0]                   
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 5 0           add_14[0][0]                     
                                                                 leaky_51[0][0]                   
__________________________________________________________________________________________________
conv_53 (Conv2D)                (None, None, None, 2 131072      add_15[0][0]                     
__________________________________________________________________________________________________
bnorm_53 (BatchNormalization)   (None, None, None, 2 1024        conv_53[0][0]                    
__________________________________________________________________________________________________
leaky_53 (LeakyReLU)            (None, None, None, 2 0           bnorm_53[0][0]                   
__________________________________________________________________________________________________
conv_54 (Conv2D)                (None, None, None, 5 1179648     leaky_53[0][0]                   
__________________________________________________________________________________________________
bnorm_54 (BatchNormalization)   (None, None, None, 5 2048        conv_54[0][0]                    
__________________________________________________________________________________________________
leaky_54 (LeakyReLU)            (None, None, None, 5 0           bnorm_54[0][0]                   
__________________________________________________________________________________________________
add_16 (Add)                    (None, None, None, 5 0           add_15[0][0]                     
                                                                 leaky_54[0][0]                   
__________________________________________________________________________________________________
conv_56 (Conv2D)                (None, None, None, 2 131072      add_16[0][0]                     
__________________________________________________________________________________________________
bnorm_56 (BatchNormalization)   (None, None, None, 2 1024        conv_56[0][0]                    
__________________________________________________________________________________________________
leaky_56 (LeakyReLU)            (None, None, None, 2 0           bnorm_56[0][0]                   
__________________________________________________________________________________________________
conv_57 (Conv2D)                (None, None, None, 5 1179648     leaky_56[0][0]                   
__________________________________________________________________________________________________
bnorm_57 (BatchNormalization)   (None, None, None, 5 2048        conv_57[0][0]                    
__________________________________________________________________________________________________
leaky_57 (LeakyReLU)            (None, None, None, 5 0           bnorm_57[0][0]                   
__________________________________________________________________________________________________
add_17 (Add)                    (None, None, None, 5 0           add_16[0][0]                     
                                                                 leaky_57[0][0]                   
__________________________________________________________________________________________________
conv_59 (Conv2D)                (None, None, None, 2 131072      add_17[0][0]                     
__________________________________________________________________________________________________
bnorm_59 (BatchNormalization)   (None, None, None, 2 1024        conv_59[0][0]                    
__________________________________________________________________________________________________
leaky_59 (LeakyReLU)            (None, None, None, 2 0           bnorm_59[0][0]                   
__________________________________________________________________________________________________
conv_60 (Conv2D)                (None, None, None, 5 1179648     leaky_59[0][0]                   
__________________________________________________________________________________________________
bnorm_60 (BatchNormalization)   (None, None, None, 5 2048        conv_60[0][0]                    
__________________________________________________________________________________________________
leaky_60 (LeakyReLU)            (None, None, None, 5 0           bnorm_60[0][0]                   
__________________________________________________________________________________________________
add_18 (Add)                    (None, None, None, 5 0           add_17[0][0]                     
                                                                 leaky_60[0][0]                   
__________________________________________________________________________________________________
zero_padding2d_4 (ZeroPadding2D (None, None, None, 5 0           add_18[0][0]                     
__________________________________________________________________________________________________
conv_62 (Conv2D)                (None, None, None, 1 4718592     zero_padding2d_4[0][0]           
__________________________________________________________________________________________________
bnorm_62 (BatchNormalization)   (None, None, None, 1 4096        conv_62[0][0]                    
__________________________________________________________________________________________________
leaky_62 (LeakyReLU)            (None, None, None, 1 0           bnorm_62[0][0]                   
__________________________________________________________________________________________________
conv_63 (Conv2D)                (None, None, None, 5 524288      leaky_62[0][0]                   
__________________________________________________________________________________________________
bnorm_63 (BatchNormalization)   (None, None, None, 5 2048        conv_63[0][0]                    
__________________________________________________________________________________________________
leaky_63 (LeakyReLU)            (None, None, None, 5 0           bnorm_63[0][0]                   
__________________________________________________________________________________________________
conv_64 (Conv2D)                (None, None, None, 1 4718592     leaky_63[0][0]                   
__________________________________________________________________________________________________
bnorm_64 (BatchNormalization)   (None, None, None, 1 4096        conv_64[0][0]                    
__________________________________________________________________________________________________
leaky_64 (LeakyReLU)            (None, None, None, 1 0           bnorm_64[0][0]                   
__________________________________________________________________________________________________
add_19 (Add)                    (None, None, None, 1 0           leaky_62[0][0]                   
                                                                 leaky_64[0][0]                   
__________________________________________________________________________________________________
conv_66 (Conv2D)                (None, None, None, 5 524288      add_19[0][0]                     
__________________________________________________________________________________________________
bnorm_66 (BatchNormalization)   (None, None, None, 5 2048        conv_66[0][0]                    
__________________________________________________________________________________________________
leaky_66 (LeakyReLU)            (None, None, None, 5 0           bnorm_66[0][0]                   
__________________________________________________________________________________________________
conv_67 (Conv2D)                (None, None, None, 1 4718592     leaky_66[0][0]                   
__________________________________________________________________________________________________
bnorm_67 (BatchNormalization)   (None, None, None, 1 4096        conv_67[0][0]                    
__________________________________________________________________________________________________
leaky_67 (LeakyReLU)            (None, None, None, 1 0           bnorm_67[0][0]                   
__________________________________________________________________________________________________
add_20 (Add)                    (None, None, None, 1 0           add_19[0][0]                     
                                                                 leaky_67[0][0]                   
__________________________________________________________________________________________________
conv_69 (Conv2D)                (None, None, None, 5 524288      add_20[0][0]                     
__________________________________________________________________________________________________
bnorm_69 (BatchNormalization)   (None, None, None, 5 2048        conv_69[0][0]                    
__________________________________________________________________________________________________
leaky_69 (LeakyReLU)            (None, None, None, 5 0           bnorm_69[0][0]                   
__________________________________________________________________________________________________
conv_70 (Conv2D)                (None, None, None, 1 4718592     leaky_69[0][0]                   
__________________________________________________________________________________________________
bnorm_70 (BatchNormalization)   (None, None, None, 1 4096        conv_70[0][0]                    
__________________________________________________________________________________________________
leaky_70 (LeakyReLU)            (None, None, None, 1 0           bnorm_70[0][0]                   
__________________________________________________________________________________________________
add_21 (Add)                    (None, None, None, 1 0           add_20[0][0]                     
                                                                 leaky_70[0][0]                   
__________________________________________________________________________________________________
conv_72 (Conv2D)                (None, None, None, 5 524288      add_21[0][0]                     
__________________________________________________________________________________________________
bnorm_72 (BatchNormalization)   (None, None, None, 5 2048        conv_72[0][0]                    
__________________________________________________________________________________________________
leaky_72 (LeakyReLU)            (None, None, None, 5 0           bnorm_72[0][0]                   
__________________________________________________________________________________________________
conv_73 (Conv2D)                (None, None, None, 1 4718592     leaky_72[0][0]                   
__________________________________________________________________________________________________
bnorm_73 (BatchNormalization)   (None, None, None, 1 4096        conv_73[0][0]                    
__________________________________________________________________________________________________
leaky_73 (LeakyReLU)            (None, None, None, 1 0           bnorm_73[0][0]                   
__________________________________________________________________________________________________
add_22 (Add)                    (None, None, None, 1 0           add_21[0][0]                     
                                                                 leaky_73[0][0]                   
__________________________________________________________________________________________________
conv_75 (Conv2D)                (None, None, None, 5 524288      add_22[0][0]                     
__________________________________________________________________________________________________
bnorm_75 (BatchNormalization)   (None, None, None, 5 2048        conv_75[0][0]                    
__________________________________________________________________________________________________
leaky_75 (LeakyReLU)            (None, None, None, 5 0           bnorm_75[0][0]                   
__________________________________________________________________________________________________
conv_76 (Conv2D)                (None, None, None, 1 4718592     leaky_75[0][0]                   
__________________________________________________________________________________________________
bnorm_76 (BatchNormalization)   (None, None, None, 1 4096        conv_76[0][0]                    
__________________________________________________________________________________________________
leaky_76 (LeakyReLU)            (None, None, None, 1 0           bnorm_76[0][0]                   
__________________________________________________________________________________________________
conv_77 (Conv2D)                (None, None, None, 5 524288      leaky_76[0][0]                   
__________________________________________________________________________________________________
bnorm_77 (BatchNormalization)   (None, None, None, 5 2048        conv_77[0][0]                    
__________________________________________________________________________________________________
leaky_77 (LeakyReLU)            (None, None, None, 5 0           bnorm_77[0][0]                   
__________________________________________________________________________________________________
conv_78 (Conv2D)                (None, None, None, 1 4718592     leaky_77[0][0]                   
__________________________________________________________________________________________________
bnorm_78 (BatchNormalization)   (None, None, None, 1 4096        conv_78[0][0]                    
__________________________________________________________________________________________________
leaky_78 (LeakyReLU)            (None, None, None, 1 0           bnorm_78[0][0]                   
__________________________________________________________________________________________________
conv_79 (Conv2D)                (None, None, None, 5 524288      leaky_78[0][0]                   
__________________________________________________________________________________________________
bnorm_79 (BatchNormalization)   (None, None, None, 5 2048        conv_79[0][0]                    
__________________________________________________________________________________________________
leaky_79 (LeakyReLU)            (None, None, None, 5 0           bnorm_79[0][0]                   
__________________________________________________________________________________________________
conv_84 (Conv2D)                (None, None, None, 2 131072      leaky_79[0][0]                   
__________________________________________________________________________________________________
bnorm_84 (BatchNormalization)   (None, None, None, 2 1024        conv_84[0][0]                    
__________________________________________________________________________________________________
leaky_84 (LeakyReLU)            (None, None, None, 2 0           bnorm_84[0][0]                   
__________________________________________________________________________________________________
up_sampling2d (UpSampling2D)    (None, None, None, 2 0           leaky_84[0][0]                   
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, None, None, 7 0           up_sampling2d[0][0]              
                                                                 add_18[0][0]                     
__________________________________________________________________________________________________
conv_87 (Conv2D)                (None, None, None, 2 196608      concatenate[0][0]                
__________________________________________________________________________________________________
bnorm_87 (BatchNormalization)   (None, None, None, 2 1024        conv_87[0][0]                    
__________________________________________________________________________________________________
leaky_87 (LeakyReLU)            (None, None, None, 2 0           bnorm_87[0][0]                   
__________________________________________________________________________________________________
conv_88 (Conv2D)                (None, None, None, 5 1179648     leaky_87[0][0]                   
__________________________________________________________________________________________________
bnorm_88 (BatchNormalization)   (None, None, None, 5 2048        conv_88[0][0]                    
__________________________________________________________________________________________________
leaky_88 (LeakyReLU)            (None, None, None, 5 0           bnorm_88[0][0]                   
__________________________________________________________________________________________________
conv_89 (Conv2D)                (None, None, None, 2 131072      leaky_88[0][0]                   
__________________________________________________________________________________________________
bnorm_89 (BatchNormalization)   (None, None, None, 2 1024        conv_89[0][0]                    
__________________________________________________________________________________________________
leaky_89 (LeakyReLU)            (None, None, None, 2 0           bnorm_89[0][0]                   
__________________________________________________________________________________________________
conv_90 (Conv2D)                (None, None, None, 5 1179648     leaky_89[0][0]                   
__________________________________________________________________________________________________
bnorm_90 (BatchNormalization)   (None, None, None, 5 2048        conv_90[0][0]                    
__________________________________________________________________________________________________
leaky_90 (LeakyReLU)            (None, None, None, 5 0           bnorm_90[0][0]                   
__________________________________________________________________________________________________
conv_91 (Conv2D)                (None, None, None, 2 131072      leaky_90[0][0]                   
__________________________________________________________________________________________________
bnorm_91 (BatchNormalization)   (None, None, None, 2 1024        conv_91[0][0]                    
__________________________________________________________________________________________________
leaky_91 (LeakyReLU)            (None, None, None, 2 0           bnorm_91[0][0]                   
__________________________________________________________________________________________________
conv_96 (Conv2D)                (None, None, None, 1 32768       leaky_91[0][0]                   
__________________________________________________________________________________________________
bnorm_96 (BatchNormalization)   (None, None, None, 1 512         conv_96[0][0]                    
__________________________________________________________________________________________________
leaky_96 (LeakyReLU)            (None, None, None, 1 0           bnorm_96[0][0]                   
__________________________________________________________________________________________________
up_sampling2d_1 (UpSampling2D)  (None, None, None, 1 0           leaky_96[0][0]                   
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, None, None, 3 0           up_sampling2d_1[0][0]            
                                                                 add_10[0][0]                     
__________________________________________________________________________________________________
conv_99 (Conv2D)                (None, None, None, 1 49152       concatenate_1[0][0]              
__________________________________________________________________________________________________
bnorm_99 (BatchNormalization)   (None, None, None, 1 512         conv_99[0][0]                    
__________________________________________________________________________________________________
leaky_99 (LeakyReLU)            (None, None, None, 1 0           bnorm_99[0][0]                   
__________________________________________________________________________________________________
conv_100 (Conv2D)               (None, None, None, 2 294912      leaky_99[0][0]                   
__________________________________________________________________________________________________
bnorm_100 (BatchNormalization)  (None, None, None, 2 1024        conv_100[0][0]                   
__________________________________________________________________________________________________
leaky_100 (LeakyReLU)           (None, None, None, 2 0           bnorm_100[0][0]                  
__________________________________________________________________________________________________
conv_101 (Conv2D)               (None, None, None, 1 32768       leaky_100[0][0]                  
__________________________________________________________________________________________________
bnorm_101 (BatchNormalization)  (None, None, None, 1 512         conv_101[0][0]                   
__________________________________________________________________________________________________
leaky_101 (LeakyReLU)           (None, None, None, 1 0           bnorm_101[0][0]                  
__________________________________________________________________________________________________
conv_102 (Conv2D)               (None, None, None, 2 294912      leaky_101[0][0]                  
__________________________________________________________________________________________________
bnorm_102 (BatchNormalization)  (None, None, None, 2 1024        conv_102[0][0]                   
__________________________________________________________________________________________________
leaky_102 (LeakyReLU)           (None, None, None, 2 0           bnorm_102[0][0]                  
__________________________________________________________________________________________________
conv_103 (Conv2D)               (None, None, None, 1 32768       leaky_102[0][0]                  
__________________________________________________________________________________________________
bnorm_103 (BatchNormalization)  (None, None, None, 1 512         conv_103[0][0]                   
__________________________________________________________________________________________________
leaky_103 (LeakyReLU)           (None, None, None, 1 0           bnorm_103[0][0]                  
__________________________________________________________________________________________________
conv_80 (Conv2D)                (None, None, None, 1 4718592     leaky_79[0][0]                   
__________________________________________________________________________________________________
conv_92 (Conv2D)                (None, None, None, 5 1179648     leaky_91[0][0]                   
__________________________________________________________________________________________________
conv_104 (Conv2D)               (None, None, None, 2 294912      leaky_103[0][0]                  
__________________________________________________________________________________________________
bnorm_80 (BatchNormalization)   (None, None, None, 1 4096        conv_80[0][0]                    
__________________________________________________________________________________________________
bnorm_92 (BatchNormalization)   (None, None, None, 5 2048        conv_92[0][0]                    
__________________________________________________________________________________________________
bnorm_104 (BatchNormalization)  (None, None, None, 2 1024        conv_104[0][0]                   
__________________________________________________________________________________________________
leaky_80 (LeakyReLU)            (None, None, None, 1 0           bnorm_80[0][0]                   
__________________________________________________________________________________________________
leaky_92 (LeakyReLU)            (None, None, None, 5 0           bnorm_92[0][0]                   
__________________________________________________________________________________________________
leaky_104 (LeakyReLU)           (None, None, None, 2 0           bnorm_104[0][0]                  
__________________________________________________________________________________________________
conv_81 (Conv2D)                (None, None, None, 2 261375      leaky_80[0][0]                   
__________________________________________________________________________________________________
conv_93 (Conv2D)                (None, None, None, 2 130815      leaky_92[0][0]                   
__________________________________________________________________________________________________
conv_105 (Conv2D)               (None, None, None, 2 65535       leaky_104[0][0]                  
==================================================================================================
Total params: 62,001,757
Trainable params: 61,949,149
Non-trainable params: 52,608
__________________________________________________________________________________________________

Loading the model and its configuration for our object detection.

In [5]:
# Loading Yolov3
net = cv2.dnn.readNet("yolov3.weights", "cfg/yolov3.cfg")
classes = []
# Reading the class names which we are focussed on predicting
with open("coco.names", "r") as f:
    classes = [line.strip() for line in f.readlines()]
layer_names = net.getLayerNames()
output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]
# Defining font and bounding box colour
colors = np.random.uniform(0, 255, size=(len(classes), 3))
font = cv2.FONT_HERSHEY_PLAIN

Visualizing the results

In [6]:
# Loading image
img = cv2.imread("test.jpg")
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
frame = cv2.resize(img, None, fx=0.4, fy=0.4)

height, width, channels = frame.shape

# Detecting objects
blob = cv2.dnn.blobFromImage(frame, 0.00392, (416, 416), (0, 0, 0), True, crop=False)

net.setInput(blob)
outs = net.forward(output_layers)

# Showing boxes on the images whose confidence of predicting an object is higher than a particular threshold
class_ids = []
confidences = []
boxes = []
for out in outs:
    for detection in out:
        scores = detection[5:]
        class_id = np.argmax(scores)
        confidence = scores[class_id]
        if confidence > 0.2:
            # Object detected
            center_x = int(detection[0] * width)
            center_y = int(detection[1] * height)
            w = int(detection[2] * width)
            h = int(detection[3] * height)

            # Rectangle coordinates
            x = int(center_x - w / 2)
            y = int(center_y - h / 2)

            boxes.append([x, y, w, h])
            confidences.append(float(confidence))
            class_ids.append(class_id)

indexes = cv2.dnn.NMSBoxes(boxes, confidences, 0.8, 0.3)

# Building the boxes on top of the image
for i in range(len(boxes)):
    if i in indexes:
        x, y, w, h = boxes[i]
        label = str(classes[class_ids[i]])
        confidence = confidences[i]
        color = colors[class_ids[i]]
        cv2.rectangle(frame, (x, y), (x + w, y + h), color, 2)

print("Detected object in Image")
plt.imshow(frame)
   
Detected object in Image
Out[6]:
<matplotlib.image.AxesImage at 0x7f069b2a64e0>
In [7]:
# Loading image
img = cv2.imread("test.jpg")
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
frame = cv2.resize(img, None, fx=0.4, fy=0.4)

height, width, channels = frame.shape

# Detecting objects
blob = cv2.dnn.blobFromImage(frame, 0.00392, (416, 416), (0, 0, 0), True, crop=False)

net.setInput(blob)
outs = net.forward(output_layers)

# Showing boxes on the images whose confidence of predicting an object is higher than a particular threshold
class_ids = []
confidences = []
boxes = []
for out in outs:
    for detection in out:
        scores = detection[5:]
        class_id = np.argmax(scores)
        confidence = scores[class_id]
        if confidence > 0.2:
            # Object detected
            center_x = int(detection[0] * width)
            center_y = int(detection[1] * height)
            w = int(detection[2] * width)
            h = int(detection[3] * height)

            # Rectangle coordinates
            x = int(center_x - w / 2)
            y = int(center_y - h / 2)

            boxes.append([x, y, w, h])
            confidences.append(float(confidence))
            class_ids.append(class_id)

indexes = cv2.dnn.NMSBoxes(boxes, confidences, 0.8, 0.3)

# Building the boxes on top of the image
for i in range(len(boxes)):
    if i in indexes:
        x, y, w, h = boxes[i]
        label = str(classes[class_ids[i]])
        confidence = confidences[i]
        color = colors[class_ids[i]]
        cv2.rectangle(frame, (x, y), (x + w, y + h), color, 2)

print("Detected object in Image")
plt.imshow(frame)
Detected object in Image
Out[7]:
<matplotlib.image.AxesImage at 0x7f062c0b0240>
In [8]:
# Loading image
img = cv2.imread("test.jpg")
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
frame = cv2.resize(img, None, fx=0.4, fy=0.4)

height, width, channels = frame.shape

# Detecting objects
blob = cv2.dnn.blobFromImage(frame, 0.00392, (416, 416), (0, 0, 0), True, crop=False)

net.setInput(blob)
outs = net.forward(output_layers)

# Showing boxes on the images whose confidence of predicting an object is higher than a particular threshold
class_ids = []
confidences = []
boxes = []
for out in outs:
    for detection in out:
        scores = detection[5:]
        class_id = np.argmax(scores)
        confidence = scores[class_id]
        if confidence > 0.2:
            # Object detected
            center_x = int(detection[0] * width)
            center_y = int(detection[1] * height)
            w = int(detection[2] * width)
            h = int(detection[3] * height)

            # Rectangle coordinates
            x = int(center_x - w / 2)
            y = int(center_y - h / 2)

            boxes.append([x, y, w, h])
            confidences.append(float(confidence))
            class_ids.append(class_id)

indexes = cv2.dnn.NMSBoxes(boxes, confidences, 0.8, 0.3)

# Building the boxes on top of the image
for i in range(len(boxes)):
    if i in indexes:
        x, y, w, h = boxes[i]
        label = str(classes[class_ids[i]])
        confidence = confidences[i]
        color = colors[class_ids[i]]
        cv2.rectangle(frame, (x, y), (x + w, y + h), color, 2)

print("Detected object in Image")
plt.imshow(frame)
Detected object in Image
Out[8]:
<matplotlib.image.AxesImage at 0x7f06085e0208>

Compiling the model using DeepC Compiler

In [ ]:
!deepCC model.h5
[INFO]
Reading [keras model] 'model.h5'
[SUCCESS]
Saved 'model.onnx'
[INFO]
Reading [onnx model] 'model.onnx'
[INFO]
Model info:
  ir_vesion : 5
  doc       : 
[WARNING]
[ONNX]: graph-node conv_1's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node conv_5's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node conv_12's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node conv_37's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: graph-node conv_62's attribute auto_pad has no meaningful data.
[WARNING]
[ONNX]: terminal (input/output) input_1's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) input_1's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) input_1's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_81's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_81's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_81's shape is less than 1. Changing it to 1.
WARN (GRAPH): found operator node with the same name (conv_81) as io node.
[WARNING]
[ONNX]: terminal (input/output) conv_93's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_93's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_93's shape is less than 1. Changing it to 1.
WARN (GRAPH): found operator node with the same name (conv_93) as io node.
[WARNING]
[ONNX]: terminal (input/output) conv_105's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_105's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) conv_105's shape is less than 1. Changing it to 1.
WARN (GRAPH): found operator node with the same name (conv_105) as io node.
[INFO]
Running DNNC graph sanity check ...
[SUCCESS]
Passed sanity check.
[INFO]
Writing C++ file 'model_deepC/model.cpp'
[INFO]
deepSea model files are ready in 'model_deepC/' 
[RUNNING COMMAND]
g++ -std=c++11 -O3 -fno-rtti -fno-exceptions -I. -I/opt/tljh/user/lib/python3.7/site-packages/deepC-0.13-py3.7-linux-x86_64.egg/deepC/include -isystem /opt/tljh/user/lib/python3.7/site-packages/deepC-0.13-py3.7-linux-x86_64.egg/deepC/packages/eigen-eigen-323c052e1731 model_deepC/model.cpp -o model_deepC/model.exe