Cainvas
Model Files
rainfall_prediction.h5
keras
Model
deepSea Compiled Models
rainfall_prediction.exe
deepSea
Ubuntu

Rainfall Prediction App

Credit: AITS Cainvas Community

Photo by nineteentwenone on Dribbble

This is an exploratory analysis to figure out whether we can predict rainfall just by getting different environmental parameters. This can be used in devices which are offline and has to decide whether to perform an action based on whether it is going to rain or not. Can be of great use in agricultural sector.

This is not a time series analysis of weather pattern. It is a binary classification problem of whether it is going to rain or not based on certain parameters. Although the data is such that it can be used to do a multivariate time-series analysis of rainfall, but that is not our goal here.

Data taken from here

In this one we will use DNN using Tensorflow with weights adjusted for the distribution of the lables. We will do this since our dataset is imbalanced, i.e, the number of 1 label is far less than 0 label. We will aslo use precison recall and other metrics along with accuracy

In [1]:
############# import necessary libraries ##########################
import pandas as pd
import numpy as np
import seaborn as sns
from datetime import datetime
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout
import os
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn import metrics
import sklearn
import zipfile
In [2]:
path = os.getcwd()
if not os.path.exists('archive.zip'):
    !wget -N "https://cainvas-static.s3.amazonaws.com/media/user_data/cainvas-admin/archive.zip"
if not os.path.exists('DataFiles'):
    zip_ref = zipfile.ZipFile('archive.zip', 'r')
    zip_ref.extractall('DataFiles/')

filenames = os.listdir('DataFiles')
print(filenames)
['bombay.csv', 'pune.csv', 'delhi.csv', 'hyderabad.csv', 'jaipur.csv', 'kanpur.csv', 'nagpur.csv', 'bengaluru.csv']
In [3]:
###########Create one common dataframe from all the files. We will ignore the city for now ##########
list_of_df = []
for file in filenames:
    df_temp = pd.read_csv(os.path.join('DataFiles',file))
    list_of_df.append(df_temp)
    
df = pd.concat(list_of_df, axis=0, ignore_index=True)
df.head()
Out[3]:
date_time maxtempC mintempC totalSnow_cm sunHour uvIndex uvIndex.1 moon_illumination moonrise moonset ... WindChillC WindGustKmph cloudcover humidity precipMM pressure tempC visibility winddirDegree windspeedKmph
0 2009-01-01 00:00:00 30 22 0.0 11.0 7 1 31 10:21 AM 10:20 PM ... 27 11 0 49 0.0 1012 22 10 20 10
1 2009-01-01 01:00:00 30 22 0.0 11.0 7 1 31 10:21 AM 10:20 PM ... 26 12 0 50 0.0 1012 22 10 18 11
2 2009-01-01 02:00:00 30 22 0.0 11.0 7 1 31 10:21 AM 10:20 PM ... 26 14 0 50 0.0 1012 22 10 16 12
3 2009-01-01 03:00:00 30 22 0.0 11.0 7 1 31 10:21 AM 10:20 PM ... 25 15 0 50 0.0 1012 22 10 14 13
4 2009-01-01 04:00:00 30 22 0.0 11.0 7 1 31 10:21 AM 10:20 PM ... 26 14 0 49 0.0 1013 22 10 28 12

5 rows × 25 columns

In [4]:
########## Dropping unnecessary columns. ################
df.drop(['date_time','sunrise','sunset','moonrise','moonset', 'totalSnow_cm','maxtempC','mintempC','sunHour'], axis = 1, inplace=True)
In [5]:
df['precipMM'] = df['precipMM'].apply(lambda x: 1 if x > 0 else 0)
In [6]:
####### Getting an idea about the data #############
pd.Index(df['precipMM']).value_counts()

### This shows us that the data is imbalanced
Out[6]:
0    703102
1     68354
Name: precipMM, dtype: int64
In [7]:
##Checking if there are any nulls in any columns
df.apply(lambda x: sum(x.isnull()))

###There are no nulls
Out[7]:
uvIndex              0
uvIndex.1            0
moon_illumination    0
DewPointC            0
FeelsLikeC           0
HeatIndexC           0
WindChillC           0
WindGustKmph         0
cloudcover           0
humidity             0
precipMM             0
pressure             0
tempC                0
visibility           0
winddirDegree        0
windspeedKmph        0
dtype: int64
In [8]:
sns.pairplot(df, vars=['pressure'], dropna = True, hue = 'precipMM')
Out[8]:
<seaborn.axisgrid.PairGrid at 0x7fad80aa3c88>
In [9]:
sns.pairplot(df, vars=['tempC'], dropna = True, hue = 'precipMM')
Out[9]:
<seaborn.axisgrid.PairGrid at 0x7fad078ec1d0>
In [10]:
sns.pairplot(df, vars=['windspeedKmph'], dropna = True, hue = 'precipMM')
Out[10]:
<seaborn.axisgrid.PairGrid at 0x7fad80aa3c18>
In [11]:
data = df.loc[:, df.columns != 'precipMM']
label = df['precipMM']

data = np.array(data)
label = np.array(label)

data.shape
Out[11]:
(771456, 15)
In [12]:
train_data, test_data, train_label, test_label = train_test_split(data, label, test_size = 0.1, random_state = 42, shuffle = True, stratify = label) ## We are keeping the test size to 10 percent because we will make one more split in the model.fit stage as well

print(train_data.shape)
print(train_label.shape)
print(test_data.shape)
print(test_label.shape)
(694310, 15)
(694310,)
(77146, 15)
(77146,)
In [13]:
METRICS = [
      tf.keras.metrics.BinaryAccuracy(name='accuracy'),
      tf.keras.metrics.Precision(name='precision'),
      tf.keras.metrics.Recall(name='recall'),
      tf.keras.metrics.AUC(name='auc'),
      tf.keras.metrics.AUC(name='prc', curve='PR'), # precision-recall curve
]
In [14]:
model = Sequential([
    Dense(32, input_shape = (train_data.shape[1],), activation = 'relu'),
    Dense(16, activation = 'relu'),
    Dropout(0.2),
    Dense(8, activation = 'relu'),
    Dense(1, activation = 'sigmoid'),
    
])

# model = tf.keras.Sequential()
# model.add(Dense(250, input_shape=(train_data.shape[1],), activation=tf.nn.relu))
# model.add(Dropout(0.4))
# model.add(Dense(200, activation=tf.nn.relu))
# model.add(Dropout(0.4))
# model.add(Dense(100, activation=tf.nn.relu))
# model.add(Dropout(0.3))
# model.add(Dense(50, activation=tf.nn.relu))
# model.add(Dense(1, activation=tf.nn.sigmoid))

model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 32)                512       
_________________________________________________________________
dense_1 (Dense)              (None, 16)                528       
_________________________________________________________________
dropout (Dropout)            (None, 16)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 8)                 136       
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 9         
=================================================================
Total params: 1,185
Trainable params: 1,185
Non-trainable params: 0
_________________________________________________________________
In [15]:
class_weights = sklearn.utils.class_weight.compute_class_weight('balanced', np.unique(train_label), train_label)
class_weights = {l:c for l,c in zip(np.unique(train_label), class_weights)}
print(class_weights)
{0: 0.5486092564527625, 1: 5.643053365626879}
/opt/tljh/user/lib/python3.7/site-packages/sklearn/utils/validation.py:70: FutureWarning: Pass classes=[0 1], y=[0 0 0 ... 0 0 0] as keyword args. From version 0.25 passing these as positional arguments will result in an error
  FutureWarning)
In [16]:
model.compile(optimizer='adadelta', loss='binary_crossentropy', metrics = METRICS)
In [17]:
# lr_schedule = tf.keras.callbacks.LearningRateScheduler(lambda epoch: 1e-5 * 10**(epoch/10))
In [18]:
history = model.fit(train_data, train_label, validation_split=0.3, batch_size = 100, epochs = 60, class_weight=class_weights)
Epoch 1/60
4861/4861 [==============================] - 32s 7ms/step - loss: 92.6010 - accuracy: 0.2383 - precision: 0.0891 - recall: 0.8197 - auc: 0.5006 - prc: 0.0891 - val_loss: 58.3844 - val_accuracy: 0.0878 - val_precision: 0.0878 - val_recall: 1.0000 - val_auc: 0.5000 - val_prc: 0.0878
Epoch 2/60
4861/4861 [==============================] - 32s 7ms/step - loss: 34.8016 - accuracy: 0.3986 - precision: 0.0827 - recall: 0.5706 - auc: 0.4745 - prc: 0.0837 - val_loss: 2.0910 - val_accuracy: 0.6356 - val_precision: 0.0557 - val_recall: 0.1977 - val_auc: 0.4257 - val_prc: 0.0703
Epoch 3/60
4861/4861 [==============================] - 32s 7ms/step - loss: 27.4634 - accuracy: 0.4888 - precision: 0.0826 - recall: 0.4697 - auc: 0.4776 - prc: 0.0835 - val_loss: 1.8409 - val_accuracy: 0.6461 - val_precision: 0.0664 - val_recall: 0.2322 - val_auc: 0.4628 - val_prc: 0.0757
Epoch 4/60
4861/4861 [==============================] - 29s 6ms/step - loss: 23.2885 - accuracy: 0.5072 - precision: 0.0871 - recall: 0.4788 - auc: 0.4951 - prc: 0.0866 - val_loss: 1.6940 - val_accuracy: 0.6477 - val_precision: 0.0797 - val_recall: 0.2858 - val_auc: 0.5001 - val_prc: 0.0819
Epoch 5/60
4861/4861 [==============================] - 28s 6ms/step - loss: 19.6643 - accuracy: 0.5230 - precision: 0.0894 - recall: 0.4748 - auc: 0.5082 - prc: 0.0891 - val_loss: 1.5713 - val_accuracy: 0.6489 - val_precision: 0.0926 - val_recall: 0.3408 - val_auc: 0.5359 - val_prc: 0.0886
Epoch 6/60
4861/4861 [==============================] - 28s 6ms/step - loss: 16.1886 - accuracy: 0.5466 - precision: 0.0942 - recall: 0.4754 - auc: 0.5213 - prc: 0.0923 - val_loss: 1.4117 - val_accuracy: 0.6546 - val_precision: 0.1029 - val_recall: 0.3803 - val_auc: 0.5679 - val_prc: 0.0950
Epoch 7/60
4861/4861 [==============================] - 29s 6ms/step - loss: 12.8992 - accuracy: 0.5601 - precision: 0.0986 - recall: 0.4845 - auc: 0.5287 - prc: 0.0941 - val_loss: 1.2710 - val_accuracy: 0.6606 - val_precision: 0.1099 - val_recall: 0.4039 - val_auc: 0.5925 - val_prc: 0.1002
Epoch 8/60
4861/4861 [==============================] - 28s 6ms/step - loss: 9.8713 - accuracy: 0.5747 - precision: 0.1031 - recall: 0.4908 - auc: 0.5383 - prc: 0.0960 - val_loss: 1.1514 - val_accuracy: 0.6722 - val_precision: 0.1247 - val_recall: 0.4543 - val_auc: 0.6269 - val_prc: 0.1093
Epoch 9/60
4861/4861 [==============================] - 28s 6ms/step - loss: 7.2227 - accuracy: 0.5930 - precision: 0.1065 - recall: 0.4838 - auc: 0.5446 - prc: 0.0974 - val_loss: 1.0777 - val_accuracy: 0.6748 - val_precision: 0.1322 - val_recall: 0.4861 - val_auc: 0.6365 - val_prc: 0.1125
Epoch 10/60
4861/4861 [==============================] - 29s 6ms/step - loss: 5.2168 - accuracy: 0.6082 - precision: 0.1107 - recall: 0.4838 - auc: 0.5525 - prc: 0.0998 - val_loss: 1.1146 - val_accuracy: 0.6645 - val_precision: 0.1434 - val_recall: 0.5675 - val_auc: 0.6566 - val_prc: 0.1230
Epoch 11/60
4861/4861 [==============================] - 28s 6ms/step - loss: 3.6240 - accuracy: 0.6239 - precision: 0.1186 - recall: 0.5020 - auc: 0.5735 - prc: 0.1070 - val_loss: 1.3895 - val_accuracy: 0.6386 - val_precision: 0.1487 - val_recall: 0.6596 - val_auc: 0.6768 - val_prc: 0.1388
Epoch 12/60
4861/4861 [==============================] - 28s 6ms/step - loss: 2.6825 - accuracy: 0.6376 - precision: 0.1263 - recall: 0.5193 - auc: 0.6018 - prc: 0.1136 - val_loss: 1.2421 - val_accuracy: 0.6617 - val_precision: 0.1555 - val_recall: 0.6443 - val_auc: 0.6895 - val_prc: 0.1377
Epoch 13/60
4861/4861 [==============================] - 28s 6ms/step - loss: 2.0337 - accuracy: 0.6514 - precision: 0.1275 - recall: 0.4996 - auc: 0.6216 - prc: 0.1168 - val_loss: 0.9800 - val_accuracy: 0.6735 - val_precision: 0.1547 - val_recall: 0.6094 - val_auc: 0.7073 - val_prc: 0.1510
Epoch 14/60
4861/4861 [==============================] - 28s 6ms/step - loss: 1.4511 - accuracy: 0.6501 - precision: 0.1419 - recall: 0.5808 - auc: 0.6581 - prc: 0.1315 - val_loss: 0.6368 - val_accuracy: 0.6708 - val_precision: 0.1806 - val_recall: 0.7779 - val_auc: 0.7647 - val_prc: 0.1870
Epoch 15/60
4861/4861 [==============================] - 28s 6ms/step - loss: 1.1066 - accuracy: 0.6184 - precision: 0.1475 - recall: 0.6882 - auc: 0.6784 - prc: 0.1342 - val_loss: 0.5292 - val_accuracy: 0.6549 - val_precision: 0.1825 - val_recall: 0.8427 - val_auc: 0.7824 - val_prc: 0.2063
Epoch 16/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.9744 - accuracy: 0.5858 - precision: 0.1479 - recall: 0.7674 - auc: 0.6854 - prc: 0.1327 - val_loss: 0.5068 - val_accuracy: 0.6437 - val_precision: 0.1817 - val_recall: 0.8731 - val_auc: 0.7874 - val_prc: 0.2171
Epoch 17/60
4861/4861 [==============================] - 27s 6ms/step - loss: 0.9145 - accuracy: 0.5688 - precision: 0.1481 - recall: 0.8096 - auc: 0.6908 - prc: 0.1340 - val_loss: 0.4787 - val_accuracy: 0.6475 - val_precision: 0.1850 - val_recall: 0.8858 - val_auc: 0.7949 - val_prc: 0.2260
Epoch 18/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.8721 - accuracy: 0.5556 - precision: 0.1478 - recall: 0.8387 - auc: 0.6959 - prc: 0.1363 - val_loss: 0.4688 - val_accuracy: 0.6471 - val_precision: 0.1863 - val_recall: 0.8967 - val_auc: 0.8006 - val_prc: 0.2315
Epoch 19/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.8415 - accuracy: 0.5522 - precision: 0.1480 - recall: 0.8482 - auc: 0.7018 - prc: 0.1393 - val_loss: 0.4646 - val_accuracy: 0.6437 - val_precision: 0.1860 - val_recall: 0.9062 - val_auc: 0.8036 - val_prc: 0.2340
Epoch 20/60
4861/4861 [==============================] - 27s 6ms/step - loss: 0.8020 - accuracy: 0.5493 - precision: 0.1484 - recall: 0.8582 - auc: 0.7123 - prc: 0.1447 - val_loss: 0.4540 - val_accuracy: 0.6450 - val_precision: 0.1871 - val_recall: 0.9100 - val_auc: 0.8079 - val_prc: 0.2384
Epoch 21/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.7748 - accuracy: 0.5472 - precision: 0.1485 - recall: 0.8639 - auc: 0.7182 - prc: 0.1476 - val_loss: 0.4352 - val_accuracy: 0.6549 - val_precision: 0.1912 - val_recall: 0.9074 - val_auc: 0.8154 - val_prc: 0.2472
Epoch 22/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.7445 - accuracy: 0.5473 - precision: 0.1488 - recall: 0.8663 - auc: 0.7272 - prc: 0.1524 - val_loss: 0.4372 - val_accuracy: 0.6500 - val_precision: 0.1896 - val_recall: 0.9125 - val_auc: 0.8216 - val_prc: 0.2551
Epoch 23/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.7116 - accuracy: 0.5500 - precision: 0.1502 - recall: 0.8717 - auc: 0.7391 - prc: 0.1593 - val_loss: 0.4366 - val_accuracy: 0.6498 - val_precision: 0.1898 - val_recall: 0.9148 - val_auc: 0.8285 - val_prc: 0.2647
Epoch 24/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.6769 - accuracy: 0.5587 - precision: 0.1534 - recall: 0.8761 - auc: 0.7497 - prc: 0.1661 - val_loss: 0.4265 - val_accuracy: 0.6550 - val_precision: 0.1921 - val_recall: 0.9144 - val_auc: 0.8338 - val_prc: 0.2729
Epoch 25/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.6462 - accuracy: 0.5675 - precision: 0.1569 - recall: 0.8831 - auc: 0.7593 - prc: 0.1720 - val_loss: 0.4292 - val_accuracy: 0.6535 - val_precision: 0.1917 - val_recall: 0.9166 - val_auc: 0.8368 - val_prc: 0.2775
Epoch 26/60
4861/4861 [==============================] - 27s 6ms/step - loss: 0.6194 - accuracy: 0.5747 - precision: 0.1595 - recall: 0.8857 - auc: 0.7668 - prc: 0.1770 - val_loss: 0.4271 - val_accuracy: 0.6556 - val_precision: 0.1925 - val_recall: 0.9156 - val_auc: 0.8379 - val_prc: 0.2792
Epoch 27/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5988 - accuracy: 0.5836 - precision: 0.1624 - recall: 0.8850 - auc: 0.7704 - prc: 0.1798 - val_loss: 0.4267 - val_accuracy: 0.6573 - val_precision: 0.1931 - val_recall: 0.9133 - val_auc: 0.8379 - val_prc: 0.2798
Epoch 28/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5781 - accuracy: 0.5973 - precision: 0.1673 - recall: 0.8869 - auc: 0.7756 - prc: 0.1859 - val_loss: 0.4414 - val_accuracy: 0.6492 - val_precision: 0.1904 - val_recall: 0.9214 - val_auc: 0.8373 - val_prc: 0.2762
Epoch 29/60
4861/4861 [==============================] - 27s 6ms/step - loss: 0.5638 - accuracy: 0.6035 - precision: 0.1696 - recall: 0.8871 - auc: 0.7764 - prc: 0.1880 - val_loss: 0.4408 - val_accuracy: 0.6496 - val_precision: 0.1907 - val_recall: 0.9224 - val_auc: 0.8361 - val_prc: 0.2680
Epoch 30/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5502 - accuracy: 0.6086 - precision: 0.1725 - recall: 0.8949 - auc: 0.7808 - prc: 0.1935 - val_loss: 0.4457 - val_accuracy: 0.6478 - val_precision: 0.1903 - val_recall: 0.9255 - val_auc: 0.8356 - val_prc: 0.2641
Epoch 31/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5441 - accuracy: 0.6097 - precision: 0.1734 - recall: 0.8990 - auc: 0.7846 - prc: 0.2013 - val_loss: 0.4471 - val_accuracy: 0.6488 - val_precision: 0.1908 - val_recall: 0.9259 - val_auc: 0.8351 - val_prc: 0.2614
Epoch 32/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5381 - accuracy: 0.6137 - precision: 0.1754 - recall: 0.9032 - auc: 0.7873 - prc: 0.2053 - val_loss: 0.4473 - val_accuracy: 0.6520 - val_precision: 0.1920 - val_recall: 0.9243 - val_auc: 0.8357 - val_prc: 0.2616
Epoch 33/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5369 - accuracy: 0.6183 - precision: 0.1772 - recall: 0.9027 - auc: 0.7898 - prc: 0.2096 - val_loss: 0.4490 - val_accuracy: 0.6529 - val_precision: 0.1923 - val_recall: 0.9235 - val_auc: 0.8356 - val_prc: 0.2612
Epoch 34/60
4861/4861 [==============================] - 27s 6ms/step - loss: 0.5315 - accuracy: 0.6205 - precision: 0.1783 - recall: 0.9047 - auc: 0.7926 - prc: 0.2126 - val_loss: 0.4473 - val_accuracy: 0.6575 - val_precision: 0.1940 - val_recall: 0.9203 - val_auc: 0.8363 - val_prc: 0.2622
Epoch 35/60
4861/4861 [==============================] - 27s 6ms/step - loss: 0.5309 - accuracy: 0.6257 - precision: 0.1799 - recall: 0.9013 - auc: 0.7928 - prc: 0.2130 - val_loss: 0.4487 - val_accuracy: 0.6587 - val_precision: 0.1945 - val_recall: 0.9195 - val_auc: 0.8359 - val_prc: 0.2606
Epoch 36/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5276 - accuracy: 0.6288 - precision: 0.1812 - recall: 0.9014 - auc: 0.7950 - prc: 0.2140 - val_loss: 0.4511 - val_accuracy: 0.6604 - val_precision: 0.1951 - val_recall: 0.9181 - val_auc: 0.8364 - val_prc: 0.2612
Epoch 37/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5255 - accuracy: 0.6339 - precision: 0.1832 - recall: 0.9004 - auc: 0.7973 - prc: 0.2179 - val_loss: 0.4445 - val_accuracy: 0.6682 - val_precision: 0.1982 - val_recall: 0.9129 - val_auc: 0.8374 - val_prc: 0.2617
Epoch 38/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5249 - accuracy: 0.6382 - precision: 0.1847 - recall: 0.8981 - auc: 0.7981 - prc: 0.2186 - val_loss: 0.4430 - val_accuracy: 0.6715 - val_precision: 0.1993 - val_recall: 0.9091 - val_auc: 0.8378 - val_prc: 0.2628
Epoch 39/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5236 - accuracy: 0.6402 - precision: 0.1856 - recall: 0.8986 - auc: 0.7990 - prc: 0.2188 - val_loss: 0.4511 - val_accuracy: 0.6674 - val_precision: 0.1978 - val_recall: 0.9129 - val_auc: 0.8382 - val_prc: 0.2639
Epoch 40/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5221 - accuracy: 0.6415 - precision: 0.1857 - recall: 0.8952 - auc: 0.7994 - prc: 0.2198 - val_loss: 0.4449 - val_accuracy: 0.6739 - val_precision: 0.2003 - val_recall: 0.9078 - val_auc: 0.8388 - val_prc: 0.2643
Epoch 41/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5204 - accuracy: 0.6450 - precision: 0.1873 - recall: 0.8956 - auc: 0.8019 - prc: 0.2230 - val_loss: 0.4390 - val_accuracy: 0.6801 - val_precision: 0.2027 - val_recall: 0.9019 - val_auc: 0.8396 - val_prc: 0.2652
Epoch 42/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5175 - accuracy: 0.6484 - precision: 0.1888 - recall: 0.8952 - auc: 0.8039 - prc: 0.2263 - val_loss: 0.4391 - val_accuracy: 0.6819 - val_precision: 0.2037 - val_recall: 0.9024 - val_auc: 0.8405 - val_prc: 0.2668
Epoch 43/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5181 - accuracy: 0.6505 - precision: 0.1892 - recall: 0.8913 - auc: 0.8032 - prc: 0.2250 - val_loss: 0.4457 - val_accuracy: 0.6783 - val_precision: 0.2022 - val_recall: 0.9053 - val_auc: 0.8405 - val_prc: 0.2672
Epoch 44/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5161 - accuracy: 0.6512 - precision: 0.1898 - recall: 0.8930 - auc: 0.8051 - prc: 0.2275 - val_loss: 0.4425 - val_accuracy: 0.6826 - val_precision: 0.2040 - val_recall: 0.9021 - val_auc: 0.8413 - val_prc: 0.2685
Epoch 45/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5152 - accuracy: 0.6543 - precision: 0.1908 - recall: 0.8907 - auc: 0.8058 - prc: 0.2287 - val_loss: 0.4482 - val_accuracy: 0.6791 - val_precision: 0.2027 - val_recall: 0.9059 - val_auc: 0.8412 - val_prc: 0.2684
Epoch 46/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5146 - accuracy: 0.6565 - precision: 0.1919 - recall: 0.8911 - auc: 0.8060 - prc: 0.2282 - val_loss: 0.4442 - val_accuracy: 0.6834 - val_precision: 0.2047 - val_recall: 0.9037 - val_auc: 0.8420 - val_prc: 0.2690
Epoch 47/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5127 - accuracy: 0.6575 - precision: 0.1924 - recall: 0.8916 - auc: 0.8080 - prc: 0.2305 - val_loss: 0.4359 - val_accuracy: 0.6909 - val_precision: 0.2076 - val_recall: 0.8952 - val_auc: 0.8426 - val_prc: 0.2701
Epoch 48/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5113 - accuracy: 0.6589 - precision: 0.1931 - recall: 0.8918 - auc: 0.8096 - prc: 0.2321 - val_loss: 0.4379 - val_accuracy: 0.6909 - val_precision: 0.2077 - val_recall: 0.8960 - val_auc: 0.8433 - val_prc: 0.2712
Epoch 49/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5114 - accuracy: 0.6593 - precision: 0.1932 - recall: 0.8907 - auc: 0.8088 - prc: 0.2314 - val_loss: 0.4378 - val_accuracy: 0.6914 - val_precision: 0.2081 - val_recall: 0.8968 - val_auc: 0.8438 - val_prc: 0.2719
Epoch 50/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5100 - accuracy: 0.6624 - precision: 0.1946 - recall: 0.8903 - auc: 0.8093 - prc: 0.2310 - val_loss: 0.4424 - val_accuracy: 0.6892 - val_precision: 0.2073 - val_recall: 0.9004 - val_auc: 0.8443 - val_prc: 0.2726
Epoch 51/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5092 - accuracy: 0.6628 - precision: 0.1948 - recall: 0.8909 - auc: 0.8103 - prc: 0.2326 - val_loss: 0.4384 - val_accuracy: 0.6924 - val_precision: 0.2089 - val_recall: 0.8987 - val_auc: 0.8447 - val_prc: 0.2724
Epoch 52/60
4861/4861 [==============================] - 30s 6ms/step - loss: 0.5090 - accuracy: 0.6632 - precision: 0.1951 - recall: 0.8912 - auc: 0.8105 - prc: 0.2334 - val_loss: 0.4344 - val_accuracy: 0.6964 - val_precision: 0.2108 - val_recall: 0.8964 - val_auc: 0.8455 - val_prc: 0.2732
Epoch 53/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5059 - accuracy: 0.6659 - precision: 0.1965 - recall: 0.8915 - auc: 0.8130 - prc: 0.2370 - val_loss: 0.4363 - val_accuracy: 0.6956 - val_precision: 0.2106 - val_recall: 0.8986 - val_auc: 0.8462 - val_prc: 0.2750
Epoch 54/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5062 - accuracy: 0.6664 - precision: 0.1968 - recall: 0.8922 - auc: 0.8132 - prc: 0.2365 - val_loss: 0.4372 - val_accuracy: 0.6957 - val_precision: 0.2107 - val_recall: 0.8986 - val_auc: 0.8460 - val_prc: 0.2741
Epoch 55/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5053 - accuracy: 0.6664 - precision: 0.1964 - recall: 0.8896 - auc: 0.8131 - prc: 0.2357 - val_loss: 0.4422 - val_accuracy: 0.6927 - val_precision: 0.2095 - val_recall: 0.9020 - val_auc: 0.8461 - val_prc: 0.2738
Epoch 56/60
4861/4861 [==============================] - 30s 6ms/step - loss: 0.5044 - accuracy: 0.6683 - precision: 0.1978 - recall: 0.8927 - auc: 0.8140 - prc: 0.2365 - val_loss: 0.4368 - val_accuracy: 0.6971 - val_precision: 0.2115 - val_recall: 0.8987 - val_auc: 0.8467 - val_prc: 0.2748
Epoch 57/60
4861/4861 [==============================] - 28s 6ms/step - loss: 0.5039 - accuracy: 0.6684 - precision: 0.1977 - recall: 0.8918 - auc: 0.8144 - prc: 0.2369 - val_loss: 0.4380 - val_accuracy: 0.6974 - val_precision: 0.2118 - val_recall: 0.8997 - val_auc: 0.8472 - val_prc: 0.2756
Epoch 58/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5031 - accuracy: 0.6698 - precision: 0.1983 - recall: 0.8912 - auc: 0.8140 - prc: 0.2365 - val_loss: 0.4382 - val_accuracy: 0.6979 - val_precision: 0.2122 - val_recall: 0.9001 - val_auc: 0.8475 - val_prc: 0.2751
Epoch 59/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5002 - accuracy: 0.6710 - precision: 0.1992 - recall: 0.8938 - auc: 0.8164 - prc: 0.2396 - val_loss: 0.4346 - val_accuracy: 0.7000 - val_precision: 0.2133 - val_recall: 0.8998 - val_auc: 0.8481 - val_prc: 0.2761
Epoch 60/60
4861/4861 [==============================] - 29s 6ms/step - loss: 0.5020 - accuracy: 0.6722 - precision: 0.1998 - recall: 0.8936 - auc: 0.8161 - prc: 0.2390 - val_loss: 0.4338 - val_accuracy: 0.7021 - val_precision: 0.2143 - val_recall: 0.8975 - val_auc: 0.8486 - val_prc: 0.2770
In [19]:
model.optimizer.get_config()
Out[19]:
{'name': 'Adadelta',
 'learning_rate': 0.001,
 'decay': 0.0,
 'rho': 0.95,
 'epsilon': 1e-07}
In [20]:
# plt.semilogx(history.history['lr'],history.history['loss'])
# plt.axis([1e-5, 1e-2, 0, 30])
# plt.show()
In [21]:
print(history.history.keys())

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
dict_keys(['loss', 'accuracy', 'precision', 'recall', 'auc', 'prc', 'val_loss', 'val_accuracy', 'val_precision', 'val_recall', 'val_auc', 'val_prc'])
Out[21]:
[<matplotlib.lines.Line2D at 0x7facfc220d30>]
In [22]:
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.show()
plt.plot(history.history['recall'])
plt.plot(history.history['val_recall'])
Out[22]:
[<matplotlib.lines.Line2D at 0x7facfc1b30f0>]
In [23]:
expected_y  = test_label
predicted_y = model.predict_classes(test_data)

print(metrics.classification_report(expected_y, predicted_y))
print(metrics.confusion_matrix(expected_y, predicted_y))
WARNING:tensorflow:From <ipython-input-23-1ae7617ec0ed>:2: Sequential.predict_classes (from tensorflow.python.keras.engine.sequential) is deprecated and will be removed after 2021-01-01.
Instructions for updating:
Please use instead:* `np.argmax(model.predict(x), axis=-1)`,   if your model does multi-class classification   (e.g. if it uses a `softmax` last-layer activation).* `(model.predict(x) > 0.5).astype("int32")`,   if your model does binary classification   (e.g. if it uses a `sigmoid` last-layer activation).
              precision    recall  f1-score   support

           0       0.99      0.68      0.81     70311
           1       0.22      0.90      0.35      6835

    accuracy                           0.70     77146
   macro avg       0.60      0.79      0.58     77146
weighted avg       0.92      0.70      0.76     77146

[[47851 22460]
 [  653  6182]]
In [24]:
model.save("rainfall_prediction.h5")

deepCC

In [26]:
!deepCC rainfall_prediction.h5
[INFO]
Reading [keras model] 'rainfall_prediction.h5'
[SUCCESS]
Saved 'rainfall_prediction_deepC/rainfall_prediction.onnx'
[INFO]
Reading [onnx model] 'rainfall_prediction_deepC/rainfall_prediction.onnx'
[INFO]
Model info:
  ir_vesion : 4
  doc       : 
[WARNING]
[ONNX]: terminal (input/output) dense_input's shape is less than 1. Changing it to 1.
[WARNING]
[ONNX]: terminal (input/output) dense_3's shape is less than 1. Changing it to 1.
WARN (GRAPH): found operator node with the same name (dense_3) as io node.
[INFO]
Running DNNC graph sanity check ...
[SUCCESS]
Passed sanity check.
[INFO]
Writing C++ file 'rainfall_prediction_deepC/rainfall_prediction.cpp'
[INFO]
deepSea model files are ready in 'rainfall_prediction_deepC/' 
[RUNNING COMMAND]
g++ -std=c++11 -O3 -fno-rtti -fno-exceptions -I. -I/opt/tljh/user/lib/python3.7/site-packages/deepC-0.13-py3.7-linux-x86_64.egg/deepC/include -isystem /opt/tljh/user/lib/python3.7/site-packages/deepC-0.13-py3.7-linux-x86_64.egg/deepC/packages/eigen-eigen-323c052e1731 "rainfall_prediction_deepC/rainfall_prediction.cpp" -D_AITS_MAIN -o "rainfall_prediction_deepC/rainfall_prediction.exe"
[RUNNING COMMAND]
size "rainfall_prediction_deepC/rainfall_prediction.exe"
   text	   data	    bss	    dec	    hex	filename
 125011	   2968	    760	 128739	  1f6e3	rainfall_prediction_deepC/rainfall_prediction.exe
[SUCCESS]
Saved model as executable "rainfall_prediction_deepC/rainfall_prediction.exe"