Traffic Signs Classification Using Convolution Neural Networks (CNN)

Whenever you heard about image classification or computer vision you must have heard the word CNN or ConvNets (Convolution Neural Networks). Now you might ask why CNN and not ANN? It is because CNN has a very good ability when it comes to image classification in Neural Networks. Simple Deep Neural Nets cannot perform well in image classification due to their own limitations like one of the biggest limitations is that they require huge computational power to work with images as each pixel of the image is fed as an input so this method is computationally expensive and if you don’t use the above method you should have to manually extract important image features and fed into the NN which is a quite unrealistic approach. So Convnets have made our life easier.

Now in this blog, we are going to see how we can classify traffic signs using CNN and also see its code. Now I will be focusing much on code (and less on concept building) and will not be reinventing the wheel. I will assume that you have some basic knowledge and understanding about CNN and that basic knowledge is enough to go through this blog. We will move step by step and understand the code.

STEPS:

1. Explore and visualize the dataset.
2. Preprocessing the dataset.
3. Augment the dataset.
4. Developing, Training, and Validating the CNN model.
5. Testing the model.
6. Testing on an image got from the web.

Now let’s dive deeper into each of these steps and learn the process better.

1-Explore and visualize dataset:

The dataset I used is a German traffic signs dataset which is available on bitbucket. It’s a small dataset almost around 153 MB so you can download it easily. The data consists of three pickle files i.e., training, validation, and testing, and one CSV file that contains metadata i.e., Numbers and names of classes. The details of the dataset are given below.

• The training set contains 34799 images.
• The validation set contains 4410 images.
• The test set contains 12630 images.
• All images are 32 x 32 x 3 (RGB color channel)
• And CSV contains 43 classes along with their names.

(Imported) Modules and (all the)code used in this project will be available in the code(GitHub link below(at the end)). Firstly we will have a look at imported libraries and modules.

There is only one line code to import the dataset. Here in my case, I used the Colab notebook you can use any available one of your choice

Now since we have pickle files, we first have to unpickle them. So below is the code to unpickle them and separate them as train, validate and test set. You can also print their(datasets) shapes to get the number of images and their dimensions e.g., if you try to print training dataset shape you will get something like this (34799, 32, 32, 3) (34799,) which means it has 34799 images each with 32 x 32 x 3 dimension. And 34799 labels i.e., one label against each image.

Now we have our dataset in usable form we will plot some of its images to understand it better. The code for the image grid is given below.

Now, what the above code will do is, basically plots 5 images of each class randomly for all 43 classes. The first loop iterates over columns while the second loop iterate over rows.

The output of the above code will look like this.

The image shows only four classes but if you run the code you will see images for all 43 classes. Since we have seen the dataset plotting, now we plot the distribution of images against each class to see that is our dataset is good i.e., balanced, or needs some effort to become a good one.

Code for plotting is given here.

The output will look like this.

So we can see our data is unbalanced for example, contain less than 250 images for class 0 but more than 2000 for the 2nd class so the model trained on this dataset will be biased toward classes with more images especially when it is unsure about prediction. To overcome this problem we will use data augmentation which is step 3 and I will explain data augmentation there in detail.

2- Preprocessing the dataset:

Now in the first step, we convert our image to grayscale because in traffic sign classification color is not an important feature as most of the signs have the same color so the color does not play an important role in image classification. The second step is to equalize(scatter) brightness all over the image instead of over particular pixels. In the third step, we normalize the image to get the pixels in a defined range i.e. 0 to 1. Here we use the map function which iterates over each image, applies preprocess function on each image, and then returns it which we get in a list and convert it to a NumPy array(for our ease).

Since gray scaling the image covert image into 2D so we must have to reshape the image to 32 x 32 x 1 because CNN requires the exact dimensions of the image to choose the filter(kernel) dimensions.

Here we also one hot encoded the labels as it is much easier to deal with one hot encoded label in our case.

Now if you display a single preprocessed image it will look something like this.

3-Augmenting the data:

As we discussed above our dataset is biased i.e more number of examples for one class and less for another so now we are going to tackle this problem using the data augmentation technique. Data augmentation simply means adding more variety to data, adding more sample images using augmentation technique i.e. rotation, shear, zoom, etc. In this project, I used the Keras built-in function ImageDataGenerator(). The code of data generator is given below.

The width_shift_range specifies the amount of horizontal sliding that can be done. The height_shift_range specifies the amount of vertical sliding that can be done. In both cases, I set it to 0.1 which means 10% of total width or height. Zoom_range specifies the amount of zoom in and zoom out here zoom out is calculated as 1-value and zoom in as 1+value.In the above case value is 0.2 so zoom out is 0.8 while zoom in is 1.2. rotation_range specifies the amount of rotation allowed in this case it is 0.1 i.e., 10%, and shear range specifies the amount of the shear that is allowed. And at the last data.fit() is a function that helps the image generator to learn statistics about the data to create a batch of images. In data.flow we set the batch size to 20 so the generator will create 20 images per batch.

4- Developing, Training and Validating the CNN model:

Now we will create a CNN model and train it. The code for the model is given below.

In this model, I used a total of 4 Convolution layers and 2 pooling layers 1 flatten and 2 dense (fully connected) layers. Now in the convolution layer first argument is the number of filters(kernels) here in this case it is 120 in the first layer the second argument is kernel matrix size here in the first layer it is 5 x 5 the third argument is image dimension here it is 32 x 32 x 1 (a grayscale image with depth of 1) and last argument is activation function here it is relu. In pooling layers, kernel size is 2 x 2. The first fully connected layer has 500 nodes and the second has 43 which is equal to the number of classes. In the last(output) layer we use the SoftMax activation function which is widely used in NN where we have to predict more than 2 classes (multiclass classification). And then we used Adam optimizer with learning rate 0.001 with loss function categorical_crossentropy which is used for multiclass classification and our metric for measuring performance is Accuracy. After that, we fit the model (model.fit ()) on the training dataset and validate it on the validation set we have a total of 10 epochs in which each epoch has almost 695 steps per epoch with a batch size of 50 per step.

Now if we plot the model accuracy and loss for training and validation set it will look something like this.

5-Testing the model:

The code for the testing model on the test set is.

Now this will give an accuracy of 95.6 which is quite good.

6-Testing on an image got from the web:

Here is the code for testing the model. Here we simply get an image from the web and get a prediction from our model and our model predicted it 14 which is right.

So now you have learned how to make a traffic sign classification model using CNN with pretty good accuracy now you can use this in your various project for example in self-driving car projects to classify traffic signs and move accordingly. A clap will surely motivate me to write more blogs like this so your appreciation will be helpful for me.

GitHub code link.