Jurnal
Teknologi

Full Paper

TRAFFIC SIGN DETECTION AND RECOGNITION: REVIEW
AND ANALYSIS
Nursabillilah Mohd Ali*, Mohd Safirin Karis, Amar Faiz Zainal Abidin,
Bahzifadhli Bakri, Ezreen Farina Shair, Nur Rafiqah Abdul Razif
Faculty of Electrical Engineering, Universiti Teknikal Malaysia
Melaka, Malaysia
Graphical abstract

Article history
Received
15 May 2015
Received in revised form
1 July 2015
Accepted
11 August 2015

*Corresponding author nursabillilah@utem.edu Abstract
Over the years, traffic sign detection and recognition systems gives extra value to driver assistance when driving, leading to more user-friendly driving experience and much improved safety for passengers. As part of Advanced Driving Systems (ADAS) one can be benefitted by using this system especially with driving incapacities by alerting and aid them about the existence of traffic signs to minimized unwanted circumstances during driving such as fatigue, poor sight and adverse weather conditions. Though a various number of traffic sign detection systems have been revised in literature; the need of design with a robust algorithm still remains open for further research. This paper purposes to design a system capable of performing traffic sign detection while considering variations of challenges such as color illumination, computational difficulty and functional constraints existed. Traffic sign detection is divided into three main parts namely; Pre-processing, Color segmentation and Thresholding. The color segmentation method is vital as it presents a detailed investigation of vision based color spaces in this case RGB, HSV and CMYK considering varying illumination conditions under different environments.
This paper further highlights possible improvements to the proposed approaches for traffic sign detection.
Keywords: Traffic sign, detection and recognition, HSV, RGB, Bhattacharyya
Coefficient
© 2015 Penerbit UTM Press. All rights reserved

1.0 INTRODUCTION
Lately, the area of traffic sign detection and recognition has caught the attention of many researchers over the past decades. The significant of this trend lies mainly on the vast amount of accidents occur throughout each year all over the globe. In addition, the main cause of these to happen is due to driver’s lack of ability to interpret and process all the visual information they received while driving [1]. Apart from that, the significant increased amount of dangers on the traffic causing the traffic signs to become more important and demanding to traffic users. There are a lot of new traffic signs that have been introduced according to the necessity to aid traffic users when driving. Those who drive vehicles need to learn to

recognize all traffic signs for traffic safety reason. For instance drivers require to process the knowledge of cyclist signs, pedestrian signs, obligatory signs and advisory signs and unawareness of any of these sign can cause possible accident hazards.
Often it is said that the increasing number of vehicles and drivers are associated with total number of traffic accidents occur. The statistics that are recorded from
2005 to 2009 from Department of Statistics Malaysia shows that traffic accidents in Malaysia has increase to
21 percent over the 5 years period which is in 2009 there were 397,194 traffic accidents recorded when compared to 328,268 in 2005 [2].
In technique done in [3] the traffic sign detection is carried out by using the RGB color space along with pattern matching and edge detection used for shape

77:20 (2015) 107–113 | www.jurnalteknologi.utm.my | eISSN 2180–3722 |

108

Nursabillilah et al. / Jurnal Teknologi (Sciences & Engineering) 77:20 (2015) 107–113

evaluation. The color based segmentation is carried after a preprocessing stage which consists of light source identification and chromatic of the image. A

comparison technique made using color based technique done in [4].
Table 1 shows taxonomy of traffic sign detection approaches. Table 1 A Taxonomy of traffic sign detection approaches

Reference
No.

Color Segmentation
Technique

Color Segmentation Method

[3]

RGB

Spectral composition

Images

[4]

RGB

Images

[5]

HSV

Pattern Matching and Edge
Detection
Fuzzy Shape Recognizer

[6]

RGB

Video Stream

[7]

RGB and HSV

Shape features with Hopfield neural network -

[8]

RGB

Background Shape Histogram

Images

[9]

RGB

Gaussian Filter

Images

[10]

RGB and HSV

Blob detection and Median Filter

Video Stream

[11]

RGB

Laplacian of Gaussian (LoG)

Images

[12]

RGB(Greyscale)

Edge Detection

Images

2.0 EXPERIMENTAL SETUP
Traffic Signs give vital information to the users especially for drivers to aid them during driving experience. The significance of these signs very much dependent on their colors and its contents. Primarily traffic signs are colors from Red, Blue, Green, Brown,
Yellow, Black or White which representing various information (shown in Table 2). For example red for obligatory signs and blue for advisory signs. In addition

Meanings

Blue

Advisory Signs/Information

Red

Prohibition/Danger

Yellow

Construction/Traffic Works

Brown

Tourist Attraction/Viewable
Places

Green

Directions

Images

Images

to that, color plays an important initial role in a typical traffic sign detection task before being handed to the recognition task. Due to varying lighting and weather conditions, the segmentation of traffic signs using color information in outdoor environment is a significantly challenging task for the designer.
Nevertheless a correct segmentation process performed to these signs is essential to ease the recognizing process of the contents and shape of the sign in the later stages [5].

Table 2 Significant of color as detection criteria

Color

Stream
Type

Example

109

Nursabillilah et al. / Jurnal Teknologi (Sciences & Engineering) 77:20 (2015) 107–113

2.1 Flowcharts of Traffic Sign Detection System

Figure 1 Flowchart of traffic sign detection process

2.2 Pre-processing
In the pre-processing process, a web camera is used to capture video or images frame by frame for the input of detection process. RGB images captured by a digital camera are not always appropriate to complete the desired image processing tasks.
Different video systems used to capture the images significantly affect the original or pixel intensity values.
2.3 Color Investigation
Three experiments are conducted to investigate the performance and robustness between RGB, HSV [6] and CMYK color spaces under varying illumination and environmental conditions. The first experiments is to investigate the conversion performance between
RGB to HSV color spaces and RGB to CMYK color space in order to determine the most reliable color space to use in traffic sign detection.
Secondly, an experiment to determine the best threshold ranges among individual color spaces that is RGB, HSV and CMYK when used with segmenting

red, green and blue signs. Third experiment is about model matching that is performed between source histogram and candidate histogram using Bhattacharyya Coefficient. The procedure for the segmentation process of traffic signs is explained in the following sections.
2.4 Experiment
2.4.1 Experiment 1: Conversion of RGB to HSV and RGB to CMYK
This experiment is to investigate the relationship between RGB, HSV and CMYK color spaces. The conversion to HSV and CMYK is obtained using equation Hr1 = 0/3=0.00
Hr2 = 3/3= 1.00
Hg = 1/3=0.33
Hb=2/3=0.66

(1)
(2)
(3)
(4)

110

Nursabillilah et al. / Jurnal Teknologi (Sciences & Engineering) 77:20 (2015) 107–113

Where r=red, b=blue, g=green color in HSV color space. Seven input RGB value was entered and the conversion results is as shown in Table 3.

be performed using model matching method. This method is performed by matching the source histogram (image database) with candidate histogram (image taken by camera) to see how closely the candidate object resembles source histogram. The technique used is Bhattacharya
Coefficient. Given two histogram with source histogram frequency, cu and candidate histogram frequency, ru. Then  is given by:

2.4.2 Experiment 2: Thresholding Image
In this experiment, an image is transformed into binary image by thresholding technique. Histogram back propagation and moments technique is the type of thresholding technique used to threshold the candidate image. Image moments can be called as function of image pixel intensity. Pixel intensity is the pixel value of the candidate image. Moments that are defined here is derived with respect to the power consisting of Zeroth Moment, First Moment and etc.
Zeroth moment is also called the area of an image and basically the target is the pixel intensity or value and the x and y coordinates to find moments. Using the value of moments gained, the centroid or centre of gravity of an image is found.
Meanwhile, histogram back projection is a technique used for generating a probability image
(back projection) of the Region of Interest (ROI). A histogram of a target image is created firstly. Then, the histogram is back projected onto the image causing an image that shows the probability of a pixel belonging to the target. By using this methods, two candidates image consisting a red STOP sign and yellow prohibition sign are transformed to binary images as shown in section 4.

m

   cu ( y )ru

(5)

u 1

By designing C# algorithm inside Microsoft Visual
Studio 2013, Bhattacharya Coefficient between the source histogram and candidate histogram is compared using equation (5). Noted that the only one source image is used since this experiment is aiming to compare histogram results. The result obtained is as shown in Table 3.

3.0 RESULT AND DISCUSSION
3.1 Experiment 1: Conversion of RGB to HSV and RGB to CMYK
The results to calculate value of RGB into HSV and
CMYK are as shown in Table 3.

2.4.3 Experiment 3: Model Matching
After the image attributes on an image have been represented using Histogram, the color detection can
Table 3 A Taxonomy of traffic sign detection approaches

RGB

HSV

CMYK

R=255, G=0, B=0 Color=Red

H = 0, S = 100, V = 100

C = 0, M =100, Y = 100, K = 0

R = 0, G = 255, B = 0 Color = Green

H = 120, S = 100, V = 100

C = 100, M = 0, Y = 100, K = 0

R = 0, G = 0, B = 255 Color = Blue

H = 240, S = 100, V = 100

C = 100, M = 100, Y = 0, K = 100

R = 128, G = 128, B = 128, Color = Grey

H = 0, S = 0, V = 50

C = 0, M = 0, Y = 0, K = 100

R = 128, G = 0, B = 0, Color = Maroon

H = 0, S = 100, V = 50

C = 0, M = 100, Y = 100, K = 50

R = 128, G = 128, B = 0, Color = Olive

H = 60, S = 100, V = 50

C = 0, M = 0, Y = 100, K = 50

R = 128, G = 0, B = 128, Color = Purple

H = 300, S =100, V =50

C = 0, M = 100, Y = 0, K = 50

3.2 Experiment 2: Threshoding Image
In this experiment, an image is transformed into binary image by thresholding technique. Historgram back propagation and moments technique is the type of

thresholding technique used to threshold the candidate image.
Image moments can be called as function of image pixel intensity. Pixel intensity is the pixel value of the candidate image. Moments that are defined here is

111

Nursabillilah et al. / Jurnal Teknologi (Sciences & Engineering) 77:20 (2015) 107–113

derived with respect to the power consisting of Zeroth
Moment, First Moment, and Second Moment etc.
Above all, Zeroth Moment is also called the area of an image and basically the target is the pixel intensity or value and the x and y coordinates to find moments.
Using the value of moments gained, the centroid or center of gravity of an image is found. Meanwhile, histogram back propagation is a technique used for generating a probability image (back propagation) of the Region of Interest (ROI). A histogram of a target is created firstly. Then, the histogram is back projected onto the image causing an image that shows the probability of a pixel belonging to the target. By using this methods, two candidates image consisting a red stop sign and yellow prohibition sign are transformed to binary images as shown in Figure 1.
3.3 Experiment 3: Model Matching
After the image attributes on an image have been represented using Histogram, then the color detection can be performed using model matching method.
This method is performed by matching the source histogram (image database) with candidate histogram (image taken by camera) to see how closely the candidate object resembles source object histogram. The technique used is Bhattacharya
Coefficient which stated as equation 5.
By designing C# algorithm inside Microsoft Visual
Studio 2013, Bhattacharya Coefficient between the source histogram and candidate histogram is compared using equation 5. Noted that the only one source image is used since this experiment is aiming to compare histogram. Section 4 discussed the results obtained. Figure 2 Program to convert RGB value to HSV and CMYK and vice versa

4.2 Experiment 2: Thresholding using Histogram Back
Projection Method
The results of thresholding candidate A and candidate B are shown as in Figure 3 and Figure 4.

4.0 RESULT AND DISCUSSION
4.1 Experiment 1: Conversion of RGB to HSV and RGB to CMYK

Figure 3 A red STOP sign transformed into binary image

The results to calculate value of RGB into HSV and CMYK is a shown in Table 4. The output of C# algorithm compiled in Microsoft Visual Studio 2013 is as shown in
Figure 2.

Figure 4 A yellow prohibition sign transformed into binary image 112

Nursabillilah et al. / Jurnal Teknologi (Sciences & Engineering) 77:20 (2015) 107–113

4.3 Experiment 3: Model Matching
The results from matching two candidates’ STOP sign, in this case the color is red with the source STOP sign
(database image) is obtained as in Figure 5 and Figure
6 for STOP sign candidates 1 and STOP sign candidates
2 respectively. The Bhattacharya Coefficient are displayed in Table 4.
Table 4 Bhattacharya Coefficient for Candidates 1 and 2

Name

Preview
Candidates 1

Bhattacharya
Coefficient
0.829644

Candidates 2

0.9244339

Candidates 3

0.5003294

Figure 7 Model matching using Bhattacharya Coefficient for
Candidate 3

4.4 Analysis of Experiments
4.4.1 Experiment 1: Conversion of RGB to HSV and RGB to CMYK
It was observed that the conversion of RGB to HSV and
RGB to CMYK using this algorithm is much faster than using the conventional method of using basic color space equation [7].
4.4.2 Experiment 2: Thresholding using Histogram Back projection Method
Figure 5 Model matching using Bhattacharya Coefficinet for
Candidate 1

In this experiment, it was observed that Histogram
Back projection was very sensitive to illumination changes and the presence of other colored area such as trees, grass and overlapping sign. In Figure 4, there were two traffic signs overlapping each other in the captured image. So, the need to assign a correct
Region of Interest (ROI) to captured image is important. It is because, Region of Interest will assigned candidates image to certain area of interest and to that area will be cropped. Then, the histogram will be obtained from the cropped image consisting of color information onside the pixels before analyzed and converted into binary image.
4.4.3 Experiment 3: Model Matching

Figure 6 Model matching using Bhattacharya Coefficient for
Candidate 2

Model matching method using Bhattacharya coefficient is a robust technique used to compare an assigned source image (from database) and candidate image. It was observed that this technique is very useful in color detection in a region as it can deal with variation of lighting and illumination. As can be seen in Figure 5 to 7 and Table 4, this method can derive all the color information effectively with variation of lighting condition and illuminations. This proved as the Bhattacharya coefficient was a nonzero value. When Bhattacharya coefficient (ranging

113

Nursabillilah et al. / Jurnal Teknologi (Sciences & Engineering) 77:20 (2015) 107–113

from 0 to 1) has a value in it, it can be assumed that the ROI is within that particular color space.

References
[1]

4.0 CONCLUSION
This paper presented ten contributions to the existing state-of-the-art approaches in Traffic Sign Detection and Recognition system. The experiments were performed according to the color behavior in conjunction to the change in illumination and with various environmental conditions. The experimental section provided segmentation results of the traffic signs by using Histogram Back projection method which is very robust in converting image to binary provided correct assigned region of pixels of interest.
It also used color matching model to compare the color attributes between a source images candidate image. Acknowledgement
The author wish to extend their thanks for the support provided by the Universiti Teknikal Malaysia Melaka.
This work was supported by the university short term grant. [2]
[3]
[4]

[5]

[6]

[7]

G. Siogkas, E. Dermatas. 2006. 1st Initial, Detection, Tracking and Classification of Traffic Signs in Adverse Conditions. IEEE
MELECON 2006. 537-540.
Mohd Ali, N., Mohd Sobran, N., Md Ghazaly, M., Ab Shukor,
SA. and Tuani, AF. 2014. Traffic Sign Detection and
Classification for Driver Assistant System (DAS). 277-283
M. Benallal and J. Meunier, 2003. Real Time Color
Segmentation of Traffic Signs. IEEE CCECE 2003 Canadian
Conference on Electrical and Computer Engineering.
Mohd Ali, N., Mohd Sobran, N., Md Ghazaly, M., Ab Shukor,
SA. and Tuani, AF. 2014. Individual Processing Speed
Analysis for Traffic Sign Detection and Recognition. Smart
Instrmentation, Measurement and Applications (ICSIMA).
2013, Kuala Lumpur, Malaysia.
Mohd Ali, N., Karis, MS. and Safei, J. 2014. Hidden Nodes of
Neural Network: Useful Application in Traffic Sign
Recognition. Smart Instrmentation, Measurement and
Applications (ICSIMA), 2014, Kuala Lumpur, Malaysia.
Mohd Ali, N., Md Rahid, NK. and Mohd Mustafah, Y. 2013.
Performance Comparison between RGB and HSV Color
Segmentations for Traffic Sign Detection. Applied
Mechanics and Materials. 393: 550-555.
Zakir, U., Leonce, NJ., and Edirisinghe, EA. 2010. Traffic Sign
Segmentation based on Color Spaces: A Comparative
Study. In the Proceedings of the 11th International
Conference on Computer Graphics and Imaging,
Innabruck, Austria, 2010.