Final Project Resume

DESIGN AND IMPLEMENTATION OF INDONESIAN SIGN LANGUAGE RECOGNITION SYSTEM BASED ON FLEX SENSOR WITH ARTIFICIAL NEURAL NETWORK

Azizah Izzatur Rahim
Program Studi D4Teknik Elektronika Departemen Teknik Elektro Politeknik Elektronika Negeri Surabaya Kampus PENS-ITS, Jalan Raya ITS Sukolilo, Surabaya 60111 Tel: (031) 594 7280; Fax: (031) 594 6114 Email : azizahirahim@gmail.com

Abstract

As a social creature, humans are very need of communication as a media to make some interaction with others. There are verbal and non verbal communication. Yet, not all of humans are able to communicate well because they don’t understand each other meaning. This kind of problem are often happened in normal people who hard to understand mute people meanings who uses sign language. Meanwhile, translator devices are very expensive and not everyone can buy or even hired a translator also needed a high cost. So, from this union of flex sensor and accelerometer with artificial neural network backpropagation method resulted a sign language translator device which text is as the output on computer that more economic with succeed percentage by 99.2% and failure percentage less by 1%. Moreover, this device can be use as learning and introduction media to normal people to knowing sign language system. Keyword : Sensor Flex, Accelerometer, Backpropagation, Isyarat 1. Background So many ways that human done to communicated each others like conversations, sign language or sentences through any capable media. But, for deaf and mute people couldn’t communicate normally because their disability. This disability makes them being separated form normal people around as a social being. There’s so many other options to help them like hire a translator or buy translator device, but both of them cost too expensive like cyberglove from Virtual Technologies Company for US$ 10.000. So I made a sign language device to translate their sign language movements into text by using some sensors and algorithm that a lot cheaper than Cyberglove. 2. Algorithm and Hardware 2.1 Artificial Neural Network Backpropagation Many hundreds of Neural Network types have been proposed over the years. In fact, because Neural Nets are so widely studied (for example, by Computer Scientists, Electronic Engineers, Biologists and Psychologists), they are given many different names. You’ll see them referred to as Artificial Neural Networks (ANNs), Connectionism or Connectionist Models, Multi-layer Perceptron (MLPs) and Parallel Distributed Processing (PDP). However, despite all the different terms and different types, there are a small group of “classic” networks which are widely used and on which many others are based. These are: Back Propagation, Hopfield Networks, Competitive Networks and networks using Spiky Neurons. There are many variations even on these themes. We’ll consider these networks in this and the following chapters, starting with Back Propagation. Most people would consider the Back Propagation network to be the quintessential Neural Net. Actually, Back Propagation is the training or learning algorithm rather than the network itself. The network used is generally of the simple type shown in figure 1. These are called Feed- Forward Networks or occasionally Multi-Layer Perceptron (MLPs). The network operates in exactly the same way as the others we’ve seen. A Back Propagation network learns by example. You give the algorithm examples of what you want the network to do and it changes the network’s weights so that, when training is finished, it will give you the required output for a particular input. Back Propagation networks are ideal for simple Pattern Recognition. As just mentioned, to train the network you need to give it examples of what you want– the output you want (called the Target) for a particular input.

Figure 1. Backpropagation Architecture Once the network has been trained, it should be able to recognise not just the perfect patterns, but also corrupted or noisy versions. In fact if we deliberately add some noisy versions of the patterns into the training set as we train the network (say one in five), we can improve the network’s performance in this respect. The training may also benefit from applying the patterns in a random order to the network. This Algorithm has 3 basics principles : 1. Feedforward phase on training pattern. 2. Calculation phase from bakpropagation’s error. 3. Weight adaptation phase. 2.2 ATMega 328 This microcontroller has some features : 1. 2. 3. 4. 5. 6. 130 executed instructions in one clock 32 x 8bit multipurposes registers. 16 MIPS speed with 16 MHz clock 32 KB flash memory 1 KB EEPROM (Electrically Erasable Programmable Read Onlu Memory) as a safe. 2 KB SRAM (Static Random Access Memory)

Figure 2. ATMega 328 2.3 Sensor Flex 4’5” Use the sensor as the input of a resistance to voltage converter using a dual sided supply op-amp. A negative reference voltage will give a positive output. Should be used in situations when you want output at a low degree of bending. The impedance buffer in the [Basic Flex Sensor Circuit] (above) is a single sided operational amplifier, used with these sensors because the low bias current of the op amp reduces errer due to source impedance of the flex sensor as voltage divider. Suggested op amps are the LM358 or LM324.

Figure 3. Basic Sensor Flex Circuit 4’5” 2.4 Sensor Accelerometer adxl345 The ADXL345 is a small, thin, low power, 3-axis accelerometer with high resolution (13-bit) measurement at up to ±16 g. Digital output data is formatted as 16-bit twos complement and is acces-sible through either a SPI (3- or 4-wire) or I2C digital interface. The ADXL345 is well suited for mobile device applications. It measures the static acceleration of gravity in tilt-sensing appli-cations, as well as dynamic acceleration resulting from motion or shock. Its high resolution (4 mg/LSB) enables measurement of inclination changes less than 1.0°. Several special sensing functions are provided. Activity and inactivity sensing detect the presence or lack of motion and if the acceleration on any axis exceeds a user-set level. Tap sensing detects single and double taps. Free-fall sensing detects if the device is falling. These functions can be mapped to one of two interrupt output pins. An integrated, patent pending 32-level first in, first out (FIFO) buffer can be used to store data to minimize host processor intervention. Low power modes enable intelligent motion-based power management with threshold sensing and active acceleration measurement at extremely low power dissipation.

Figure 4. Accelerometer adxl345

2.5

Diagram Block System
Mikrokontroler AtMega 328

I2C

Accelerometer adxl345 Sensor Flex

Visual Basic Serial Training ANN

Sign Language System

Figure 5. Diagram Block System

This system works when those 5 flexs sensor on glove that sewed first in each fingers do the bending according to the sign language movements and combined with 3-axes hand acceleration would make dataset of ADC 10-bit on computer’s serial monitor. Then those dataset would be normalized with backpropagation algorithm by training them with visual basic software on laptop. From training data could be examined which movement is nearly correct with it and shown up the desired sign language (alphabet and number). But, the correct one is first compared by the smallest error. If it correct then the dialogue box would shown “BENAR” and if it wrong then “COBA LAGI”.

(a) Figure 6. (a) Sign Language Software display on Visual Basic (b) Hand movement on hardware : data input from flex sensor and accelerometer : backpropagation data (epoch, learning rate, error max and target) : sign language output : error list : indicator whether it’s true or not : sign language picture as an example

(b)