Use of Unobtrusive Human-Machine Interface for
Rehabilitation of Stroke victims through Robot
Assisted Mirror therapy
Gautam Narangi, Arjun Narang2, Soumya Singhi

luhani Lempiainen

Electrical and Electronics Engineering, Bharati

Managing Director

Vidyapeeth's College of Engineering, New Delhi, India

Deltatron Oy Ltd.

Department of Electronics and Instrumentation, Birla

Helsinki, Finland

Institute of Technology and Science, Pilani, India

jle@deltatron.fi

gautam2410@gmail.com, arjun.narang09@gmail.com, soumya.singh1001@gmail.com Abstract- Stroke is one of the leading causes of long-term disability worldwide.

Present

techniques

employed

One technique employed to effectively rehabilitate stroke

for

victims, especially those suffering from partial paralysis or

rehabilitation of victims suffering from partial paralysis or loss

loss of function, is using mirror therapy. Mirror therapy is a

of function, such as mirror therapy, require substantial amount of resources, which may not be readily available. In traditional mirror therapy, patients place a mirror beside the functional limb, blocking their view of the affected limb, creating the illusion that

both

the

limbs

are

working

properly,

which

strategy that has been used successfully to treat phantom pain after amputation and recovery from hemiplegia after a stroke.
In traditional mirror therapy, patients place a mirror beside the functional limb, blocking their view of the affected limb, thus

enhances recovery by enlisting direct simulation. This paper

creating the illusion that both the limbs are working properly,

proposes an alternate robot based concept, named Wear-A-BAN,

which enhances recovery by enlisting direct stimulation. In

where the rehabilitative task will be carried out by a normal

therapy of stroke patients, trials have reported improved

articulated industrial robot. During the proposed rehabilitative

prognosis if therapy is started within 20-30 days after the

procedure, the patients are made to wear a smart sleeve on the functional limb. Movement of this limb is monitored in real-time, by wireless Body-Area Network (BAN) sensors placed inside the sleeve, and copied over the sagittal plane to the affected limb.
This procedure results in considerable savings in terms of money and personnel, as even though this procedure does not make the

stroke. This time frame often proves to be challenging due to limited personnel resources in healthcare.
In robot assisted mirror therapy, visual illusion and motoring function is enhanced by robot assisted actual movements of

rehabilitation process autonomous, but one therapist can monitor

the affected limbs. This method has several advantages over

various patients at a time. The industrial robot used is suitable

conventional methods in terms of clinical and biomechanical

for this purpose due to safety aspects naturally existing in the

measures [3]. During the proposed rehabilitative procedure,

robot, is relatively cheap in price, and allows comprehensive 3-D

the patients are made to wear a sleeve on the functional limb.

motions

of

the

limb.

Also,

unlike

traditional

therapy,

this

procedure allows actual movement of the affected limb. The sensors can also be used for other applications, such as gaming and daily life personal activity monitoring.

Movement of this limb is monitored with wireless body-area network (WBAN) sensors placed inside the sleeve. Movement of the functional limb is monitored in real time, and copied over the sagittal plane to the affected limb.

Keywords- Stroke; Rehabilitation; Mirror Therapy; Ultra Low
Power WEANs; Sensory nodes; Smart Textile/Fabric

In the Wear-A-BAN project for rehabilitation, the task of moving the affected limb is carried out by a normal 6 degrees

I.

INTRODUCTION

With an ageing population in an industrialized world, the global burden of stroke is staggering. Hemiparesis is one of the most common and disabling consequences of stroke [1]. 1 in 6 people worldwide will have a stroke in their lifetime. 15 million people worldwide suffer stroke each year and 5.8 million die from it. Stroke claims a life every 6 seconds. It is the second leading cause of death for people above the age of
60, and the fifth leading cause in people aged 15 to 59 [2]. It is assumed that 40% of all stroke patients benefit remarkably from limb therapy.

978-1-4673-6225-2/13/$31.00 ©2013

IEEE

of freedom industrial robot for the acute phase brain strokes.
The main effort in this case is the rehabilitation of the upper limb and

to

improve

the

very

basic

needs

of

human

independent living. Upper and lower limb robotic tools for traditional neuro-rehabilitation are effective in reducing motor impairment but they are limited in their ability to improve real world function.

There

is

a

need

to

improve

functional

outcomes after robot assisted mirror therapy. Improvements in the effectiveness of these environments may be achieved by incorporating into their design and control strategies important elements key to induce motor learning and cerebral plasticity

and

The WBAN sensor nodes used are based on generic design for

complex problem solving [4].

different scenarios. They incorporate three complementary

Also, from a technological point of view, industrial robots are

motion sensors: a 3 DOF digital accelerometer, a 3 DOF

such

as

mass-practice,

feedback,

task-engagement,

a cheap, accurate and flexible choice as compared to many

digital gyroscope, and a 3 DOF digital magnetometer, the

alternate approaches.

combination of which allows for accurate motion and position

II.

sensing.

DESCRIPTION OF SENSORY NODES

A

digital

microphone

is

also

used

to

capture

emotional data. The design of the WBAN node was based on

treatment

the sensor board containing all of the motion sensors, the

consisting of active, highly repetitive movements is one of the

microprocessor, boot ROM and battery/JTAG adapter and

most

fabric antenna interface, on a PCB of only 20mm x 32mm.

High

intensity, effective and

task

approaches

specific to upper

arm-

and

limb hand- function

restoration. [5] Hence, in mirror therapy with robots, it is essential to know the exact movements of the guiding limb.
For this reason, there are three sensor nodes in the sleeve glove on the guiding limb. There are scientific theoretical models that combine three of these nodes in one limb to be enough to calculate its exact position. The reverse kinematics has been adopted in the project and thus, the limb movements to control the robot can be measured with good accuracy.
A

low

power

programmable

processor

was

designed,

combining features of a digital signal processor (DSP) and a microcontroller unit (MCU). Implemented as a synthesizable
VHDL software intellectual property core,

Fig. 2 Complete node system consisting of a generic Wear-A-BAN module and JTag Programmer board

the processor

implements a broad range of power saving features including

III.

its customizable architecture and reconfigurable instruction set

DESCRIPTION OF SMART TEXTILES

[6].

"Smart

The Wear-A-BAN project uses the sensory nodes that consist

integrated

of a low power radio,

includes materials (e.g. cotton, polyester etc.), treatments (e.g.

a system on chip (SoC),

a 3D

Fabric" with is

defined

active

as

a

traditional

functionality.

fabric

"Traditional

with

fabric"

accelerometer, a 3D gyroscope and magnetometer. The main

dyes, polyvinyl alcohol, etc.), and manufacturing techniques.

task

Body-Area

Active functionality could include power generation and

Network BAN, which can process and transfer a large amount

storage, human interface elements, sensing devices, radio

was

to

make

the

information

network,

of sensor data wirelessly to computer for further use.

frequency

(RF)

functionality, or assistive technology [7]. The

network will generate research and development programs and
The nodes are programmed separately. The HDK enables easy

will form the framework of new research center in intelligent

validation of software for the processor on a SoC circuit. The

product research [8]. A key driver for smart textile research is

HDK is composed of a motherboard on top, of which up to 3

the fact that both textile and electronics fabrication processes

daughter boards can be plugged. The board includes a

are capable of functionalizing large-area surfaces at very high

(reprogrammable) circuit, which enables it to connect the

speeds [9].

daughter connections to the other daughter slots in a different way. Hence, it is possible to add, on the available slots, the

The technology used to manufacture the textile antenna is

application level daughter board. The sensor board is made of

weaving technology. The introduction of conductive yams

all required sensors for the WAB project, and an interface

into conventional manufacturing systems (shed weaving and

board allows connecting it to the board. The application level

knitting) allows the production of highly-conductive textile

drivers have been developed on this system.

structures

which

are

much

more

permeable

than

those

manufactured using other technologies such as laminating.
These fabrics have high color- fastness, flexibility, rigidity, durability and strength. The main drawback of these structures is the difficulties encountered in obtaining complex designs such as the textile antenna which were being developed by this project. This necessitated the search for other complementary technologies in order to ensure the successful outcome of the development. The system has a removable textile antenna. The nodes are placed in sleeve which is worn by the patient on the functional limb. Because all electronic parts and textile

antenna are placed inside a pocket, the smart garment will be comfortable, usable,

flexible

and

hence,

wearable.

The

garments are developed using comfortable materials (highly
Fig.! Central node with receiver antenna

appropriated for skin contact).The final sensory node with

communications; Central node functionalities for the device

smart textile antenna is shown in Fig.3.

connected to the PC; and Sensor node functionalities i.e.
WAB nodes have to synchronize on the beacon, to request to he coordinator an association and a BAN specific short address and to request to the coordinator GTS slots.
V.

APPLICA nON TO REHABILIT A nON USING
WEAR-A-BAN DEMONSTRATOR

Full motion capture for this scenario is illustrated.
•

An

application

scheduling

the

recuperation

of

Sensors data is defined.
•

The values of WAB sensors is recuperated.

•

The

WAB

MAC

is

used

to

schedule

the

communications.
•

The WAB radio of the WAB node is used to transmit the data packet.

•

The central node receives the data using the WAB radio and the WAB MAC.

•

Fig. 3 Complete sensor node with textile antenna

IV.

The data packet goes through the different layers until the UART framer.

•

NODE COMMUNICA nON PROTOCOL

The communication protocol enables forwarding of distributed

thanks to the FTDI of the USB JTAG KEY WAB.
•

BAN sensor data such that individual data transmitted by each is easily identified, received without error and transmitted to a central device for further processing and visualization. The design of nodes consists of three major goals to enable

The data packet goes from the UART to the USB
In the PC1, the data is transferred from the USB to
COM and from the COM to UDP respectively.

•

In the PC2, the data is used by the WAB GUI using the UDP port.

experimentation: minimal power consumption, easy usage, and increased software and hardware robustness [10]. When relaying functionalities are not used or communications with the central node are good, the WAB has a star topology.
Nevertheless,

some

communications

can

suffer

from

shadowing effect and WAB nods can cooperate to provide satisfactory QoS requirements. In this case, the topology becomes a star mesh hybrid topology, where sensor node to sensor node communication is possible.

PC
Driver USB lO COM
-----------------------------------

----------------------------------------

________________________�:��:��_�::O ���!_________________________�_;;� __ .
UA

ramer

A

Remote Pro rammm
A

II cation

lIealion framer

�
\,. ,I

Coordinator

;.

WAS node MAC stracnon AS

ensors

WA

Radio

Fig. 5 Complete Node placement

WAB1Imer

Fig. 4 Communication Protocol Diagram for WBAN

The motion capture demonstrator works as follows: one WAB

In this communication WAB nodes can have different roles:
Coordinator functionalities i.e. the device can send a beacon for synchronization and superframe management, accept or refuse node

associations

and

allocate

timeslots

for

node is the central node and the coordinator of the network. It is connected to the pc. A second WAB node is a sensor node on the body. This node sends its data in real time according to the movement of the guiding limb during the rehabilitation

scenario. It is assumed that a set-up of 3 sensor nodes is

.1

typically enough to calculate the motions of the limb for the

....(,

. �=

....... 1I-.

control system of the robot by inverse kinematics. This application uses monitoring flow for real time mode, and the expired monitoring data is retransmitted later using a logging mode. The setup during the development process is shown in
Fig. 6.

Fig. 8 ABB Robot Studio window used for simulation
Intelligent
textile sleeve with 3 node positions Fig. 6 Setup during the system development

Graphic User Interface is provided, showing the node on the body, and showing the log when a node is associated, and with a button to start or stop the BAN.
The robot used to move the affected limb corresponding to the movement of the guiding limb is ABB IRB140. This is an industrial class robot, and is controlled using ABB Robot
Studio. The values of the accelerometer,

gyroscope and

magnetometer are processed through this software.

ce:J leti

Fig. 9 Terminal window showing data received at the central node

Wear-A-BAN
There are two tasks being done simultaneously by the ABB
Robot Studio. The background task sets up a TCP/IP socket communication with the software. This background task will then listen

and

receive

coordinate

data

(translation

and

rotation in quaterions) from the software and will analyze this received data. The received data is stored to a robtarget persistent, which will be used by the main task. This main task will check the position of the guiding limb as stored in the robtarget, and move the ABB IRB140 to that position, if change from previous received position is detected. There are also short wait times at the end of the program to avoid unwanted loop programs.
1
» 71 1 5 6.014.010 12.0 -3 .0-5.0-26.0336.0 -2GlJ.0 -737 .0
•
•

7 1 : F rame type, moti on se rISe r fr.• me with 3 A3G3 M [Ox47 in h ex adec imal j .
1 : Add ress oHh e no de that it h.""e s€ nt t he frame

·5: Ti me st amp
•

Fig. 7 Wear-A-BAN demonstrator window

·6 .01 4 .01 012.0: Ac � e le ro m €t€ rval u e

•

-3 .0 -5.0 -26.0: G y rcsmpeval u€

•

336.0 -260.0-737.0: M€g neto m e er valu€ t Fig. 10 Real time values of accelerometer, gyroscope and magnetometer received from nodes.

The main hurdles to overcome in the developed sensor prototype are:
•

Battery capacity must be increased.

•

Accelerometer sensor sensitivity must be fine-tuned for this application
Gyroscope sensor must be tuned to achieve less

•

power consumption.
VII.

FUTURE APPLICA nONS AND DEVELOPMENT

The rehabilitation system powered by industrial robot is a promising concept. The growth in this market segment is obvious because

of

the

new

intelligent

technology,

and

increase in the number of people who need rehabilitation.
Commercialization of the robot based therapy concept to many physiotherapy modalities gives room for more markets.
Fig. 1 1 The rehabilitation scenario in real scale

VI.
This

rehabilitation

The

CONCLUSION

scenario

shows,

in

use

of

robot

as

an

aid

tool

in

many

other

kind

physiotherapy solutions will be profitable. When the price of sensor node technologies goes down along with a long reality, the

good

potential of the developed wireless technology in the vicinity

production series, the robot assisted rehabilitation technology will spread into physiotherapy clinics.

of the human body. The limb movements can be monitored

The new body area network communication technology opens

and various product concepts can be generated based on this

new

Wear-A-BAN technology.

interaction (HMI). For example, hand movements could be

The industrial robot used in Wear-A-Ban is suitable for the

used as a gesture language.

solutions

for

proposed rehabilitative purpose due to safety aspects naturally

new

modalities

for

Human-Machine

ACKNOWLEDGMENT

existing in the robot, being relatively cheap in price and allowing comprehensive 3D motions for the arm, all three

The

translations

working

Research for the benefit of SMEs Associations, under contract

envelope, speed and load capacity can easily be adjusted

number 243473. The developed sensor techniques are available

according to the needs of this scenario.

at Deltatron Ltd. in Helsinki, Finland.

and

all

three

rotations.

The

robot

Wear-A-Ban

project was funded

but

all

present

rehabilitation

machines

EU's

FP7

REFERENCES

There is no existing concept based on this industrial based technique, by the

are

[1] Altschuler, E.; Wisdom, S.; Stone, L.; Foster, c.; Galasko,

complicated, expensive and more or less tailor made for rehab

D.; Llewellyn, M.; Ramachandran, V.; "Rehabilitation of

purposes. The intelligence capability of the industrial robot

hemiparesis after stroke with a mirror", The Lancet, 2035-

can be utilized while adapting the machinery for the different

2036, 1999

body sizes of the patients, and also for compensating the

[2] http://www.world-stroke.org/

physiological non-Iinearities of the limb movements in a

[3] Lum, P. S.; Burgar, c.; Shor, P.; Majmundar, M.; Van der

human body. This is a case in, for example, the human

Loos, M.; "Robot Assisted Movement Training Compared

shoulder, where the upper limb doesn't rotate around one

with Conventional Therapy Techniques for Rehabilitation of

fixed point, but withdraws horizontally according to the height

Upper-Limb

of the limb. When calculating the reverse kinematics for the

Physical Medicine and Rehabilitation, 952-959, 2002

Motor

Function

after

Stroke",

Archives

of

robot according to the sensory data, these non-linearities can

[4] Johnson, M.J.; "Recent trends in robot-assisted therapy

be taken into account without any physical mechanisms.

environments to improve real life functional performance after

The measurement of the rehabilitated limb's movements can

stroke", Journal of NeuroEngineering and Rehabilitation, 2006

be easily displayed and the progress of the rehabilitation can

[5]

thus be monitored on PC software. This concept cannot make

Hermens, H.; IJzerman, M.; "Systematic review of the effect

Prange,

G.;

Jannink,

M.;

Groothuis-Oudshoorn,

c.;

the therapy autonomous, but one therapist can work with

of robot-aided therapy on recovery of the hemiparetic arm

several

after

patients

at

the

same

time

and

this

increases

rehabilitation capacity and power. Beside visual illusion, it also provides a tactile sense of movement of the affected limb.

stroke",

Journal

of

Rehabilitation

Research

and

Development, 171-184, 2006
[6]Arm,C., Gyger, S. ;

Masgonty, J.-M.; ; Nagel, J.-L. ;

Piguet, c.; Rampogna, F. ;

Volet, P.

"Low-Power 32-bit

The potential end-users of this robot based rehabilitation

Dual-MAC 120

scenario will be interested to further develop and rest-run the

IEEE Journal of Solid State Circuits, 2055-2064, 2009

W/MHz l.0 V icyflexl DSP/MCU Core",

robotized rehabilitation concept. The developed technology

[7] Simon, c.; Potter,E.; McCabe, M. ;Beggerman, c.; "Smart

has now been proven to work in this environment.

Fabrics Technology Development", NASA innovation fund project, 2010

[8]

Baurley,

S.L.

"Smart

Textiles

for

future

intelligent

[9]

Cherenack,

K.;

van

Pieterson,

L.;

Smart

Textiles:

Challenges and Opportunities", Journal of Applied Physics,
Volwne 112, Issue 9, 2012

[10] Polastre, J.; Szewczyk, R.; Culler, D.; Telos; "Enabling ultra-low consumer products", lEE Eurowearable, 73-75, 2003

power

wireless

research",

4th

International

Symposium on Information Processing in Sensor networks,
364-369, 2005