Visible Light Communications: Recent Activities in Japan
Shinichiro Haruyama, Professor The Graduate School of System Design and Management, Keio University, Yokohama, Japan
Smart Spaces: A Smart Lighting ERC Industry - Academia Day at BU Photonics Center, Boston University February 8, 2011
Shinichiro Haruyama
1
Contents 1. Visible light communication 2. Methods of visible light communication 3. Areas to which visible light communication may be applied 4. Conclusion
Shinichiro Haruyama
2
1. Visible light communication
Shinichiro Haruyama
3
1. Visible light communication
Visible Light communication is a wireless communication technology that uses light that is visible to humans.
Typical example of visible light communication: the use of LED illumination as a transmitter
LED Lights
Receiver
Why do we want to use visible light for communication?
Shinichiro Haruyama 4
Reason 1: Visible light communication is not regulated.
Radio Wave and Light Wave
Frequency (Hz) Ultraviolet light Visible light Infrared light Light Wave Radio Wave 16 10 15 10 1014 13 10 3THz 12 1THz 10 11 10 1010 9 1GHz 10 108
Visible light communication
Frequency above 3THz is NOT regulated by the Radio Regulation Law .
Wireless LAN Cellular phone TV FM radio
The use of radio frequency below 3THz is regulated by the Radio Regulation Law .
7 10 1MHz 106
AM radio
Visible light communication is not regulated, so it is fairly easy to introduce new products , though safety regulation of high-power LED light has been already discussed.
Shinichiro Haruyama 5
Reason 2: Visible light LED will be used everywhere.
Infrared light is already used for communication, such as wireless remote control , IrDA, Infrared wireless LAN, and infrared interbuilding communication. However, visible light LEDs are beginning to be used in every home and office, which makes visible light LEDs ideal for ubiquitous data transmitter.
Data source: Nikkei Electronics
Data source: OIDA
Prediction of LED lighting in the global market
Prediction of market share of lighting types: SSL(Solid State Lighting or LED lighting), incandescent lamp, and fluorescent light
The growth rate of LED lighting is expected to triple from 2009 to 2012, and the market share of LED lighting will be more than 30 percent of total lighting market in 2016.
Shinichiro Haruyama 6
Reason 3: Image sensors can be used as receivers
The use of image sensors as receivers makes it possible detect not only incoming data, but also the accurate direction of the incoming vector from a transmitter to a receiver. The image sensor technology will allow visible light communication to various new applications as shown below, which cannot be realized by radio wave technology: ◎ ◎ ◎ ◎ Indoor navigation Augmented reality Accurate control of robots or vehicles Accurate position measurement
Shinichiro Haruyama
7
2. Methods of visible light communication
2.1. Devices used for Visible Light Communication 2.2. Communication using Image Sensors
Shinichiro Haruyama
8
2.1. Devices used for Visible Light Communication
Visible Light
Transmitter device
Receiver device
Shinichiro Haruyama
9
Transmitter device of visible light communication
-Visible light LED LED light intensity is modulated by controlling its current. data rate: low speed to very high speed (up to several hundred Mbps)
-Fluorescent lamp FSK modulation of high frequency fluorescent light data rate: up to several kilo bps
Shinichiro Haruyama
10
Receiver device of visible light communication
-PIN photo diode -high speed reception up to 1Gbps
-Avalanche photo diode
-very sensitive reception
-Image sensor -simultaneous image acquisition and data reception
Shinichiro Haruyama 11
2.2. Communication using Image Sensors
Data can be received with image sensors.
Shinichiro Haruyama
12
Principles of communication using image sensor
LED light is modulated lens recorded images by an image sensor
Received data:1
Received data:0
Received data:1 time
Camera (receiver) continuously takes images of a scene with an LED light and a receiver detects the optical intensity at a pixel where the LED light is focused on.
Shinichiro Haruyama 13
Principles of communication using image sensor (continued)
Visible Light Communication
LED
Image Sensor
Signal discovery and bit decode
ID data transmit with fast blinking
Even if multiple visible light sources send data simultaneously, an image sensor is able to receive and demodulate all the data simultaneously without any interference between them.
Shinichiro Haruyama 14
Merits of communication using image sensor
Photo diode communications
Number of signal
Image sensor communications
○ Multiple
△ Single × △ ～ １０ｍ △ Direction
Robustness Distance
Space Resolution
○
(no cross talk/ no Interference)
○ Very Long （2km） ○ Each pixel
Info. A
Zone Info. Ｂ
Zone Info. Ａ Info.B
Suitable for Zone informati on
Suitable for Object Information with position
InfoA
Info B
Shinichiro Haruyama
15
3. Areas to which visible light communication technology may be applied
Shinichiro Haruyama
16
3. Areas to which visible light communication may be applied
3.1. Applications that do personal area communication 3.2. Applications that enable users to know users’ locations in several meter accuracy 3.3. Applications that enable users to know users’ locations in several millimeter accuracy 3.4. Applications that use augmented reality 3.5. Applications that cannot be achieved by radio-wave technology
Shinichiro Haruyama 17
3.1. Applications that do personal area communication
Shinichiro Haruyama
18
Download of needed information for digital signage applications
The Infrared Data Association (IrDA) (located in California, USA) and Visible Light Communications Consortium (VLCC) (located in Tokyo, Japan. Chairman: Prof. Haruyama of Keio University) has been working together since September, 2008 for a visible light communication using IrDA protocol stacks. [1]
Prototype made by VLCC (Visible Light Communications Consortium) in Tokyo September 2009
User downloads information from a digital signage board whose backlight LEDs sends data
Shinichiro Haruyama 19
100Mbps Full-duplex multi-access visible light communication system[2] by Nakagawa Laboratory, Tokyo, Japan
アクセスポイント
Ｉ／Ｆ
Network controller
ＰＣ
子機
子機
ＰＣ
Ｉ／Ｆ ＆
Optical axis control
Network controller
Optical CSMA/CD (carrier sense multiple access with collision detection) method is used for accesses from multiple terminals. The communication distance is 3 meters. This technology was based on 100Mbps infrared LAN system whose standard was proposed by ICSA (Infrared Communication Systems Association) of ARIB (The Association of Radio Industries and Businesses), Japan.
Shinichiro Haruyama 20
High-Speed Parallel Wireless Visible Light Communications System Using 2D Image Sensor and LED Transmitter [6] by Keio University and Sony Kihara Laboratory
Each of 64 (8 x 8) LEDs transmits different data, which are projected onto a different pixel of a high-speed image sensor. About 50 times increase of data rate was achieved. (Because of the overhead, 64 times increase was not possible.)
Shinichiro Haruyama 21
Communication using LED spotlight
“Hikari Guide System” by Shimizu Corporation, Japan
Spot lights sends data such as picture information in a museum as shown above and a receiver guides guests by voice sound.
Shinichiro Haruyama 22
Visible light information broadcast system
Prototype made by NEC and Matsushita Electric Works, members of VLCC
Information of products at a supermarket is obtained by a visible light receiver that is installed in a shopping cart.
Shinichiro Haruyama 23
Application of visible light communication to ITS (Intelligent Transport System)
System with LED traffic light transmitter and image sensor receiver [3] by VLCC (Visible Light Communications Consortium, Japan) and The Japan Traffic Management Technology Association
VLCC (Visible Light Communications Consortium), March 2010
Data from multiple LED traffic lights are received by an image sensor receiver.
Shinichiro Haruyama 24
Application of visible light communication to ITS (Intelligent Transport System)
Information broadcast system using LED traffic lights
The Nippon Signal Co., Ltd. and VLCC, at Japan Shop 2006
The transmission method is based on Japan JEITA CP-1222 standard.
Shinichiro Haruyama 25
Multiple transmission using multiple colors
Joint project of Keio University and JVC
“Sound Spot” Different color lights send different data
Shinichiro Haruyama 26
Application of visible light communication to amusement
By Sony and Agilent at CEATEC 2004, Japan
SoundSpot
Demonstration of music broadcast system
Shinichiro Haruyama 27
3.2. Applications that enable users to know users’ locations in several meter accuracy
Shinichiro Haruyama
28
Global location service that uses visible light ID system
Visible Light ID
Visible Light ID
Visible Light ID
Visible Light ID
Prototype made by NEC and Matsushita Electric Works
Internet Server of location information
It accesses the Internet by first obtaining code from a visible light source such as LED lights. It then accesses the location server from the cellular phone in order to obtain location-related information.
Shinichiro Haruyama 29
Standardization of format of location information
Geospatial Information Authority (GSI) of Japan’s Ministry of Land, Infrastructure, Transport and Tourism started the activity of standardization of format of location information in 2010. 17companies including Keio University has been doing research collaboration with GSI until 2012.
Shinichiro Haruyama
30
Standardization of format of location information
The idea is to assign a unique number for 3 meter by 3 meter square everywhere. If such a unique code is used for IC tag, ubiquitous location services will be possible. At outdoor locations, GPS and other GNSS satellites can be used to detect user’s location. However, it is still very difficult to detect user’s accurate position indoor. If this standardized format is widely used for IC tag, ubiquitous indoor navigations infrastructure will be made.
Keio University is working with GSI to use this format as GSI’s code of sending location information from visible light source such as LED lighting.
Shinichiro Haruyama 31
Flow planning survey system for a store
Visible light ID from LED lights is received by a receiver attached at the bottom of a shopping cart. LED light transmitters are installed in each passage in a supermarket. Shoppers’ movement is analyzed based on the received ID history.
Nakagawa Laboratories, Inc.
Nakagawa Laboratories, Inc.
Visible light transmitter using LED light
Shinichiro Haruyama
Visible light receiver attached at the bottom of a shopping cart
32
Flow planning survey system for a store (continued)
Example of flow planning of shoppers in a supermarket
Average speed fast
slow
The thickness of lines indicate the traffic amount in each passage, and the color indicates how fast shoppers walked on the average.
This system is able to analyze how many shoppers passed in each passage and how fast they walked.
Shinichiro Haruyama 33
3.3. Applications that enable users to know users’ locations in several millimeter accuracy
Shinichiro Haruyama
34
Three dimensional position measuring system using visible light communication [4] by Keio University, Sumitomo Mitsui Construction Co., Ltd., and Nakagawa Laboratory, Japan
Objects can be measured by receiving and detecting the direction of visible light signal with an image sensor. The error of position for a 100 meter object such as a bridge in the photo is about 5mm.
Shinichiro Haruyama 35
Example of three dimensional position measuring system using visible light communication
August, 2008
LED lights attached to a highway bridge under construction in Japan
When an object in located in the 40 meter by 40 meter area, this technology is able to measure three-dimensional position of the object in several millimeter accuracy.
Shinichiro Haruyama 36
Another example of three dimensional position measuring system using visible light communication
November, 2009
The technology is applied to measure the displacement which is caused by the increase of temperature of the aluminum dome roof due to the sunshine in the daytime. 12 LED lightings were attached to the roof. We were able to do automatic measurement of every 20 minutes for 24 hours and confirm the displacement of several millimeters.
Shinichiro Haruyama 37
Advantages of three dimensional position measuring system using visible light communication • It can measure positions automatically • It can measure temporal change of positions by continuous measurement. • It can measure positions even at night. • It can measure multiple positions at the same time. • Its system is inexpensive than conventional survey equipments.
Shinichiro Haruyama
38
Accurate position detection of a transmitter or a receiver[5] by Keio University Control of a robot by detecting its accurate position using visible light and image sensor
Position data is transmitted from visible light LEDs
High resolution image sensor detects accurate direction of incoming light sources
December 2009
PC
LED lighting with a transmitter A robot with two image sensors calculates its three-dimensional position in real time.
Shinichiro Haruyama
PC calculates threedimensional position High speed image sensor demodulates data sent from LED lightings
39
Video of the accurate position detection of a transmitter or a receiver
LED lights send position data, and an image sensor of a robot receives the data as well as the image. The robot then does image processing to do phtogrammetry calculations to figure out its own position.
Shinichiro Haruyama 40
3.4. Applications that use augmented reality
Shinichiro Haruyama
41
Augmented Reality of real world image added with personal information by Casio Computer Co., Ltd., Japan
Casio Computer Co., Ltd., September 2009
Data from LED badges are received by an image sensor which records images at the same time
Shinichiro Haruyama 42
Lighthouse visible light communication by Visible Light Communications Consortium (VLCC)
Maritime Safety Agency Research Center of the Japan Coast Guard requested VLCC (Visible Light Communications Consortium) to do research about visible light communication using lighthouse or buoy lights in 2008, and VLCC member companies (Casio Computer Co., Ltd., NEC, and Toshiba) are doing experiments.
We succeeded in the experiment of 2km communication using an image sensor in September, 2008.
Shinichiro Haruyama
43
Lighthouse visible light communication (continued)
Received image and data
LED light in a photograph image
LED light for a buoy
1300 kilo bits per second 2780 frames of pictures per second LED light attached to a buoy
Casio Computer Co., Ltd.
The purpose of this project is to develop a new technology that enables the visible light communication using visible light from lighthouses and buoys. An image sensor of a camera on a boat demodulates the incoming data from lighthouses and buoys and displays its content on a display monitor.
Shinichiro Haruyama 44
3.5. Applications that cannot be achieved by radio-wave technology
Shinichiro Haruyama
45
Underwater Visible Light Communication[7]
By Rise, Nakagawa Laboratories, Inc., Keio University, October 2010
Radio waves do not propagate for a long distance under water. We were able to demonstrate that the flashlight visible light transmitter was able to transmit signals for 30 meter distance. A diver can communicate with a buddy diver using voice. LED flashlight’s light is intensity-modulated. A photo diode is attached next to LEDs. Rise is planning to sell the product in 2011.
Shinichiro Haruyama 46
4. Conclusion
There are many applications that can be realized using visible light communication. Careful consideration will be needed to introduce it into a market.
Shinichiro Haruyama
47
References
[1] http://press-releases.techwhack.com/25210-irda [2] Hirohashi, Ikawa, Lin, “High-Speed Full-Duplex Multiaccess System for LEDBased Wireless Communications Using Visible Light”, International Symposium on Optical Engineering and Photonic Technology: OEPT 2009, July 2009, Orlando, Florida, USA [3] http://www.tmt.or.jp/research/pdf/kashiko_21.pdf (in Japanese) [4] Uchiyama, Yoshino, Saito, Nakagawa, Haruyama, Kakehashi, Nagamoto, “Photogrammetric system using visible light communication”, IEEE 34th Annual Conference of Industrial Electronics (IECON), pp.1771 – 1776, November 2008, Orlando, Florida, USA [5] Tanaka, Haruyama, “New Position Detection Method Using Image Sensor and Visible Light LEDs”, IEEE Second International Conference on Machine Vision (ICMV 2009), pp. 150 – 153, December 2009, Dubai, the United Arab Emirates (UAE)
Shinichiro Haruyama
48
References
[6] Masanori Ishida, Satoshi Miyauchi, Toshihiko Komine, Shinichiro Haruyama, Masao Nakagawa: "An Architecture for High-Speed Parallel Wireless Visible Light Communications System Using 2D Image Sensor and LED Transmitter" Proc. of International Symposium on Wireless Personal Multimedia Communications (WPMC'05) (CD-ROM). 1523-1527 (2005) [7] Yousuke Ito, Shinichiro Haruyama, Masao Nakagawa, Rate-Adaptive Transmission on a Wavelength Dependent Channel for Underwater Wireless Communication using Visible Light LEDs, IEICE technical report, SIP, 105(636), 127-132, 2006-02-28
