Upload
hoanghuong
View
222
Download
3
Embed Size (px)
Citation preview
Application of Wearable Sensors and Augmented Reality in
Safety and Maintenance functions at Tata Steel Limited
Abhishek Kumar, Abhilash Agnihotri
System Integration Group, Automation Division, Tata Steel Limited
Jamshedpur, Jharkhand -831001.
Phone- +919234511868
E-mail: [email protected]
In any manufacturing industry, maintenance crews work in hazardous areas such as
around moving conveyors , live power cables ,open pits and trenches or even poisonous
gases and fumes to name a few.
To mitigate this methods and apparatus for Personal Protective equipment (PPE) such as
Helmets and High Visibility jackets fitted with wearable sensors which can detect and alert
proximity to live power cables andtrack and monitor health and environment of the worker
indoors and outdoors has been developed. Helmet mounted camera system transmits live
video to the control room with immersive viewing facilities.
The alerts and hazards generated from the Smart PPE are monitored centrally and expert
viewing the live video feed provides assistance immediately to the field operator from the
central control room. This system provides equipment based specific condition monitoring
and maintenance data, thus increasing productivity and safety.
INTRODUCTION
Rapid advances have been made in digitalization, cybernetics and micro –electromechanical
Sensor technologies .Internet of Things leveraging the ubiquitous cellular and Radio
Frequency backbone with highly miniaturized electronics have now made possible wearable
technologies and Augmented Reality applications which were in the realms of science
fiction only a few years back.
ABSTRACT
Inevitably as these nascent technologies evolve, they find multitudinous applications in the
Manufacturing Industry. Some areas of applications where these technologies are getting
rapidly deployed are for inventory tracking through the supply chain, condition monitoring
of equipment, for Preventive maintenance and for personnel safety. As low hanging fruits
these technologies can be quickly and cheaply deployed to give organizations sustainable
competitive advantages and new insights to improve their operational efficiencies.
Several solutions leveraging the above technologies have been developed and deployed in
Tata Steel as part of its digitization initiative. This paper discussed a system which has been
developed recently which utilizes Wearable Sensors, Augmented Reality and associated
technologies for plant maintenance and personnel safety.
RELATED TECHNOLOGIES
Augmented Reality:-
Augmented Reality or Mediated Reality can be defined as Real World/Physical environment
supplemented or Augmented with sensory data (Sight /Sound/Smell/Vibration etc.) with the
help of computers and electronics. A Typical example/application ofaugmented reality
technology is the Heads UP display provided in the helmets of fighter pilots which project
critical flight data on the windscreen of the Helmets worn by fighter pilots so that they do
not have to look down at the control panel. Now this technology is commercially available
(See Fig.1.HUD for a Car) and has changed the paradigm of the conventional Human
Machine Interfaces or the computer screens familiar to us. From Interacting conventionally
with a computer HMI using a keyboard and mouse the operator can be shown the entire
process in 3 dimensions projected on their consoles and the keyboard can be projected on
their desk or any flat surface (See Fig1B--Laser Keyboard). They can rotate, dissect and
inspect the 3 dimensional holographic image of the equipment (See Fig.1.C&D) to diagnose
problems and also obtain expert help at remote locations using these immersive viewing
/display technologies.
Fig.1.0: Examples of Augmented Reality
Wearable Sensors:–
With miniaturization of micro controllers and sensors it is now possible to have a button
sized micro controllers with 8 I/O‟s sown on the Protective jacket or pasted on the helmet of
any personnel. These low cost micro controllers can be used to monitor the ambient
temperature, Air quality and health of the personnel and also transmit in real time situational
data such as locationGPS or LBS and position(upright or fallen) from micro accelerometers
and GPS devices embedded in the Garments. This information transmitted to the control
room can provide lifesaving assistance if such as case arises.
Wearable sensors such as products from „Adafruit‟ (Fig 2) and Grover provide low cost
wearable wireless sensors used in this system which can be stitched into clothes using
conductive wires and fabric these low powered devices connect to micro controllers using
the Arduino programming environment. These micro controllers can be stacked with various
devices and modules known as shields which range from sensors to WiFi, Zigbee, Blue
tooth or other communication modules as well. These are open source systems and are
available for integration off the shelf.
A B
C D
Fig.2.0: Wearable Sensor from Adafruit Industries.
Tracking/Tagging /Positioning Systems –
The essence of the augmented reality system for maintenance is to provide contextual,
relevant information to the maintenance personnel based upon their location. Location
information is also necessary to track / monitor the personnel working in hazardous areas.
Of the several technologies which have been incorporated in the system, some have been
available for several years such as RFID, GPS (Global Positioning System) and LBS and
Barcoding and these do not need much explanation. Some new Technologies which have
been adapted for this system areBluetooth Beacon technology called Eddystone API which
is an open platform developed by Google, originally for proximity based advertising
tracking
Fig.3.0: QR Code and NFC Beacon Schematic
Blue tooth beacons using the “Eddystone” profile (See Fig.4) developed by Google provide
similar location and beacon functionality. These are used to broadcast advertisements and
store locations and special offers inside shopping malls or location of booths in a conference
with many stalls. These beacons can deliver advertisements to the Bluetooth enabled smart
phones nearby.
Fig. 4.0: Google Eddystone Beacon Data Format and Encoding
QR codes or 2D which are an advanced version of the Bar codes and NFC(Near Field
Communication Tags (See Fig.3) are also used for providing tagging or location based
information in the system as a cheaper alternative to the Bluetooth Beacons or RFID (Radio
Frequency Identification) systems .
Telemetry Networks
For data transfers from the field to the Central control room multiple RF technologies have
been used for Telemetry and data communication .As cell phones are available with all
personnel, the wearable sensors piggyback data on the GSM/GPRS networks of the cell
phone through blue tooth connectivity to the cell phone. For the Immersive viewing and
remote monitoring components of the system, 5.8 Ghz License Free ISM band has been
utilized for video transmission .Blue tooth Technology and Wi-Fi on 2.4 Ghz have been
used for indoor tracking of personnel.
Table 1.0: Performance characteristics of various Telemetry technologies
Multiple RF bands have been selected on the basis of their propagation characteristics such
as Range, bandwidth and resilience to interference .For critical information with low data
volume Cellular networks have been preferred while for high Data volume short range Wi-
Fi and Wi-Max networks have been deployed.
Inductive Proximity Sensing
Inductive fencing systems have been developed by companies for controlling live stocks and
pets .Companies such as pet safe market products in which a wire buried underground
transmits a signal which is received by a receiver. More recently wireless systems which are
located centrally and radiate an Omni directional signal .These do not require the
cumbersome process of burying an antenna wire. Such systems work on the Chirp spread
spectrum technology implementation.
Immersive Viewing Systems
Micro CCTV system with head tracking and First Person viewing using 5 Ghz or 2.4 Ghz
band have been developed by Aero-modeling companies for hobbyists flying drones and
remote control air crafts. These devices are marketed by companies such as Boscam and
Fatshark . The GoPro Company manufactures cameras and gimbals which can be mounted
on helmets and mountain bikes or skate boards while playing adventure sports or even for
water sports but do not have a video transmission or pan tilt facility.
Fig: 5.0: Immersive viewing and Head Tracking System
FEATURES OF THE SYSTEM
System Overview
In a manufacturing industry such as steel industry carrying out maintenance work is fraught
with hazards. It is also desired to track the work team if they are working in such hazardous
locations. While doing maintenance work to improve productivity it is desirable to provide
the correct documents and SOP (Standard Operating procedure) for the specific equipment
maintenance history of the equipment and location and availability of spares and
replacement schedules etc. This can improve the productivity of the equipment and
maintenance teams by reducing maintenance times. It is also desirable to track the people
working in underground cable tunnels for their safety.
There are certain work stations in a plant, where round the clock vigilant monitoring of the
equipment is required. In case personnel leave their station for any reason, a reliever should
be present or the system should be shut down safely, examples being conveyors and
crushers in mining industry.
To address these problems a system was conceived where- in wearable sensors are
incorporated in the mandatory safety equipment worn by the workers. The sensors on the
invention can detect when the worker is approaching a hazard or leaving a work place.
If a person falls or become incapacitated the system triggers a SOS to the control room
automatically.
Live streaming video over an immersive viewing system provides for remote assistance and
diligent monitoring of the work by experts from a central location. It tracks people when
they are working under ground by means of beacons and above ground by GPS.
During maintenance work pertinent and relevant maintenance manuals and documentation
are made available to the team when in proximity of equipment by means of the beacons,
NFC tags or QR Code stickers. By Scanning the QR code or NFC tag, the Hand Held device
of the Maintenance crew is automatically connected to the Maintenance Server. The relevant
maintenance checklist is downloaded and information on Maintenance history, list of earlier
problems, spares availability, dismantling guide etc. is downloaded on the Hand Held device
of the Maintenance crew.
Using Proximity based tags; it is also ensured that the maintenance team has to be physically
present near the machine to access the checklist, thus preventing proxy or offline entries.
Also by means of the Beacons and Wireless Networks , location of persons working
underground or indoors can be tracked, while movement outside can be tracked using GPS
sensors. These sensors stitched on the High Visibility jackets of the workers. Accelerometer,
Gyroscope, pulse sensors indicate to the control room if the person has fallen or
incapacitated in any way. Carbon Monoxide detectors alert the wearer and control room of
gas leakage.
A micro CCTV system enables live video transmission to a safe location from where
assistance or expert help can be provided. This system thus enhances safety and productivity
of workers in a hazardous environment.
Another objective of the Smart PPE system is to implement a positive isolation system
whereby hazardous areas are demarcated with a buried conductor or a conductor embedded
in a rope or metal cordon, which acts as a radiating antenna. The buzzer vibrates to alerts the
wearer of the receiver when it nears this cordon. This antenna may also be a centrally
located Omni directional radiator without requiring a need for burying a conducting cable.
This allows users to provide variable lengths of cordoned area possible by enabling
transmission of a proportional frequency through the tunable Signal generator depending on
cordon length which is the antenna, (antenna length being a function of frequency) and
thereby reception of several frequencies by the receiver simultaneously through mechanisms
of filters and associated circuitry.
This is done so that alarms are generated only from those area/power lines considered as
hazardous and create false positive alerts are reduced.
As process areas are inherently noisy, audible visual alarms may be overlooked, therefore a
piezo buzzer is embedded inside the helmet which vibrates near the temple of the wearer
and cannot be ignored or overlooked.
The Smart Helmet also has a feature toprovide live video transmission to an expert located
centrally. The expert can remotely control the pan tilt of the camera and can have an
immersive viewing experience by using immersive viewing goggles. This live video can
also be seen on a video monitor and recorded
To conserve the battery and also to monitor if the personnel are wearing the smart PPE,
conductive cloth has been used along with pressure switches to fabricate the Chin strap. The
movement the chin strap goes slack, information is triggered to the control room that the
Smart PPE has been removed and after a defined interval the system is turned off
automatically.
Description of the Wearable Sensors and Smart PPE system
The Smart Helmet (Fig6) incorporates a battery pack 11 with a USB charging port 10,an on
off switch 8, a battery indicator 9 and charging port 10.This battery pack contains a suitable
Lithium polymer battery in explosion proof casing, which powers the other devices a micro
-camera 13 with pan tilt features and a proximity sensor /receiver 6 .This battery pack 11 is
so designed that it can be installed and removed easily for replacement.The micro camera
with pan tilt facility is used to transmit video and receive data respectively from an
Immersive viewing goggles 4 with head tracking facility and a LCD monitor with Diversity
antenna 15, over a 5Ghz frequency band.
Fig.6.0: Schematic of the Smart Helmet System
The range of this system is typically 300 meters in line of sight outdoors or about 100
meters indoors.A conducting media such as thin single strand copper or metal wires of
armored sheath can be used as the transmitting media or the signal generator can be
connected to a monopole antenna which transmits radially from a central location. The
signal generator 2 can be battery operated or connected to the mains power. The receiver 6
denoted as an inductive proximity sensor mounted on the said helmet 14 receives this
transmitted frequency when it nears the radiating antenna 3 which earmarks the boundary or
fence of the hazard area. An amplifier circuit embodied in the receiver 6 and induction coil
amplify the signal as the fence is neared. Depending upon the proximity of the receiver 6 an
escalating level of vibration and buzzer intensity can be provided to the wearer. The chin
strap of the helmet14 incorporates pressure sensor which can detect if the helmet if removed
from the persons head and turn off the system to conserve power, while sending an alert
message to control room.
Fig.7.0: Smart Jacket with Wearable Sensors
As summarized in Fig 7 a high visibility jacked 22 has incorporated within several wearable
sensors. These sensors known as shields are stitched using conducting thread and cloth. The
sensors such as Accelerometer, CO gas detector, Gyroscope, pulse monitor etc. 19 are
connected to the input ports of a micro controller 21. This micro controller is enabled with a
Wi-Fi/GSM module which is used to transmit the sensor data to the control station. A GPS
sensor 20 is also incorporated in the jacket and connected to the micro controller 21. These
devices are powered by a battery pack which hasa trickle charger and is connected with a
solar panel 18 stitched into the jacket.
The Central control system is on the LAN 17 and runsproprietary software which denotes
the sensor status, location data received from each person wearing the Invention, reports etc.
Fig 8.0: Indoor Personnel Tracking System using Beacons and Wi-Fi infrastructure
As shown in 8.hen the worker is moving indoors in a cable tunnel or confined space, Blue
tooth beacons enumerated as 25 in .8 will be used to track location. The beacon 25 based on
the Eddystone profile developed by Google Inc. are devices which can detect blue tooth
low energy beacons and transmit the same over a Wi-Fi network 27 or transmit a beacon
with parameters such as location id, signal and health of the device and other messages. A
person can be tracked in such location by either carrying a beacon which is detected and
transmitted over Wi-Fi network to the central station or by carrying a smart phone which
can receive these beacons broadcasts. Also as indicated in the figure 8 the beacon 25 when
located near any equipment can enable the smart phone to connect to the Wi-Finetwork and
connect to the central computer. The Smart PPE and Augmented Reality system may be
configured in a variety of ways and is not limited to the representation described here
INDUSTRIAL APPLICATIONS AND CONCLUSION:
The Smart PPE and Augmented Reality based Maintenance system can be used in a multiple
configurations and combinations. The beacon, smartphone Wi-Fi combination can be
utilized for effective maintenance of the equipment. The helmet and Beacon system can be
used to ensure positive isolation of hazardous areas and prevent unauthorized entry at such
locations while the CCTV system can be used to provide remote expert assistance and
guidance.
This system can be used to track and monitor workmen entering hazardous area with
chances of exposure to poisonous gases and in locations where conventional GPS cannot
work in confined spaces, indoors and underground.
Fig.9.0:Integrated Plant Maintenance using Augmented Reality Technologies
The system can be marketed as a combined unit or in separate modules at different price
points for enhancing safety and productivity for workmen exposed to hazardous
environments in all industries/agencies engaged in maintenance of equipment generally and
steel industry specifically.
The challenge is to drive down cost by economies of scale and modular design to make this
affordable and cost effective proposition. The cost of the system using discrete components
is around 3000 INR, excluding the immersive viewing system which alone costs around
Rs.30,000.The challenge is to bring this price to below 300 Rs. using indigenous
components.
REFERENCES
1. https://en.wikipedia.org/wiki/Augmented_reality
2. Newman, Richard L. “Head Up Displays: Designing the Way Ahead” , ISBN 0-291-39811-1
3. T. Olsson, D. Gaetano, J. Odhner, and S. Wiklund.”Open Softwear 2nd
Edition”.
4. https://github.com/google/eddystone
5. Denso.”QR Code Essentials”.
6. Robert M Dondero . ” Programming the Webserver”
7. J Annuzi, Lauren Darcy, Shane Conder .”Advanced Android Development,4th
Edition .Addison-
Wesley”
8. PanagiotaFatourou.” Introduction to Sockets Programming in C using TCP/IP”
9. Jacob Millman,ChristosHalkias,Chetan D Parikh.”Integrated Electronics AnalogAnd Digital Circuits
And Systems Paperback (English) 2011”.
Abhishek Kumar