© 2002 - 2013 V2COM - Hardware + Software. All rights reserved Building a Reliable Remote Communication Device with Java ME JavaOne San Francisco 2013

© 2002 - 2013 V2COM - Hardware + Software. All rights reserved

Building a Reliable Remote Communication

Device with Java MEJavaOne San Francisco 2013 - CON8004

Leonardo Lima – V2COM ([email protected])

Tiago Padovani – V2COM ([email protected])

mailto:[email protected]


Introduction


About the presenter (Leonardo)• Software Development Manager at V2COM, leading the

development of both embedded software for edge/field devices and server-side software that receives field data and enables fast communication with remote devices.

• Eight years of development and architectural experience on mobile, embedded and server platforms to enable management and operation of highly scalable and fast telemetry and network control systems.


About the presenter (Tiago)• Lead Software Developer for Embedded at V2COM, leading the

development of embedded software for edge/field devices.

• Six years of development and architectural experience on embedded Java platforms, building reliable communication devices that stays on the field for years.


About the presentationExplore how to build reliable remote communication

devices that must stay online, connected, and transmitting field data without interruptions. Using Java ME, a solid foundation was built that can be used to deploy many kinds of applications in the field, with fast, reliable development. In this session, see what the difficulties, ideas, and insights were that led to this foundation, which is currently in use in more than 50,000 devices in Brazil and Latin America to connect more than 800,000 field devices.


• V2COM is a leading Latin American provider of Smart

Grid technologies and Advanced Metering Infrastructure

(AMI). Our offer includes hardware, software and services

that can reduce losses and increase water and energy

efficiency, currently connecting more than 700.000

devices.

• Besides the economic gains for our customers, we

develop technologies with significant impact on the

environment (avoiding losses of important natural

resources) and with social responsibility (our Smart Grid

systems can be used to offer free internet access in

challenging areas)


• Awarded 2013 Java Business

Innovation Award

• JCP member, nominated for EC election• More interaction with community to foster

Java ME development in this age of IoT!


V2COM clients

Meter Reading

Network Control


The scenario


IP Network Local Network

Legacy Systems Intelligenceware Suite

WAN Terminal LAN Devices

Wireless Network


Use cases• Automated Meter Reading (AMR)

• Read “dumb meters” and send data over to Utility• Both energy and water meters• Can be any meter, really!

• Like our Zigbier refrigerator’s temperature sensor, with alarms for too cold or hot!

• Network control• Remotely connect SCADA systems to field devices like reclosures• “transparent pipe” between SCADA and Device• DNP3/IEC104 protocols and proprietary as well


Use cases

136 4 25b 5a


The techonology usedGateway:

• Oracle Java Micro Edition 1 (Java profile IMP-NG version 2 & CLDC 1.1 Hl), embedded in a Gemalto TC65i module.

• Started with Siemens TC45, 6 years ago!

Intelligenceware Suite:

• Oracle Java Enterprise Edition 5

• Oracle Java Standard Edition 6

• Oracle Utilities Meter Data Management 2.0


Living an embedded life


The 3 commandments• “If the device has a power from source, it must be on”

• Our first commandment is to keep the device always on.• If it can be on, it must be on, no matter what.

• “The firmware must be remotely upgradable”• One of few certainties in software developments is that bugs always

happen!• So, we must be able to OTAP (over-the-air provision) the device

• “If it’s on, it must be online and connected (or trying to be...)”• It doesn’t suffice to be on. It must be also be avaiable so that

communication can happen


Hardware choices• WatchDog

• One of the few always-must-have circuit for any device is a watchdog to turn it on in case it turns off or there’s a lock-up

• Dual Sim Card• GPRS infrastructure is not very reliable, so it should be able to have

option in case of carrier failure

• Dual Serial• Many field devices have two serial ports, so being able to connect to both

at the same time is very important

• Java ME• Being able to run Java ME allows many advantages over C


Software choices - Separation of concerns

• “Hardcore”• Abstracts board specifics details (like GPIO)

• “Zion”• Handles common infrastructure (backend connection, message

queues, locks and so on)• Abstracts Modem commands, so there’s one for each avaiable module

(Cinterion/Gemalto; Motorola; Telit)

• “App”• Holds application logic - As tailored as clients need it to be• Ranges from a simple Transparent (TN) to Automated Meter Reading

(AMR)


Hardcore• Handles board specifics

• Modem/AT Connection

• SMS/GPRS logic

• GPIO• Watchdog


Zion• Defines command shell

• Handle TCP communications

• Kitchen sink utils (math, string, locks, queue)

• Event Bus

• Extendable to handle “fatter” functionalities• CSD• SNMP monitoring


Application• Handles only business logic and protocol

• Fast changes to acommodate business logic• After the first few cicles, it stabilizes


Package Diagram• mobi.v2com.zion...

• Application• Control• Hardcore• Communication• Event• Protocol• Tools• Utilities• ... and application specifics as well


ChallengesProblems faced, solutions provided!


How to keep the device on?• Problem:

• Module can throw Exceptions (EXIT CODEs) that shuts down the application

• Solution:• Hardware Watchdog!

• Dedicated QG8 to handle only this (power on/off)• 10 minutes timeout: either the firmware “swings” the GPIO, or it gets powered off

then on

• Many other software watchdogs...• Many other conditions can mean a “lock up”


How to keep the device online?• Problem:

• GPRS connections can be unstable, specially when needed most: thunderstorms and harsh weather condition

• Solution:• Dual-SIM cards, so if a network fails, there can be another• Many soft watchdogs to retry attachments

• Turn on/off modem, like our phones• Attach/Deattach


How to get behind public APN firewall?

• Problem:• Can’t use a server socket while communicating on public networks!

• Solutions:• Device polls server, asking for new commands

• This doesn’t work for always-on connection needs• Expensive on resources

• Software Switch (server side)!• Meet-in-the-middle approach


How to keep the TCP connection up?• Problem:

• You don’t know the connection is broken until you have a blocking read() or write()

• TCP keep-alive is not enough• No module support for this...

• Solution: • Keep-Alive packets

• Application-level bytes sent to make sure connection is not off


How to keep the firmware updated?• Problem:

• There are MANY devices deployed, each with a software version• No one wants to mass-deploy...• Not all updates are required for all devices

• Solution: • Axon has control of each version• Updates are dispatched as needed and/or cleared by client


How to debug in-device?• Problem:

• No simple logging framework to write log to• Log redirection works for serial port only, not remote connections

• Solution: • Built a simple logging framework that handles debug level, message,

threads


How to debug off-device?• Problem:

• How can we run/test outside the framework?• Many classes you depend on are hardware specific (Files,

Connections)

• Solution: • Built a simple “java abstraction layer” that implements

javax.microedition classes (and proprietary ones too) that runs on Java SE on the desktop.


How to keep historical, debugging info?

• Problem:• No long-term storage on devices to keep logs• Flash memory can hold at most 30 minutes of logs• The logs you need are the ones when device is unreachable, but alas,

you can’t download it.

• Solution: • Have a “black box” flight recorder• Small, coded files that can hold critical information for weeks


How to synchronize access to resources?

• Problem:• Access to system resources must be synchronized

• Serial device, system files

• synchronized can lead to unknown lock ups• Some race conditions hard to track ando also some bugs...

• Solution: • Custom implementation of Java SE’s Lock

• ReentrantLock inspired• Added few methods for convenience

• tryLock, so that you lock if you can, but don’t fail if you can’t• fastLock, for locks that should last than 2 secods


How to default parameters?• Problem:

• Same firmware for different clients with different needs, lots of parameters

• Parameters for APN, Serial, Timings and such

• Flash memory can get corrupt and you lose your parameter files!

• Solution: • Use JAD parameters

• JAD can be custom tailored to each client• Can provide good defaults


Pending challenges...... because you can’t win them all...


Many pain points are still there• Make smaller binary fragments?

• And small updates over the air.

• Run multiple applications at the same time?

• More “permanent” and/or remote logging

• Security issues• Better management of keys• Anti-tampering measures


Incoming changesJava ME (r)evolution


Java ME 8 has much new features!• Java ME 8 seems to have addressed some, if not most, this

problems and abstractions• Device I/O• Access Point• NIO• ... And many others!

• Some still need works• No concurrency support might mean no locks...

• We’ll work with JSRs to try and address the rest of them• V2COM is joining the JCP EC to help foster Java ME evolution


Q?


[email protected]

www.v2com.mobi

mailto:[email protected]

Documents

© 2002 - 2013 V2COM - Hardware + Software. All rights reserved Building a Reliable Remote Communication Device with Java ME JavaOne San Francisco 2013