Mobisys 2005 1

SOS - Dynamic operating system for sensor networks

Simon Han, Ram Kumar, Roy Shea, Eddie Kohler and Mani Srivastava

http://nesl.ee.ucla.edu/projects/sosModified for Yale EENG 460a Lecture 4

Andreas Savvides (09/08/05)

Mobisys 2005 2

Minimal Program on an Embedded Processor

int main(void) {

Init_All(); for (;;) {

IO_Scan(); // bar-code scannerIO_ProcessOutputs(); KBD_Scan(); // keyboardPRN_Print(); // printerLCD_Update(); // displayRS232_Receive(); // serial portRS232_Send(); TIMER_Process(); // timer

} // should never ever get here

}

Mobisys 2005 3

Programming an embedded processor

What are the problems with the previous code fragment?

Some things about programming microcontrollers No memory management unit – be careful with

memory usage/protection Need to worry about peripherals Need to decide where to locate code on the

microcontroller memory Why does OS make a difference?

Mobisys 2005 4

Some Background…

TinyOS NesC

Mobisys 2005 5

Re-programming Challenges

Severe resource constraints on nodes 4 KB RAM, 128 KB FLASH, 2 AA batteries

Avoiding crashes Unattended operation - Crashed node is useless No architecture support for protection e.g. MMU

Balancing flexible and concise updates Update applications, services and drivers Energy efficient distribution and storage

Why do you need reprogrammability?

Mobisys 2005 6

Sensor Network OS State of the Art

TinyOS - Application specific OS Application, OS and drivers are NesC components Select app components, statically analyze and optimize Extensive set of well-tuned components Supports full binary upgrades

Maté - Application specific Virtual Machine Domain specific bytecode interpreter on TinyOS Programs are small scripts containing VM instructions Better suited for application specific tuning Interpreter updates require fallback to TinyOS

Others: Mantis, VM*

Mobisys 2005 7

Towards general purpose sensor OS

TinyOS and Maté Application and OS are tightly linked

Design Goal: An application independent sensor OS Independently written & deployed apps run on one network Towards traditional kernel space/user space programming

model Re-programming via binary modules

Risk: Lose safety provided by static analysis or dynamic interpreter

Design Challenge Provide general purpose OS semantics on resource

constrained embedded sensor nodes

Mobisys 2005 8

SOS Operating System

Dynamic operating system for sensor networks Kernel and dynamically-loadable modules Ported to Mica2, MicaZ, XYZ and Telos

Convenient, yet compact, kernel interface Dynamic function links - 10 bytes overhead/function

Safety features through run-time checks Type safe linkage, Memory overflow checks

Performance No worse than TinyOS for real world applications

Mobisys 2005 9

Contributions Framework for binary modular re-programming

Dynamic linking Message Passing Dynamic Memory

Inexpensive safety mechanisms for an embedded OS Type safe linking Monitored memory allocation Garbage collecting scheduler and error stub Watchdog mechanism

General purpose OS semantics on sensor nodes

Mobisys 2005 10

Architecture Overview

DynamicMemoryDynamicMemory

MessageSchedulerMessageScheduler

DynamicLinker

DynamicLinker

KernelComponents

SensorManagerSensor

ManagerMessaging

I/OMessaging

I/OSystemTimer

SystemTimer

SOSServices

Radio*Radio* I2CI2C ADC*ADC*DeviceDrivers

Tree RoutingModule

Tree RoutingModule

Data CollectorApplication

Data CollectorApplication

Photo-sensorModule

Photo-sensorModule

DynamicallyLoaded modules

Static SOS Kernel

* - Drivers adapted from TinyOS for Mica2

Mobisys 2005 11

SOS Overview

Programmed entirely in C

Co-operatively scheduled system

Event-driven programming model

System provides no memory protection

Mobisys 2005 12

Designing Safety Features

Dynamically evolving system Unspecified behavior resulting from transient states

Goals Ensure system integrity Graceful recovery from failures

Design Minimal set of run-time checks Designed for low resource utilization Does not cover all failure modes

Mobisys 2005 13

<Empty Space>

Installing Dynamic Modules Modules implement specific function or

task

Position independent binary

Loader stores module at arbitrary program memory location

Minimal state maintenance 8 bytes per module Stores module identity and version

SOS Kernel

Bootloader

Module 1

FLASH Layout

<Empty Space>

Mobisys 2005 14

Inter-module Communication

Module FunctionPointer Table

DynamicLinking

MessagePassing

MessageBuffer

Module AModule A Module BModule B

Module AModule A Module BModule B

Dynamic Linking Synchronous communication Blocking function calls that

return promptly

Message Passing Asynchronous communication Long running operations

Mobisys 2005 15

Dynamic Linking Overview Goals

Low latency inter-module communication comparable to direct function calls

Functional interface is convenient to program

Challenges Safety features to address missing and updated

modules

Constraints Minimize RAM usage

Mobisys 2005 16

Dynamic Linking Design Publish functions for the other parts of system to use Subscribe to functions supplied by other modules Indirection provides support for safety features Dynamic function call overhead

21 cycles compared to 4 cycles for direct function call

Module AModule A Module BModule B

<foo, B, FOO_ID, Type> <foo, B, FOO_ID, Type>

Function Control Block Table (FCB)

Subscribe Publish

Mobisys 2005 17

Dynamic Linking Safety Features

Module AModule A

<foo, B, FOO_ID, Type> <foo, B, FOO_ID, Type>

Function Control Block Table

Module BModule B

Error StubError Stub

Run-time Type Checking Module updates can introduce new function prototype Type mismatches are detected, error flag is raised

Mobisys 2005 18

Tree RoutingModule

Tree RoutingModule

Data CollectorApplication

Data CollectorApplication

SystemTimer

SystemTimer

MESSAGE<Dest. Addr><Dest. Mod. Id><Message Type><Payload> …

Kernel - module communication

Inter-module communication

System Scheduler

Message Passing System

Scheduler looks up handler of destination module Handler performs long operations on message payload

Mobisys 2005 19

Messaging Safety Features High priority messaging

Signal timing sensitive events (For e.g. hardware interrupts) Prevent interrupt chaining into the modules Concurrency management by kernel Eliminates race conditions in modules

Watchdog support Co-operatively scheduled system Long running message handlers trigger watchdog reboot Kernel terminates execution of the buggy module

Mobisys 2005 20

Module-Kernel Communication

Kernel services available as system calls Jump table redirects system calls to handlers Update kernel independent of modules System Call Overhead - 12 clock cycles

Data Collector ModuleData Collector Module

SystemJump table

PriorityScheduler

SOSKernel

Hardware

System Call

HW specific API Interrupt Service

System Messages

Mobisys 2005 21

Dynamic Memory Allocation

Need to allocate module state at run-time Design Choice - Fixed-partition allocation

Performance - Constant low allocation time (69 cycles) Resources - 1 byte overhead per block, 52 blocks

SOS provides memory safety features Guard bytes detect memory overflow Ownership tagging to track buggy modules

16 byte blocks 32 byte blocks 128 byte blocks

Guard Byte

Mobisys 2005 22

Garbage Collection Memory leakage problem Garbage collection on failed message delivery Destination module needs to signal ownership

Use the return code of the message handler SOS_OK - Kernel frees the dynamic memory SOS_TAKEN - Destination module owns memory

Module A

Message

PayloadDynamic Memory

Module BMessage Passing

Mobisys 2005 23

Surge: What does it do?

Small application that organizes nodes in a spanning tree, rooted at the basestation

Based on broadcast messages Each node maintains the ID of its parent and its

depth in the tree Depth advertised with every message

Nodes select their parents as the nodes with the least depth – and best signal

Root initiates the creation of the spanning tree by periodically broadcasting messages

Mobisys 2005 24

Evaluation Design Goal

Provide general purpose OS semantics Low resource utilization

Hypothesis Performance no worse TinyOS Update cost closer to Maté

Experiment Setup Surge data collection and tree routing on 3 hop network Low duty cycle application Mica2 motes: AVR 8-bit microcontroller

Mobisys 2005 25

Application Performance Comparison

Data Transfer Delay Packet Delivery Ratio

Application performance is nearly identical for TinyOS, SOS and Mate

Mobisys 2005 26

Performance Overhead

CPU Active Time - Metric to measure OS overhead Measured by profiling Surge for 1 min. on real nodes Averaged over 20 experiments for each system

SOS has 1% overhead relative to TinyOS Surge has minimal application level processing (“worst” case OS

overhead) Insignificant variation of average power consumption

Surge application has a very low CPU utilization System level energy: E(CPU) << E(Radio) Duty Cycling - Idle energy dominates over active energy

TinyOS SOS Maté

4.58% 4.64% 5.13%

29.92 29.94 30.02

Active Time (%)

Average Power(mW)

Mobisys 2005 27

Update Costs

Method Energy Cost

Entire binary upgrade (TinyOS) 784.14 mJ High

Modular binary upgrade (SOS) 12.25 mJ Moderate

Virtual Machine scripts (Maté) 0.34 mJ Low

Re-programming cost involves Communication Energy - Transfer the new code Storage Energy - Write the code to RAM/FLASH etc.

Impact on system level energy Depends significantly upon frequency of updates Difference in update cost amortized over the interval between updates Idle energy in the interval between updates dominates Idle energy consumption does not depend on the OS

Mobisys 2005 28

Lessons Learnt

Focus on duty cycling all parts of the system Standardize the API for power management of peripherals Performance optimization of the CPU is secondary

Account for update energy and frequency

Choose an OS based on the features it provides SOS - Flexibility of general purpose OS semantics TinyOS - Full system static analysis Mate VM - Efficient scripting interface

Mobisys 2005 29

Future Work New models for application development

Independent re-usable loadable binary modules

Hierarchy of re-configuration Maté VM ported to SOS - Extensible virtual machines Upgrade SOS kernel using TinyOS whole image technique

Staged checkers Combination of static and run-time checks for code safety

FLASH wear and tear management using SOS

Mobisys 2005 30

Programming

Programmed in C Function Registration

char tmp_string = {'C', 'v', 'v', 0}; ker_register_fn(TREE_ROUTING_PID, MOD_GET_HDR_SIZE, tmp_string,(fn_ptr_t)tr_get_header_size);

Mobisys 2005 31

Network Capable Messagestypedef struct { sos_pid_t did; // destination module ID sos_pid_t sid; // source module ID uint16_t daddr; // destination node uint16_t saddr; // source node uint8_t type; // message type uint8_t len; // message length uint8_t *data; // payload uint8_t flag; // options} Message;

Messages are best-effort by default.

No senddone and Low priority Can be changed via flag in

runtime

Messages are filtered when received.

CRC Check and Non-promiscuous mode

Can turn off filter in runtime

Mobisys 2005 32

SOS Messaging API// send message over netint8_t post_net( sos_pid_t did, sos_pid_t sid, uint8_t type, uint8_t length, void *data, uint8_t flag, uint16_t daddr);// send long messageint8_t post_long( sos_pid_t did, sos_pid_t sid, uint8_t type, uint8_t length, void *data, uint8_t flag);

// send message

int8_t post(Message *msg);

// short message struct

typedef struct {

uint8_t byte;

uint16_t word;

} MsgParam;

// send short message

int8_t post_short(

sos_pid_t did,

sos_pid_t sid,

uint8_t type,

uint8_t byte,

uint16_t word,

uint8_t flag);

Mobisys 2005 33

Messaging Example: Ping_pong

enum {

MSG_LONG_BALL = MOD_MSG_START,

MSG_SHORT_BALL = (MOD_MSG_START + 1),

};

enum {

PLAYER1_PID = DFLT_APP_ID0,

PLAYER2_PID = DFLT_APP_ID1,

};

typedef uint8_t ball_t;

typedef struct {

ball_t next_seq;

} player_t;

Mobisys 2005 34

Messaging Example: Ping_pongint8_t player(void *state, Message *msg){ player_t *s = (player_t*)state;

switch (msg->type){ case MSG_INIT: { //! initialize the state s->next_seq = 0; //! start with short ball if(msg->did == PLAYER1_PID) { post_short(PLAYER2_PID, PLAYER1_PID, MSG_SHORT_BALL, s->next_seq, 0, 0); } return SOS_OK; }

Mobisys 2005 35

Messaging Example: Ping_pong case MSG_SHORT_BALL: { MsgParam *p = (MsgParam*)(msg->data); s->next_seq = p->byte + 1; DEBUG("%d get short ball %d\n", msg->did, p-

>byte); if(p->byte % 2) { post_short(msg->sid, msg->did, MSG_SHORT_BALL, s->next_seq, 0, 0); } else { post_net(msg->sid, msg->did, MSG_LONG_BALL, sizeof(ball_t), &(s->next_seq), 0, ker_id()); } return SOS_OK; }

Mobisys 2005 36

Messaging Example: Ping_pong case MSG_LONG_BALL: { ball_t *b = (ball_t*)(msg->data); s->next_seq = (*b) + 1; DEBUG("%d get long ball %d\n", msg->did, *b); if((*b) % 2) { post_long(msg->sid, msg->did, MSG_LONG_BALL, sizeof(ball_t), &(s->next_seq), 0); } else { Message m; m.did = msg->sid; m.sid = msg->did; m.daddr = ker_id(); m.saddr = ker_id(); m.type = MSG_SHORT_BALL;m.len = sizeof(ball_t); m.data = &(s->next_seq); m.flag = 0; post(&m); } return SOS_OK; }

Mobisys 2005 37

Messaging Example: Ping_pong

default:

return -EINVAL;

}

}

void sos_start(void){

ker_register_task(DFLT_APP_ID0,

sizeof(player_t), player);

ker_register_task(DFLT_APP_ID1,

sizeof(player_t), player);

}

Mobisys 2005 38

Synchronous Communication

Module can register function for low latency blocking call (1).

Modules which need such function can subscribe it by getting function pointer pointer (i.e. **func) (2).

When service is needed, module dereferences the function pointer pointer (3).

Module FunctionPointer Table

Module A Module B

3

12

Mobisys 2005 39

Synchronous Communcation API

typedef int8_t (*fn_ptr_t)(void);

// register function

int8_t ker_register_fn(

sos_pid_t pid, // function owner

uint8_t fid, // function id

char *prototype, // function prototype

fn_ptr_t func); // function

// subscribe function

fn_ptr_t* ker_get_handle(

sos_pid_t req_pid, // function owner

uint8_t req_fid, // function id

char* prototype) // function prototype

Mobisys 2005 40

Memory Management Modules need memory to store state information

Problems with static memory allocation Worst case memory allocation – every variable is global Single packet in the radio stack – can lead to race conditions

Problems with general purpose memory allocation Non-deterministic execution delay Suffers from external fragmentation

Use fixed-partition dynamic memory allocation Memory allocated in blocks of fixed sizes Constant allocation time Low overhead

Memory management features Guard bytes for run-time memory over-flow checks Semi-auto ownership tracking of memory blocks Automatic free-up upon completion of usage

Mobisys 2005 41

SOS Memory API

// allocate memory to id

void *ker_malloc(uint16_t size, sos_pid_t id);

// de-allocate memory

void ker_free(void* ptr);

Mobisys 2005 42

Messaging and Dynamic Memory Messaging is asynchronous operation. Attaching dynamic

memory in post() results transfer of ownership. Bit Flag is used to tell SOS kernel the existence of dynamic

memory. SOS_MSG_DYM_ALLOC -- data is dynamically allocated SOS_MSG_FREE_ON_FAIL -- free memory when post fail. SOS_DYM_MANAGED = SOS_MSG_DYM_ALLOC | SOS_MSG_FREE_ON_FAIL

Dynamically allocated message payload will be automatically freed after module handling. This is the default. You can change it by return SOS_TAKEN

instead of SOS_OK to take the memory. Message header belongs to the kernel, and it will be recycled.

If you need them, make a deep copy.

Mobisys 2005 43

Asynchronous Module Kernel Interaction

Kernel provides system services and access to hardware Kernel jump table re-directs system calls from modules to kernel handlers Hardware interrupts and messages from the kernel to modules are dispatched through a

high priority message buffer Low latency Concurrency safe operation

Module A

SystemJump Table

Hardware

System Call

High PriorityMessage

Buffer

HW Specific API Interrupt

System Messages

SOS Kernel

Mobisys 2005 44

Schedule Message with Software Timer

Two priority: high and low. Normal timer has high priority while slow timer is not.

Two types: periodic and one shot

Module A

SystemJump Table

Delta Timer

Timer syscall

High PriorityMessage

Buffer

Timer API Interrupt

Timer Messages

SOS Kernel

Mobisys 2005 45

SOS Timer APIenum { TIMER_REPEAT = 0, // high priority, periodic TIMER_ONE_SHOT = 1, // high priority, one shot SLOW_TIMER_REPEAT = 2, // low priority, periodic SLOW_TIMER_ONE_SHOT = 3, // low priority, one shot};

int8_t ker_timer_start( sos_pid_t pid, // module id uint8_t tid, // timer id uint8_t type, // timer type int32_t interval // binary interval);int8_t ker_timer_stop( sos_pid_t pid, // module id uint8_t tid // timer id);

Mobisys 2005 46

Timer Example: Blink

#include <sos.h>#define MY_ID DFLT_APP_ID0int8_t blink(void *state, Message *msg) { switch (msg->type) { case MSG_INIT: //!< initial message from SOS //! 256 ticks is 250 milliseconds ker_timer_start(MY_ID, 0, TIMER_REPEAT, 256); return SOS_OK; case MSG_TIMER_TIMEOUT: //!< timeout message arrived ker_led(LED_RED_TOGGLE); return SOS_OK; default: return -EINVAL; }}void sos_start() { ker_register(MY_ID, 0, blink);}

Mobisys 2005 47

Sensor Manager

PeriodicAccess

Request

Sensor Manager

Module A Module B

Sensor 2

Data Policy

Sensor 1

Data

DataPolledAccess

Enables sharing of sensor data between multiple modules

Presents a uniform data access API to many diverse sensors

Underlying device specific drivers register with the sensor manager

Device specific sensor drivers control Calibration Data interpolation

Sensor drivers are loadable Enables post-deployment configuration of

sensors Enables hot-swapping of sensors on a running

node

Mobisys 2005 48

Network Simulation Support

Source code Level Network Simulation Pthread is used to simulate hardware. UDP is used to simulate communication with perfect radio

channel. Support user defined topology and heterogeneous software

configuration. Useful for verifying the correctness of the implementation of

the application.

Avrora: Instruction Level Simulation Instruction cycle accurate simulation. Simple perfect radio channel. Useful for verifying timing information. See http://compilers.cs.ucla.edu/avrora/

Mobisys 2005 49

Discussion

For more information look at SOS Project Website

SOS tutorial & Sample code

XYZ Manual for details on using dynamic modules