Embed Size (px)
Citation preview
Architectural Patterns In
IoT Cloud PlatformsRoshan Kulkarni
Cloud. Mobility. Internet of Things.
IoT PaaS Platforms Galore!
Are we spoilt for choice?
Why patterns?
§ Understand the recurring nature of design problems in IoT.
§ Capture essential aspects of a successful IoT implementation.
§ Reuse components to accelerate your solution development.
§ Understand feature-parity and differentiators across various IoT platforms.
§ Checklist to validate architectural readiness of your solution.
‘An architectural pattern is a general, reusable solution to a
commonly occurring problem in software architecture
within a given context…’
Patterns we’ve observed…Patterns Patterns
1. Edge Connectivity Patterns 7. COTA
2. Device Identity & Enrollment 8. FOTA
3. Brokered Communication 9. Virtual Device Representation
4. Control Plane 10. Stream Processing
5. Heartbeat 11. Telemetry Ingestion
6. Unified Endpoint Management 12. Event-Driven Micro Services
* Not claimed to be an exhaustive list, as the domain is rapidly evolving. :)
Personal Area
Networks
Pattern 1: Edge-Connectivity Patterns
Smart IP Capable Devices
IoT Field Gateway
PAN and Low Power Devices
Custom Cloud
Gateway (Protocol
Translation)
Legacy IP Capable Devices
IoT Cloud Gateway
IP-Capable ButLow Compute
Legacy IP Capable Devices
IoT Cloud Backend >><< Field / WAN
A
B
C
Proprietary IP Capable Devices
Pattern 1A: IoT ‘Cloud Gateway’
§ Termination for IoT Protocols:
§ MQTT, AMQP
§ Web Sockets, BSD Sockets…
§ Terminate Persistent, Transport-layer, Device Connections.
§ Secure Tunnel for Data in Motion:
§ TLS / SSH / IPSec.
§ Enforce Security Safeguards:
§ Firewalls, DDOS, Rate Limits.
§ Authorization and Admission-Controls for Devices.
§ Access Policy Enforcement:
§ Permission to Topics, Queues…
§ Tenant Metering:
§ Enforce Quotas and Throughput.
§ Horizontally Scalable:
§ Support Millions of Devices.
§ TCP / HTTP Load Balancing.
§ Process Per Connection
§ Thread Per Connection
§ Non-blocking IO.
Pattern 1B: IoT ‘Field Gateway’
When Applicable?
§ Devices Without a TCP/IP Stack:
§ PAN Protocols
§ BT, ZigBee, Z-Wave, ModBus, I2C.
§ Devices with Low Compute Power:
§ No support for tunnels or crypto.
§ Legacy Devices With IP Stack:
§ Without Support for IoT Protocols.
§ SNMP Managed Devices…
§ How do they hook to IoT Cloud?
Primary Responsibilities…
§ Support Near-field PAN Protocols, RS 232, RS 485 Interfaces.
§ Edge Filtering, Data Shaping, Data Aggregation, Edge Analytics!
§ Monitor Sensor & Actuator Health.
§ Localized ‘Control Loops’:
§ Local Decision Tree.
§ Provide A Secure Tunnel to the IoT Cloud Gateway.
Pattern 1C: ‘Protocol Translation Gateway’
§ Legacy Devices and Protocols:
§ Ex: Modbus TCP/IP.
§ Devices which Need:
§ Custom Authentication
§ Custom Authorization
§ Custom Encryption, Compression
§ Protocol Impedance MismatchBetween Devices and IoT Cloud Gateway.
§ Support Low-Latency, In-flight Protocol Translation.
Legacy IP Capable Devices
IoT Cloud Gateway
Protocol Translation Gateway
Legacy: BSD Sockets, Custom Authentication, Custom Authorization
Contemporary: MQTT, AMQP, X.509 Certificates
Pattern 2: Device Identity Patterns
§ Primary Concerns:
§ Device Identity Provisioning?
§ Identity Uniqueness?
§ Cloning and Impersonation?
§ Identity Lifecycle M’gmt?
§ Physical Device Identity:
§ SoC Identifier
§ MAC Address
§ M2M SIMs / MIMs
§ Trusted Platform Modules (TPM)
§ Logical Device Identities:
§ Device UUID
§ User Credentials / Binary Credentials
§ X.509 Client Certificates
Pattern 2: Device Enrollment Patterns
§ ‘Foundry Assisted’ Enrollment:
§ Keys + Certs Embedded During The Firmware Build Process.
§ ‘Factory Assisted’ Enrollment:
§ Keys + Certs Embedded at Unit Manufacturing Time.
§ ‘Provider Assisted’ Enrollment:
§ Using TPM or UUID.
§ JIT Enrollment at First Boot.
§ ‘User Assisted’ Enrollment:
§ Unit Deployment Time.
§ Companion App (BT, WiFi).
§ Companion Web Interface.
Pattern 2A: Foundry Assisted Enrollment
Certificate Authority
Firmware Build Process
Device
Client Certificate, Private Key
4. Out of Band Installation of
Certificate (Burning the
Standard Firmware)
2. Request Certificate (CSR)
1. Generate Key Pair and CSR
3. Issue Certificate
• Same keys & certificates across all devices!
• File system based key-store.
• MCU - JTAG fuse blown after burning the firmware.
• Certificate lifecycle management?
• Anti-pattern?
Device
Pattern 2B: Factory Assisted Enrollment
Certificate Authority
Factory Buildout Process
Client Certificate, Private Key 4. Out of Band
Installation of a Unique Certificate During the Build-out of Each Unit.
2. Request Certificate (Public Key, CSR)
1. Generate Key Pair and CSR
3. Issue Certificate
• Unique Key-Pair for Each Device.
• Complicates the manufacturing process!
• File system based key-store?
• MCU - JTAG fuse blown after burning the key and device certificate.
• Certificate lifecycle management?
Device
Pattern 2C: Provider Assisted JIT Enrollment
Enrollment API
Client Certificate, Private Key
4. Request Certificate (CSR)
3. Verify Fingerprint, TPM Attestation
5. Issue Certificate
2. Enrollment Request (Provisioning Keys, Device Fingerprint,
CSR)
Certificate Authority
6. Enrollment Response (Client Cert)
1. Generate Key Pair, CSR Device
Registry
TPM
• Just-in-Time Enrollment to the IoT Cloud.
• TPM-based Key Store: Key Pair Generation, Crypto Functions.
• Easier Certificate Lifecycle Management (Revocation, Renewal).
Device
Pattern 2D: User Assisted Enrollment
Enrollment API
Client Certificate, Private Key 6. Request Certificate
(CSR)7. Issue Client
Certificate
3. Enrollment Request (User Creds, Fingerprint, CSR)
Certificate Authority
8. Enrollment Response (Client Cert)
Companion App
Directory Service
1. Generate Key Pair, CSR
2. PAN Pairing
4. Verify User Credentials
5. Verify Fingerprint
Device Registry
• Not a ‘Headless’ Enrollment Mechanism!
• Companion App used to Validate User Identity Before Device Enrollment.
• Client Certificates Transferred to the Device Over a Local PAN Interface (Bluetooth)
Pattern 3: Brokered Communication Model
§ Communication Models:
§ Request-Response
§ Need a Bi-directional Asynchronous Communication Model.
§ Decoupling of Parties is a Critical Requirement.
§ Durable Messaging Queue / Pub-Sub / Service Bus:
§ Accommodate ‘Latency Mismatch’ Between Producers and Consumers.
§ Primary Messaging Backbone within most IoT Cloud Platforms.
§ Message Delivery Semantics:
§ At most once / At least once / Exactly once.
§ Ordered vs Unordered Delivery.
Pattern 4: Device Control Plane
§ Polling by Device – Inefficient!
§ Need a Push-based Mechanism.
§ Push Commands & Notifications.
§ Intermittent Device Connectivity:
§ Hibernation > Power Conservation.
§ Choppy WAN Connectivity.
§ ‘Inbox’ Pattern for Offline Devices.
§ Typically Uses:
§ At Least Once Delivery Semantics.
§ Highest QoS Level.
§ Target Addressing Modes:
§ Unicast (Device UUID).
§ Multicast (Tags or Device Groups).
§ Broadcast (All Tenant Devices).
Pattern 5: Device Heartbeat
§ Would the TCP Keep-Alive Mechanism Suffice?
§ An Optional Feature in TCP. Some Hosts Do Not Implement It.
§ Off By Default.
§ Would a Application-layer Protocols Suffice?
§ MQTT => Keep Alive with PING => LWT Notifications
§ Keep-Alive Implementation at the Application Layer:
§ Periodic Call Home.
§ Piggyback Real-time Health Information of the Device + Device Fingerprint.
§ Notify Application When Devices go Offline:
§ Ex: No Heartbeat for 5 mins.
Pattern 6: Unified Endpoint Management
§ Device Registry / Inventory:
§ Device Meta-data and Attributes.
§ Manufacturer, Firmware Revision, Capability Matrix.
§ Device Lifecycle Management / Fleet Management:
§ Remote Logging.
§ Remote Reboot.
§ Remote Lock, Unlock.
§ Wipe Device State / Factory Reset.
§ Device Fingerprint Request.
§ Remote Firmware Upgrade.
§ Track Audit Trail of Device Lifecycle Changes
Pattern 7: Configuration Over the Air (COTA)
§ Schema-Driven Configuration Management:
§ Schema Definition and Versioning.
§ Ex - Apache Avro.
§ Layered Configuration:
§ Default Configuration Upon Enrollment.
§ Override with Tag-specific Configuration (Logical Device Groups).
§ Override with Device-specific Configuration (Based on User Identity etc).
§ Configuration Push Mechanism:
§ Exchange Configuration Master Frames.
§ Exchange Deltas Only.
Pattern 8: Firmware Over the Air (FOTA)
§ Why?
§ Move Fast and Break Things!
§ Shorter Time to Market
§ Granularity of Updates:
§ Root FS + Boot FS
§ Application Layer Only
§ Application Dependencies
§ Planning Firmware Images:
§ Ensuring ‘Brick-safe’ Rollouts
§ Checksums / HMAC
§ Well Defined Versioning Scheme
§ Chunked Transfers
§ Silent vs Assisted
§ Process:
§ Canary Releases
§ Automatic Roll Back Capability
§ Efficient ‘Delta’ Mechanism:
§ Ubuntu Snappy Core (Snaps)
§ Container Deltas (Layers)
Silently Propagate Desired State
Pattern 9: Virtual Device Representation
§ How Cloud App interacts with device?
§ Push Command > Modify Device State.
§ Receive Confirmations Asynchronously.
§ Async paradigms increase Cloud application complexity.
§ Can we have simpler ‘device abstraction’ visible to your cloud applications?
§ Virtual Devices in the Cloud:
§ State of IO Pins and Flags on Device.
§ Represented as JSON / Cloud Object.
§ Silent State Synchronization.
Last Known State
Desired State
Delta
Client SDK
Cloud Application
Physical Reality
Virtual Device Representation (Cloud)
‘Instate’ the Desired State on
the Device
Pattern 10: Stream Processing
§ Hot Path!
§ Why?
§ Identify Real-time Events
§ Generate Actions & Alerts
§ Metering & Analytics
§ Nature of Streaming Data:
§ JSON, CSV, TSV, XML
§ High Volume
§ High Velocity
§ Consume Secondary Streams
§ Service Considerations:
§ HA of the Streaming Service
§ High Performance: Sub-1-second
§ Data Resilience
§ Used For…
§ Data Cleansing.
§ Range Normalization
§ Linearization
§ Sensor Temperature Drift
§ Sensor Hysteresis
§ Schema Normalization
§ Control Decisions & Thresholds
§ Data Aggregation
§ Interleaving Contextual Data
Pattern 10: Stream Processing
IFTT Paradigm:IF <condition>THEN <do something>
Compute Topology:
SQL / DSL Paradigm:SELECT color AS red WHERE furnace.temp > 112FROM ’path/topic'
Examples:§ AWS IoT Rules Engine
§ AWS Kinesis + KCL
§ Apache Kafka + Storm
Pattern 11: Telemetry Ingestion
§ Cold Path!
§ Remember Everything
§ Why?
§ Offline Analytics
§ Predictive Maintenance Alerts
§ Retroactive Insights
§ Big Data Stores:
§ Durable, Replicated, Distributed
§ Sliding Window or Time Series
§ Micro-Batching: Load Stream to Store
§ Examples:
§ Kinesis > FireHose > Redshift
§ Hadoop & MapReduce
Pattern 12: Event-Driven Micro Services
§ Server-less Architecture:
§ AWS Lambda Functions
§ Invoke ‘Stateless Functions’ On-Demand
§ No Capacity Planning, No Cloud Ops!
§ Lambda Function Encapsulates Connectors For:
§ Enterprise Application Integration
§ Third-party Micro Services
§ Notification Services
Bare Metal
Virtual Machines
Containers
Lambda Functions
Evolution…
Next?
§ Discover More Patterns…
§ Notifications, Analytics
§ Machine Learning
§ CloudOps and IoT Ops
§ Formal Abstractions…
§ Fear of PaaS Lock-In?
§ Provider-agnostic Baseline APIs
for IoT PaaS.
§ Commoditization enables easier
‘Lift-and-Shift’ between Providers.
§ We’ve seen this happen before in
POSIX and IaaS!
THANK [email protected]
