View
2
Download
0
Category
Preview:
Citation preview
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Real-Time USB Communication in the Quest
Operating System
Eric Missimer, Ye Li, Richard West
Eric Missimer, Ye Li, Richard West Real-Time USB 1
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Table of contents
1 Contributions
2 USB 2.0 and EHCI Outline
3 USB Scheduling Problem
4 Experimental Evaluation
5 Conclusions
Eric Missimer, Ye Li, Richard West Real-Time USB 2
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Quest USB Subsystem
Quest:
Boston University’s in house operating system
Developed for real-time and high-confidence systems
Major Contributions:
Bandwidth allocation guarantees when opening connectionswith endpoints
Dynamic reordering of transfer requests to avoid unnecessaryrejections
Real-time guarantees for both asynchronous and periodictransfers
Eric Missimer, Ye Li, Richard West Real-Time USB 3
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Motivation - Why USB?
Precise timing: 125 µs time granularity
To date, USB 2.0 is the highest bandwidth I/O on SBC(BeagleBoard, PandaBoard, Raspberry Pi)
USB 3.0 is just starting to appear on such SBC
One common bus for I/O devices and communication
Towards a real-time interconnect for distributed embeddedsystems
Eric Missimer, Ye Li, Richard West Real-Time USB 4
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
USB 2.0
Master-Slave Protocol
Time is broken up by 125 µs chunks called micro-frames
8 micro-frames make up one frame (1 millisecond)
Transactions cannot cross micro-frames
Four types of USB transactions:
(1) Bulk and (2) Control - Asynchronous(3) Isochronous and (4) Interrupt - Periodic
For each micro-frame at most 80% can be used for periodictransactions
Eric Missimer, Ye Li, Richard West Real-Time USB 5
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
USB Transaction Types
Endpoint Data Integrity Guaranteed Example
Type Ensured Transfer Rate
Control Yes No All Devices
Bulk Yes No Mass Storage
Isochronous No Yes Camera
Interrupt Yes Yes Keyboard
Eric Missimer, Ye Li, Richard West Real-Time USB 6
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
USB 2.0 Host and Device
USB Host:
Host controller makes all scheduling decisions
More complex circuitry than a device
USB Device:
Device contains one or more endpoints
Each endpoint has a:
NumberDirectionTransaction Type
Packet Size
Interval (for periodic transactions, ×2n micro-frames)
Eric Missimer, Ye Li, Richard West Real-Time USB 7
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
USB Device Stack
Eric Missimer, Ye Li, Richard West Real-Time USB 8
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI - Enhanced Host Controller Interface
Information for performing USB transactions are contained intransaction descriptors (TDs)
Handles asynchronous and periodic transactions differently
Separate scheduling data structures for each class oftransactions
Isochronous TDs (iTDs) and Queue Head (QH) plus queueTDs (qTDs)
Eric Missimer, Ye Li, Richard West Real-Time USB 9
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling
Eric Missimer, Ye Li, Richard West Real-Time USB 10
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Processing TDs
At the start of the micro-frame the host controller does thefollowing:
Use the frame index register to index into the periodic list
Process the iTDs and QHs/qTDs in the periodic list pointedto by the periodic list entry
Process the next QH in the asynchronous circular list until theend of the micro-frame
Eric Missimer, Ye Li, Richard West Real-Time USB 11
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 12
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 13
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 14
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 15
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 16
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 17
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 18
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 19
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Scheduling: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 20
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
USB Scheduling Problem
Periodic USB requests can be represented as a tuple (wi , ti ).
wi time to send transaction i
ti interval of transaction i
Scheduling problem is given a set of tuples{(w1, t1) , (w2, t2) , . . . , (wn, tn)} is there an assignment ofUSB transactions to micro-frames such that no micro-frame isover-committed.
If a request is assigned to micro-frame f it is also assigned tomicro-frames f + n ∗ ti , n ∈ N
Eric Missimer, Ye Li, Richard West Real-Time USB 21
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Scheduling Example
Eric Missimer, Ye Li, Richard West Real-Time USB 22
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Scheduling Example
Eric Missimer, Ye Li, Richard West Real-Time USB 23
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Scheduling Example
Eric Missimer, Ye Li, Richard West Real-Time USB 24
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Invalid Scheduling Example
Eric Missimer, Ye Li, Richard West Real-Time USB 25
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Quest USB Scheduling
Heuristic that parallels first fit decreasing algorithm for bin-packingand rate monotonic scheduling
Sort requests by interval from smallest to largest, breaking tiesby sorting transmission delay from largest to smallest
Assign sorted requests to first available micro-frame
1e+0
0
1e+0
1
1e+0
2
1e+0
3
1e+0
4
1e+0
5
1e+0
6
1e+0
7
1e+0
8
Quest
Linux
# Failed Schedules1 to 5 requests
Intervals: 2, 4, 8, 16
Packet Sizes: 32, 64, . . . , 1024
Quest ≈ 150, 000
Linux ≈ 95, 000, 000
Eric Missimer, Ye Li, Richard West Real-Time USB 26
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Experimental Evaluation
Beagleboard:
Angstrom LinuxCustom USB-GadgetDevice Driver
Functionally equivalentLinux and Quest devicedrivers
Eric Missimer, Ye Li, Richard West Real-Time USB 27
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Maximum Throughput
9 isochronous endpoints with an interval of 1 and 9 bulkendpoints, packet size for all is 512-bytes
0
10
20
30
40
50
60
Linux Quest
Dat
a R
ate
(KB
/ms)
35.94
47.87
35.99
48.00
PeriodicAsynchronous
The design of the Quest USB subsystem is not detrimental tooverall throughput
Eric Missimer, Ye Li, Richard West Real-Time USB 28
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation Experiment
Eric Missimer, Ye Li, Richard West Real-Time USB 29
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation Experiment
Eric Missimer, Ye Li, Richard West Real-Time USB 30
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation Experiment
Eric Missimer, Ye Li, Richard West Real-Time USB 31
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation Experiment
Eric Missimer, Ye Li, Richard West Real-Time USB 32
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation Experiment
Eric Missimer, Ye Li, Richard West Real-Time USB 33
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation Experiment
Eric Missimer, Ye Li, Richard West Real-Time USB 34
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Bandwidth Allocation: Results
Tokens Passed Tokens Passed Third Beagleboard
OS Without Third With Third Isoc Endpoints
Beagleboard Beagleboard Opened
Quest 1499 1499 5
Linux 1499 300 7
Eric Missimer, Ye Li, Richard West Real-Time USB 35
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Dynamic Reordering of Transactions
System can reorder transactions in the periodic list to admitnew USB requests that would otherwise be rejected
Reordering algorithm is uses heuristic for USB schedulingproblem
Transactions are reordered while maintaining endpointconstraints
Endpoints violate their poll/push rate for one interval at most
Eric Missimer, Ye Li, Richard West Real-Time USB 36
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Dynamic Reordered of Transactions: Example
Eric Missimer, Ye Li, Richard West Real-Time USB 37
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Dynamic Reordered of Transactions
First Ordering: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Second Ordering: 4, 5, 6, 7, 8, 9, 10, 0, 1, 2, 3
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Isoc 0-3 Interval = 2
Isoc 4-8 Interval = 1
Isoc 9-10 Interval = 1
Dat
a R
ate
(KB
/ms)
Endpoint
Quest Ordering 1Quest Ordering 2Linux Ordering 1Linux Ordering 2
Eric Missimer, Ye Li, Richard West Real-Time USB 38
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Real-Time Asynchronous
In most systems, asynchronous transactions (Bulk & Control)are typically handled in a best effort manner
Guaranteed at least 20% of each micro-frame
In Quest, asynchronous transactions can be assigned aninterval value just as periodic (specified by the device driverinstead of the endpoint)
Eric Missimer, Ye Li, Richard West Real-Time USB 39
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Real-Time Asynchronous: Experiment and Results
Seven bulk, four isochronous and three interrupt endpointsAll have an interval of one and a packet size of 512-bytes
0
1
2
3
4
5
Bulk 0-6 Isoc 7-10 Interrupt 11-13
Dat
a R
ate
(KB
/ms)
Endpoint
QuestLinux
Eric Missimer, Ye Li, Richard West Real-Time USB 40
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Conclusions
Heuristic algorithm that outperforms Linux’s first fit approachfor the USB scheduling problem
Bandwidth allocation guarantees when opening connectionswith endpoints
Dynamic reordering of transfer requests to avoid unnecessaryrejections
Real-time guarantees for both asynchronous and periodictransfers
Eric Missimer, Ye Li, Richard West Real-Time USB 41
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Thank You
Thank you for your time
More information can be found at:http://www.cs.bu.edu/~richwest/quest.html
Eric Missimer, Ye Li, Richard West Real-Time USB 42
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Questions?
Questions?
Eric Missimer, Ye Li, Richard West Real-Time USB 43
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Transfer Descriptors
Isochronous Transfer Descriptor(iTD)
Only used for Isochronoustransactions
Contains the information forat most 8 isochronoustransactions
Multiple iTDs for a singleendpoint in the periodic list
Eric Missimer, Ye Li, Richard West Real-Time USB 44
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
EHCI Transfer Descriptors cont.
Queue Head (QH) and QueueElement Transfer Descriptor(qTD)
Used for Control, Bulk andInterrupt transactions
Contains the information formultiple transactions.
Forms a linked list withother qTDs
Head of the list is a QH
Eric Missimer, Ye Li, Richard West Real-Time USB 45
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Real-Time Asynchronous cont.
One caveat: Asynchronous transactions are in a circular linkedlist
We have to schedule all asynchronous transactions as havingan interval equal to the smallest asynchronous interval
Can then leave room in periodic schedule for asynchronoustransfers
Eric Missimer, Ye Li, Richard West Real-Time USB 46
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Dynamic Reordered of Transactions cont.
Eric Missimer, Ye Li, Richard West Real-Time USB 47
ContributionsUSB 2.0 and EHCI Outline
USB Scheduling ProblemExperimental Evaluation
Conclusions
Future Directions
USB OTG and USB device to PCI (e.g. Net2280) support
USB 3.0 - XHCI host controller driver
USB FPGA-based router for P2P communication over USB
Eric Missimer, Ye Li, Richard West Real-Time USB 48
Recommended