Embed Size (px)
DESCRIPTION
As users demand greater scalability from Citrix XenServer, the transmission of performance data from guests via xenstore is increasingly becoming a bottleneck. Future use of service domains is likely to make this problem worse. A simple, efficient way of transmitting time-varying datasets between userspace components in different domains is required. This talk will propose a lock-free mechanism to allow interdomain reporting of performance data without relying on continuous xenstore usage, and describe how it fits into the XAPI toolstack.
Citation preview
XAPI projectXenAPIXAPIXCP/XenServer toolstack
XAPI projectHigh level OO API around XenHTTP interfaceCollects performance data from dom0/guests
Performance dataNumericChanges regularly - requires samplingIf we sometimes lose a sample - that's OK
Interdomain performance dataCurrently one metric per domU: free memoryTransmitted via xenstoreEven this has been enough to cause performance problems!
Improvements so farData collection caused laggy API response from XAPI-> XAPI disaggregation: rrdd, networkd and xenopsd
Improvements so farHigh xenstore load reduced bootstorm performance
-> Watch xenstore instead of polling
What next?Take xenstore out of the data pathCreate a generic API for:
Driver domain monitoringApplication monitoring...
rrdd plugins - dom0
rrdd plugins - interdomain
rrdd plugins - protocol v1header 10 bytesdata length (ASCII) 8 bytesmd5sum (ASCII) 32 bytesjson data inc. timestamp ? bytes
rrdd plugins - protocol v1DATASOURCES <- header00000a79 <- data length6cc0472a94896d245dcea04a37c26474 <- data md5sum{ "timestamp": 1407715758, "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value": "0.55", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } }}
rrdd plugins - protocol v1DATASOURCES <- header00000a79 <- data length6cc0472a94896d245dcea04a37c26474 <- data md5sum{ "timestamp": 1407715758, "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value": "0.55", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } }}
rrdd plugins - protocol v2Separate data from metadataPack as much as possible as binary
rrdd plugins - protocol v2header 10 bytesdata crc32 4 bytesmetadata crc32 4 bytesdatasource count (int32) 4 bytestimestamp (int64) 8 bytesdatasource values n * 8 bytesmetadata length (int32) 4 bytesjson metadata ? bytes
rrdd plugins - protocol v2header 10 bytesdata crc32 4 bytesmetadata crc32 4 bytesdatasource count (int32) 4 bytestimestamp (int64) 8 bytesdatasource values n * 8 bytesmetadata length (int32) 4 bytesjson metadata ? bytes
rrdd plugins - protocol v2DATASOURCES <- header.... <- data CRC.... <- metadata CRC.... <- datasource count........ <- timestamp........ <- datasource value.... <- metadata length{ "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } }}
rrdd plugins - protocol v2DATASOURCES <- header.... <- data CRC.... <- metadata CRC.... <- datasource count........ <- timestamp........ <- datasource value.... <- metadata length{ "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } }}
rrdd plugins - protocol v2V2 protocol gives about a 10x speedup in the "usual" case, i.e. set ofexported datasources doesn't change.
Demo
Further infoProtocol:Frontend:Slides:IRC: freenode #xen-apiEmail: [email protected]
github.com/xapi-project/rrd-transportgithub.com/xapi-project/ocaml-rrdd-pluginjohnelse.github.io/xen-summit-2014
