Time-Synchronized Visualization of Arbitrary Data Streams ... · vi, the rapid window manipulation...

Time-Synchronized Visualization of Arbitrary Data Streamsfor Real-Time Monitoring and Historical Analysis⇤

Paul Z. KolanoNASA Advanced Supercomputing Division

NASA Ames Research Center, M/S 258-6

Moffett Field, CA 94035 U.S.A.

paul.kolano@nasa.gov

AbstractLarge installations involve huge numbers of interactingcomponents that are subject to a multitude of hardwarefailures, transient errors, software bugs, and misconfigu-ration. Monitoring the health, utilization, security, and/orconfiguration of such installations is a challenging task.While various frameworks are available to assist withthese tasks at a high level, administrators must more of-ten than not revert to using command line tools on indi-vidual systems to get a low-level understanding of sys-tem behavior. The output from such tools can be difficultto grasp on even a single system, so when taken acrossa large number of hosts, can become completely over-whelming.

A variety visualization tools and techniques have beenproposed to increase the amount of information that canbe processed by humans at once. Existing tools, how-ever, do not provide the flexibility, scalability, or us-ability needed to assist with all the varied informationstreams possible in large installations. In particular, thesetools often require data in a specific format and/or in aspecific location with interfaces that have little relationto the underlying commands from which the data origi-nates.

Savors is a new visualization framework for the Syn-chronization And Visualization Of aRbitrary Streams.The goal of Savors is to supercharge the command-linetools already used by administrators with powerful vi-sualizations that help them understand the output muchmore rapidly and with far greater scalability across sys-tems. Savors not only supports the output of existingcommands, but does so in a manner consistent with thosecommands by combining the line-editing capabilities ofvi, the rapid window manipulation of GNU screen, thepower and compactness of perl expressions, and the ele-gance of Unix pipelines. Savors was designed to support

⇤This work is supported by the NASA Advanced SupercomputingDivision under Task Number ARC-013 (Contract NNA07CA29C) withComputer Sciences Corporation

impromptu visualization, where the user can simply feedin the commands they were already using to create alter-nate views with optional on-the-fly aggregation of infor-mation across many systems. In this way, visualizationbecomes part of the administrator’s standard repertoireof monitoring and analysis techniques with no need fora priori aggregation of data at a centralized resource orconversion of the data into a predefined format.

Savors can show any number of data streams eitherconsolidated in the same view or spread out across mul-tiple views. In multi-data scenarios, data streams canbe synchronized by time allowing even distributed datastreams to be viewed in the same temporal context. Insingle-data multi-view scenarios, views are updated inlockstep fashion so they show the same data at the sametime. Together with its integrated parallelization capabil-ities, this allows Savors to easily show meaningful resultsfrom across even very large installations.

Savors consists of three components: a console, somenumber of data servers, and some number of views.The console is responsible for user interaction, spawningdata servers and views according to the given commandpipelines, and controlling synchronization between datastreams. The data servers are responsible for spawningand interacting with the commands that generate data,manipulating the data as specified, and sending the datato the console and views. Finally, the views are respon-sible for visualizing the data as specified on one or moredisplays.

Savors is open source and available for download athttp://savors.sf.net.

1

Proc. of the 28th USENIX Large Installation System Administration Conf., Seattle, WA, Nov. 9-14, 2014