New Media for Scientific Visualization

8/15/2019 New Media for Scientific Visualization

1/126

New MediA for

scientific data

visualization

Andrea Fasolo Rao


2/126


3/126

New Media for scientific

data visualization

Andrea Fasolo Rao

279245

Supervisor:Marco Ferrari

A.A 2014/2015

Sessione di LaureaMarzo 2016

Università Iuav di Venezia

Dipartimento di Progettazione e pianificazione in

ambienti complessi

Corso di Laurea Magistrale in Design del Prodotto edella Comunicazione Visiva

Curriculum Comunicazioni visive e multimediali


4/126


5/126


6/126

6 New Media for Scientific Data Visualization

Abstract

Italiano

Questa tesi affronta il tema dell’interpretazionee della visualizzazione dei dati scientifici, su come siè sviluppata negli ultimi decenni, il suo legame conla tecnologia informatica e la posizione nei riguardi

dei principi del design dell’informazione più tradizio-nale, oltre all’enorme influenza che il fenomeno OpenData sta avendo sul mondo della visualizzazione ingenerale. Verranno analizzati inoltre vari progettibasati sulla visualizzazione dati, ma che la affrontanoda punti di vista diversi: quello puramente scienti-fico, quello del design e quello artistico. Quest’ultimotipo di progetti potrebbe essere fondamentale perl’evoluzione della visualizzazione scientifica e di datiin generale: mettendo al primo posto valori esteticipiuttosto che informativi esplora nuove modalità diapproccio ai dati precedentemente non prese in con-

siderazione Inoltre, l’utilizzo di strumenti consideratiinnovativi (video, new media, interattività, suono)può dare nuova vita alla divulgazione scientifica ealla visualizzazione di dati. La complessità e la vastitàdei fenomeni e dei dati coinvolti, applicando algoritmidi “traduzione” in immagini e suoni, può contribuire acostruire esperienze di valore sia divulgativo che es-tetico. Sfruttando questo valore estetico si può avvici-nare il pubblico a tematiche apparentemente comp-lesse ma affascinanti, che normalmente passano performe di divulgazione più tradizionali che, negli annia venire, rischiano di perdere “appeal” agli occhi di unpubblico sempre più esigente, in termini di entertain-ment e “spettacolarità”.


7/126

7

Abstract

English

This thesis deals with the interpretation andvisualization of scientific data, on how it has devel-oped in recent decades, its connection with computertechnology and its position regarding the principles

of the more traditional information design, and alsothe huge influence of the Open Data phenomenon ishaving on the visualization world in general. Vari-ous data visualization projects will be analyzed, thatapproaches it from different angles: the purely sci-entifically one, the design one and the art one. Thislast kind of project could be fundamental for the sci-entific, and data too, visualization evolution: with itsprimary focus on aesthetic values rather than infor-mative it explores new ways to approach data; in ad-diction the use of innovative instruments (video, newmedia, interactivity, sound) can breathe new life in

science popularization and to data visualization. Thecomplexity and breadth of the involved phenomenaand data, together with appropriate “translating” al-gorithms to sounds and images, can create informa-tive, popularizing and aesthetic experiences. Lever-aging this aesthetic value, it can be possible to bringthe public closer to apparently complex but fascinat-ing themes, since the more traditional popularizationand communication means will lose of “appeal” to analways more demanding and difficult to astound pub-lic, that is becoming used to always more spectacularand entertaining means of communication.


8/126


Contents

1. Introduction

Why to investigate Scientific Visualization

2. Scientific Visualization

What is Scientific Visualization

Historic Scientific Visualization

Scientific Visualization, computers and computer graphics

3. Open Data

Beginning of the Open Data era

Early Open Data examples

The birth of public open data

The future: Linked Data

4. Art, Science and Design

Separation of Science and Aesthetics

Evaluating attractiveness

The role of art

10

12

14

16

18

24

32

34

36

37

40

44

46

48

50


9/126

9

5. Case Studies

Case studies selection

Ryoji Ikeda, Supersymmetry

Nathalie Miebach, Weather scores

Alex H. Parker, Worlds

Aaron Koblin, Nik Hafermaas, Dan Goods, eCLOUD

Soo-in Yang, David Benjamin, Living Light

Lorenzo Pagliei, Les Invisibles

JeongHo Park, LHC CERN Data Audio Generator

Jan Willem Tulp, Goldilocks

Kurzgesagt, In a Nutshell

Nasa Scientific Visualization Studio

Nasa Hyperwall Visualizations

6. A Design-centered evolution

Design as the unifying force

Usage of new mediaCoverage of contemporary and popular science topics

Usage of datasets and real-time data sources

Keeping the data understandable

Application of design reasoning and verifiable methodology

7. Project: Sounds from the Sun

Sound and Psychoacoustics

Application to data

The Solar wind and its consequences Solar wind observation

Sound rendition of solar wind data

Technical aspects

Testing process

Resources

Conclusions

Thanks

Colophon

52

54

56

60

64

66

68

70

72

74

76

78

82

88

90

9193

94

95

96

98

100

102

103104

106

108

110

114

118

121

122


10/126


11/126

1. Introduction


12/126


One of the most rapidly growing field of designin the last decade has surely been data visualization.This happened, and is still happening, for a variety ofreasons, including the always-increasing availability

of any sort of dataset, the rise of visual journalism,but also the incredible computational power that wecan have inside the few kilograms of a nowadayslaptop computer, when compared to a 20, or even 10,years ago workstation.

Thanks to these few conditions, we have thepossibility, power and will to create progressivelymore useful and appealing visualizations that can fitin different aspects of our lives.

Parallel to design-driven and “artistic” data vi-sualizations, in the last three decades another field ofrepresentation was born and growing exponentially,

both in usage and importance: Scientific visualiza-tion, or SciViz.This particular kind of computer graphics is

traditionally aimed at rendering three-dimensionalphenomena of various kind, with an emphasis on the“realistic” representation of features like volumes orsurfaces, depending on the aim of the piece. This kindof artifacts, however, are usually developed in highlyspecific scientific or academic environments, focusingon very different parameters from the artworks weare more used to see, aiming at a very small group ofexperts from a specific field and often created with-out the design knowledge involved in the generallymore accessible works of data visualization.

However, things are rapidly changing: in thelast years the open data revolution caused a greatnumber of research centers and agencies to maketheir datasets available to the public. For the firsttime in history each one of us have the possibility towork on the very same numbers analyzed by the topscientists and researchers of NASA or CERN, just tomention two of many. This created a lot of interest invarious design and art communities, and made pos-

Why to investigate

scientific

visualization


13/126

13Introduction

sible some extremely interesting works of “data ren-dition”, ranging from static visualization to real-timeimmersive interactive installations, covering the vastarc that goes from design to art.

This could turn out to be a great possibility forScientific visualization: it has the chance to learnfrom the great mass of projects involving open data. Itcould incorporate, for example, a more conscious andconsistent design process, to improve usability and

readability for the specialists, but also to look at themore experimental and aesthetics-oriented works ofart, that could be made into powerful popularizationinstruments for the general public.


14/126


15/126

Scientific

Visualization2.


16/126


If Scientific visualization is approached, the firstthing that should be made clear is if and how it is dif-ferent from what we usually call “data visualization”:is it a subfield, a completely different thing or some-

thing in between?Traditionally, Scientific visualization has always

been considered a group of techniques to transformsome kind of science-based data into visual informa-tion through a reproducible process, with the funda-mental difference from data visualization that thedata is intrinsically spatial and physically based. Inother words, in Scientific visualization the spatial co-ordinates are part of the data, usually being betweenthe independent variables; while in information anddata visualization designers often deal with abstractdata, so they can freely decide how to manage the

two or three spatial dimensions.1

The same consid-eration is for time: in Scientific visualization it’s oftengiven and intrinsic to the data points, while in datavisualization it’s another parameter that the designercan choose to manipulate.

This difference can be further clarified whenthe most common visualization techniques used inSciViz are taken into account, considering data ofdifferent nature:2 bi-dimensional dataset usuallyrepresents scalar or vector fields; in the case of scalardata (for ex., temperatures) color mapping, wireframeplots, scatter plots or contour lines can be used, whilevector fields (like winds or magnetic fields) take ad-vantage of streamlines or line integral convolution;

For tri-dimensional data the subdivision be-tween scalar and vector fields still adapts: in case ofscalar values, volume rendering or isosurfaces can beused, while vector data uses streamlines, line integralconvolution, particle tracing and topological meth-ods. To visualize tensor fields instead, which are morecomplex, a technique denominated “hyperstream-lines” were developed.3

What is Scientific

Visualization

1: T.M. Rhyne, North Carolina State University,“Information and Scientific Visualization:

Separate but Equal or Happy Together at Last”,2011

2: The Ohio State University, “A Critical Historyof Computer Graphics and Animation”

https://design.osvu.edu/carlson/history/lesson18.html

3: T. Delmarcelle, L. Hesselink, “Visualizing second-order tensor fields with hyperstreamlines”, Computer

Graphics and Applications, IEEE, 1993


17/126

17Scientific Visualization

Example of line integral convolution appliedvisualizing heat movements in a hot water pot

Example of hyperstreamlines applied to tensor field visualization

Example of mesh visualization of a 3D functio n


18/126


For how SciViz is defined and for its mechan-ics, it is tightly bounded to computers: it requires toelaborate and translate to images always increasingamounts of data, or to run complex simulations that

may take up to years of calculations on huge super-computers. Therefore, it may be difficult to imagineScientific visualization before the digital revolutionthat happened in the ’80. In fact, usually we con-sider SciViz to be born in the early ’80, and becom-ing prominent at the end of the same decade, after a1987 report released by SIGGRAPH for the NationalScience Foundation, “Visualization in Scientific Com-puting”. That report was crucial for the history of Sci-entific visualization, bringing to the newborn field alot of attentions, that concretized themselves in newfunding, workshops, conferences and publications in

the following years.Before the NSF paper there are anyway someremarkable examples of visualizations, developedbeginning from the ’60, and there are even more im-pressing examples of “pre-digital” Scientific visual-ization: in fact, the century from 1850 to 1950 gave agreat push to the developing of various data represen-tation techniques. The most important example, butalso quite singular, is considered to be the Maxwell’sthermodynamic surface, a clay sculpture created in1874 by the mathematician James Clerk Maxwell.He is considered the father of electromagnetism, buthe was active also in other physics and mathematicsfields.

His surface was, in fact, a three-dimensional ab-stract plot of the thermodynamic states of a fictitioussubstance, similar to water, putting in relation its vol-ume, entropy and energy on the x, y and z axes, withisothermal and isopiestics lines traced on the surface.With Maxwell’s own words, it allowed “the principalfeatures of known substances [to] be represented ona convenient scale”4 : usually, in thermodynamics,when it is required to analyze the state of any fluid

Historic Scientific

Visualization

4: J.C. Maxwell, “Letter to James Thomson”,The Scientific Letters and Papers of James Clerk

Maxwell, volume 3, p. 230-231


19/126


20/126


Example of Pression / Entropythermodynamic diagram for water

Example of Entropy / Enthalpythermodynamic diagram


21/126


the sewer system had still to reach it. This wasn’t,anyway, a big concern to the population: at the time itwas a wide spread idea that infectious diseases spreadthrough some kind of miasma, or “bad air”, thereforemost people had no idea of the importance of propersanitary services. What really changed the situation,saving many other people from the infection andbringing a new understanding about hygiene, wasthe work of John Snow, a physician: he started trac-ing on a map all the cases of cholera he was hearingabout from the population, and thus noticing thatmost cases were localized in a precise area. Furtherinvestigations brought him to conclude that most ofthe infected had been drinking water from a par-ticular street pump. His research led to the closing ofthat pump, that later was discovered to be pollutedby a nearby cesspit, and created in the population theawareness that some diseases are carried by infectedwater, thus anticipating Louis Pasteur’s germ theory

Snow’s original cholera spreading map


22/126


by 7 years.5 It’s an incredible milestone for scientificvisualization: it was one of the first time, and a proof,that visualizing a phenomena on a map in a usefulway can cast a new light on events that may seemunconnected between them.

Another notable example that are considered

precursors of SciViz is the “Diagram of the causes ofmortality in the army in the East” by Florance Night-ingale, from 1873. She was as a British nurse managerduring the Crimean War, founder of the first secu-lar nursing school in London and also a proficientwriter, with various publications to spread medicalknowledge. She also worked on and popularized thegraphical presentation of statistical data, leading to abreakthrough in scientific visualization. During hercareer she used to produce various diagrams, with adesigned that she created and called “coxcombs”, ornowadays polar area diagrams, that she often usedto present statistical data and observations to peo-ple who may had difficult to understand traditionalstatistical reports, for example for her reports to themembers of the Parliament on the state of the medi-cal care in the Crimean War.

One of these diagrams visualized the seasonalmortality by cause of her patients, noting that a bignumber of deaths was linked to the bad state of Brit-ish army camps in India, particularly: bad drainage,contaminated water, overcrowding and poor ventila-tion. So, she started a campaign to popularize the idea

Florence Nightingale’s “Diagram of the causes ofmortality in the army in the East”

5: Steven Johnson, The Ghost Map: The Story of London’s Most Terrifying Epidemic – and How it

Changed Science, Cities and the Modern World, Riverhead Books, 2006


23/126


that it was necessary to improve the sanitary condi-tions in the country to increase the life quality andexpectance for both the Indian population and theBritish army.

With support from the Royal Sanitary Com-mission, she convinced the minister responsible torequire every house owner to connect its building tothe main drainage system, greatly enhancing sani-tary conditions.

As in Snow’s example, also this case shows howa proper and well-design way to represent abstractdata can help to establish connections that are other-wise difficult to be noted, and to spread knowledge onproblems that are difficult to understand for whoevermay be a stranger to a particular field of knowledge.


24/126


One of the most remarkable example of earlycomputer-generated scientific visualizations, and alsothe first computer graphics film, is “A two gyro grav-ity gradient attitude control system”. Dated 1963, it

comes from the Bell Laboratories, at the time ownedby AT&T. It was created by Edward E. Zajac, an en-gineer working on various long-range, trans-oceanicand planetary communications projects.

At the beginning of the sixties Zajac was work-ing on Bell’s communication satellites program, in-volving a new kind of satellite equipped with twogyroscopes, that would have solved some stabilityand control problems that emerged with the previoussatellite designs. While the mathematics required toprove its theory are thoughtfully explained in its pa-per “A Two-Gyro, Gravity-Gradient Satellite Attitude

Control System”, (J.A. Lewis, E.E. Zajac, The Bell sys-tem technical journal, 1964), he felt that the data, thegraphs and the static images in that dissertation werenot really enough to explain the complex rotationsthat its theorized satellite would have. So, togetherwith his colleague Frank Sinden, they started to workon an innovative visualization technique: a perspec-tive video simulation, programmed in FORTRAN andfinalized using a custom-made software called ORBIT,designed for this very purpose.

The realization process is to be considered in-teresting, for today’s standards: the original computa-tions were fed via punch cards to a IBM 7090 super-computer, and then printed directly to microfilm witha General Dynamics Electronics Stromberg-Carlson4020 microfilm recorder.

While the animation looks clear but extremelyelementary, each minute of the resulting 16 FPS-vid-eo rendering took up to eight minutes of computationon what was one of the most performing supercom-puter at the time.6

In those early days of computing resources wereincredibly strict, thus the aesthetics of most visual-

Scientific

Visualization,

computers and

computer graphics

Composition of frames from“A two gyro gravity gradient

attitude control system”

6: Steven Johnson, The Ghost Map: The Story of London’s Most Terrifying Epidemic – and How it

Changed Science, Cities and the Modern World,Riverhead Books, 2006


25/126


izations were tied just to black and white, since colorwould have consumed significantly more memory.The first example of the use of color came neverthe-less in 1969, by Dr. Louis Frank, who was workingwith the Space Plasma Physics Research group of theUniversity of Iowa on the data collected by the LEPE-

DEA instrument of a satellite.7

He plotted time on thex axis, while the energy spectra in on the y axis andthe different color hues represents the amount of par-ticle hits per second, effectively summarizing threedimensions of data in two-dimensional plot.

Going back again to animation, in 1977 theComputer Graphics Research Group (CGRG) of OhioState University, with Bob Reynolds and Wayne Carl-

IBM 7090 supercomputer

Dr. Frank’s time / energy graph

7: htt p://www-pi.physics.uiowa.edu/~frank/


26/126


Still frame from “Interacting Galaxies”

The Mice Galaxies


27/126


son, created a milestone visualization of interactinggalaxies that was also featured on the Carl Sagan Cos-mos TV show.8 The importance of this work lies inthe fact that it was a display of a very complex nu-merical simulation, that actually was repeated manytimes with different parameters to show how differ-ent types of interaction can create very differentlyshaped galaxies, explaining for example the tidaltails formations we observe in the Antennae Galax-

ies and in the Mice.9 The simulation was programmedin FORTRAN by Dr. Alan Toomre, and even if quietsimple for the number of particles simulated in thesystem, proved how this kind of simulation were al-ready a valuable research tool in this particular field:in fact, when dealing with this kind of dynamics, sim-ulations are the only way to visualize the evolutionof galaxies, since observation can show us only “stills”of galaxies in various stage of their life, consideringthe time involved in these processes. For example,his simulations led Toomre to suggest that ellipticalgalaxies are remnants of the major mergers of spiral

galaxies,10

which became a widely accepted idea fromthe scientific community.Visualization and animation proved its value

also with another historic example, this time in thefield of programming. One of the classic problemsposed to computer science students is the implemen-tation of sorting algorithms, which are various waysto put in order a big series of randomly – distributed

Still frames from “Sorting out Sorting”

8: http://excelsior.biosci.ohio-state.edu/~carlson/history/ACCAD-overview/overview-old.html

9: http://excelsior.biosci.ohio-state.edu /~carlson/history/ACCAD-overview/overview-old.html

10: http://excelsior.biosci.ohio-state.edu/~carlson/history/ACCAD-overview/overview-old.html


28/126


numbers. Mathematics and researchers created inthe years a lot of different sorting algorithms, reach-ing an incredible level of efficiency, but also involv-ing very complex mathematics processes that makevery different to imagine how an algorithm actuallyworks and to estimate its efficiency. With this idea inmind, Ronald Baecker from the University of Torontocreated in 1980, and presented at Siggraph in 1981,“Sorting out Sorting”, the first computer-generatedprocess-visualization animation that showed the ap-plication of different algorithms in real-time, making

their inner workings evident, with clear comparisonson time and efficiency.11 The video was a great suc-cess, and it’s still used today to illustrate that kind ofprocesses.

The fact that really pushed forward the at-the-time young Scientific Visualization field was the pub-lication of a report compiled for the National ScienceFoundation from a SIGGRAPH panel in 1987, called“Visualization in Scientific Computing”: first of all itacknowledged and defined “officially” Scientific Vi-sualization, poetically describing it as “a tool for ap-plying computer to science, [that] offers a way to seethe unseen”,12 and then proceeded to explain therole of visualization in science, which processes areinvolved, how it should expand and even mentionedthe already underrated importance of data, especiallyfrom high-volume sources. The paper identifies Visu-alization as a multidisciplinary field, that at that timeinvolved computer graphics, image processing, com-puter vision, computer-aided design, signal process-ing and user interface studies, but also recognizesthe need to expand that, including engineering andscientists dedicated to develop new visualization sys-

Still frames from “Sorting out Sorting”

11: R. Baecker, “Sorting out Sorting: A CaseStudy of Software Visualization for Teaching

Computer Science”, Software Visualization: Programming as a Multimedia Experience, MIT

Press, 1998.

12: Bruce McCormick, Thomas DeFanti, MaxineBrown, Visualization in Scientific Computing , p. vii


29/126


tems, artists, for their education in visual communi-cation and inventive about visual representations,cognitive scientist and psychologists to design betterinterfaces and try new concepts and steer research innew directions.13

Throughout the paper it’s underlined manytimes how visualization in general is fundamentalto human beings, and that a lot of different kinds ofinformation are better communicate in a visual way:for example, when dealing with big amounts of datathat the brain cannot process in numerical form, or

for structures like DNA, molecular models or otherkinds of simulation. An interesting fact is that theauthors note the need for interaction in Scientific vi-sualization: at the time most elaborations run on su-percomputers via batch processing, so the user couldonly start the process and wait for the result, with-out knowing what the calculator was doing. Theytheorized Interactive visual computing , where scientistcould “communicate with data by manipulating its vi-sual representation during processing. The more sophis-ticated process of navigation allows scientists to steer, ordynamically modify, computations while they are occur-ring. These processes are invaluable tools for scientificdiscovery.”14 This change should be implemented de-veloping and giving high-quality visualization toolsdirectly to the researchers, via a proposed federally-funded initiative. The first step should be to abandonthe supercomputing model, in favor of “workstations,minicomputers and image computers […] that shouldbe placed on the desks of each and every researcher”with specialized graphics processors, that could bemuch more efficient, with each of these machinescosting between 5,000 $ and 100,000 $. Along with

Still frame from the last section of“Sorting out Sorting”

13: Bruce McCormick, Thomas DeFanti, MaxineBrown, Visualization in Scientific Computing , p. 11

14: Bruce McCormick, Thomas DeFanti, MaxineBrown, Visualization in Scientific Computing , p. 5


30/126


this new hardware, research centers should providededicated software to develop visualizations, easyenough to use to not require the user to know pro-gramming or have particular graphical knowledge.

The SIGGRAPH report apparently got its point,causing the interest of the research world, attractingfunding agencies and triggering conferences, work-shops and various publications on the topic. Mostimportantly, various software developers proposed

a variety of specific and generic visualization soft-ware environments, for example IBM Data Explorer,Wavefront Advanced Visualizer and SGI IRIS Explor-er. Most of them did not use code or textual instruc-tion, but rather developed a dataflow paradigm thatdid not require programming or graphics knowledge:the user had to connect modules between them, eachone performing a specific operation. It was howeverpossible to create custom modules with traditionalprogramming languages like C or FORTRAN.

Screenshot from the Wavefront Advanced Visualizer


31/126


Screenshot from the IBM Data Explorer software

Screenshot from apE from the OhioSupercomputer Graphics Project at Ohio State


32/126


33/126

Open data3.


34/126


A phenomenon that is slowly influencing andchanging various aspect of our lives, even if most ofus are still not really aware of it, is Open Data: with itsidealistic push and novel way of seeing information

we can consider it to be the most recent revolution inthe history of knowledge.

For centuries data, particularly governmental,was considered something to keep away and evendangerous for the public security, until 2008, whenthe Obama administration realized that opening uptheir data to the public could bring to more participa-tion and trust in the government.

But if we consider the attitude towards theopenness of scientific data and results, it has alwaysbeen more positive than in the political world, withvarious researchers theorizing that every scientific

result should be as more widespread and accessible aspossible. Even with this disposition, until the 1990 de-cade there wasn’t really any active projects to publishor share scientific data.

The specific term “Open data” was first men-tioned in May 1995, in a proposal for the Congress ofthe World Meteorological Organization, concerningthe disclosure of geophysical and environmental data.From the same paper we can read: “Atmosphere datatranscends borders, they promote open exchange ofinformation between countries, a prerequisite for theanalysis of this global phenomena”. While this seemsalmost obvious nowadays, at the time even the shar-ing of “naturally global” atmospheric and weatherdata was not really taken into consideration.

But the concepts behind Open data are way old-er than that, always considering the scientific world:from a theoretical point of view, in 1942 Robert KingMerton, considered one of the fathers of sociology,wrote: ““Communism,” in the nontechnical and extend-ed sense of common ownership of goods, is a secondintegral element of the scientific ethos. The substantive

findings of science are a product of social collaboration

beginning of

The Open Data era

Robert King Merton


35/126

35Open Data

and are assigned to the community. They constitute acommon heritage in which the equity of the individual

producer is severely limited. […] Property rights in scienceare whittled down to a bare minimum by the rationaleof the scientific ethic. The scientist’s claim to “his” intel-lectual “property” is limited to that of recognition and es-teem which, if the institution functions with a modicumof efficiency, is roughly commensurate with the signifi-cance of the increments brought to the common fund of

knowledge. […] The institutional conception of science as part of the public domain is linked with the imperative for the communication of findings. Secrecy is the antith-esis of this norm; full and open communication its en-actment. […] The communism of the scientific ethos is in-compatible with the definition of technology as “private

property” in a capitalistic economy ”.15

While it does not clearly mention Open Datain the way we consider it nowadays, he is endors-ing it way ahead of its time, and even touching thetheme of intellectual property, a topic that will bevery dear to the Open source movement in the last

decades of the century, that was also a great promoterof the Open Data ideals. Another important theoreti-cal endorser of Open Data was the American politi-cal economist and Nobel prize Elinor Ostrom, that in2009 stated that information commons are similar topublic goods, because their use by one do not impedethe use by other. Instead, information commons canbe considered of a new kind: their use does not de-plete the common stock, but rather it enriches it.16

Elinor Ostrom

15: Robert King Merton, “The NormativeStructure of Science”, The Sociology of Science:Theoretical and Empirical Investigations

16: Charlotte Hess, Elinor Ostrom, “ IDEAS, ARTIFACTS, AND FACILITIES: INFORMATION AS A COMMON-POOL RESOURCE ”, 2003


36/126


37/126

37Open Data

Going back to consider Governative Open Data,the most important event on this side was the Sebas-topol meeting: a reunion of thirty activists and think-ers near San Francisco, aiming to define the concepts

of Open public Data, hoping to make it adopt to theUS presidential candidates. Between them therewere various important figures: Tim O’Reilly, mostknown for popularizing the concepts of open sourceand of Web 2.0; Lawrence Lessing, a professor ofLaw at Stanford Univeristy and founder of the Cre-ative Commons licenses; Aaron Swartz, hacktivistand inventor of RSS, sadly known for his suicide in2013 after being found guilty of wire fraud for tryingto download and share a great number of academicjournal articles.

The Sebastopol meeting defined that public data

should be considered a common property, just like sci-entific ideas, and it should be of interest to everyonefor it to be shared and used. This concept did not comeout of nothing, but it is strongly based on the conceptsof open source software: openness, participation andcollaboration. Coming from that environment most ofthe participants were aware of how good this modelis working for free software development: program-mers collaborate and are invited to publicly sharetheir code, both to let others learn from them, and tocreate a collective exvvpertise. There are not reallyhierarchical rules, and the work between developersis based on a peer-to-peer collaboration model thatpromotes competence and reputation. Synthetizing,Tim O’Reilly said that “we must apply the principles ofopen source and its working methods to public affairs”.

The meeting achieved even more than whatthey were expecting: in 2009 the newly elected presi-dent of the US Barack Obama signed three presiden-tial memoranda about open government and OpenData, taking the open source culture at the heart ofpublic actions, stating that “Openness will strengthenour democracy ”. Precisely, he defined that the Govern-

The birth of

public open data

Aaron Swartz at a Creative Commons event, 2008


38/126


ment should be transparent , to promote accountabilityand because information is a national asset; participa-tory , to enhance public engagement, soliciting publicinput, thus enhancing the government effectivenessand improving its decision-making; collaborative, us-ing innovating tools to create collaboration betweenGovernment, public and private sectors.18 This had ahuge impact also for scientific data, because it trig-gered the opening of NASA and other governmental

agencies datasets to the public.The situation of Open Data in governments is

positively evolving and many states worldwide aremoving towards policies similar to the one used in theUS, even if the cultural changes required for it maybe difficult to accept in some context. The benefits areanyway vastly superior that any possible concern,both in politics, allowing better transparency andcitizenship participation, and economics terms, hav-ing created a whole new branch of possible activitiesbased on data.

What is incredibly relevant about Open Data

for the Scientific Visualization field is that makingscientific datasets available for the public created agreat interest in designers and artist: being able to ac-cess that amount of data made possible to create en-tirely new data-driven concepts and projects both inart and design, that are creating fascinating ways tomake use of those numbers, often crossing with fieldslike new media, interactive and installation art.

Anyway, Open Data is not spreading as fast as itcould, and it is facing some issues: the most important

Open data scores per country, evalued from theGlobal Open Data Index - http://index.okfn.org/

18: Barack Obama, “Transparency and OpenGovernment - Memorandum for the Heads of

Executive Departments and Agencies”, https:// www.whitehouse.gov/the_press_office/

TransparencyandOpenGovernment


39/126


40/126


Tim Berners-Lee is an incredibly important fig-ure, that influenced the very way we communicateand perceive information nowadays: in March 1989,while working at CERN he defined and prosed a new

information management system that became, in theNovember of the same year, the World Wide Webthat we all know, successfully implementing the firstHTTP communication.19 Currently he is director ofthe World Wide Web Consortium (W3C) and founderof the World Wide Web Foundation.

Starting from 2009, he worked with the Britishgovernment to support their Open Data project, thatconcretized itself in the data.gov.uk portal.

Tim Berners-Lee research nowadays is focusingtoward the future of Open Data: he is proposing the“Linked Data” standard, as a mean to solve the prob-

lems that traditional Open Data are facing.Linked data must respond to four simple rules:1- Use Uniform Resource Identifiers (URIs) as

names for things, e.g. http://dbpedia.org/resource/Brus-sels can be used for referring to the city of Brussels.

2- Use HTTP URIs, so that people can look up thosenames.

3- When someone looks up a URI, provide usefulinformation, using the standards (i.e. RDF, SPARQL).

4- Include links to other URIs, so more things canbe discovered, e.g. from http://dbpedia.org/resource/

Brussels a link is available to http://dbpedia.org/re-source/Belgium. 20

To make clear and “rate” how much LinkedOpen Data is considered good and useful, a rating sys-tem has been developed:

1 Star: The Data is available on the Web with anOpen license.

2 Stars: Data is in a machine-readable format, likean Excel file instead of a scanned image.

3 Stars: Data is in a non-propertary format, like.csv instead of Excel.

4 Stars: Data uses a W3C open standard to iden-

The future:

Linked Data

Tim Berners-Lee

19: https://www.w3.org/History/1989/proposal.html

20: htt ps://joinup.ec.europa.eu/community/semic/news/understanding-linked-data-

example


41/126

41Open Data

tify things, making the data easily linkable.5 Stars: Data is linked to others dataset to provide

context.21

This system is addressing mainly governmentalagencies and any other entity that can produce andspread Open and Linked Data, to make clear whichis considered to be the best and most useful way toshare their data, focusing on shareability and possi-bility to link different contents.

The most notable example of Linked Data now-adays is DBpedia22: it is a project born in 2007 fromthe Universities of Berlin and Leipzig and in collabo-ration with OpenLink Software, aiming at extractingall the structured content from Wikipedia, to makeit available on the web and semantically searchable.

This last aspect is a major difference over more“traditional” forms of Open Data: semantic researchallows to query for associative and contextual in-formation, both explicitly and implicitly, allowing toprovide a single, precise piece of information, but alsoanswer to open questions through techniques like

pattern matching and digital reasoning. With thiskind of technology, the value of information growsexponentially when adding new connections, al- DBpedia mapping in 2014

21: https://www.w3.org/DesignIssues/LinkedData.html

22: https://www.ted.com/talks/tim_berners_lee_on_the_next_web/transcript?language=en#t-485180


42/126


lowing each dataset to be interconnected, effectivelybuilding a much more effective network of knowl-edge than what is possible today.

To give an example, as of February 2016, theEnglish DBpedia describes 5.9 million resources, with4.3 million abstract, 425 thousand geo coordinatesand 1.45 million depictions. 4 million resources arepart of a consistent ontology and consists of 2.06 mil-lion persons, 682 thousand places, 376 thousand cre-

ative works, 278 thousand species and 5 thousanddiseases. There are 128 languages in which DBpediais localized, for a total amount of 38.3 million entries.All of this data is automatically selected, extracted,categorized and linked together by automaticallyscraping Wikipedia.23

Another remarkable example of Linked Data isrepresented by GeoNames, a geographical databasecontaining 10 million geographical names, corre-sponding to more than 7.5 million unique features24;and UMBEL (Upper Mapping and Binding ExchangeLayer), a knowledge graph of 35 thousands concepts

that can be used in information science to relate in-formation from different sources. UMBEL in particu-lar makes heavily rely on the semantic research andinteroperability for concept search, concept defini-tions and ontology consistency checking.25

23: https://www.ted.com/talks/tim_berners_lee_on_the_next_web/

transcript?language=en#t-485180

24: http://www.geonames.org/

25: http://umbel.org/resources/news/umbel-v-1-20-released/


43/126

43Open Data


44/126


45/126

art, science

and design4.


46/126


Observing both the historic examples and themore contemporary case studies taken into accountfrom scientific visualization, it can be noted that theyfocus mostly on solving functional requirements, of-

ten overlooking on concepts like aesthetics or userexperience. Also, they are often created by and aimedto experts in a particular scientific sector, requiringsome level of familiarity and knowledge with thetopic of the visualization.

On the opposite side, the increasing public avail-ability of datasets, APIs, and other forms of Open Datais creating a kind of “parallel universe” of artistic proj-ects involving visualization, centered mostly on theaesthetic value, reversing the more traditional utili-tarian approach, and attracting much attention froma new and wider kind of public, that maybe never re-

ally approached science and are just generally inter-ested in art and culture: in this way, suddenly, theyare experiencing a work of art but also dealing withsubatomic particles, pulsars and weather data. “Comefor the art, stay for the science”, it might be said.

A cause of the low consideration of aesthet-ics in SciViz resides in the perceived impossibility toevaluate and rate objectively the attractiveness ofany visualization, privileging different parameters,specificially utility and soundness. These three prin-ciples are considered the elementary requirements ofdesign, and were identified for the first time in 25 BC,by the Roman architect Vitruvius in his book “De Ar-chitectura”. Relating these principles to informationvisualization:

Utility is associated with functionality, effective-ness and efficiency, and can be considered the opti-mization of a visualization, presenting in the clearestway as much information as possible within the giv-en constraints. It is often the main focus of academicresearch, since it can be evaluated by comparison ofdifferent methods and concepts.

Soundness is a quality related to the algorithm

separation of

science and

aesthetics


47/126

47Art, Science and Desig

used to generate a visualization, mainly to its reli-ability and robustness; but also takes a wider sense,including how efficient the code is, if it can take dif-ferent kind of data, or tolerate errors. In an academicenvironment it is expected that the algorithm behinda visualization is clearly explained, and must fulfillsome elementary stability and reliability require-ments. It can be evaluated both with mathematicaland empirical methods.

Attractiveness refers to the aesthetic quality ofa visualization, relating primarily to the appeal orbeauty, but can be extended to consider other aspectslike originality, novelty, and can also encompass thewhole user experience matter. The problem with thisprinciple is that, at least traditionally, there is no rec-ognized way to objectively evaluate or it, thereforeacademic research in scientific visualization does nothave any expectation in that regard26.

This kind of reasoning has a fundamental flawanyway: while it is obvious that “beauty” cannot bemeasured, it is also true that various studies proved

that, when dealing for example with computer in-terfaces, the aesthetic quality can support utility, im-proving the user’s performances and satisfaction27.

26: Andrew Vande Moere, Helen Purchase, “Onthe role of design in information visualization”

27: D. Norman, “Emotion & Design: AttractiveThings Work Better.” interactions 2002


48/126


It has been proved that the perceived usabilityof an interface is related to the subjective aestheticjudgment by the user28; another study revealed thatusers tend to spend more time using a software with

a more appealing interface and are also more for-giving in case of errors or bugs29.Another study em-pirically demonstrated that “visual embellishments”considered non-utilitarian, like metaphors, positivelyinfluence long term recall in infographic charts30.

Transferring the same reasoning to visualiza-tion, a highly valuable aesthetic applied to data maycompel the user to engage with it, giving a higher val-ue to the information presented and enabling a bettercommunication.

When it comes to evaluate attractiveness for vi-sualization, some theoretical efforts to identify some

kind of criteria have been made, most notably fromLau and Vande Moere. They identified three possiblecharacteristics that determine the engagement ofaudiences on a visualization: design quality, gener-ally including visual style and user experience; datafocus as the core of the visualization is to communi-cate meaning instead of simple facts or trends; userinteraction, that involves the aspects related to fluid-ity, engagement and social collaboration31.

Another effort is from David Chek Ling Ngo,that proposed 14 different metrics to quantify theaesthetic layout of an interface: balance, equilibrium,symmetry, sequence, cohesion, unity, proportion,simplicity, density, regularity, economy, homogeneity,rhythm, order and complexity; backed by mathemat-ical models and empirical analysis32.

This extremely rational approach of course isnot shared by everyone: others researchers believethat aesthetics can be evaluated without any measur-able characteristics, and is ultimately founded in thehuman experience. So people can find an artwork ora visualization aesthetically pleasing without know-ing “why”, in a non-discursive way33.

Evaluating

attractiveness

28: M. Kurosu and K. Kashimura, “ApparentUsability vs. Inherent Usability: Experimental

Analysis on the Determinants of the ApparentUsability”. Conference on Human factors in

Computing Systems (CHI’95)

29: N. Tractinsky, D. Ikar, “What is Beautiful isUsable”, Interacting with Computers

30: S. Bateman, R. Mandryk, C Gutwin, “UsefulJunk? The Effects of Visual Embellishment onComprehension and Memorability of Charts”.

Conference on Human Factors in ComputingSystems (CHI’10)

31: A. Lau and A. Vande Moere, “Towards aModel of Information Aesthetic Visualization.“

International Conference on InformationVisualisation (IV’07)

32: David Chek Ling Ngo, Lian Seng Teo, John G.Byrne, “Modelling interface aesthetics”

33: Oliver Conolly, Bashar Haydar, “AestheticPrinciples”


49/126


50/126


The role of Art About the role of art and artist, there is a veryinteresting case study from Daniel F. Keefe, David B.Karelitz, Eileen L. Vote, and David H. Laidlaw at theBrown University Visualization Research Lab, where

they investigated both the potential role of artists indesigning new visualization techniques and in theimpact that virtual and augmented reality technolo-gies can have on the future of visualization. They gottogether programming, art and design students, anda team of scientists to explain the various tasks andtopics they had to cover. They started from simple2D fluid flow visualization, to scale up with increas-ingly difficult task as 3D and visualization in a Vir-tual Reality environment. It quickly became clearthat artists can be very useful in the initial designand conceptualization stages, as they often provided

new insights to the team. Working together with sci-entists they achieved a very effective feedback loopwith them, also influencing the way data was col-lected to better suit the artist’s proposals. Cooperat-ing with designers and programmers was also veryproficient for they artists, since they often lacked thetechnical knowledge to convey their concepts. A use-ful tool that brought all the participants together wasthe critique and discussion moments, both on a visualand on a scientific standpoint.

Proceeding in the work, some difficultiesemerged when approaching VR: the main obstaclewith it is the lack of prototyping tools that allow non-experts to work with it. But since VR is such a uniquemedia, it is almost impossible to evaluate and critiquea design proposal without trying it in the appropri-ate environment, and nowadays this requires daysof specialized programming work, thus breaking thepossibility of quickly testing, alter or discard concepts,a methodology widely used and of proven effective-ness in the nowadays design practice. To work aroundthis difficulty, the team realized a quick augmentedreality sketching system for the artists, that involved


51/126

51Art, Science and Design

a fictional paintbrush that traces “virtual 3D strokes”,that effectively sped up the work and allowed for abetter evaluation and refinement process.

Working with VR and data has proven to be adelicate work of balance, because while it allows torepresent much more information in an amazing im-

mersive environment, it must be clear from the be-ginning how a design idea will work with the actualdata to display: it can happen to prototype a conceptbased on a personal perception of the data to findout that it seems effective, to alter discover that thatconcept does not go well with the actual data once it’simplemented, and considering how VR is difficult toimplement right now it may turn out to be a consider-able waste of time.

In conclusion, this experiment shows a few im-portant points on the relationship of art, science anddesign: artists proved that their creativity can bevery useful in the conceptual and initial designingprocess to create new techniques, better if in a mul-tidisciplinary team. When it come to virtual realityand similar novel and specific medias it also must benoted that the developing of rapid prototyping envi-ronments should be a priority in the next future, toallow for the experimentation of novel visualizationtechniques34.

Virtual Reality bloodflow visualization, createdwith the visualization prototyping systemdeveloped at Brown University’s Visualization Research Lab

34: Keefe D, Karelitz D, Vote E, et al. “ArtisticCollaboration in Designing VR Visualizations”.IEEE Computer Graphics and Applications 2005


52/126


53/126

CASE STUDIES5.


54/126


There is currently a notable number of data-based projects that crosses ways with art, design andscience, using both more “traditional” forms of ex-pression (graphics, video, sculpture…) and new and

“non-conventional” medias (sound, interaction, instal-lations). There is a selection of works in different field,including some “institutional” projects created direct-ly by research center and agencies, and some worksthat does not directly involve data, but acts as meansof science popularization, with a very good responsefrom audience. Each project has tags to distinguish itsfield and features:

case studies

selection

1 - R. Ikeda, Supersimmetry - Art, Installation, Particle physics, Music,Projections

2 - Miebacht, Weather Scores - Weather data, Sculpture, Music, Art

3 - A. Parker, Worlds - Video, Planetary Science, Design

4 - eCloud - Weather data, Installation, Art

5 - Living Light - Weather data, Installation, Design

6 - L. Pagliei, Les Invisibles - Music, Particle physics, Art

7 - J. Park, LHC CERN Data Audio Generator - Music, Particle physics,Application, Design, Science

8 - J. W. Tulp, GoldiLocks - Planetary Science, Web, Design

9 - In a nutshell - Web, Video, Animation, Design

10 - NASA Scientific Visualization Studio - Weather data, Planetary sci-ence, Science, Map

11 - NASA Hyperwall Visualizations - Planetary science, Science, Weath-er Data

Please note that some tags will be counted morethan once for tha last two cases, which embed

more small projects.


55/126

55Case Studies

10

1102

04

06

08

03

05

07

09

01

Design

Art Science

Case studies

mapping

Tags recurrencyScienceWeather Data

Planetary Science

Animation

Design

Art

Music

Map

Installation

Particle Physics

Video

Projections

Sculpture

Application


56/126


Ryoji ikedaSupersymmetry

Ryoji Ikeda is a Japanese sound and visual art-ist, whose musical interest is mainly in the most“raw” states of sounds, like fundamental tones andvarious kinds of noise, that he often uses to play withthe limits of human earing, with extremely low orhigh-pitched sounds. His visual aesthetics is based on

similar principles, often visualizing the sound he pro-duces with massive, abstract and geometric black andwhite projection. He often used data, mathematicsand physical concepts as “sources” for the compositionof his works, and is considered a pioneer of minimalelectronic music35.

His project “Supersymmetry” was developedduring his residence at CERN in Geneva in 2014 and2015, where he had the chance to cooperate with theresearchers working on the supersymmetry theory:at the time they were running experiments at theLHC particle accelerator to find some yet-undiscov-ered fundamental particles, like the Higgs Boson,that would prove the reliance of the Standard Model.Even if this work does not include data directly fromthe LHC experiments, it is trying to describe in an ar-tistic and “experiential” way what these physics theo-ries are and imply, so it is more of a conceptual projectthan an actual data visualization system.

The installation itself is constituted of two sec-tions, named “experiment” and “experience”: the firstone revolves around observation of natural phenom-ena, which is exemplified with a number of small

Ryoji Ikeda during a performance

Audio visual installation, 40 projectors,

40 computers w/ monitor, speakers, 2014

35: http://www.ryojiikeda.com/biog raphy/


57/126

Supersymmetry (experience), whole view

Supersymmetry (experiment), particular


58/126

pellets of different materials, moving on three light-ened surfaces that are constantly changing their in-clination. The pallets move following the angle of thesurfaces, colliding with each other. A laser scanningsystem detects the position of each particle and sendsit to the second section, “experience”, where soundand images are produced according to movementsand collisions detected between the pellets. This cre-ates a kind of detachments in the user’s perception:

while they try to follow and understand at least oneof the events, hundreds of other events are happen-ing each second around them, in a way simulatingthe enormous amount of data and complexity thatparticle physics researchers have to face with eachexperiment36.

Supersymmetry (experience), one of the monitors

36: http://special .ycam.jp/supersymmetry/en/work/index.html


59/126

Supersymmetry (experiment)

Supersymmetry (experience)


60/126


Nathalie’s work focuses on the intersection ofart and science, taking data from astronomy, ecol-ogy and meteorology and transforming it into wovensculptures: in this way she uses the natural “grid” thatbasket weaving creates as a coordinate system to dis-place three-dimensional data. In Nathalie’s opinion,

her work pushes the boundaries of traditional waysto interpret and visualize data, playing with the ex-pectations of audience about what is considered to be“science” and “art”37.

Additionally, since 2009 she collaborated withmusicians and composers, to transform a series ofsculptures she based on weather data into musicalscores for them to play: this both gives a diverse emo-tional level to her primarily visual work, and mightalso reveal patterns in the data that may otherwiseremain unnoticed.

Her work can be considered quite rare in dataart, mixing together media that are not really muchused in the field, thus creating a unique way to seeand experience the datasets she uses. Her work alsomaintains a very “analogic” feeling, relying on an artthat requires a lot of manual work and skill as basketweaving and preferring traditional scores and hu-man musicians for her music, rather than relying onelectronics music or computer-guided composition asmost sound/data artists uses to.

Nathalie miebachWeather Scores

Scuplture, music

Nathalie Miebach

37: http://nathaliemiebach.com/statement.html


61/126

61Case Studies

The “Weather scores” exposition

Musical score for “Duet of Blizzards”


62/126


“External Weather, Internal Storms” 2009


63/126

63Case Studies

“O Fortuna Sandy Spins,” 2013


64/126


Alex H. ParkerWorlds

Alex Harrison Parker is a planetary astronomerat Southwest Research Institute, currently collaborat-ing with NASA’s New Horizons mission38. He is activealso as a science popularizer, creating illustrations,video and musical pieces based both on scientific dataand on topics from his research.

For example, “Worlds: The Kepler Planet Candi-dates” is an animation video he created were he visu-alizes all the Kepler planet candidates around a singlestar, each one of them with their real diameter, orbit,and speed. It was awarded with the CinèGlobe “DataVisualization Award” in 201339.

The value of this work resides in the verystraightforward and clear way it presents such a richtopic: it can be considered a very good work of designrather than art, but nevertheless very interesting,giving in little more than 3 minutes a good perspec-tive on the incredible amount and variety of exoplan-ets that the Kepler space telescope is discovering.

Other projects from him include “Painted Stone”,a visualization of 100,000 asteroids and orbits, ob-served by the Sloan Digital Sky Survey, with the colorrepresenting their material composition; “Kepler 11: ASix-Planet Sonata”, a sonification of the transits of thesix planets in front of their star in the Kepler 11 plan-etary system, that demonstrate through sound theorbital resonances that system; “Beyond Neptune”, ananimation about trans-Neptunian objects, visualizingwhen they were discovered, their size and orbit.

Dr. Alex H. Parker

Animation, Music

38: http://www.alexharrisonparker.com/

39: http://www.alexharrisonparker.com/datavisualization/


65/126

Worlds

Painted Stonemid: Beyond Neptunebottom: Worlds


66/126


Aaron Koblin,Nik Hafermaas,

Dan Goods

eCLOUD

eCLOUD is a dynamic sculpture made fromsuspended polycarbonate tiles that can change fromtransparent to opaque, to simulate the volume andbehavior of an idealized cloud. It was created in col-laboration with NASA’s Jet Propulsion Laboratoryand the Art Center Collage of Design. The system

takes real-time weather data from NOAA and simu-lates how a cloud would behave in the conditions ofthe picked location40.

The project includes a dynamic display systemthat shows a preview of the simulated cloud behavior,the current location, and details about the weatherdata received.

eCLOUD was created for and is installed at theSan José International Airport as a centerpiece art-work, as part of the Art & Technology program41.

This is a valuable example of the possible inter-sections of data, art and new media: it uses an inno-vative technology to display real-time data througha very effective visual metaphor, that people passingby can easily understand, together with a notable andpositive aesthetic value.

Dynamic sculpture, 2010

40: http://www.ecloudproject.com/tech.html

41: http://www.ecloudproject.com


67/126

67Case Studies

eCloud installed in the San Jose Airport

eCloud dynamic display mid: eCloud installed in the San Jose Airport bottom: demonstration of the liquid crystal tiles technology


68/126


Soo-in YangDavid Benjamin

Living Light

Living Light is a functional art installation lo-cated in Seoul, created to inform the citizens aboutthe air quality in the city. The canopy has a city mapetched on its glass surface, and is divided in 27 differ-ent blocks representing the different areas of the citywhere the air monitoring stations are situated. When

an improvement in the air quality of a district is de-tected, the corresponding block of the system lightsup; otherwise the system is automatically updatedevery 15 minutes, lighting up all the blocks in order,from the one with the best air quality to the worstone. The structure also responds to text messages,blinking and then sending the data about the specificlocation that the user requests. “Living Light” is fullyintegrated with the already existing air-quality moni-toring system active in Seoul, receiving data from itssensors network42.

This is a notable example of public art: its beau-tiful design became an attraction for the city itself,and it is also very useful for the population, inform-ing and raising awareness about the condition of airand pollution in the city.

Public art installation, 2009

42: http://www.interactivearchitecture.org/living-light-2.html


69/126

Living Light installed at Peace Park, Seoul


70/126


Lorenzo PaglieiLes Invisibles

Lorenzo Pagliei is an Italian composer, electronicmusician, pianist and orchestral conductor, currentlyliving and working in Paris at IRCAM as composerand researcher. His work focuses on sounds not re-alizable with acoustical instruments and innovativeways to interact and control sound43.

In 2001 he took part in a collaboration effortmade from the Centre des Arts of Paris, aiming atmaking scientists and artists work together on a pieceof art involving concepts from physics and nanotech-nology: Lorenzo worked with the Centre national dela recherche scientifique – CNRS, that was currentlyworking on nuclear magnetic resonance. It’s a newtechnique to investigate the properties of moleculesand nanoparticles, that works by analyzing the par-ticle’s oscillations when hit by an electromagneticpulse. This kind of data is almost directly translatableinto sound, since particles and molecules threated thisway behave like regular objects that get hit, vibratingand thus generating sound. Analyzing this oscillatingresponse from molecules allow to deduce its composi-tion and various other properties.

Each particle generates a different sound orrhythm, that were used as the composing materialfor a 1-hour music piece created for the exposition.“Les Invisibles” was performed again in October 2015,in the Crypt of the Ville d’Orsay, this time as a real-time performance controlled live by the composer44.

Sound Installation, 2001 / 2015

Lorenzo Pagliei

43: http://www.lorenzopagliei.com/site/bio.html

44: Personal interview with Lorenzo Pagliei,February 2016


71/126


72/126


JeongHo ParkLHC CERN Data

Audio Generator

Data analysis Software

Jeongho Park is a developer and audiovisualartist that created various data-oriented and interac-tive projects, applications and installations45.

This project is a software application that gen-erates a tridimensional visual models and a sonifica-tion from event data of the LHC particles accelerator

at CERN in Geneve. Some events are bundled withthe application, and new ones can be download fromCERN website, since the application is natively com-patible with their databases.

Each line of the visualization is the path of aparticle, and when an event is “played” also a soundfrom each one of them is generated, with a differentfrequency according to its mass and properties.

Tools like this prove the didactic and researchvalue that an aesthetic approach can have, giving auseful and explorable 3D view of an event, while thesound gives information in an intuitive way, makingeasy to spot the general layout of an event, or quicklypinpointing abnormal or strange particles, that willstand out for their different tone.

45: htt p://jeonghopark.de/z

46: https://github.com/jeonghopark/lhcCernMac


73/126

73Case Studies

Screenshot from the application

Interface of the application


74/126


Jan Willem TulpGoldilocks

Data Visualization, website, 2015

The “Goldilocks” name comes from the hom-onym effect, that states that some phenomena mustfall within certain margins rather than reachingextremes: this principle is applied to astrobiology todetermine when a planet could theoretically supportlife, since it depends on a series of parameters that

must fall within certain boundaries47

.This interactive visualization displays all theknown exoplanets (1942 confirmed, at October 9 th,2015) and their host stars. Size and orbital parametersof the planets are based on the observed data, as wellas star size, color and position.

Alongside the main visualization, displayingall planets around their stars in their respective posi-tions, there are five other visualizations, more data-oriented, focused on: distance of each planet fromtheir star, underlying the ones in their system’s hab-itable zone; Earth Similarity Index; atmosphere andcomposition compared to Earth; temperature andmass; temperature of their stars and amount of en-ergy they receive from them.

47: http://tulpinteractive.com/goldilocks/


75/126

Various visualizations from

http://goldilocks.info/


76/126


kurzgesagtIn a Nutshell

YouTube channel

“In a nutshell” is a YouTube channel started in2013 by a Munich design studio, working with ani-mation and information design48. Their videos dealwith various topics, like physics, sociology, computerscience, biology, astronomy and politics, and alwaysexplain a particular subject related to some contem-

porary matter, to mention the most recent topics: thewar on drugs, what are red dwarf stars, what is aquantum computer, how Facebook steals video views,and the refugee crisis. It’s always some kind of de-manding argument, that may somehow “scare away”people looking for entertainment: but the channelhas nevertheless almost two million subscribers andis getting close to a hundred million visualizations,thanks to the their engaging style of drawing and an-imations, and for the easy but informative way theydeal with these topics49.

Very little of their content is related with dataitself, even if they use data visualization from timeto time, but the value of this case is in the way theyare doing science popularization and information,in a way demonstrating that it is possible to talk toregular people about “difficult”, niche and specializedarguments with the right design language.

48: http://kurzgesagt.org/profile/

49: https://www.youtube.com/user/Kurzgesag t


77/126

77Case Studies

Various still frames from “In a nutshell” videos


78/126


Nasa ScientificVisualization

Studio

NASA Scientific Visualization Studio works topopularize various contemporary science topics cov-ered at NASA, like Earth’s climate or planetary ob-servations. They also cover various techniques, fromsimple picture elaborations or combinations to ani-mations and various data visualizations.

Projects between the most popular include:Human Fingerprint on Global Air Quality:

thanks to to high-resolution global satellite maps,NASA tracked air pollution trends in the last decadearound the world. They compiled a map summariz-ing that data, and a video to explaining how and whyair quality is changing aroud the globe.

This project demostrates how valuable emissioncontrol laws are being in the United States and in Eu-rope, observing how general air quality improved inthese regions in the last years, while emerging andless regulated countries like China or India are seeinga decrease in air quality. Air pollution is linked to oth-er factors too: for example since the beginning of thecivil war Syria is decreasing in population due to emi-gration and most economical activities are shuttingdown, thus, ironically, leading to a better air quality.

Data is collected with the Ozone MonitoringSystem aboard the NASA Aura satellite50.

50: https://svs.gsfc.nasa.gov/cgi-bin/details.cgi?aid=12094


79/126

79Case Studies

“Human Fingerprint on Global Air Quality” map


80/126


Dial-a-Moon: This project is mainly a picture vi-sualizer, that takes a date and time as input and out-puts an image of the Moon in that moment, plus sometextual informations about phase, apparent diameter,distance and precise positioning. The very high reso-lution and detail of the pictures comes from the factthat they are taken from the Lunar ReconnaissanceOrbiter, a NASA probe orbiting the Moon, that pairsits LROC camera to a laser altimeter, LOLA, allowingfor a precise mapping of our satellite’s surface. It is alsopossible to download a high resolution map for eachpicture, with the name of each location written on it.

Together with the visualization are presentedsome video explaining related topics, for examplehow the Moon rotates around Earth and what its li-bration is52.

Five-Year Global Temperature Anomalies from

1880 to 2015: This project from NASA Scientific Visu-alization Studio shows the temperature anomalies onEarth’s surface from 1880 to 2015: this means that thecolors does not represent the actual temperature, butthe difference from the normal temperature.

Data is taken from the GISTEMP (GISS SurfaceTemperature Analysis, NASA Goddard Institute forSpace Studies) analysis system, combining data fromNOAA GHCN v3 (meteorological stations), ERSST v4(ocean areas), and SCAR (Antarctic stations)51.

Frames of “Five-Year Global Temperature Anomalies from 1880 to 2015”



81/126

81Case Studies

Space Weather: This animation shows the hy-pothetical voyage of a photon of light from the centerof the Sun to the Earth’s atmosphere. The video dis-plays also various phenomena that are encounteredin such a trip: it shows nuclear fusion and ignitionprocesses inside the Sun’s core; loops, flares, promi-nences and coronal material ejections on the surface;and then follow the solar particles until they reachEarth’s magnetic field, causing aurora53.

Still frames from the “Space Weather” video


53: htt ps://svs.gsfc.nasa.gov/Gallery/SpaceWeather.html


82/126

Nasa HyperwallVisualizations

The HyperWall is a visualization system with8x16 = 128 LCD modular screens, backed up by 128computational nodes and connected to the Pleiadessupercomputer (a 11312 multi-CPU nodes calculator).

NASA internally develops various kind of me-dia and simulation for this system, leveraging both on

the huge screen size available and the massive com-puting power behind it.

The visualizations include:Dionysius Crater: It’s a series of pictures taken

from the Lunar Reconnaissance Orbiter of a craterin the Sea of Tranquility, interesting for non-uniformand striated materials of which the crater seems to becomposed. Dionysius Crater’s rim


83/126

83Case Studies

Pluto’s surface composition: A series of pictureof the nano planet Pluto, taken with various instru-ments aboard the New Horizons probe. Togetherwith regular pictures, other instruments toke imagesin other wavelengths to investigate Pluto’s surfacecomposition. Also, High-resolution b/w images andlow-resolution color images where combined to ob-tain some of the most iconic pictures released.

NASA’s Hyperwall

Observations on Pluto’s surface


84/126


85/126

85Case Studies

GEOS-5 Nature Run Collection: This series ofmassive visualizations displays data from varioussimulations about weather and changes in Earth’satmosphere, run with the NASA Goddard Earth Ob-serving System Model (GEOS-5). Topics covered in-clude the evolution of surface temperatures, cloudsformation and movements, presence of water vapor,dispersion of aerosol from dust, biomass burning, fos-sil fuel emission and volcanoes, and winds.

The full simulation took 2 years to complete,running on 3750 processors on the Discover super-computer at the NASA Center for Climate Simulation,producing 400 terabytes of data.

On the other page, top to bottom: Earth’s surfacetemperature (colors) and outgoing radiation (white);surface (white) and upper level (colors) w inds; Dust (red), sea salt (blue), organic/black carbon(green), and sulfates (white).On this page, top: total precipitable water (white)and rainfall (colors, red is highest);middle: aerosol and human-initiated burningsimulation.


86/126


NOAA Coral Reef Watch 2015: A series of-animated Data Visualizations showing the real timeconditions of reef environments, helping to identifywhere the corals are losing their symbiotic algae,so losing their color and bleaching: a condition that

heavily endangers the survival of entire colonies.Data is taken from a variety of satellites, includingthe polar orbiters Suomi-NPP/VIIRS and MetOp-B/AVHRR, and the geostationary satellites MSG-3, MT-SAT-2, GOES-East, and GOES-West. Data itself consistmainly of temperatures, from which various otherindexes are calculated, according to average or in-stantaneous values.

top: sea-surface temperatures visualizationmid left: sea-surface temperature anomalies

visualizationmid-right: accumulated thermal stree visualization


87/126

87Case Studies

Solar Wind Strips the Martian Atmosphere:While today Mars is covered by a dry desert and has avery thin atmosphere, scientists agree it used be quitedifferent, with a thick atmosphere and probably bod-ies of liquid water like seas, lakes and rivers, similarly

to modern Earth. The change happened because theatmosphere dispersed into space, and the most prob-able cause is solar wind: in this video the simulationsmade to verify this theory are confronted with theMAVEN probe’s observations of ions leaving Mars’atmosphere to disperse in space.

top: Solar wind hitting Marsmid left: Mars atmosphere stripped away mid-right: Ions speeds observed by MAVEN


88/126


89/126

a design-centered

evolution6.


90/126


Design as the

unifying force

Analyzing projects from both scientific visu-alization and data-driven art, looking at their his-tory and at some case studies, it emerges that, whilethese disciplines underwent a great evolution and

has reached very remarkable levels of advancement,there is still room for notable improvement, especiallyif we consider the role that design methodology couldhave in the future of these disciplines.

Nowadays, in fact, SciViz and data art projectsare running in two different and almost parallel di-rections, with almost none contact points; but theycould both benefit a lot from some kind of interac-tion between them. Design could be the force drivingthem to cooperate, being almost “equidistant” fromboth of them, but still close enough to be able to in-teract with both. Recognizing the potential of design

for these fields and involving more designers into thedeveloping processes of scientific visualizations anddata art would bring a more reliable design method-ology, that would improve, for scientific visualization,important aesthetic qualities like the overall experi-ence and the design of interfaces, easing the work forresearchers and allowing the creation of better popu-larizing and generally public-distributed material; atthe same time data-oriented art projects could rely ona more solid work methodology and on a good tech-nologic know-how.

There are various examples of designers and re-searchers endorsing the potential of art: to mentionone, in the report made for a project experimentingvisualization techniques for Virtual Reality, DanielKeefe and his colleagues stated concluded that “artists[have] key roles in working closely with scientists todesign novel visual techniques for exploring data andtesting hypotheses”, and strongly suggest a interdis-ciplinary approach to visualization, “Teaching art tocomputer scientists, computer science to artists”.


91/126

91A Design-centered evolution

With the technological evolution of the lastdecades and the popularization of developing andeasy-to-use creative tools, it’s always more commonto see projects that involve interaction, immersive

video projections, sound and similar state-of-the-artfeatures. In this way the general audience is becom-ing more and more acquainted to this forms of media,and always more difficult to impress.

This implies that any forward-looking projectmust take into serious consideration the usage ofthe most interesting and appealing of these technol-ogies: it must not be modernism just for the sake ofit of course, but it must be taken into account duringthe design process that it may be more useful andfar more compelling for the audience to rzepresenta certain phenomenon, for example, with sound, or

giving the possibility to explore a particular datasetthrough virtual or augmented reality.It’s obvious that a weak design idea won’t magi-

cally acquire value thanks to the use of any tech-nology, but the right combination of a well-thoughtconcept and the appropriate technical mean willmuch more easily appeal and attract the public, andthe more time they will spending interacting or ob-serving the project the more it will possible for the tobecome interested in the underlying information and

Usage of New Media

Example of projection mapping, a new media thatcan be seen more and more in public events


92/126


topic.It must also be considered the value of new me-

dia for researchers and professionals, for example: inthe medical field the use of virtual reality has alreadyproven its outstanding value both for surgical train-ing and as assistance during surgeries54, and some

theorize that technologies such as Microsoft Hand-pose, an extremely precise hand-tracking system,will be paired with already-existing robotic preci-sion surgical arm to use during operations, allowingmuch more accurate actions and preventing humanmistakes55.

The use of sound is also proving its usefulnessin the field of astrophysics research: NASA is usingdata sonification through a software called “Xsonify”to translate observations made in various fields intosound. For example, they translated x-ray observa-tions, radio signals from Jupiter and micrometeoritesimpacts on the Voyager II probe56. The project wasborn to help visually impaired researchers, but it hasbeen found to be useful also for normally seeing in-dividuals: for example, for x-ray observations datawas usually visualized, but that method proved to bescarcely effective due to the high level of noise andthe difficulty to visually identify events and patterns.Moving to a sound-based system proved to be a bet-ter solution to pinpoint different kinds of events, thatpresents themselves as different timbres, or patterns,audible as rhythms57.

Interactive virtual reality simulation of a braintumor removal surgical operation.

54: Teodor P Grantcharov, “Is Virtual RealitySimulation an Effective Training Method

in Surgery?” - http://www.medscape.com/viewarticle/575183

55: https://virtualrealityreporter.com/virtual-reality-surgical-medical-application

56: Robert M. Candey, Anton M. Schertenleib, Wanda L. Diaz Merced, “Xsonify sonification

tool for space physi

Documents

New Media for Scientific Visualization