Upload
willow
View
42
Download
4
Embed Size (px)
DESCRIPTION
Visualization Knowledge ( VisKo ): Leveraging the Semantic Web to Support Visualization. University of Texas at El Paso Computer Science. Velocity Model Visualization. A set of isosurfaces extracted from a seismic velocity model Covers a region in southern New Mexico. 3 km/s. Depth. - PowerPoint PPT Presentation
Citation preview
Visualization Knowledge (VisKo): Leveraging the Semantic Web to Support Visualization
University of Texas at El PasoComputer Science
Velocity Model Visualization
• A set of isosurfaces– extracted from a seismic velocity model– Covers a region in southern New Mexico
8 km/s
3 km/s
Dept
h
Visualizing the Velocity Model
• Generated by custom Java application– relied on Visualization Toolkit (VTK) for rendering– VTK was developed for rendering 3D visualizations– VTK is supported by Sandia, Los Alamos, ARL, and others
• Writing a custom visualization application:– may rely on third party package to support rendering– may need to perform some transformations on input
dataset before rendering
Program For Velocity Visualization
vtkImageReader rdr = new vtkImageReader();rdr.SetFileName(inputDatasetFilePath);rdr.SetDataScalarTypeToUnsignedShort();rdr.SetDataByteOrderToLittleEndian();rdr.SetFileDimensionality(3);rdr.SetDataOrigin(0,0,0);rdr.SetDataSpacing(1,1,1);rdr.SetDataExtent(0,230,0,25,0,68);rdr.SetNumberOfScalarComponents(1);rdr.FileLowerLeftOn();rdr.Update();
vtkContourFilter contours = new vtkContourFilter();contours.SetInput(rdr.GetOutput());contours.GenerateValues(35,0.0,9000.0);
vtkPolyDataMapper contMapper = new vtkPolyDataMapper();contMapper.SetInput(contours.GetOutput());contMapper.SetScalarRange(0.0,9000.0);
vtkActor contActor = new vtkActor();contActor.SetMapper(contMapper);contActor.RotateX(105);
vtkRenderer ren1 = new vtkRenderer();ren1.AddActor(contActor);ren1.AddActor2D(outlineActor);ren1.SetBackground(1,1,1);
vtkRenderWindow renWin = new vtkRenderWindow();renWin.SetOffScreenRendering(1);renWin.AddRenderer(ren1);renWin.SetSize(300,300);renWin.Render();
vtkJPEGWriter img = new vtkJPEGWriter();img.SetInputConnection(renWin.GetOutputPort());img.SetFileName(outputDatasetFilePath);img.SetQuality(100);
What Information is Needed?
• Regarding visualization, users need to know:– What view suits their data (e.g., contours, map, surfaces)– What properties should the view exhibit (e.g., orientation,
projection, color, size)
• Regarding datasets, users need to know:– The semantic type of the data (e.g., gravity, velocity)– The type of data helps with setting parameter arguments– The format the data is encoded in (e.g., netCDF, ESRI)
What Information is Needed?
• Regarding third party rendering software:– Can it generate the required renderings?– Can it ingest my data in the format it resides in?– Does it satisfy my performance requirements?– What language can I use to interface with it?– What dependent packages do I need to install?
Where to Get Information
• Regarding the view:– some user may prefer to see isosurfaces whereas another
user prefers volume renderings (its personal)• Regarding the data:– provenance may declare what format and type of dataset– Provenance Markup Language (PML) has fields for
recording both type and format• Regarding the third party rendering software:– Reading, reading, and more reading– May have a visualization expert assist you
Writing the Program
• Once you have answers for the previous questions, you can begin coding
• Some portion of the code will transform input datasets into formats that can be rendered– Most renderers format specific (exception Protovis)
• The rest of the code will:– Gather suitable arguments for the rendering– Invoking the renderer with the arguments
Overview of the Visualization Program
• The visualization program that generated the velocity model visualization:– Relies on VTK for rendering– Renders the data as isosurfaces– Ingests data in format binaryFloatArray– transforms binaryFloatArray to VTKImageData– Ingests data of type 3DVelocityModel– Is a serial application– Is written in Java
Presentation Overview
1. Toolkits and Visualization Pipelines2. Visualization Query3. Automated Generation of Visualization pipelines4. Ontological Description of Visualization Pipelines5. Execution of Visualization Pipelines6. Giovanni and VisKo
Visualization Toolkits
• Visualization Toolkit (VTK) was used to render the velocity visualization
• VTK is a toolkit that provides functions such as:– Filtering– Gridding/interpolating– Mapping (i.e., transform data into views like isosurfaces)– Rendering the views
• Functions are referred to as operators– Generic mapping tools (GMT): 60 operators– VTK: hundreds of operators
Visualization Pipeline Model• VTK requires that users write pipelines– the output of an operator feeds into the operator next in
the pipeline sequence– first operator in pipeline is usually data reader– final operator in pipeline is usually renderer
• Thus the Java program that visualizes the velocity model can be seen as a pipeline of VTK operators
• It is up to the users to write these pipelines…
VTK Java Pipeline For Velocity Model
vtkImageReader rdr = new vtkImageReader();rdr.SetFileName(inputDatasetFilePath);rdr.SetDataScalarTypeToUnsignedShort();rdr.SetDataByteOrderToLittleEndian();rdr.SetFileDimensionality(3);rdr.SetDataOrigin(0,0,0);rdr.SetDataSpacing(1,1,1);rdr.SetDataExtent(0,230,0,25,0,68);rdr.SetNumberOfScalarComponents(1);rdr.FileLowerLeftOn();rdr.Update();
vtkContourFilter contours = new vtkContourFilter();contours.SetInput(rdr.GetOutput());contours.GenerateValues(35,0.0,9000.0);
vtkPolyDataMapper contMapper = new vtkPolyDataMapper();contMapper.SetInput(contours.GetOutput());contMapper.SetScalarRange(0.0,9000.0);
vtkActor contActor = new vtkActor();contActor.SetMapper(contMapper);contActor.RotateX(105);
vtkRenderer ren1 = new vtkRenderer();ren1.AddActor(contActor);ren1.AddActor2D(outlineActor);ren1.SetBackground(1,1,1);
vtkRenderWindow renWin = new vtkRenderWindow();renWin.SetOffScreenRendering(1);renWin.AddRenderer(ren1);renWin.SetSize(300,300);renWin.Render();
vtkJPEGWriter img = new vtkJPEGWriter();img.SetInputConnection(renWin.GetOutputPort());img.SetFileName(outputDatasetFilePath);img.SetQuality(100);
Pipeline of Visualization Operators
vtkImageReader rdr = new vtkImageReader();rdr.SetFileName(inputDatasetFilePath);rdr.SetDataScalarTypeToUnsignedShort();rdr.SetDataByteOrderToLittleEndian();rdr.SetFileDimensionality(3);rdr.SetDataOrigin(0,0,0);rdr.SetDataSpacing(1,1,1);rdr.SetDataExtent(0,230,0,25,0,68);rdr.SetNumberOfScalarComponents(1);rdr.FileLowerLeftOn();rdr.Update();
vtkContourFilter contours = new vtkContourFilter();contours.SetInput(rdr.GetOutput());contours.GenerateValues(35,0.0,9000.0);
vtkPolyDataMapper contMapper = new vtkPolyDataMapper();contMapper.SetInput(contours.GetOutput());contMapper.SetScalarRange(0.0,9000.0);
vtkActor contActor = new vtkActor();contActor.SetMapper(contMapper);contActor.RotateX(105);
vtkRenderer ren1 = new vtkRenderer();ren1.AddActor(contActor);ren1.AddActor2D(outlineActor);ren1.SetBackground(1,1,1);
vtkRenderWindow renWin = new vtkRenderWindow();renWin.SetOffScreenRendering(1);renWin.AddRenderer(ren1);renWin.SetSize(300,300);renWin.Render();
vtkJPEGWriter img = new vtkJPEGWriter();img.SetInputConnection(renWin.GetOutputPort());img.SetFileName(outputDatasetFilePath);img.SetQuality(100);
Op 1
Op 2
Op 3
Op 5
Op 6
Op 7
Op 8
Different Types of Operators
vtkImageReader rdr = new vtkImageReader();rdr.SetFileName(inputDatasetFilePath);rdr.SetDataScalarTypeToUnsignedShort();rdr.SetDataByteOrderToLittleEndian();rdr.SetFileDimensionality(3);rdr.SetDataOrigin(0,0,0);rdr.SetDataSpacing(1,1,1);rdr.SetDataExtent(0,230,0,25,0,68);rdr.SetNumberOfScalarComponents(1);rdr.FileLowerLeftOn();rdr.Update();
vtkContourFilter contours = new vtkContourFilter();contours.SetInput(rdr.GetOutput());contours.GenerateValues(35,0.0,9000.0);
vtkPolyDataMapper contMapper = new vtkPolyDataMapper();contMapper.SetInput(contours.GetOutput());contMapper.SetScalarRange(0.0,9000.0);
vtkActor contActor = new vtkActor();contActor.SetMapper(contMapper);contActor.RotateX(105);
vtkRenderer ren1 = new vtkRenderer();ren1.AddActor(contActor);ren1.AddActor2D(outlineActor);ren1.SetBackground(1,1,1);
vtkRenderWindow renWin = new vtkRenderWindow();renWin.SetOffScreenRendering(1);renWin.AddRenderer(ren1);renWin.SetSize(300,300);renWin.Render();
vtkJPEGWriter img = new vtkJPEGWriter();img.SetInputConnection(renWin.GetOutputPort());img.SetFileName(outputDatasetFilePath);img.SetQuality(100);
Op 1
Op 2
Op 3
Op 5
Op 6
Op 7
Op 8
Transformer
View Mapper
Renderer
Transformer
Transformer
Transformer
Transformer
Operators are Parameterized
vtkImageReader rdr = new vtkImageReader();rdr.SetFileName(inputDatasetFilePath);rdr.SetDataScalarTypeToUnsignedShort();rdr.SetDataByteOrderToLittleEndian();rdr.SetFileDimensionality(3);rdr.SetDataOrigin(0,0,0);rdr.SetDataSpacing(1,1,1);rdr.SetDataExtent(0,230,0,25,0,68);rdr.SetNumberOfScalarComponents(1);rdr.FileLowerLeftOn();rdr.Update();
vtkContourFilter contours = new vtkContourFilter();contours.SetInput(rdr.GetOutput());contours.GenerateValues(35,0.0,9000.0);
vtkPolyDataMapper contMapper = new vtkPolyDataMapper();contMapper.SetInput(contours.GetOutput());contMapper.SetScalarRange(0.0,9000.0);
vtkActor contActor = new vtkActor();contActor.SetMapper(contMapper);contActor.RotateX(105);
vtkRenderer ren1 = new vtkRenderer();ren1.AddActor(contActor);ren1.AddActor2D(outlineActor);ren1.SetBackground(1,1,1);
vtkRenderWindow renWin = new vtkRenderWindow();renWin.SetOffScreenRendering(1);renWin.AddRenderer(ren1);renWin.SetSize(300,300);renWin.Render();
vtkJPEGWriter img = new vtkJPEGWriter();img.SetInputConnection(renWin.GetOutputPort());img.SetFileName(outputDatasetFilePath);img.SetQuality(100);
Op 1
Op 2
Op 3
Op 5
Op 6
Op 7
Op 8
P1
P2
P5
P6
P7
P8
P9
P3
P4
What Kind of Skills are Currently required by a user?
• Knows about different views• Knows what toolkits support a particular view• Knows what toolkits operate on a particular data• Knows how to install a visualization toolkit• Knows what language the toolkit is built on• Knows what operators need to compose a pipeline• Knows suitable arguments for the operators• Knows how to develop software
What Kind of Skills are Currently required by a user?
• Knows about different views
• Knows what toolkits support a particular view• Knows what toolkits operate on a particular data
• Knows how to install a visualization toolkit• Knows what language the toolkit is built on• Knows what operators need to compose a pipeline• Knows suitable arguments for the operators• Knows how to develop software
scientist
VisualizationExpert
Engineer
Presentation Overview
1. Toolkits and Visualization Pipelines2. Visualization Query3. Automated Generation of Visualization pipelines4. Ontological Description of Visualization Pipelines5. Execution of Visualization Pipelines6. Giovanni and VisKo
Declarative Requests
• Many user skills needed to visualize data• Aside from cognitive aspects of visualization, a large
part of the problem is engineering• Stems from fact that we generate visualizations
imperatively (i.e., write code)
Can we provide a means for users to generate visualizations declaratively (i.e., specify what visualization they
want without having to code)?
Visualization Query
(AND (hasView ?VIS isosurfaces)(hasContent ?VIS http://vel.3d)(hasFormat ?VIS floatArray)(hasType ?VIS velocityData)(viewedBy ?VIS mozilla-firefox)(hasValue numContours 36))
Desired View URL of Data to visualize
Semantic Type of datasetWDO types (UTEP)
Format of dataset
Parameter Argument Viewer
• The velocity model visualization was actually a result of a visualization query
Requested Visualization
Visualization Queries and SQL
• Visualization queries mirror SQL queries– query request is specified declaratively– request is then translated into a query plan– query plan computes the result requested by the query
• Information specified in visualization queries is used to derive pipelines rather than query plans
• The pipeline in turn generates the visualization requested in the query
Visualization Query Challenges
What kind of knowledge is needed to generate pipelines that answer
visualization queries?
What infrastructure can leverage the knowledge to support the generation
and execution of the pipelines?
VisKo Project Claims
• VisKo supplements user skills with visualization knowledge to compose pipelines
• VisKo is a framework for:– Encoding user skills into visualization knowledge– managing visualization knowledge– Executing pipelines
Old Way vs VisKo
vtkImageReader rdr = new vtkImageReader();rdr.SetFileName(inputDatasetFilePath);rdr.SetDataScalarTypeToUnsignedShort();rdr.SetDataByteOrderToLittleEndian();rdr.SetFileDimensionality(3);rdr.SetDataOrigin(0,0,0);rdr.SetDataSpacing(1,1,1);rdr.SetDataExtent(0,230,0,25,0,68);rdr.SetNumberOfScalarComponents(1);rdr.FileLowerLeftOn();rdr.Update();
vtkContourFilter contours = new vtkContourFilter();contours.SetInput(rdr.GetOutput());contours.GenerateValues(35,0.0,9000.0);
vtkPolyDataMapper contMapper = new vtkPolyDataMapper();contMapper.SetInput(contours.GetOutput());contMapper.SetScalarRange(0.0,9000.0);
(AND (hasView ?VIS isosurfaces)(hasContent ?VIS http://vel.3d)(hasFormat ?VIS floatArray)(hasType ?VIS velocityData)(viewedBy ?VIS mozilla-firefox)(hasValue numContours 36))
Presentation Overview
1. Toolkits and Visualization Pipelines2. Visualization Query3. Automated Generation of Visualization pipelines4. Ontological Description of Visualization Pipelines5. Execution of Visualization Pipelines6. Giovanni and VisKo
A Pipeline Synthesized by VisKoda
ta fl
ow
operators
VisKo Pipeline Composition
• The query tells the system:– the input format– the target view
• From target view:– Identify operator that generates view (i.e. view mapper)
• From operator:– identify format it operates on (i.e., target format)
• Find sequence of operators that transforms the input format to target format
Information From Query
hasView
BinaryFloatArray
hasF
orm
at
Dataset
Information from Query
Isosurfaces view
Knowledge about Toolkit Operators
vtkContour
Filter
vtkPolyData
in format
out f
orm
at
generatesViewhasView
vtkImageData
BinaryFloatArray
hasF
orm
at
Dataset
Information from Query Knowledge about toolkit operators
Isosurfaces view
The Knowledge is Linked
vtkContour
Filter
vtkPolyData
in format
out f
orm
at
generatesViewhasView
vtkImageData
BinaryFloatArray
hasF
orm
at
Dataset
Information from Query Knowledge about toolkit operators
Isosurfaces view
Both query and operator reference “isosurfaces”
Format Transformation?
vtkContour
Filter
vtkPolyData
in format
out f
orm
at
generatesViewhasView
vtkImageData
BinaryFloatArray
hasF
orm
at
Dataset
Information from Query Knowledge about toolkit operators
Isosurfaces view
Required Pipeline
vtkContour
Filter
vtkPolyData
in format
out f
orm
at
generatesViewhasView
vtkImageData
BinaryFloatArray
hasF
orm
at
Dataset
Information from Query Knowledge about toolkit operators
Isosurfaces view
Multiple Results Example
Visualization Query
No view specified!
Input format is ESRI Gridded
Data is of type Gravity Data
Multiple Results Example
Query Results
VisKo was able to generate three different pipelines, given the query and visualization
knowledge currently loaded
Multiple Visualizations Example
Query Results
Multiple Visualizations ExampleVisualization Query
Query Results
PDF vs PNG Format
View was left unspecified in query, so system visualized data by any means
Composition by Rules
• Pipeline composition is actually derived through application of rules– rules are applied to statements comprising our
visualization knowledge– rules are simple horn clauses
• Rules are not described in this seminar• Before we can apply rules, we need to know what
statements comprise our knowledge base
Presentation Overview
1. Toolkits and Visualization Pipelines2. Visualization Query3. Automated Generation of Visualization pipelines4. Ontological Description of Visualization Pipelines5. Execution of Visualization Pipelines6. Giovanni and VisKo
VisKo Visualization Language
• VisKo provides a language to describe operators and how they can be composed into pipelines
• The language’s expressivity is focused on describing:– Views and view properties– Different operator types– Parameters associated with operators
• The language is defined by ontologies encoded in Ontology Web Language (OWL)
VisKo Language Layers• The VisKo language encompasses three different
ontologies to describe toolkit operators from different perspectives
Visko-Service(services, parameters, and types)
Visko-Operator(operator function + composition rules)
Visko-Views(views and properties)
Execution
Visualization
Spec
trum
Encoding Velocity Model View
• Velocity model is visualized as a set of isosurfaces, so this view is defined as a set of ESIP surfaces
• We need to describe this resource isosurfaces in terms of the ontology:
Isosurfaces isa SurfacesIsosurfaces isa GeometryIsosurfaces isa AtomicViewIsosurfaces isa View
Isosurfaces description
Note: VisKo relies on Resource Document Framework (RDF) for encoding statements
Encoding Velocity Model Operator
• A contouring operator generated the isosurfaces• The contouring operator– operated on data in format 3DImageData– generated the view isosurfaces– output plot in format PolyData
contouringOperator isa MappercontouringOperator operatesOn 3DImageDatacontouringOperator transformsTo PolyDatacontouringOperator mapsTo isosurfaces
vtkContourFilter description
Note: contouring operator is conceptual and cannot be executed
Encoding Velocity Model Service
• The contouring operator was implemented by the VTKContourFilter service
vtkContourFilter isa ServicevtkContourFilter implements contouringvtkContourFilter supportedBy VTK
vtkContourFilter hasInput contourSpacingvtkContourFilter hasInput numberOfContoursvtkContourFilter hasGrounding wsdlGrounding
vtkContourFilterdescription
VisKo-Service
OWL-S
Executable VisKo service implements operator contouring
Presentation Overview
1. Toolkits and Visualization Pipelines2. Visualization Query3. Automated Generation of Visualization pipelines4. Ontological Description of Visualization Pipelines5. Execution of Visualization Pipelines6. Giovanni and VisKo
Pipeline Execution
• VisKo infrastructure is also responsible for executing the derived pipelines
• Pipeline Execution Requirements– All operators must have a Web service implementation– Any operator that does not have a service implementation
cannot be executed, and neither can its parent pipeline
Visualization Language Layer Cake• VisKo language does not express execution details• VisKo is therefore is built on other
languages/standards that specialize in execution of services and pipelines
WSDL
GMT VTK IDL
OWL-S
VisKo
Executable Toolkit Operators
Web Services
Semantic Web Services
Visualization Language
SEMANTIC
SYNTACTIC
PROGRAMINGLANGUAGE
Execution layers
Presentation Overview
1. Toolkits and Visualization Pipelines2. Visualization Query3. Automated Generation of Visualization pipelines4. Ontological Description of Visualization Pipelines5. Execution of Visualization Pipelines6. Giovanni and VisKo
Giovanni vs VisKo
Giovanni VisKo Capability
Visualization
Configurable views
Pipeline (i.e., workflow)
Multiple Toolkits
Provenance Capture
Dimension Compression
Dynamic Workflow
Plug and play
Automatically Adaptable
Giovanni From GrADS to NCL
• Had to rework Giovanni when NCL was introduced• Workflow may have needed to be modified:– Added operators to convert to new format required by
new operator– Configure or change actor to work with protocol of new
visualization operator– Input new parameters to the visualization module
Giovanni-VisKo
• VisKo could mediate some issues associated with:– Adding new visualization capabilities– Modifying existing capabilities– Workflow versioning
• Underlying toolkits and pipelines will constantly change but the visualization query should remain the same
• So the interface to VisKo will never change
Giovanni-VisKo (1)
• Giovanni’s Kepler workflow is performing two functions:– Scientific workflow: bias correction, filtering, gridding– Visualization: possibly converting to format that NCL
renderer can process
• If Giovanni workflow could be partitioned into two sub workflows:– It could focus only on the scientific data processing– And delegate the visualization responsibilities to VisKo
Thanks for listening!
VisKo Ontologies
VisKo-View Ontology
• Defines different views and view properties
infovis Propertiesscivis
Imported from ESIP Data-type ontology
ESIP Views
• Defines geometries of scientific data with different dimensionality of spatial components
VisKo-Operator Ontology
• Defines different kinds of visualization toolkit operators
Operators
Format
Imported from PML-P provenance ontologyImported from VisKo-View
Note: operator concepts are conceptual and cannot be executed
VisKo-Service Ontology
• Defines relations between an operator and OWL-S• Imports OWL-S Service (service)• Imports OWL-S Process (parameters)
Cardinality: an operator can be implemented by many services
Cardinality > 1
Data type
Pipeline Procedure and Rules
Pipeline Composition Procedure
• Non-logic based description of algorithm
Using our knowledge about visualization operators:1. Find operator A that works on format of input data2. Find operator B that works on format output from
operator A3. Chain A and B4. A B; Repeat from 2 until output format of B can be
ingested into operator that generates requested view
Composition Rules
• Rules that derive pipelines are applied to– statements about the visualization toolkit operators– Statements asserted in the query
• The composition rules are encoded in OWL2– Inverse properties– Property chains– Property transitivity
OWL Composition Rules
canBeTransformedTo(?Fmt1, ?Fmt2) :-isOperatedOnBy(Fmt1, A),transformsTo(A, Fmt2).
A
Legend:
Asserted PropertiesoperatesOntransformsTo
Inferred PropertiesisOperatedOnBycanBeTransformedTo
B
Fmt1
Fmt2
Fmt3
OWL Transitive Property Chain
Transformer
isOperatedOnBy(?Fmt1, ?A) :- operatesOn(A, Fmt1).
OWL Inverse Property
OWL-S and WSDL
The Role of OWL-S
• Visualization toolkits:– Have different protocols for interfacing with operators– Are implemented as executable, C/C++ libs, Java libs etc.
• OWL-S adds a layer over the different protocols and “levels the playing field” so to speak…
VTK(C++) GMT(.exe)
OWL-S
VisKo interfaces with only OWL-S
WSDL
Declarative
Imperative
The Role of WSDL
• Web Service Description Language is an XML format for describing service endpoints….syntactically
• OWL-S to bridges semantic/syntactic gap• Toolkit operators must be exposed as WSDL services
to be considered in the VisKo framework
WSDL Document