View
233
Download
3
Category
Tags:
Preview:
Citation preview
Architect-to-Architect Series
■ Series of five webinars over 3 months
■ Deeper look into Denodo Platform
■ Architectural Overview
■ Performance
■ Scalability (today’s session)
■ Data Discovery and Governance
■ Security
Denodo Express
■ Denodo Express
■ Free to Download
■ Fully functioning Data Virtualization Platform
■ Single user, supports common data sources
■ Many of the same capabilities of Denodo Platform
■ Performance, Data Discovery, Governance, internal Security, Publishing, …
Scalability – HA and Clustering
■ Denodo Cache
■ The Denodo Platform incorporates an advanced cache system to improve performance
■ For data materialization it makes use of an external JDBC database (physical or in-memory)
■ Denodo High-Availability
■ Set of features provided by the Denodo Platform to support High Availability and Load Balancing
■ Denodo Cluster Management
■ Tool for monitoring and managing a cluster of Denodo servers
Scalability – HA and Clustering
■ Singleton
■ High Availability Architecture
■ Active-Passive
■ Active-Active
■ Vertical Scalability
■ VM Support
■ Metadata Sharing
■ Horizontal Scalability
■ Clustering
■ Shared Cache or Individual Cache
• A single Data Virtualization server instance
…
Data sources
LDAP Auth
Cache Database
Client ApplicationESB/BPM/Portal…
Singleton
…
Data sources
LDAP Auth
Cache Database
Client ApplicationESB/BPM/Portal…
HA – Active-Passive
Load Balancer
HA – Active-Passive
■ All queries sent to Active server
■ Passive server running, but not handling requests
■ Both servers ‘pinged’ by Load Balancer with appliance health check or using ping script
■ When Load Balancer detects Active server failure
■ Requests redirected to Passive server
■ Passive server becomes ‘Active’
■ When former Active server restored
■ Load Balancer determines new roles…
…
Data sources
LDAP Auth
Cache Database
Client ApplicationESB/BPM/Portal…
HA – Active-Active
Load Balancer
HA – Active-Active
■ Load Balancer distributes queries between both servers
■ Both servers running and handling requests
■ Both servers ‘pinged’ by Load Balancer with appliance health check or using ping script
■ When Load Balancer detects server failure
■ All requests redirected to other server
■ When ‘failed’ server restored
■ Added back to load balancing pool
■ Starts receiving queries
Vertical Scaling
■ Java limitations inhibit vertical scaling for singletons
■ Virtualization
■ Run Denodo Platform instances in VM
■ VMWare, Hyper-V, Xen, …
■ Very common scenario
■ Multiple Instances
■ Install multiple instances of Denodo Platform
■ Run multiple instances of Denodo Platform from a single Denodo Platform installation
■ Share metadata and cache
Vertical Scaling (Cont’d)
■ First started instance is ‘master’
■ All others are ‘slaves’
■ Only ‘master’ instance can modify metadata
■ Slaves read metadata at startup time
■ Pros
■ Only one copy of metadata
■ Cons
■ Port configurations need modifying
■ SPoF
Horizontal Scaling – Clustering
Load Balancer
…
Dat
a so
urc
es
Virtual ServerVDP Cluster: 192.168.0.10:9999Web Container Cluster: 192.168.0.10:9090
VDP Cluster:• Denodo1:9999• Denodo2:9999• Denodo3:9999• Denodo4:9999• Denodo5:9999
Web Cont. Cluster:• Denodo1:9090• Denodo2:9090• Denodo3:9090• Denodo4:9090• Denodo5:9090
Clustering – Shared Cache
Load Balancer
Virtual ServerVDP Cluster: 192.168.0.10:9999Web Container Cluster: 192.168.0.10:9090
VDP Cluster:• Denodo1:9999• Denodo2:9999• Denodo3:9999• Denodo4:9999• Denodo5:9999
Web Cont. Cluster:• Denodo1:9090• Denodo2:9090• Denodo3:9090• Denodo4:9090• Denodo5:9090
Cache Database
Clustering – Separate Caches
Load Balancer
Virtual ServerVDP Cluster: 192.168.0.10:9999Web Container Cluster: 192.168.0.10:9090
VDP Cluster:• Denodo1:9999• Denodo2:9999• Denodo3:9999• Denodo4:9999• Denodo5:9999
Web Cont. Cluster:• Denodo1:9090• Denodo2:9090• Denodo3:9090• Denodo4:9090• Denodo5:9090
Cache Database
Cache Database
Cache Database
Cache Database
Cache Database
Clustering - Caches
■ Shared Cache
■ Any server can populate cache
■ All servers can access cached data
■ Increased ‘hit ratio’
■ Cache can become performance bottleneck
■ Separate Caches
■ Each server has its own cache
■ Avoids potential performance bottleneck
■ Access only data it has cached
■ Removes SPoF
Distributed Environments
■ Local caches help to achieve performance goals when servers are in geographically dispersed locations
New York City
Omaha
San Francisco
Houston
Distributed Environments (Cont’d)
■ A remote Denodo server can be viewed as a data source to another server
■ Promotes multi-layered architecture
■ Queries can be delegated to remote servers to reduce network data movement
■ Same query delegation mechanism as talked about in last session
Distributed Environments (Cont’d)
■ Remote query delegation…
Local Data Sources
Cache Cache
Local Data Sources
New York City HoustonClient
10 M
rows
5 M
rows
JOIN Result:
20,000 rowsJOIN Result:
2,500 rows
2 M
rows
Distributed Environments (Cont’d)
■ Shared views…
Local Data Sources
Cache Cache
Local Data Sources
New York City HoustonClient
10 M
rows
5 M
rows
JOIN Result:
20,000 rowsJOIN Result:
2,500 rows
2 M
rows
New York City Houston
Disaster Recovery
■ Business Continuity policies usually require that critical IT systems are backed up to a DR center
■ Denodo supports scripted synchronization of metadata with DR systems
■ Using Export and Import tools
■ Sync up can be scheduled (time-based) or on an event (promotion to production)
Scalability - Summary
■ Denodo Platform scales from single instance to geographically distributed system
■ Grow system without reconfiguring
■ Supports HA configurations
■ Active-Passive, Active-Active
■ Vertical scalability options
■ Support for DR site synchronization
Data Virtualization – Next Steps
Move forward at your own pace
Download Denodo Express –
The fastest way to Data Virtualization
Denodo Community: Documents, Videos, Tutorials, and more.
Attend Architect-to-Architect Series
Performance
Scalability
Move forward with one of our Data Virtualization experts
Phone: (+1) 877-556-2531 (NA)
Phone: (+44) (0)20 7869 8053 (EMEA)
Email: info@denodo.com | www.denodo.com
Data Discovery and Governance
Security
Five In-depth Technology and Architecture Sessions on Data Virtualization
Thank You!
Next SessionSession 4
Denodo Platform: Data Discoveryand Governance
Recommended