Embed Size (px)
Citation preview
Next Generation Next Generation Internet ServicesInternet Services
Dean JacobsTechnische Universität München
Talk OutlineWeb 2.0 and Consumer ServicesEnterprise Software as a ServiceCase Study: salesforce.comDatabase Virtualization
What Is Web 2.01. The Web As Platform2. Harnessing Collective Intelligence3. Data is the Next Intel Inside4. End of the Software Release Cycle5. Lightweight Programming Models6. Software Above the Level of a Single Device7. Rich User Experience
Tim O'Reilly 2005
1. The Web As PlatformSeamless cooperation (mashups) between
services Google maps + Craig’s List = housingmaps.com An ecosystem of interdependent services
Reach the “long tail” of users Exploit self-service and algorithmic data
management Google AdSense rather than DoubleClick ($3.1B)
Easy to experiment Low cost for failure Fostering a huge wave of innovation
2. Harnessing Collective Intelligence
People perform tasks for themselves, the community, or both (reputation building)
Computers amplify the data into significant value
Google Page rank Ebay Supply chain Wikipedia, MySpace, Flickr User-generated content Flickr, del.icio.us Tagging for classification ESP Game Two-person game that tags photos Digg Vote to determine importance of news stories Last.fm Song recommendations from listening patterns HSX Hollywood predictive market Amazon Customer ratings and reviews Amazon Mechanical Turk Hire people for simple tasks
3. Data is the Next Intel InsideOwnership Enhancemen
t
Maps NavTeq Google Maps
Satellite Images Digital Globe Google Maps
Book Numbers* ISBN Amazon
Home Valuations*
County Records
Zillow
Stock Valuations*
Stock Markets E*Trade
Product Reviews*
Amazon Amazon
Photographs* Photographer Flickr
Market Basket* ?? Amazon
Browsing Activity*
* Streaming Data: “The Live Web”
4. End of the Software Release Cycle
Software is offered as a service not a product
Operations become a core competency As important to Google as PageRank
Perpetual beta Weekly or even daily updates Real-time monitoring of user behavior
Dynamic scripting languages (Perl, Python, PHP, Ruby) are more nimble and therefore more suitable
5. Lightweight Programming Models
Started with HTTP/HTML and lead to Web Services Also applies to the design of Web Services APIs
Loose coupling is essential Make few assumptions about the other end Be self-describing and resilient to changes
Simple, flexible protocols are better than complex, specialized protocols Amazon reports 95% of their Web Services traffic
uses REST (XML over HTTP) rather than SOAP SOAP and WS-* are designed for Enterprise
Application Integration
6. Software Above the Level of a Single Device Applications should span servers, desktops,
laptops, cell phones, PDAs, and special devices Example: iPod/iTunes reaches from the handheld
device to a massive web back-end with the PC acting as a local cache and control station
7. Rich User Experience Rich Internet Applications started with browser
Applets Realized today as AJAX“AJAX isn't a technology. It's really several technologies, each flourishing
in its own right, coming together in powerful new ways. AJAX incorporates:• standards-based presentation using XHTML and CSS; • dynamic display and interaction using the Document Object Model; • data interchange and manipulation using XML and XSLT; • asynchronous data retrieval using XMLHttpRequest; • and JavaScript binding everything together.“
Jesse James Garrett - Adaptive Path
Talk OutlineWeb 2.0 and Consumer ServicesEnterprise Software as a ServiceCase Study: salesforce.comDatabase Virtualization
Enterprise Software Models
Data Center
Application Service Provider
System Integrators
Customers
Open Source
Platform Vendors
System Integrator
s
On-Premises Software
Software as a Service
Application Vendors
Customer
Data Center
Web 2.0 Checklist
Data Center
Application Service Provider
Customers
1. The Web As Platform2. Harnessing Collective Intelligence3. Data is the Next Intel Inside4. End of the Software Release Cycle5. Lightweight Programming Models6. Software Above the Level of a Single
Device7. Rich User Experiences
51
5
System Integrators
1 76
43
2
51
HostedServices
On-premisesApplications
Lower Total Cost of Ownership
Leverage economy of scale Capital expenditures – hardware, software Operational expenditures – bandwidth, personnel
Leverage open-source software Natural fit for a service-oriented model
One version of the software Runs on only one platform Upgrade all users at the same time (rolling upgrade)
Tight coupling of operations and support Resolve customer issues on the production system Perform professional services on the production
system
An application is more attractive to host if … Leverages Internet connectivity On-premises solutions expensive Less data and simpler operations Simpler configuration Simpler backend integration Weaker transactional requirements Weaker security requirements Less mission critical
Apply more to
People AppsSales, Marketing, HR,Help Desk, Portal
than
Process AppsPlanning, Purchasing,Inventory, Financials,Manufacturing
Hosting Sweet Spots
Apply more to small to mid-sized businesses
Opening up new markets
Limits on functionality
Sacrificing Functionality
Functi
onali
ty Total Cost of Ownership
Sca
labi
lity
Using on-premises software
The sweet spot
Not worth building
Not possible to build
Talk OutlineWeb 2.0 and Consumer ServicesEnterprise Software as a ServiceCase Study: salesforce.comDatabase Virtualization
The Primary ApplicationCustomer Relationship Management (CRM)
Marketing and Campaign Automation Salesforce Automation Customer Service and Support Analytics and Reporting
Hosted CRM Market Share- IDC Worldwide On-Demand CRM Vendor Analysis, 2005
50%
7%
All others
14%7%
salesforce
18%
4%
#2 #3#4
#5
The Web As PlatformWeb Services API to access salesforce objects
End users: strongly-typed for organization data model
Developers: weakly-typed to span organization models
Simple and scalable REST model to reduce server-side state Bulk operations to reduce communication overhead
Used to integrate with the ecosystem of partner services on-premises applications in enterprise data centers clients: browser (AJAX), off-line edition, mobile
devices, spreadsheets, calendar
Harnessing Collective Intelligence
A set of extensions can be packaged into an installable application by customers and SIs
Project management, expense tracking, budgeting, purchasing, HR, education, manufacturing, …
Targets the “long tail” of custom applications
1999 2000 2001 2002 2003 2004 2005
- Based on publicly available data. Bars represent fiscal quarters.
Organizations
Subscribers
30,000
650,000
March 2007
Service Growth
Organizations
Subscribers
20,000
350,000
October 2005
Scalability and Performance
0
200
400
600
800
1,000
1,200
1,400
1,600
1,800
2,000
FY05 Q1 FY05 Q2 FY05 Q3 FY05 Q4 FY06 Q1 FY06 Q2 FY06 Q3
0
200
400
600
800
1,000
1,200
1,400
1,600
1,800
2,000
Average Response
Time (milliseconds)
Transactions per quarter(millions)
- Based on publicly available data. Points represent fiscal quarters.
Talk OutlineWeb 2.0 and Consumer ServicesEnterprise Software as a ServiceCase Study: salesforce.comDatabase Virtualization
High-Level GoalsConsolidate multiple businesses (tenants)
onto the same operational systemReduce total cost of ownership
Pooling resources Improving management efficiency
Support Web 2.0 style collaborationTo the extent possible, hide multi-tenancy
from developers and system administrators
Specific RequirementsPool database resources to improve their
utilization Avoid provisioning each tenant for their maximum load Breaks down isolation: weakens security, increases
resource contention, interferes with optimizationsProvide a tenant-aware administrative framework
Manage farms of individual multi-tenant databases Support DML and DDL operations across tenants Support tenant migration within and across farms
Support Web 2.0 style collaboration Allow shared extensions to the base schema Allow shared public data with private updates3
2
Multi-Mode DeliveryThree ways of delivering software to the
enterprise On-premises The business controls the application Hosted The service provider controls the application Hybrid The application runs in the data center of
the business in an appliance and is remotely administered by the service provider
Customers should be able to migrate between them Or keep a warm back-up that is ready to use
Sweet spot is open source software, since that is commonly used in the hosted and hybrid models
Consolidation OptionsShared Machine
Shared Process
Shared Table
Isolation
Resource Pooling
Shared Machine
Cannot scale beyond tens of tenants per server
Appropriate for applications with a smaller number of larger tenants, e.g., for banking
PostgresSQL
MaxDB COTS 1 COTS 2 COTS 3
55 MB 80 MB 171 MB 74 MB 273 MBMemory requirements for a database
with one empty CRM schema instance
Shared Process
Scalability limited due to the amount of metadata But it is redundant for the base schema
Should scale up to thousands of tenants If each tenant gets their own table space then
migration entails simply moving files
PostgresSQL
MaxDB COTS 1 COTS 2 COTS 3
79 MB * 80 MB 616 MB * 2061 MB 359 MBMemory requirements for a database
with 10,000 empty CRM schema instances
* extrapolated
Shared TableData from many tenants in the same tables
Add a tenant id column Tenant queries must fix the value for this column
Extend the base schema using generic columns May be varchar or a mix of types The database must compactly represent sparse
tables Account
0021
0029
Name
Acme
Ball
...TenId
1041
1041
1053
1053
0016
0049
Gump
Wonk
Val0
1/3/95
3/7/72
red
blue
Val1
----
----
35
18
... Val100
----
----
----
----
Shared TableThe Good News - Everything is pooled
Processes, memory, connections, prepared statements Easy DML and DDL operations across tenants Add, remove, and extend tenants with DML (not DDL)
The Bad News - Isolation is very weak Irrelevant data infects query processing
• Optimization Statistics• Table Scans• Data locality
No indexes or integrity constraints on generic columns Migration requires querying the operational system
Possible ImprovementsShared Process – increase resource pooling
Keep one copy of the meta-data about the base schema
Allow prepared statements over table names Allow queries over table names Dynamically set the principal for a database
connectionShared Table – increase isolation
Take the tenant ID into account in query optimization, indexing, and data placement
Could be done from inside the database or from outside in a SQL transformation layer
Our Current ResearchAdapt an open source database for multi-tenancyMeasure its scalability and performance using a
benchmark for multi-tenant databases
Lead Campaign
Line Item
Product
Case
Contact
Contract
Account
Opportunity
Asset
conversion
source
source
parent
reports to
signed by
Simple CRM
Schema
