Upload
intuit
View
3.312
Download
6
Tags:
Embed Size (px)
DESCRIPTION
Curious to know how QuickBase really works? Learn about QuickBases high level architecture to the essential details on how QuickBase requests are processed. You'll gain insights on what makes the QuickBase platform consistently reliable, as well as learn real-world tips on performance analysis and optimization.
Citation preview
How QuickBase WorksBehind the Scenes with the Platform Architect
Jim Salem
04/11/2023
PERFORMANCE TIPS
QUICKBASE ARCHITECTURE
Q & A
04/11/2023
Architecture - Services View
All services run on redundant machines with load balancing and 24x7
monitoring.
QuickBase provides an integrated set of reliable
web services with a high-performing database
engine.
04/11/2023
Architectural Layers
WebLayer Web Server
• Buffers and manages requests
• Load balanced
• Proxy to external services
Load Balancer(MCP/SCP)
App Server
AppLayer
Services
Layer
• Application engine
• Proprietary in-memory db
• Active-active redundancy
Charting Full-Text Search
• Integrated services
• Clustered/Redundant
AuthLDAP
Task Scheduler
System Data Store
File Storage
Native Web UI
Easy-to-use, Web-based
XML-based APIWeb service for developers
Interfaces • Native UI for “spreadsheet users”
• API targeted at developers
Web Server Web Server
App Server App Server
…
…
04/11/2023
Web Server
Load Balancer(MCP/SCP)
App Server
Charting Full-Text Search
AuthLDAP
Task Scheduler
System Data Store
File Storage
Native Web UI
Easy-to-use, Web-based
XML-based APIWeb service for developers
Web Server Web Server
App Server App Server
The Life of a Request
WebLayer
AppLayer
Services
Layer
Interfaces
1
2
3
1
2
3
Customer clicks on a link in QuickBase or calls an API. That request is passed to one of our web servers
The Web server asks the MCP which App Server process is handling the application
The Web server passes the request to the App Server for processing.
If the App Server fails for any reason, the web server will inform the MCP and retry the request.
4
4
If necessary, the App Server will request additional information from the service layer
App server sends response to web browser via web server
5
5
5
…
…
04/11/2023
Database Engine
• In-memory database– Extremely fast (proprietary C++ engine)– Respond to most transactions without disk I/O– Designed for workgroup-sized applications
• “Auto-Tuning”– Smart lookup and formula caching
• By transaction, row, column, short/long term– Automatic indexing
• Backing Store– All changes immediately written to disk– Data paged out of RAM when not actively
in use
• Accessed via– Native UI – XML “RESTful” API
Backing Store
API
Interface
Mgmt
UI
Schema
Table Data
Forms and Reporting
Engine
App Server
• Multi-Tenant Architecture– Real-time load balancing
• Security designed in from the start– Rich, role-based permissions model– Consistent permission checking framework
• High level: UI/API engine requires permission definition for each interface• Low level: Checks at the data access layer
• Redundant and Resilient – Real-time replication to disaster recovery site– No single points of failure
• Multiple internet connections; multiple power sources• Active-active replication for all app servers
– 24x7 Monitoring and management
• Web Service Integration• SDKs for Java, Javascript, Perl, VB, C#, Ruby, etc.
7
Designed for the Web
Primary Data Center
PROLIANT PROLIANT
PROLIANT PROLIANT
PROLIANT PROLIANT
High Speed Link
DR Data Center
PROLIANT PROLIANT
PROLIANT PROLIANT
PROLIANT PROLIANT
QuickBase Statistics
– Performance• 225+ requests/sec (peak)•< 0.025 secs response time for most requests- Measured at data center edge (median)
– Uptime• Including planned and unplanned- Last 90 days: 99.98% (< 2 minutes/week)- Last 5 years: > 99.9% (< 10 minutes/week)
• Enabled by- World-class operations team- Reliable, resilient architecture- Extensive logging and monitoring infrastructure- Ongoing capacity planning and analysis
– Scalability Goals• Peak load always < 50% capacity• Clear and easy path to 10-100X growth
Intuit Confidential Page: 8
04/11/2023
PERFORMANCE TIPS
QUICKBASE ARCHITECTURE
Q & A
04/11/2023
QuickBase Performance
• The QuickBase architecture delivers excellent performance across a wide range of users and applications– 225+ requests/sec (peak)– Faster than “human perception”•Median response less than 0.025 seconds
• Application sizes– Up to 350MB of structured data– 100s of GB of attachments
• When might users experience delays?–When there is a combination of:• High usage (e.g., > 10,000 requests/day)• Large structured data size (e.g., 100s of MBs)• Very complex (e.g., 100s of fields and/or tables; dozens of relationships)
1. Use a faster browser
2. Archive old data
3. Reduce wide-spread application dependencies
4. Optimize searches
5. Simplify!
11
Top Tips to Optimize Your Performance
If everything is running fine, don’t do anything!
• QuickBase performs best on browsers with fast javascript support and sufficient cache– Don’t disable your browser’s cache!
• Typically:– Safari, Chrome and FireFox 3.5 outperform Internet Explorer (IE)– IE8 outperforms IE7 and IE6
12
Tip #1: Use a faster browser
Source: www.k840.com
(Smaller numbers are better)Source: Lifehacker.com
(Larger numbers are better)
• Search speed is roughly proportional to the size of the data• Consider moving the oldest data to an “archival copy” of the
application:1. Make an copy of your application (in QuickBase)2. In the original app, delete records that are no longer needed.
13
Tip #2: Archive Old Data
View Old RecordsCopy Application(under Customize > Application)
• Some customers share a table between many of their apps– Using the “cross-application relationship” or “shared multiple
choice” features– Examples: Department number tables, employee lists, etc.
• This forces all of those applications onto a single app server process.– Increases resource contention and queuing
• Solutions– Reduce/Eliminate the widely-used cross-application dependencies– Use multiple copies of the shared data.• You can automatically synchronize the copies once a day using API_ImportFromCSV
14
Tip #3: Reduce wide-spread dependencies
• Turn off search for most fields• The order of your filter criteria is critical
• QuickBase processes filter criteria sequentially• Eliminate the most records with the first filters.
• Use “IS” before “Contains”• Exact matches are faster
• Filter native data fields before derived fields• Native: text, number, etc.• Derived: formula, lookup, etc.
• Sorting and grouping add handling time• Especially for large result sets
15
Tip #4: Optimize Searches
• Apps grow organically–Often fields are never deleted, even if they are no longer needed.– This complicates maintenance and can slow performance– The field usage tab is your friend–> 100 fields is a lot
• De-clutter forms– Ease-of-use– Remove unneeded form rules
• Simplify complex permissions– Permissions based on values within each records• Requires significantly more computation when filtering records
– Permissions based on a user’s email address• Requires an email address look up
–Measuring performance impact of permissions• How much slower is a report for users with custom permissions vs. a user with access to all records?
16
Tip #5: Simplify!
1. Use a faster browser
2. Archive old data
3. Reduce wide-spread cross-application relationships
4. Optimize searches
5. Simplify!
17
Top Tips to Optimize Your Performance
If everything is running fine, don’t do anything!
04/11/2023
PERFORMANCE TIPS
QUICKBASE ARCHITECTURE
Q & A