18
How QuickBase Works Behind the Scenes with the Platform Architect Jim Salem

How QuickBase Works Behind The Scenes - Jim Salem

  • Upload
    intuit

  • View
    3.312

  • Download
    6

Embed Size (px)

DESCRIPTION

Curious to know how QuickBase really works? Learn about QuickBases high level architecture to the essential details on how QuickBase requests are processed. You'll gain insights on what makes the QuickBase platform consistently reliable, as well as learn real-world tips on performance analysis and optimization.

Citation preview

Page 1: How QuickBase Works Behind The Scenes - Jim Salem

How QuickBase WorksBehind the Scenes with the Platform Architect

Jim Salem

Page 2: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

PERFORMANCE TIPS

QUICKBASE ARCHITECTURE

Q & A

Page 3: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

Architecture - Services View

All services run on redundant machines with load balancing and 24x7

monitoring.

QuickBase provides an integrated set of reliable

web services with a high-performing database

engine.

Page 4: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

Architectural Layers

WebLayer Web Server

• Buffers and manages requests

• Load balanced

• Proxy to external services

Load Balancer(MCP/SCP)

App Server

AppLayer

Services

Layer

• Application engine

• Proprietary in-memory db

• Active-active redundancy

Charting Full-Text Search

• Integrated services

• Clustered/Redundant

AuthLDAP

Email

Task Scheduler

System Data Store

File Storage

Native Web UI

Easy-to-use, Web-based

XML-based APIWeb service for developers

Interfaces • Native UI for “spreadsheet users”

• API targeted at developers

Web Server Web Server

App Server App Server

Page 5: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

Web Server

Load Balancer(MCP/SCP)

App Server

Charting Full-Text Search

AuthLDAP

Email

Task Scheduler

System Data Store

File Storage

Native Web UI

Easy-to-use, Web-based

XML-based APIWeb service for developers

Web Server Web Server

App Server App Server

The Life of a Request

WebLayer

AppLayer

Services

Layer

Interfaces

1

2

3

1

2

3

Customer clicks on a link in QuickBase or calls an API. That request is passed to one of our web servers

The Web server asks the MCP which App Server process is handling the application

The Web server passes the request to the App Server for processing.

If the App Server fails for any reason, the web server will inform the MCP and retry the request.

4

4

If necessary, the App Server will request additional information from the service layer

App server sends response to web browser via web server

5

5

5

Page 6: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

Database Engine

• In-memory database– Extremely fast (proprietary C++ engine)– Respond to most transactions without disk I/O– Designed for workgroup-sized applications

• “Auto-Tuning”– Smart lookup and formula caching

• By transaction, row, column, short/long term– Automatic indexing

• Backing Store– All changes immediately written to disk– Data paged out of RAM when not actively

in use

• Accessed via– Native UI – XML “RESTful” API

Backing Store

API

Interface

Mgmt

UI

Schema

Table Data

Forms and Reporting

Engine

App Server

Page 7: How QuickBase Works Behind The Scenes - Jim Salem

• Multi-Tenant Architecture– Real-time load balancing

• Security designed in from the start– Rich, role-based permissions model– Consistent permission checking framework

• High level: UI/API engine requires permission definition for each interface• Low level: Checks at the data access layer

• Redundant and Resilient – Real-time replication to disaster recovery site– No single points of failure

• Multiple internet connections; multiple power sources• Active-active replication for all app servers

– 24x7 Monitoring and management

• Web Service Integration• SDKs for Java, Javascript, Perl, VB, C#, Ruby, etc.

7

Designed for the Web

Primary Data Center

PROLIANT PROLIANT

PROLIANT PROLIANT

PROLIANT PROLIANT

High Speed Link

DR Data Center

PROLIANT PROLIANT

PROLIANT PROLIANT

PROLIANT PROLIANT

Page 8: How QuickBase Works Behind The Scenes - Jim Salem

QuickBase Statistics

– Performance• 225+ requests/sec (peak)•< 0.025 secs response time for most requests- Measured at data center edge (median)

– Uptime• Including planned and unplanned- Last 90 days: 99.98% (< 2 minutes/week)- Last 5 years: > 99.9% (< 10 minutes/week)

• Enabled by- World-class operations team- Reliable, resilient architecture- Extensive logging and monitoring infrastructure- Ongoing capacity planning and analysis

– Scalability Goals• Peak load always < 50% capacity• Clear and easy path to 10-100X growth

Intuit Confidential Page: 8

Page 9: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

PERFORMANCE TIPS

QUICKBASE ARCHITECTURE

Q & A

Page 10: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

QuickBase Performance

• The QuickBase architecture delivers excellent performance across a wide range of users and applications– 225+ requests/sec (peak)– Faster than “human perception”•Median response less than 0.025 seconds

• Application sizes– Up to 350MB of structured data– 100s of GB of attachments

• When might users experience delays?–When there is a combination of:• High usage (e.g., > 10,000 requests/day)• Large structured data size (e.g., 100s of MBs)• Very complex (e.g., 100s of fields and/or tables; dozens of relationships)

Page 11: How QuickBase Works Behind The Scenes - Jim Salem

1. Use a faster browser

2. Archive old data

3. Reduce wide-spread application dependencies

4. Optimize searches

5. Simplify!

11

Top Tips to Optimize Your Performance

If everything is running fine, don’t do anything!

Page 12: How QuickBase Works Behind The Scenes - Jim Salem

• QuickBase performs best on browsers with fast javascript support and sufficient cache– Don’t disable your browser’s cache!

• Typically:– Safari, Chrome and FireFox 3.5 outperform Internet Explorer (IE)– IE8 outperforms IE7 and IE6

12

Tip #1: Use a faster browser

Source: www.k840.com

(Smaller numbers are better)Source: Lifehacker.com

(Larger numbers are better)

Page 13: How QuickBase Works Behind The Scenes - Jim Salem

• Search speed is roughly proportional to the size of the data• Consider moving the oldest data to an “archival copy” of the

application:1. Make an copy of your application (in QuickBase)2. In the original app, delete records that are no longer needed.

13

Tip #2: Archive Old Data

View Old RecordsCopy Application(under Customize > Application)

Page 14: How QuickBase Works Behind The Scenes - Jim Salem

• Some customers share a table between many of their apps– Using the “cross-application relationship” or “shared multiple

choice” features– Examples: Department number tables, employee lists, etc.

• This forces all of those applications onto a single app server process.– Increases resource contention and queuing

• Solutions– Reduce/Eliminate the widely-used cross-application dependencies– Use multiple copies of the shared data.• You can automatically synchronize the copies once a day using API_ImportFromCSV

14

Tip #3: Reduce wide-spread dependencies

Page 15: How QuickBase Works Behind The Scenes - Jim Salem

• Turn off search for most fields• The order of your filter criteria is critical

• QuickBase processes filter criteria sequentially• Eliminate the most records with the first filters.

• Use “IS” before “Contains”• Exact matches are faster

• Filter native data fields before derived fields• Native: text, number, etc.• Derived: formula, lookup, etc.

• Sorting and grouping add handling time• Especially for large result sets

15

Tip #4: Optimize Searches

Page 16: How QuickBase Works Behind The Scenes - Jim Salem

• Apps grow organically–Often fields are never deleted, even if they are no longer needed.– This complicates maintenance and can slow performance– The field usage tab is your friend–> 100 fields is a lot

• De-clutter forms– Ease-of-use– Remove unneeded form rules

• Simplify complex permissions– Permissions based on values within each records• Requires significantly more computation when filtering records

– Permissions based on a user’s email address• Requires an email address look up

–Measuring performance impact of permissions• How much slower is a report for users with custom permissions vs. a user with access to all records?

16

Tip #5: Simplify!

Page 17: How QuickBase Works Behind The Scenes - Jim Salem

1. Use a faster browser

2. Archive old data

3. Reduce wide-spread cross-application relationships

4. Optimize searches

5. Simplify!

17

Top Tips to Optimize Your Performance

If everything is running fine, don’t do anything!

Page 18: How QuickBase Works Behind The Scenes - Jim Salem

04/11/2023

PERFORMANCE TIPS

QUICKBASE ARCHITECTURE

Q & A