Yupeng Fu Kian Win Ong Kevin Zhao Yannis Papakonstantinou Vicky Papavasiliou Michalis Petropoulos The FORWARD SQL-Based Declarative Framework for Ajax

Yupeng FuKian Win Ong

Kevin ZhaoYannis Papakonstantinou

Vicky PapavasiliouMichalis Petropoulos

The FORWARD SQL-Based Declarative Framework

for Ajax Applications

Vast Majority of Web Applications: Issue simple SQL commands, reflect new state on browser

Web Application

Database

Browser

INSERT

UPDATE

DELETE

SELECT

Submit

2

App server datain main memory

SessionObjectsSessionObjects

SessionObjects Email

Paypal charge

The Paradox

Most web applications are fundamentally simple;yet their development requires large amounts of code,both server-side* and client-side (JavaScript)

* Java, PHP, C#, Ruby etc.3

Where is the extraneous code expended?

Low-level code addressing frictionsof the web application architectures

pre-2005: Pure server-side programming

post-2005: Ajax

post-2007: browser went mobile

4

Pre-2005: Pure server-side programming: Code to plumb across layers; code to optimize data access

Browser

Application Server

Database

HTMLDOM

Action

INSERT INTO reviews …

Java

Page Computation

SELECT *FROM proposals…SELECT * FROM reviews …

Java

Code with JDBC/SQL calls to combine database and application-server data

Code to combine database and application server data into HTML

RequestParameters

Database Server

SessionObjectsSessionObjects

SessionObjects

5

HTMLDOM

JavaScriptComponentsData access optimization questions:

-A single query that gets both proposals & reviews of reviewer currently in session

- or a single “proposals” query followed by one “reviews” query for each proposal

- or a single “proposals” query with one “reviews” query that lists all relevant proposal id’s

2005: Enter Ajax:A desktop feel to cloud-based applications

• Asynchronous partial update of the page

• Fancy JavaScript/Ajax componentsmaps, calendars, tabbed windows

6

RequestParameters

Ajax: Mundane data coordination betweenpage state and server state (app server + database)

JavaScript

Action

INSERT INTO reviews …

Java

collect_args();response_handler = function(){ partial_renderer(); }send_request();Browser

Application Server

Database Server Database

Page Computation

SELECT *FROM proposals…SELECT * FROM reviews …

JavaDelta

Queries

Diffs

SessionObjects

HTMLDOM

JavaScriptComponents

SessionObjects

JavaScript/Java code reflects page data & request parameters to the server

JavaScript components have different programmatic interfaces

JavaScript/Java code tuned to incrementally update HTML DOM & JavaScript components

7

For each action, yet another incremental page computation code and partial renderer

BrowserApplication Server

Pure Server-side Model Ajax Model

Page Computation Page Computation 1

Page Computation 2

Page Computation 3

Partial Renderer 1

Partial Renderer 2

Partial Renderer 3

Partial update requires different logic and imperative programming for each user action

Action 1

Action 2

Action 3

INSERT …

DELETE …

UPDATE …

8

Action 1

Action 2

Action 3

INSERT …

DELETE …

UPDATE …

9

Mobile, iPad & HTML5

• form factor• Client data & devices; geolocation

• Slower & more unreliable connections– higher need for client autonomy & low latency

• HTML5back to web-based standardizationbrowser-based databasemultithreaded asynchronous processing abilities

HTML5: Data and computations at the browser.And yet another integration, coordination exercise

Action

Browser

Application Server

Database Server Database

Page Computation

SessionObjects

HTMLDOM


SessionObjects

10

HTML5storage

Action

Action

Page Computation

Action Page Computation

Action and page computation may have enough browser-side data to fully avoid server access

Low latency: Browser-side computation returns immediately available data instantly, while asynch requests to the serverinitiated for reads or writes

Goal of theFORWARD Web Application Programming Framework

• Remove architectural frictions & manual optimizations– Plumbing code addressing language heterogeneities– Data coordination, optimization code

• Novelty: SQL-based declarative programming1. “Unified application state” virtual database, comprising

entire application stateaccessible via SQL++ distributed queries

2. Application semantics: Action -> Page cycle3. Ajax page as a rendered SQL++ view that is automatically

and incrementally maintained at each roundActions and pages can use just SQL++, no Java or JavaScript

• Make easy things easy while difficult things possible11

FORWARD Interpreter (e.g., http://forward.ucsd.edu )

FORWARD Application: Set of declarative configurations, interpreted in a location-transparent way

FORWARD Application nsf_reviewingAction ConfigurationAction Configuration save_review

Page ConfigurationPage Configurationreview

Source Configurations & SQL++ Schema Definitions

12

Browser-side FORWARD JavaScript librariesHTMLDOM


Unified SQL++ Application State

Yannis

Please fix the image.

FORWARD Application:Operation

13


FORWARD Application nsf_reviewing



Action ConfigurationAction Configuration insert-review

Page ConfigurationPage Configurationreview-proposals



Actions Access Unified Application State via SQL++

14





Unified SQL++ Application StateAction

Outputs

Core PageRequest

ParametersDatabaseSessionObjects




Core page & request parameters: Abstraction of relevant part of browser state

SQL++ includes:NestingVariabilityIdentityOrdering

Yannis


Actions in PL/SQL dialect over the Unified Application Statewhich access the main memory also (request, session)

15

Java

// Insert submitted review into reviews table of the hosted databaseINSERT INTO hosted.reviews(proposalId, comment, rating)VALUES (request.pid, request.myComment, request.myRating)

Javaactions

Pages as Rendered SQL++ Views over the Unified Application State

16









Yannis


<fstmt:for query=" // proposals assigned to // currently logged-in reviewer SELECT p.id AS pid, p.title FROM hosted.proposals AS p WHERE EXISTS( SELECT * FROM hosted.assignments AS a WHERE a.proposal = p.id AND a.reviewer =

session.user"> <tr> ... <td>{pid}</td> ... </tr></fstmt:for>

Page Configuration Follows Popular Frameworks:HTML + FORWARD statements & expressions, using SQL++

Overall structure reminiscent of XQuery and SQL/XML

17

<fstmt:for query=" SELECT ..."><tr> ... <td> <funit:bar_chart> <bars> <fstmt:for query="SELECT …"> <bar> <id> {gid} </id> <value> {score} </value> </bar> </fstmt:for> </bars> </funit:bar_chart> </td> ...

Page Configuration: Ajax/JavaScript Components Simply by FORWARD Unit Tags

18

After each action, the FORWARD interpreter automatically & incrementally updatesHTML and JavaScript components

Core Page and Request Parameters:Mirror of named page data

<fstmt:for name="proposals" query=" SELECT p.id AS pid, p.title FROM hosted.proposals AS p WHERE ..."><tr>... <funit:editor name="myReview"> ... </funit:editor> ... <funit:select name="myRating“ type=“integer”> ... </funit:select> ... <funit:button onclick="save_review"/></tr></fstmt:for>

Core Page

proposals

Automatically inferred or explicitly declared

pid my_review rating

19

20

FORWARD Demo

21

FORWARDInterpreter

Program

Email, etc.Service

Browser State

System Architecture and Optimizations

Browser

Server

Page Configuration

Visual Units

Page Query in SQL++

HTMLDOM

JavaScript Components

INSERT, UPDATE, DELETEServices

UnifiedApplication State

DatabaseState

Session State

Page State

Argument Collectors1

Program Invoker2

Modification Log

Partial Renderers4

Incremental View Maintenance

3Incremental view maintenance of data using modification log

Incremental rendering of JavaScript components

Visual Page State = Page Query + Constants (visual template)

Core DataRequest Data

Visual Page State

22

Reduction of web programming problems into data management problems with rich prior work

• Incremental Ajax page maintenance

• Resolving data heterogeneities between the logic visual layers of the application

• Query optimization for distributed queries over small main memory sources and a persistent database

• Location transparent code development and low latency optimizations for mobile

23

Incremental Ajax maintenance

• Use incremental view maintenance as an efficient implementation of the page queries

• Data model features– Nesting (decorrelation)– Variability (decorrelation)– Ordering (ordering attributes)

• Deferred view maintenance• Large number of small view

instances• Deletions & updates as commands

that utilize provenance IDs

Partial renderers ofvisual units

HTML + Javascript State

Page Configuration

Visual Units

Page Query in SQL++

Incremental View Maintenance of

Page Query

24

Incremental Rendering of JavaScript Components

new bar_chart();...

Insert Bar Chart

BarChartUnit

bar_chart.destroy();...

Delete Bar Chart

bar_chart.getBar(id);bar_chart.replace(...);...

Replace Bar

Browser State

HTMLDOM


Render

53

barchart

bar

valueid Int

Tuple

Int

Diffs

Find most specific renderer for unit diff

Visual units are provided by framework, and have renderers that call into methods of JavaScript components.

25

Incremental Rendering of JavaScript Components

new bar_chart();...

Insert Bar Chart

BarChartUnit

bar_chart.destroy();...

Delete Bar Chart Browser State

HTMLDOM


Render

53

bar

valueid Int

Tuple

Int

Diffs

Simulate renderers when necessary

Resolving heterogeneities between logical and visual aspects of the page

26

Output of Page Query

Visual Page

Visually typedCore Page

Request

Report attribute types and logical structure

dictated by page query

Input attribute types dictated by input units

Typed core page

ConfigurationPage

(logical abstraction of page)

q2cc2v

v2c

c2c

c2t

t2r

Atomic typesdictated by what developer

wants; validation errors

Types dictated by input units

27

Query optimization for distributed queries over small main memory sources and a persistent database

Unified SQL++ Application StateAction (PL/SQL)

Variables

Core PageRequest

ParametersDatabaseSessionObjects

Action (PL/SQL)

Distributed Queries

Page Configuration

Page query

Replacing the manual specification of data access optimizations that one does In Ruby-on-Rails, Hibernate

Related Industrial Work on Removing Cross-Layer Frictions

Browser

Database server

Application server

SQL Data

Browser Mirror Java (C#) ObjectsJava (C#)

Database Mirror Java (C#) ObjectsApp Server

Objects

?

28

HTMLDOM


ASP.NET,GWT

Hibernate,Entity Framework

29

Data management research community efforts: A progression towards ever more powerful declarative web application specs

• Strudel– HTML pages as rendered views

• WebML– From schemas to forms and reports, easily

• Relational transducer– Hinted that the business logic of many web applications

is captured within SQL• HILDA– Unit-based pages & logic– Client-server computation

Future work

• More on algorithms that emulate what “a good developer would do”

• Client side computation– enabled by declarativeness– location transparency– deliver low latency, disconnected operation

• Optimizations for updating a myriad views– leveraging pub-sub works

• Visual Do-It-Yourself development30

Documents

Yupeng Fu Kian Win Ong Kevin Zhao Yannis Papakonstantinou Vicky Papavasiliou Michalis Petropoulos The FORWARD SQL-Based Declarative Framework for Ajax