FlowTalk

Harriet Truscott,

Antranig Basman,

CARET, University of Cambridge

FlowTalk – Use CasesWhy did we build it?

Many varied use cases for extremely flexible discussions software:

• Student exercises with multiple rounds of feedback, different messages visible at different stages. Learning designer would like highest flexibility in designing workflow of teaching exercise.

• Discussions for “outreach” – schoolchildren where highly flexible moderation policies necessary, including “pre-moderation”. Should also be possible via email.

• Discussions amongst distributed research groups, self-organising into small “special interest groups”. Should becapable of functioning as a pure email list as well as web board.

FlowTalk - Technical goals

• Learning environments move fast – should be capable of integrating as a “first-class” tool with everything out therenow, or likely to be in the future.

Integration, Integration, Integration.

(Right now with Sakai, Stanford’s Coursework – future with LAMS, CopperCore, ??)

Issues: SkinningUser IdiomInstallationAUTH

Skinning

• Dealt with by RSF• In fact, historically, the reason RSF was

created• Today, pure HTML templates are

swappable dynamically• May even support direct upload of user’s

own templates into the FlowTalk hierarchy• RSF presented in other talks, see wiki at http://www2.caret.cam.ac.uk/rsfwiki

Separate or In-JVM?• Pros:

– Centralised deployment of ONE FlowTalk per site, with resulting smaller admin burden.

– Increased stability – during host crash/restart, services remain available (email list &c)

– Sakai footprint is already big enough! Tools in their own space is a Good Thing.

• Cons:

– Development is harder – more emphasis on serialisation/marshalling

– Difficult access to host services (probably a good thing for an integration platform)

– Performance – TCP and dispatch overheads per cycle.

Separate JVM

Important to economise effort – having created ONE tool capable of remote interoperation with multiple VLEs/portals, try to reuse this effort by packaging as libraries:

We decided to go with separate JVM model. Of course,very much easier to rebundle a remote tool as a local oneif required, reverse is much harder!

CaretAuth

FlowTalkClientPackage

Abstracts user + groups model, together withhierarchical permissions structure and resolution.

Operates user synchronisation protocol, request dispatchand consumer rendering request requirements.

FlowTalk architecture

Request dispatch cycleSakai

ToolSinkTunnelServlet

DirectWebServiceFilterRequestDispatch

HTML/HTTP requestFlowTalk

RequestDispatch

InformationServlet

XML/HTTP request

RemoteSyncServerUserManager

)SiteService/RealmService

( SakaiUIP

(copy bytes)ViewRender

*

(Serialized ConsumerRequestInfo)

* = At most once per request, and preferably never!

Notes on Dispatch Cycle

• Must avoid “chatty” protocol, at most one “backchat” per cycle, preferably none.

• The “general request” is NOT slowed down by backchat. Consumer info sent with request includes staleness timestamps, and client user structure is cached on FlowTalk side.

• More problematic answering incoming email request on FlowTalk side – must have more slack “staleness” guarantees (e.g. 10 minutes lag)

• Ideal case: ALL page rendering is done at producer, Sakai tool can start copying bytes to client with no latency. In fact achieve this with IKAT (see later)

Integration – Users/Resources

• “CaretAuth” library abstracts permissions structure. Has successfully mirrored both CourseWork and Sakai structures, so will probably generalise again!

• Permissions resolution is done on FlowTalk side with “equivalent effect” to Sakai resolution, no communications required.

• FlowTalk maintains “global” resource tree, with one node per consumer. User information is scoped to nested resource nodes.

• FlowTalk Permissions System

FlowTalk Global Resource HierarchyGlobal resource root (per FlowTalk installation)

Directly servedresources

CourseworkClient 1 root

SakaiClient 1 root Sakai

Client 2 root

Sakai Sites

Sakai Pages/Placements(=“Forums”)

Role/Realm info(as Groups, PermissionLists)

Per-consumerinfo (URLs, IDs)

Consumer per-resource Permissions (none for Sakai)

CourseWork“Nexus”

CourseWorkSection

Forums/threads

Forums/threads

FlowTalk Permissions System• Is probably on the verge of become sentient and taking over

the world• Permissions are resolved on

– User/Group– Permission name (lock)– Resource type (forum/message)– Flow state (which have hierarchical relation!)

• Permission resolution proceeds recursively up the resource hierarchy

• Different user/group status at each level of the hierarchy! – Firstly because UserManager is scoped to the client– Secondly because indirect groupings (e.g. Sakai Site-wide Role) is mapped as a

FlowTalk scoped group membership

• Incredible flexibility, but needs to be presented very carefully to avoid complete bafflement (not least of us)

Notes on Resource Hierarchy/Events

• FlowTalk stores resources of arbitary (MIME) types – “resource” is (generally) uninterpreted bytestream.

• General resource metadata includes resource “state” forming basis of event-driven architecture leading to workflow.

• Each change in event state fires off a (currently synchronous) event which propagates up the resource hierarchy being handled by listeners.

• Event handlers may induce further changes in resource state, firing recursive notification.

• An “event handler” is actually just a Spring bean, referred to by name.

Example Listeners (Workflows)• Default Workflow (simply moves message from

“init” to “active” status)• Mail Notification (for all users with

EMAIL_NOTIFY “permission”, sends out email for a message entering “active” status

• Moderated Workflow (instead of moving from “init” to “active”, moves to “moderating” – which is a kind of “inactive” status)

• PostMakesVisible Workflow (for any user making a post, adds them to the group “flowtalk:readers” within the parent forum)

FlowTalk Load Testing Results

• Testing uncovered a variety of issues with Session leaks, and memory leaks in underlying libraries (all fixed now!)

• As a result of its reliance on RSF, FlowTalk renders even complex or bulk views very quickly

• Despite (or perhaps because of) using the filesystem, FlowTalk shows extremely high concurrency and low latency.