Sakai: The proof of the pudding is in the eating

Dr. Stanley J. PortierSakai Conference, 12-14 June 2007, Amsterdam

University of Twente Campus

Profile of the university

• Founded in 1961• Research University• Entrepreneurial Research University

– >600 spin-off companies

• First (and only) Dutch campus university• Focus on technological developments and

their management in the knowledge society

Education• 5 Faculties• 23 Bachelor programmes• 31 Master programmes

• Bachelor & Master students: 7357• PhD’s: 703• International students: 344• International PhD’s: 285

History: ELO Advice• Since 2000: TeleTOP is our VLE (campus wide use)

– Problems:• Integration, standards, based on ‘traditional’ instructional design principles, isolation in Dutch Higher

Education

• Blackboard or Moodle was not an option– All-in-one; no components, not very much service oriented, no/limited framework

– But: reasonale support of standards

• Sakai as our future VLE?– For and by education (community)

– Architecture: stable, robust, scalable, etc.

– Note: Open source was not the main reason

– Large number of international partners (>100)

– Features comparable to commercial products

• So: let’s do some pilots to see whether Sakai ‘meets our demands’

• www.utwente.nl/elo

Campus Blend Using Sakai (CBUS) – pilot phase

• Research and evaluation of Sakai– Criteria

• Sakai Community: How strong? Future expectations? What’s happening in- and outside the Netherlands?

• Technical trials: (open) standards, (web)services for integration purposes

• Pilot projects in education

• Financial aspects

• August 2006 – June 2007• Final deliverables due on June 26th

Sakai community• Facts and figures

– Community established in the US– Emphasis on English-speaking countries (as well

as their educational system and methods)– 16 production sites in Europe, 46 in the US– NL: UvA, UT, Hotelschool Den Haag, >10 ROC’s,

LOI, Elsevier Opleidingen, Stoas, Portfolio4U, Omix, A New Spring (join Wytze Koopal’s presentation tomorrow)

– Sakai SIG at SURFfoundation– >100 developers worldwide– New releases twice a year

Sakai community (2)• Conclusions

– World-wide community has (too much) focus on software development

– Shift towards ‘usability’ and ‘educational / pedagogy’ issues (slowly)

– Community Source model is ‘the best of both worlds’: open source and commercial software

– No formal NL development-community, there is a Sakai NL user-community (SIG)

Technical analysis• Integration:

– Open standards and specification (what needs to be exchanged)

– Service Oriented Architecture (how to exchange)

– Across applications (VLE-VIST-SIS)– Across domains (education / organisation)– Accross institutes (e.g., 3TU)– By means of SOA based on web services

Services needed for Integration

Demonstrator• Reading from VIST (Course information

system)• Search for a course in VIST and display result

in demo application• Acquire course title and description from VIST• Edit course description (if required)• Create Sakai site and select relevant tools• The demonstrator works!

Modelling in Archimate

Embedded in a 4-layer architecture

Technical analysis• Conclusions:

– Support of e-Learning standards is on it’s way, but still average (SCORM, IMS QTI , IMS TI etc)

– Support of other standards (XML, Webdav, OKI OSID’s etc): good

– Sakai is SOA by design. Excellent architecture, internal and external service orientation

– Sakai code is enterprise scalable Java code– New knowledge and expertise required (BPEL,

WSDL, XML, XSD, SOAP etc..)

Pilot projects within our facultiesFac Course Ba/Ma Period Student

pop (N)Student sample (n)

BS Complex systems for Learning and Knowledge Management

Ma Q1 12 7

CTE Governing Product Development (GPD)

Ma Q1 32 10

CTE Project K Ba2 Q2 101 21

MG European Social Integration

Ba1 Q3 57 27

Total 202 65

Pilot projects

• Different faculties (3/5)• Bachelor and Master courses• Different instructional paradigms:

– Instructor- and student driven– Different tool selection in sites

• Different level of ICT experience among teachers

Pilot projects (2)• Procedure:

– Intake with instructors– Advice on configuration of Sakai site– Quick start– Introducing documentation and help information– Instructor works more or less indepedently

• Limited amount of support required

– After each pilot: web survey (students) and interview (instructor)

Pilot projects (3)• Functional analysis CBUS team

– ‘Longlist’– Comparison with TeleTOP– Identifying improvement issues– Setting priorities

• Other experiences– UT Information meetings (3x)– Sakai Playground (> 100 accounts)– External relations

• TSM Business School (EMBA 18)• ITC

Results – students / teachers (1)• Educational

– Can support different educational / instructional design paragdigms– Different site types (course- & projectsites)

• Performance– Robust, only few error messages– Slow during and after large uploads

• Improved after patch in 2.2.0, later 2.3.1

• Positive:– Announcements (notifications)– Assignments– Dropbox (submitting and feedback -> student <-> teacher)– E-mail archive (list & archive)– Resources– Wiki (concept of a collaborative environment)

Results – students / teachers (2)• Needs improvement:

– Assignments (submitting group assignments)– Access levels (more roles required)– Resources (not suitable as a group environment)– Schedule (integrate with Outlook calendar)– Wiki (editor and layout)

• User interface and usability– Non-intuitive and sometimes inefficient– Needs improvement before implementation

• Compared with TeleTOP– TT user ratings are more positive– Sakai is recognized as a potential follow up

‘Longlist’ CBUS team (+)• Positive (t.o.v. TeleTOP):

– Site management instructors– Projectsites– Online Help information: +– Flexible access rights for tools– Different types of notifications (announcements, resources, e-mail archive, syllabus) at site

level and aggregrated on Home– Separate assignments tool (n/a in TT)– Downloading all studentwork in one zip file– Easy copy, paste and moving of resources– Easy reusability from other sites– Multiple upload using WebDAV– E-mail archive– Dropbox (student can submit, teacher provided feedback)– News / RSS feed– Gradebook– Wiki– Message center

‘Longlist’ CBUS team (-)Priority 1:• Integration:

– VIST– Outlook– Ephorus (plagiarism)– SIS

• Course management & catalogue• Tasks and assignments• Participant overview per site (esp. for assigning participants to groups)• International issues:

– Course codes– Time and date format– First day of a week– Organisation of academic year– Grading

Listed improvements are considered ‘must haves’

Pilot projects• Conclusions:

– The pilot projects and our own analysis show that Sakai 2.3.1 can be a useful VLE for the University of Twente. It provides additional functionality compared to TeleTOP and may support different instructional design paradigms (as addressed in our ELO Advice report)

– However, the analyses also show that Sakai does not yet meet the functional requirements to be implemented as a campus wide production site. The improvement issues that were identified must be solved before implementation.

€ : Total cost of ownership• Annual costs Sakai

– Hardware: k€ 50-60 (3 yr use) [estimate, incl. personnel]– No license fee, but k$ 10 Sakai membership fee a year– 1 fte software-development: k€ 85– 0,5 fte requirements/community: k€ 40– 0,5 fte educational (technology) support: k€ 40 – Work within faculties: PM– Total: k€ 235

• Compare TeleTOP– Annual costs ITBE: k€ 250-300– Work within faculties: PM

Follow up• Deliverables will be final by June 26th• Board of directors have to make a decision• Pilot-site remains up and running until

december 2007• Sakai Core team

– Community development– Technical developments (e.g., upgrading to 2.4.0)– Support of other, potential pilots

• Publish final reports

Conclusions combined

Sakai Community Technical perspective

Functional analysis / Educational use

Financial

Thanks for listening!

• Questions & Comments?

• Links:– http://www.sakai-pilot.utwente.nl/sakaiwiki/– http://www.utwente.nl/elo