TURKU UNIVERSITY OF APPLIED SCIENCES

Business Information Systems (MBA)

Master's Thesis Presentation

May 2014

By

Youssef Hammouche

� The rapid changes of information technology, present a new challenge and awareness of the necessity to adopt new tools and processes to improve the quality of education system.

� In confronting this challenge it became obligatory to know the complexity of the education system from student’s perspective and related problems that should be addressed.

� every education system must have an independent strategy of changes to determine the affecting factors of failure or success in education system.

� The usability testing represent one of the success factors to be implemented into any LMS to evaluate the examined interfaces by testing it on users.

� It became an essential component of LMS development process similar to software testing and system design.

� LMS might be very difficult to use by students if usability problems are not identified in early stage

� Many LMS projects have failed during last few years to stay among the top existing systems, reasons could be numerous but lack of usability usability usability usability testing testing testing testing is considered as one of the most common reasons.

� especially that students interactions with system is highly impacted with various barriers: lack of available support by teachers, poorly motivated to understand system's features, students are unsure in the use of ICT.

� Therefore it is mandatory to conduct usability testing during the system development cycle.

CONTENTCONTENTCONTENTCONTENT

LIST LIST LIST LIST OF ABBREVIATIONS (OR) SYMBOLSOF ABBREVIATIONS (OR) SYMBOLSOF ABBREVIATIONS (OR) SYMBOLSOF ABBREVIATIONS (OR) SYMBOLS

1. INTRODUCTION1. INTRODUCTION1. INTRODUCTION1. INTRODUCTION1.1. Problem Statement and Research Objectives

1.2. Research Methodology

1.3. Optima LMS

2222. LEARNING MANAGEMENT SYSTEMS . LEARNING MANAGEMENT SYSTEMS . LEARNING MANAGEMENT SYSTEMS . LEARNING MANAGEMENT SYSTEMS LMSsLMSsLMSsLMSs2.1. Open Source LMS

2.2. CLOUD LMS

2.3. Steps in Selecting a LMS

2.4. Key Factors of Successful LMS

3. THE USABILITY TESTING PLAN OF OPTIMA 3. THE USABILITY TESTING PLAN OF OPTIMA 3. THE USABILITY TESTING PLAN OF OPTIMA 3. THE USABILITY TESTING PLAN OF OPTIMA LMSLMSLMSLMS3.1. The Concept of Usability Testing3.1. The Concept of Usability Testing3.1. The Concept of Usability Testing3.1. The Concept of Usability Testing

3.1.1. What is Usability?3.1.2. What is Usability Testing 3.1.3. The Purpose of Usability Testing3.1.4. What to Test?

3.2. Usability Test Plan for Optima LMS3.2. Usability Test Plan for Optima LMS3.2. Usability Test Plan for Optima LMS3.2. Usability Test Plan for Optima LMS3.2.1. Problem Statement and Test Objectives

3.2.2. User Profile 3.2.3. Methodology

3.2.3.1. Participants 3.2.3.2. Length of Sessions3.2.3.3. Quantitative & Qualitative Data 3.2.4. Scenarios and Tasks

3.2.5. Test environment and Equipment

4. EMPIRICAL USABILITY TESTING IN OPTIMA 4. EMPIRICAL USABILITY TESTING IN OPTIMA 4. EMPIRICAL USABILITY TESTING IN OPTIMA 4. EMPIRICAL USABILITY TESTING IN OPTIMA LMSLMSLMSLMS4.1. Data Collection4.1. Data Collection4.1. Data Collection4.1. Data Collection

4.2. Test Results and Analysis4.2. Test Results and Analysis4.2. Test Results and Analysis4.2. Test Results and Analysis

4.2.1. Top Negative Finding

4.2.1.1. Website Layout and Login Process 4.2.1.2. System Features and Editing Options

4.2.2. Top Positive Finding4.2.3. Findings Log from the Observer4.2.3.1. Task Completion Rate4.2.3.2. Time on Task4.2.3.3. Problems and Helps required

4.2.4. Presenting Results Based on Post-task and Post-test

Questionnaires

5. USABILITY RECOMMENDATIONS FOR OPTIMA 5. USABILITY RECOMMENDATIONS FOR OPTIMA 5. USABILITY RECOMMENDATIONS FOR OPTIMA 5. USABILITY RECOMMENDATIONS FOR OPTIMA LMS DEVELOPERSLMS DEVELOPERSLMS DEVELOPERSLMS DEVELOPERS

6. DISCUSSION 6. DISCUSSION 6. DISCUSSION 6. DISCUSSION

REFERENCESREFERENCESREFERENCESREFERENCES

APPENDICESAPPENDICESAPPENDICESAPPENDICESAppendix 1. Pre-test Questionnaire. Appendix 2. Post-task Questionnaire. Appendix 3. Post-test Questionnaire. Appendix 4. Usability Testing Video Permission Form

� Discendum is the optima system developer; they consider it as a high-quality system with an easy tool to be used by users to build a perfect online environment.

� Optima service aimed to become one of the best existing platforms for eLearning courses, information management, networking, and supporting learning on the job.

� Actually, more than 200 educational institutions and companies have implemented Optima to support their education and projects.

� Actually, more than 200 educational institutions and companies have implemented Optima to support their education and projects.

� Based on the cited quality of Optima and growing interest of customers, the system is not yet classified among the top learning management systems.

� Only a few academic literatures have been written under the name of Optima.

� often those documents belong to libraries of universities which kept a right for only their users to access into it.

� This limitation makes the case more complicated and stands as a barrier of sharing Optima researches to external readers of Finnish universities

� Additionally, lack of usability testing researches has been conducted on the subject, which makes my research a bit difficult to collect necessary information and to reach my study goals.

� On the other hand, similar learning management system has taken a wide attention across the world, despite that Optima has improved all interfaces by merging versatile tools, flexible online environment, and high security access "Moodle serve over 70 million users“

� Do Optima system provide enough functionality to users to help them achieve their study goals?

� Is the slow success of Optima due to the usability of system which would reflect the users’ satisfaction and dissatisfaction to the system?

� Is Optima system afford enough information and help to users with less effort and less clicks of navigation?

� Is Optima Easy –to- use?

� In order to get responses to the previous questions, a usability testing should be taken into consideration to find out how real users use the system in real life, and which kind of difficulties and problems users might encounter during their daily tasks?

� This thesis concentrates onto students interface only, to design a test plan based on the literature of Carol M. Barnum (Usability Testing Essentials Ready, Set…Test), and Joseph S. Dumas (A Practical Guide to Usability Testing), and other resources as well.

Constructive research,Constructive research,Constructive research,Constructive research, it is considered as one of the options currently available for a case researcher, and it has been widely developed in the field of business administration and IS.

It has gained significant results by implementing new construction to solve problems faced in real world, and this involves the analysis of the users and performance of the system design in order to understand, explain, and improve the designed systems.

� This research method matched much on my study research, where I am supposed to understand Optima system environment, and try to find usability problems encountered by users.

� The new construction issue in this research is to design a test plan based on theoreticaltheoreticaltheoreticaltheoreticalpart which frequently includes a test objectives, methods, and scenarios.

� The practicalpracticalpracticalpractical part is reserved to apply the test plan and to collect data based on the selected methods, for a goal to be analysed, interpreted, and presented in form of recommendations to project stakeholders.

� Optima is a web-based learning platform that supports learning environment and delivers extensive training programs, managing sales trainings, and offers opportunities to utilize different learning models and pedagogical ideas.

� Optima is a Finnish company developed by Discendum together with its customers, company was established in 2001 for a goal to create and implement online learning managements and related training consultation services.

� Company has delivered over 200 online learning solutions to various sorts of customers such as institution of higher education, companies, and other organizations.

� The Company turnover of the fiscal year 2011 was approximately 1, 6 million euros; currently company employ 18 people in cities of Helsinki, Tampere, Rauma, and Oulu. Optima headquarters are located in Oulu.

What is UsabilityWhat is UsabilityWhat is UsabilityWhat is Usability????

ISO 9241ISO 9241ISO 9241ISO 9241: "The extent to which a product can "The extent to which a product can "The extent to which a product can "The extent to which a product can be used by specified users to achieve specified be used by specified users to achieve specified be used by specified users to achieve specified be used by specified users to achieve specified goals with effectiveness, efficiency, and goals with effectiveness, efficiency, and goals with effectiveness, efficiency, and goals with effectiveness, efficiency, and satisfaction in a specified context of use.” satisfaction in a specified context of use.” satisfaction in a specified context of use.” satisfaction in a specified context of use.” still wider:

Specified Specified Specified Specified users:users:users:users: it concerns the persons to whom the product is designed for.

Specified Specified Specified Specified goals:goals:goals:goals: Are the shared goals between the users and product.

� A specified context of use:A specified context of use:A specified context of use:A specified context of use: Is the environment where the product should work by the users.

� EX:“website, tools, mobile devices, software application, or anything designed for people.

� Whitney Whitney Whitney Whitney QuesenberyQuesenberyQuesenberyQuesenbery'''' has given an easy definition to remember, called 5Es:

� Effective:Effective:Effective:Effective:" How the user could complete the task and reach the goals?"

� EfficientEfficientEfficientEfficient:::: "How fast the task can be done?"

� Engaging:Engaging:Engaging:Engaging: "Is the interaction between users and designed interface, and how satisfied are they about it?“

� Error tolerantError tolerantError tolerantError tolerant: "How many errors committed by users? How easily do users can recover from it?"

� Easy Easy Easy Easy to learn:to learn:to learn:to learn: "How easy could the user accomplish the basic tasks for the first time they encounter a new product or service?" (Whitney Quesenbery. February 2004.)

� What is Usability Testing? What is Usability Testing? What is Usability Testing? What is Usability Testing?

� Represents all activities that focuses on observing users testing a product or service in certain working conditions,

� the observer is supposed to watch, listen and take notes while the participants are trying to complete tasks

� for a goal to identify usability problems, collect qualitative and quantitative data, and to determine participants satisfaction and dissatisfaction with the product or service. Generally the usability testing has two forms:

� Formative testing:Formative testing:Formative testing:Formative testing: while the product is still in the development process, for a goal to diagnose the problems and try to fix them before represented to the consumers.

� Summative Summative Summative Summative testing:testing:testing:testing: commonly when the product is finished, with a goal to validate if the product meets the consumers requirements. (www.usability.gov).

The Purpose of Usability The Purpose of Usability The Purpose of Usability The Purpose of Usability TestingTestingTestingTesting

� UT became a necessary condition for a Website to survive.

� Web visitors are not trying seriously to spend much effort to understand what the Web offers for them as information or service, they leave the website as soon as they find difficulties to use it or to figure out what the web interface is trying to present for them.

� But the case might be totally different if you are forced to use a website or system which you are not familiar with and it has not been well tested.

� therefore the usability testing should be adopted for any system to support users and to identify problems based on their experiences.

� Reasons to test optima LMS:

� the number of conducted UT into Moodle CMS is incomparable with any other LMS

� although Optima has almost the same features as Moodle

� it has been used during a long period by our school educators and students

� it has gained a wide success by the users due to the free of charge of using the system by the open universities, university staff, and for the academic research purposes as well.

� Other reason is to find out how students could easily use Optima platform for e-learning courses, support learning tools, information management, and as a space for interaction and collaboration between students and supervisors.

What to Test?What to Test?What to Test?What to Test?

� Due to many reasons I have selected student as an interface to be tested during this usability testing to understand their experiences with the system and to determine whether the usability of it matches their expectations and support their goals.

� Students are confronting more usability problems and difficulties compared to other users.

� Easy to include students in testing process than supervisors.

� number of students that are using Optima system are extremely higher compared to other interface users.

� The common statement in usability testing is to inform the participants that "We are "We are "We are "We are testing the product, not you". testing the product, not you". testing the product, not you". testing the product, not you".

� Will document all the essential test planning steps and methods that will be used by me, to verify and ensure that Optima system meets the requirement of testing objectives.

� will be presented as a formal report to record all the important activities related to the tested interface,

� to figure out how the system would be tested, and to know the persons who will be recruited for this testing. The final report will be as a part of my final thesis work

� The test plan will include the sections below:

Title page, Table of Contents,

List of Illustration, Abbreviation & Figures

Executive Summary, Problem Statement and Test Objectives, Methodology, User Profiles

Participant Incentive, Screeners, Scenarios & Tasks, Evaluation Methods, Test environment & Equipment, Presentation of Findings,

Pre-test Questionnaire, Post-task & Post-test Questionnaire, Recommendations, & Appendices

� is to discover whether Optima system is easy is easy is easy is easy to be usedto be usedto be usedto be used and learned from students interface,

� to find out if they could perform proposed tasks within a prepreprepre----determined time determined time determined time determined time frame and how well they could understand it,

� to ensures if the system design works as intended and meets users expectations.

� To find serious usability problems which are related to system navigation, work functionality, message errors and system features.

� Finally is to explore if Optima LMS is effective effective effective effective and efficientand efficientand efficientand efficient as it should be, by knowing how users can find information they need to complete tasks without help.

� Of course I didn’t test every possible task that user could do with the system due to many constraints such as nature of research, limitation of human and monetary resources.

� For this matter we created Scenarios and tasks to help us determine what we want to learn by observing six participants whom matched the testing profile to achieve tasks in the Work Informatics User Experience WIUX Work Informatics User Experience WIUX Work Informatics User Experience WIUX Work Informatics User Experience WIUX Laboratory in Turku ICT buildingLaboratory in Turku ICT buildingLaboratory in Turku ICT buildingLaboratory in Turku ICT building. . . .

� Four kind of scenarios in total were planned for this testing, each one contained three tasks to be accomplished in a predetermined time frame,

� each time participant was done with scenario he or she was given the post-task questionnaire and then the next scenario,

� Tasks for testing Optima LMS are in this link https://optima.turkuamk.fihttps://optima.turkuamk.fihttps://optima.turkuamk.fihttps://optima.turkuamk.fi and it included:

� Website layout: How easily can users understand the website layout?

� Optima Login process

� Access to the desired degree program.

� Finding Information & help gained from the Help icon.

� Navigation, Satisfaction & Dissatisfaction

� Advanced Options: number of object, Cloud services…etc.