1

USING EXPERT SYSTEMS USING EXPERT SYSTEMS TECHNOLOGYTECHNOLOGY

FOR STUDENT EVALUATIONFOR STUDENT EVALUATIONIN A WEB BASED EDUCATIONAL IN A WEB BASED EDUCATIONAL

SYSTEMSYSTEM

Ioannis Hatzilygeroudis, Panagiotis Chountis, Christos Giannoulis and Constantinos Koutsojannis

University of Patras, Dept of Computer Engin. & Informatics

26500 Patras, Hellas (Greece)

2

OutlineOutline

1. Introduction 2. System Architecture 3. Domain Knowledge 4. Learning Process 5. Domain Knowledge Management6. Student Evaluation 7. Helping the Tutor 8. Implementation issues 9. Conclusion

3

1. Introduction1. Introduction

Web-based intelligent educational systems (WBIESs) use Artificial Intelligence (AI) Techniques in order to adapt mainly to student needs for self-study.

An important function of such systems is student evaluation, which provides results for adapting to both the students and the tutor. There are many techniques used for student evaluation.

However, a few of them use an expert system approach

4

Introduction (2)Introduction (2)

To help the students and the tutors in our Department, we constructed an intelligent e-learning system to assist learning and teaching in the context of the course of Artificial Intelligence (course 451).

The system provides means to the tutors for constructing questions and tests in a structured way. Also, with the help of an expert system, evaluates the students and provide useful statistics to the tutor.

5

2. System Architecture2. System Architecture

The system consists of five units:

the Identification Unit (IU), the Student Model (SM), Stereotypes for initial

student profile the Tutoring Unit (TU), the Evaluation Unit (EU)

and the Virtual Agent (VA)

IdentificationUnit (IU)

Student Model (SM)

TutoringUnit (TU)

EvaluationUnit (EU)

Stereotypes ExpertSystem (ES)

VirtualAgent (VA)

6

System Architecture(2)System Architecture(2)•SM contains all the information about students, like their preferences, interests, knowledge level, etc

•TU is responsible for the teaching process. A student can select a learning goal from the domain knowledge tree

•The main goal of EU is to evaluate student's progress due to his/her interaction with the system. This evaluation is achieved through testing.

•VA has as main goal to motivate the students and improve system's effectiveness. In our case, VA is an avatar (called Merlin) that provides information to the students in a natural and recreating way. To achieve this, VA gets information from the SM and communicates with the student in the same way that a human tutor would do.

7

User InterfaceUser Interface

Navigation area

Content area

Start – Previous - next

Info-bar

www.mmlab.ceid.upatras.gr/aigroup/undergrad/

8

3. Domain Knowledge3. Domain KnowledgeThe domain knowledge tree, as far as the

subsections level, is displayed in the navigation area of the user interface (at the left side of the screen).

From that tree the student can choose a learning goal (subsection).

Each subsection corresponds to a learning page, which is a php page. That is, only subsections correspond to displayable material.

The learning page of the selected subsection is currently presented in the content area (which resides at the centre and the right part of the screen).

Each learning page deals with a number of concepts.

Each concept is linked to the corresponding concept page. Concept pages constitute the real teaching material.

Subject

Chapter 1 Chapter N

Section 1

…

…

Subsection 1

Section N

Subsection N

Concept 1 Concept N…

…

9

Domain Knowledge(2)Domain Knowledge(2)

The teaching material, apart from concept pages, however, includes all the available questions, which are stored in a database and are used for the creation of the tests.

Each learning page is associated with a test. Each test consists of a number of multiple choice

type questions that examine the concepts of the associated learning page.

10

4. Learning Process4. Learning Process

The learning method (implicitly followed) is based on the traditional theory-examples-exercises paradigm (although the user can follow his own method).

That is, for each concept, the theory is first presented. Then, some examples are given. Finally, the student is called to make some exercises.

The student is also not forced to follow the system’s way of teaching, but he can make his own choices for studying. For example, one can jump to complex concepts without having had a look at simpler ones. In many concept pages there are links to other concepts that are prerequisite to the concept of the page.

11

5. Domain Knowledge 5. Domain Knowledge ManagementManagement

The system offers to the tutor the capability of domain knowledge management. The tutor can deal with management of the domain tree and the teaching material (learning pages, concept pages, questions and tests).

The most important part of teaching material management is the management of the questions. A question is considered as a structured object that consists of the following parts:

the name of the question the body of the question, the concept it concerns (as well as its corresponding path in the

domain tree), its difficulty level (one of easy, medium and difficult), its possible answers (1 to 4) each followed by a mark and an

explanation and a hint (to help the student to answer correctly).

12

Screen for Question CreationScreen for Question Creation

(Chapter)(Section)(Subsection)(Concept)(Difficulty level)

Answer1-4 Mark1-4 Explanation1-4

13

6. Student Evaluation6. Student Evaluation

A student can be evaluated at two levels:(a) the concept-level and (b) the topic-level. The concept-level evaluation deals with the level of

understanding of the concepts of a learning page test, whereas the topic-level evaluation deals with the level of understanding of the topic of a learning page, i.e. the test as a whole.

Student Evaluation implemented via the expert system (ES).

14

The structure of the ESThe structure of the ES The knowledge level of a student, as

far as both a concept and a topic are concerned, is classified in one of the following five categories:

(a) excellent (86-100), (b) very good (71-85), (c) good (51-70), (d) average (31-50) and (e) low (0-30). Within the parentheses are the

corresponding ranges of the marks to be achieved

The rules are implemented via the expert system (ES).

ES is a rule-based expert system implemented in Jess, which is an expert system shell.

FactBase

RuleBase

JESSInference

Engine

KB

15

Rules for answer markingRules for answer marking Difficulty Level No of tries Use of Hint Answer Status Mark

easy 1 0 Correct 85

easy 1 1 Correct 70

easy 1 0 Wrong 30

easy 1 1 Wrong 0

easy >2 0

medium 1 0 Correct 92

medium 1 1 Correct 77

medium 1 0 Wrong 40

medium 1 1 Wrong 15

medium 2 1 Wrong 15

medium >2 15

difficult 1 0 Correct 100

difficult 1 1 Correct 85

difficult 1 0 Wrong 50

difficult 1 1 Wrong 30

difficult 2 0 Correct 70

difficult 2 1 Correct 50

difficult 2 0 Wrong 50

16

Question weight distributionQuestion weight distribution Easy(E)

Medium(M)

Difficult(D)

Question (s)weight(s) (/100)

1 1 1 20 (E), 30 (M), 50 (D)

1 1 0 40 (E), 60 (M)

1 0 1 30 (E), 70 (D)

0 1 1 40 (M), 60 (D)

1 0 0 70 (E)

0 1 0 80 (M)

0 0 1 100 (D)

17

7. Helping the tutor7. Helping the tutor

The system can present some statistics about students, which are the following:

The percentage of the correct answers to questions per difficulty level per concept per student.

The percentage of the correct answers to questions per difficulty level per test per student.

The percentage of the correct answers to questions per difficulty level per concept for all students.

The percentage of the correct answers to questions per difficulty level per test for all students.

18

8. Implementation issues8. Implementation issues

The system is web-based, so it works only through the WWW and cannot be downloaded and function on a student's computer. It is implemented in .php

The VA (avatar), one of the interesting parts of the system, is implemented in VBScript. To use the avatar, we first have to call its object from the Windows Controller. Then we have to connect to that object and load it. We are then able to call its functions, show it on the screen and make it interact with the user.

The ES is implemented in Jess, an ES cell implemented in Java. After the student has finished a test, a fact for each individual question of the test is saved in a file. Then the expert system is called to evaluate the student. The expert system, after having taken the facts from the file, returns the results in another file.

19

9. Conclusions9. Conclusions We concentrate on the knowledge management and

student evaluation aspects of the system. Knowledge management mainly refers to test questions construction and management.

Student evaluation refers to the evaluation of the knowledge level of a student with regards to taught concepts. A rule-based expert system helps in student evaluation. A number of statistics provided by the system give valuable information to the tutor.

The system, as it is now, adapts to the student needs, but not to the tutor ones. It is only adaptable as far as the tutor is concerned. So, a further effort is to make it adaptive to the tutor needs as well.

20

Thank youThank you

for your attention !

21

USING EXPERT SYSTEMS TECHNOLOGYUSING EXPERT SYSTEMS TECHNOLOGYFOR STUDENT EVALUATIONFOR STUDENT EVALUATION

IN A WEB BASED EDUCATIONAL SYSTEMIN A WEB BASED EDUCATIONAL SYSTEM

Ioannis Hatzilygeroudis, Panagiotis Chountis, Christos Giannoulis and Constantinos Koutsojannis

University of Patras, Dept of Computer Engin. & Informatics26500 Patras, Hellas (Greece)