LTeL - Language Technology for eLearning -

Paola Monachesi, Lothar Lemnitzer, Kiril Simov, Alex Killing, Diane Evans,

Cristina Vertan

LT4eL - Language Technology for eLearning -

1-• EU-IST-FP6 Project 2005 - 2008• The LT4eL project uses multilingual language

technology tools and semantic web techniques for improving the retrieval of learning material. The developed technology will facilitate personalized access to knowledge within learning management systems and support decentralisation and co-operation in content management.

LT4eL - Language Technology for eLearning -

2-• Start date:; 1 December 2005• Duration: 30 months• EU finacing: 1.5 milion Euro• Type project: STREP IST-4• Coordination: Paola Monachesi (Utrecht

university)• Contact for information:

Paola.Monachesi@let.uu.nl

LT4eL - Partners• Utrecht University (UU), The Netherlands• University of Hamburg (UHH), Germany• University “Al.I.Cuza” of Iasi (UAIC), Romania• University of Lisbon (FFCUL), Portugal• Charles University Prague (CUP), Czech Republic• Institute for Parallel Processing, Bulgarian Academy of Sciences (IPP-BAS),

Bulgaria• University of Tübingen (UTU), Germany• Institute of Computer Science, Polish Academy of Sciences (ICS-PAS), Poland• Zürich University of Applied Sciences Winterthur (ZHW), Switzerland• University of Malta (UOM), Malta• University of Cologne (UCO), Germany• Open University (OU), United Kingdom

LT4eL - Languages• Bulgarian• Czech• Dutch• German• Maltese• Polish• Portugese• Romanian• English

LT4eL -Aims• Improve retrieval of learning material• Facilitate construction of user specific

courses• Improve creation of personalized content• Support decentralization of content

management• Allow for multilingual retrieval of content

LT4eL- Objectives -1-• Scientific and Technological Objectives

– Creation of an archive of learning objects and linguistic resources

– Integration of language technology resources in eLearning

– Integration of semantic Knowledge in eLearning– Integration of functionalities in open source LMS– Validation of enhanced LMS

LT4eL- Objectives -2-• Political objectives

– Support multilinguality– Knowledge transfer– Awareness raising– Exploitation of resources– Facilitate access to education

LT4eL - Workpackages• ▪ WP1 - Setting the scene - WP leader: University AI. I. Cuza

of Iasi• ▪ WP2 - Semi-automatic metadata generation driven by

Language Technology resources - WP leader: University of T 歟 ingen

• ▪ WP4 - Integration of the new functionalities in the ILIAS Learning Management System - WP leader: University of Cologne

• ▪ WP3 - Enhancing eLearning with semantic knowledge - WP leader: IPP, Bulgarian Academy of Science

• ▪ WP5 - Validation of new functionalities in the ILIAS Learning Management System - WP leader: Open University (England)

• ▪ Multilinguality - Leader: University Hamburg

LexikonCZ

CZENCONVERTOR 1

Documents SCORMPseudo-Struct.

Basic XML LING. PROCESSOR

Lemmatizer, POS, Partial Parser

CROSSLINGUAL RETRIEVAL

LMS User Profile

Documents SCORMPseudo-Struct

Metadata (Keywords)

Ling. Annot XML

Ontology

CONVERTOR 2

Documents HTML

LexikonPT

LexikonRO

LexikonPL

LexiconGE

LexikonMT

LexikonBG

LexikonDT

LexiconEN

PLGE

BG

PT

MTDT

ROEN

Documents User(PDF, DOC, HTML,

SCORM,XML)

REPOSITORY

Glossary

Collection of Learning materials • collection of the learning material (uploads & updates at

http://consilr.info.uaic.ro/uploads_lt4el/ - passwd protected)• ▪ IST domains for the LOs:▪

– 1. Use of computers in education, with sub-domains:▪ • 1.1 Teaching academic skills, with sub-domains:▪ • 1.1.1 Academic skills▪ • 1.1.2 Relevant computer skills for the above tasks (MS Word, Excel, Power Point,

LaTex, Web pages, XML)▪ • 1.1.3 Basic computer skills (use of computer for beginners) (chats, e-mail, Intenet)▪ • 1.2 e-Learning, e-Marketing▪ • 1.3 The I*Teach document (Leonardo project, http://i-teach.fmi.uni-sofia.bg/)• 1.4 Impact of use of computers in society• 1.5 Studies about use of computers in schools / high schools▪ • 1.6 Impact of e-Learning on education▪

– 2. Calimera documents (parallel corpus developped in the Calimera FP5 project, http://www.calimera.org/ )

Collection of learning materials and linguistic

tools• normalization of the learning material▪ commonly agreed DTD and convertors from html/txt to basic XML format▪ Inventarization and classification of existing tools (http://consilr.info.uaic.ro/uploads_lt4el/tools/all.php?) relevant to:– ▪ the integration of language technology resources in eLearning (WP2)– the integration of semantic knowledge (WP3)▪

• Inventarization and classification of existing language resources▪ corpora and frequencies lists ﾊ : http://consilr.info.uaic.ro/uploads_lt4el/menu/all.php

• ▪ lexica: http://www.let.uu.nl/lt4el/wiki/index.php/Lexica_Joint_Table

LexikonCZ

CZENCONVERTOR 1

Documents SCORMPseudo-Struct.

Basic XML LING. PROCESSOR

Lemmatizer, POS, Partial Parser

CROSSLINGUAL RETRIEVAL

LMS User Profile

Documents SCORMPseudo-Struct

Metadata (Keywords)

Ling. Annot XML

Ontology

CONVERTOR 2

Documents HTML

LexikonPT

LexikonRO

LexikonPL

LexiconGE

LexikonMT

LexikonBG

LexikonDT

LexiconEN

PLGE

BG

PT

MTDT

ROEN

Documents User(PDF, DOC, HTML,

SCORM,XML)

REPOSITORY

Glossary

WP2: Integration of language resources in eLearning

Aims of the Workpackage• supporting authors in the generation of

metadata for Los• improving keyword-driven search for LOs• supporting the development of glossaries

for learning material

Metadata• metadata are essential to make LOs visible

for larger groups of users• authors are reluctant or not experienced

enough to supply them• NLP tools are supposed to help them in

that task• the project uses the LOM metadata

schema as a blueprint

Task 1: Identification of keywords

• Good keywords have a typical, non random distribution in and across Los

• Keywords tend to appear mor often at certain places in texts (headings etc.)

• Keywords are often highlighted / emphasised by authors

Modelling Keywordiness

• Residual Inverse document frequency used to model inter text distribution of KW

• Term burstiness used to model intra text distribution of KW

• Knowledge of text structure used to identify salient regions (e,g, headings)

• Layout features of texts used to identify emphasised words and weight them higher

Challenges

• Treating multi word keywords (suffix arrays will be used to identify n-gramsof arbitrary length)

• Assigning a combined weight which takes into account all the aforementioned factors

• Evaluation– manually assigned keywords will be used to

measure precision and recall of key word extractor against

– inter annotator agreement will be tested to get a upper bound for keyword assignment task

Task 2: Identification of definitory contexts

• This task makes use of the linguistic annotation of Los• The approach is empirical• Identification of definitory contexts is language specific• Workflow

– Definitory contexts will be searched and marked in LOs (manually) – Local grammars will be drafted on the basis of these examples– The linguistic annotation will be used for these grammars– The grammars will be applied to new Los Integration– The tools will be integrated as additional functions to the ILIAS LMS– The tools will also be available for integration in other LMS– We consider making the tools available as web services

LexikonCZ

CZENCONVERTOR 1

Documents SCORMPseudo-Struct.

Basic XML LING. PROCESSOR

Lemmatizer, POS, Partial Parser

CROSSLINGUAL RETRIEVAL

LMS User Profile

Documents SCORMPseudo-Struct

Metadata (Keywords)

Ling. Annot XML

Ontology

CONVERTOR 2

Documents HTML

LexikonPT

LexikonRO

LexikonPL

LexiconGE

LexikonMT

LexikonBG

LexikonDT

LexiconEN

PLGE

BG

PT

MTDT

ROEN

Documents User(PDF, DOC, HTML,

SCORM,XML)

REPOSITORY

Glossary

WP3:ontology based cross-lingual retrieval

• Generic approach• For each domain :

– Using computers for beginners– Impact of eLearning in Society

• we built a domain ontology• For consistency reasons we consider also an upper ontology (DOLCE)• Lexical material in all 9 languages is mapped on the ontology and on

the upper ontology• According to :

• types of relations in the ontology and• Uses cases • Similarity (predefined ontological chunks)

we define some search patterns for the user interface

Domain Ontology• First built starting with English documents• Concepts are based on :

– Extracted keywords in WP2 and– Glossaries for the given domains

• Concepts have generic names with parts in English (for readability reasons) e.g: C11_editors

• For each concept we provide labels with explanation of the concept in english and ideally in all other languages

• Types of relations:– Is_a– Part_of– Here we need some informations about what people are

searching • The ontology will be encoded in OWL- DL

Mapping multilingual resources on the domain

ontology -1-• Trivial for words having exact a correspondent in the ontology

• Problems appear when:1. One word in a language sub-sums two or

more concepts in the ontology 2. One word in a language sub-sums two or

more concepts in an ontology but only in relations with some other concepts

3. One word has a much restrictive meaning not present in the ontology

Mapping multilingual resources on the domain

ontology -2-• Solution to 1:– Express the lexical items in OWL-DL

expressions: disjunction, conjunctions of classes (give example)

• Solution to 2:– Express the lexical items in OWL-DL using

together with operations on classes also relations between the involved concepts

• Solution to 3:– Insert new concept in the ontology

Ontology enrichment• If one word cannot be mapped directly on the

ontology look if a similar meaning can be retrieved in some other languages.

• If this seems to be not an isolated case insert the new concept in the ontology.

• In any case assign to each concept a label indicated the languages in which this concept is lexicalised

• The insertion of a new concept will be done with FACT or RACER

Linking lexicon, domain ontology and upper

ontology• Domain ontology concepts will be mapped on the upper ontology.

• This will ensure that all important properties of main classes are considered.

• Not relevant senses of some lexical items could be also mapped directly on the upper ontology

LexikonCZ

CZENCONVERTOR 1

Documents SCORMPseudo-Struct.

Basic XML LING. PROCESSOR

Lemmatizer, POS, Partial Parser

CROSSLINGUAL RETRIEVAL

LMS User Profile

Documents SCORMPseudo-Struct

Metadata (Keywords)

Ling. Annot XML

Ontology

CONVERTOR 2

Documents HTML

LexikonPT

LexikonRO

LexikonPL

LexiconGE

LexikonMT

LexikonBG

LexikonDT

LexiconEN

PLGE

BG

PT

MTDT

ROEN

Documents User(PDF, DOC, HTML,

SCORM,XML)

REPOSITORY

Glossary

WP4: Tasks• Integration of LT4eL Tools for semi-

automated metadata generation, definitory context extraction and ontology supported extended data retrieval into a learning management system (prototype based on ILIAS LMS)

• Developing and providing documentation for a standard-technology-based interface between the language technology tools and learning management systems

WP4: Objective - Fostering Re-Use of LT-Tools

LMS 1(ILIAS)

Language Technology

Tools

LT-InterfaceXML-RPC /

Web Service

Language Technology

Tools

LT-InterfaceXML-RPC /

Web Service

LMS 2(e.g. Moodle)

Language Technology

Tools

LT-InterfaceXML-RPC /

Web Service

LMS 3(e.g. ATutor)

• Simple-as-possible, well-documented and standards-based interface

WP4: Using LT-Tools in Learning Managements Systems

• Possible Use Case Scenarios– Author annotates learning object with

keywords– Author generates glossary for learning object– Tutor searches for learning objects– Learner searches for learning material in

multiple languages– Learner browses through learning material

with ontology based information

WP4: Example ILIAS-LT-Tools Use Case

Scenario: Keyword Generation1. Author adds new learning object to the LMS (e.g. HTML file)2. ILIAS displays a form including input fields for title,

language and filename3. Author enters title and language, selects a local .pdf file

and hits Upload File4. ILIAS+LTTools display the (LOM) metadata input form,

including a list of auto-generated, suggested keywords5. Author selects some of the suggested keywords, enters

some new keywords and hits Save6. ILIAS saves the metadata

LexikonCZ

CZENCONVERTOR 1

Documents SCORMPseudo-Struct.

Basic XML LING. PROCESSOR

Lemmatizer, POS, Partial Parser

CROSSLINGUAL RETRIEVAL

LMS User Profile

Documents SCORMPseudo-Struct

Metadata (Keywords)

Ling. Annot XML

Ontology

CONVERTOR 2

Documents HTML

LexikonPT

LexikonRO

LexikonPL

LexiconGE

LexikonMT

LexikonBG

LexikonDT

LexiconEN

PLGE

BG

PT

MTDT

ROEN

Documents User(PDF, DOC, HTML,

SCORM,XML)

REPOSITORY

Glossary

WP5: Validation of enhanced LMS.

• Challenge is to answer these questions:– How does this compare with what can already be done

with existing systems? – What added value is there? – What is the educational / pedagogic value of these

functionalities?• Problem is to evaluate the functionality and

separate from issues of usability or unfamiliarity with the LMS platform.How can we expect users to identify any benefit?

How can we expect users to identify any benefit?

• Present them with tasks to complete using LMS• With no project functionality• With project functionality

– Partial– Full

• Identify potential users– Course Creators– Content Authors or Providers– Teachers– Sudents

• studying in their own language• studying in a second language

Create outline User Scenarios

• We define scenarios, in this context, as– a story focused on a user or group of users which

provides information on• the nature of the users,• the goals they wish to achieve and• the context in which the activities will take place.

– They are written in ordinary language, and are therefore understandable to various stakeholders, including users.

– They may also contain different degrees of detail.

Example Outline Scenario for a student• A student has just completed studying in English a topic on 'The use of computers

in Schools'.• They are interested in finding more information on the use of this topic within

their subject domain. • Their first language is German• Suggested search approaches might be:

– standard search as available within the LMS not using any of LT4eL functionality.

– add in the lexicon– add in the multi-linguality– add in the ontology

• Users will be given guidance / familiarisation activities in using each of the tools beforehand.▪

• User Scenarios are under development for all the identified users.• Each scenario will focus on one or more of the new functionalities

dependent on the roles of a particular user.

Possible Teachers /Course creators tasks

• Add new content to new course structure • Search for existing content and add to course structure• Add new content to existing course• Add supplementary content (could be another language)• Modify existing content• Create new content and make available to the system.

Feedback from Users

• Sessions will be used to gather some initial feedback using– individual interviews– group plenary– questionnaires

Project plan

• Preparatory work in place (May 06).• Development functionalities complete

(November 2006). • Integration functionalities in LMS complete

(May 2007)• First cycle integration functionalities in LMS

and their validationcomplete (November 2007)• Second cycle integration functionalities in LMS

and their validationcomplete (May 2008)