WCC/IFIP 2010

Authors:

Sílvio César Cazella (UFCSPA,UNISINOS)

Eliseo Berni Reategui (UFRGS)

Patricia Alejandra Behar (UFRGS)

Recommendation of Learning

Objects Applying Collaborative

Filtering and Competencies

WCC/IFIP 2010

Sumary

� Challenge

� Goals and contribuition

� Competencies

� Recommender System

� Collaborative Filtering

� Model

� Prototype and experiments

� Results

� Conclusion

� Future Work

WCC/IFIP 2010

� The greatest challenge with which every educator faces is the organization of content and activities aimed at the development of certain competencies in students.

� This challenge is intensified when we try to identify and recommend different materials, customized to each student based on individual needs, interests and skills to be developed.

Challenge

WCC/IFIP 2010

� This paper describes a model for recommender systems that is able to suggest learning objects relevant to undergraduate students, focusing on competencies to be developed in the disciplines.

� The main contribution of this paper is to present this model and its implementation and evaluation with a group of students.

Goal and Contribuition

WCC/IFIP 2010

Competencies

WCC/IFIP 2010

� In all definitions, we can easily see the relationship between the concept of competence and skills (know- how), knowledge and attitudes.

� Within this research, therefore, the question arises as to how, when and how we can make a recommendation of learning objects that enable students to: � build knowledge related to specific issues,

� develop particular skills related to given contents,

� develop in students a critical awareness about the importance of competence to understand how and when

to use it.

Competencies

WCC/IFIP 2010

� Is Recommender System a

possibility to be used in this

context?

YES...

WCC/IFIP 2010

� Recommender systems have emerged, focusing on the search for relevant information in accordance with User's own characteristics.

� Different techniques are applied in recommender systems to find the most appropriate content for users. In this research we applied Collaborative Filtering (CF).

Recommender Systems

BobBobBobBob

WCC/IFIP 2010

Proposed Model

WCC/IFIP 2010

� A prototype of the model was developed in order to evaluate its efficiency in making appropriate predictions.

� Initially, some students were invited to participate in a few experiments for the evaluation of learning objects (in this casescientific papers) that were recommended by the system.

Prototype and experiments

WCC/IFIP 2010

� The evaluation of the prototype was made through two experiments with a sample by convenience (not probabilistic) of 10 students at the end of the undergraduate course of Computer Engineering.

� Learning objects used to recommend paperswere selected by a specialist teacher in thearea, and were directly related to thecompetencies to be developed in the discipline of database.

Research Method

WCC/IFIP 2010

� The experiments had the following objectives:

� To evaluate whether the prediction rate calculated by the prototype was able to match or approximate to real students’rates, using the evaluation metric MAE(Mean Absolute Error);

� To evaluate the accuracy of the recommendations made by the system through the metrics Recall (coverage) and Precision (precision).

Experiments and results

WCC/IFIP 2010

� Goal:

� Evaluation of Pre-Selected Items

� Description:

� Students were then requested to evaluate papers that had been allocated randomly;

� Use the tool prototype;

� Calculation of Pearson's coefficient.

1º Experiment

WCC/IFIP 2010

� 27.59% of the computed correlations between the students using Pearson's coefficient were considered strong (these students had "tastes" that were similar to the objects evaluated);

� 20.69% were considered weak (these students had " tastes "different from the objects evaluated);

� 51.72% of the correlations computed, nothing could be said.

1º Experiment: Results

WCC/IFIP 2010

2º Experiment

� Goal:

� Generating Predictions;

� Description:

� Performing the computation of the correlation valuesand prediction of similarity;

� The rules of competencies;

� Recommendations to users;

� Evaluation Metrics.

� Sample 10 students

� Likert scale of 5 points

WCC/IFIP 2010

Results of the experiment concerning Precision

16

WCC/IFIP 2010

Results of the 2º Experiment

WCC/IFIP 2010

� Through experiments with a group of

undergraduate students in Computer Engineering,

it was found that the degree of precision achieved

by the recommendations generated by the

prototype was satisfactory.

� The accuracy of 76% showed that the system was

able to recommend learning objects that satisfied

the students for their studies, without neglecting

the competencies required in the summary of the

course during the semester.

Conclusion

WCC/IFIP 2010

� As for the evaluation metrics Precision and Recall, it can be said that:

the prototype succeeded to get the students to have access to those materials that were relevant

to the competencies to be developed in that moment, within the set of learning resources

available.

Conclusion

WCC/IFIP 2010

� we intend to test the system with other types of learning objects to verify if its performance remains satisfactory, using information from the metadata of learning objects to select them according to specific requirements also related to competencies development (e.g. level of difficulty, level of interaction, etc.);

� include the relevance of the opinion of a User to complement the process of recommendation;

� we are also working on the formation of virtual communities that have a similarity coefficient within an acceptable range.

Future work

WCC/IFIP 2010

Contact: Silvio Cesar Cazellacazella@unisinos.brsilvioc@ufcspa.edu.br

That is all folks!

Questions?

Thanks!!!