Digital examination, forms and tools for aggregation of information and cognition. The dynamics that emerges in large student groups and the phenomenon "the wisdom of crowds"

RES URS 1KTH – RESOURCE CENTRE FOR NET BASED EDUCATION [RCN] www.math.se www.kth.se/rcn 1

Johan Thorbiörnson

PhD, Assistant Professor in MathematicsDirector MATH.SE

Director RCN - Resource Centre for Netbased [email protected]

Royal Institute of Technology, Stockholm

http://www.kth.se/rcnhttp://www.math.se

[email protected]

Digital examination, forms and tools for aggregation of information and cognition. The dynamics that emerges in large student groups and

the phenomenon "the wisdom of crowds".

RES URS 2KTH – RESOURCE CENTRE FOR NET BASED EDUCATION [RCN] www.math.se www.kth.se/rcn

Sammanfattning Skandinaviskt seminarium 22 april 2009 om Nätbaserad värdering och

examination

Titel: Digital examination, forms and tools for aggregation of information and cognition.The dynamics that emerges in large student groups and the phenomenon "the wisdom of crowds".

Plattformar för e-learning har huvudsakligen varit teknologier för att organisatoriskt kunna hantera stora studentgrupper som är skilda åt i tid och rum. Detta görs fortfarande men sådan undervisning sker inte alltid enligt den traditionella hierarkiska formen, utan iscensätts även i nya undervisningsformer. Denna presentation tar utgångspunkt i James Surowieckis begrepp "the wisdom of crowds" för att diskutera formerna för aggregering av kunskap och den dynamik som uppstår i kunskapssökande grupper. Webbplattformen Math.se komer att presenteras liksom nya initiativ inom e-learning på KTH.

Litteratur:James Surowiecki "The wisdom of crowds"

http://www.randomhouse.com/features/wisdomofcrowds/ http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds


Innehåll Begreppet och fenomenet ”The wisdom of crowds”, tillämpningar inom

lärande

Historik: hierarkisk och icke-hierarkisk struktur för kursverksamhet

Presentation av MATH.SE och RCN

Funktioner och moderna verktyg för aggregering av information och omdömen inom grupper

Högskolans primära uppgift – att examinera med ackreditering

Nätbaserad värdering och examination som stöd för formativ examination, avancerad uppföljning och feedback. Hur kan tiden som spenderas utnyttjas bättre, hur kan kvalitet skapas i studentens lärande, hur kan föreläsningar och handledning optimeras?

Lärande i ett globalt perspektiv

Framtiden, KTH Virtuellt campus

Diskussion


Fenomenet ”The wisdom of crowds”

James Surowiecki: "The wisdom of crowds" http://www.randomhouse.com/features/wisdomofcrowds/

http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

Crowd-sourcing

RES URS 5KTH - RESURSCENTRUM FÖR NÄTUNDERVISNING [RCN] www.kth.se/rcn

MATH.SE – web based university courses in European collaboration


Stockholm UniversityUppsala UniversityLinköping UniversityÖrebro UniversityUniversity of GävleMälardalen UniversityJönköping University

Imperial College, LondonTechnische Universität Berlin (TU Berlin)

MATH.SE is a collaboration managed by RCN at KTH Stockholm, consisting of 8 Swedish partner universities and from now also Imperial College and TU Berlin.


RCN and MATH.SE– backgroundand historySweden's oldest and largest technical university Royal Institute of Technology (KTH) has more than 25 years of experience in e-learning.

Every year 20% of all beginners in Sweden study a national bridge course in mathematics or other subjects before they start university.

The same service will now be offered to other countries.


RCN – Resource Centre for Netbased Education [Royal Institute of Technology] www.kth.se/rcn

RCN organizes MATH.SE


Hierchical and non-hierarchical structures for organizing courses

Person of the year 2006: You!The Person of the Year 2006 in Times Magazin was "You" (meaning that "you" control the Information Age). The organizational structure within MATH.SE and its built-in revenue sharing model along with a high level of Quality of Experience makes a big step towards approaching the new global market in education and the need for mass-individualization.

The first university, Plato's academy, has been a model for all the following traditional universities, based on the idea of one teacher per group of students. This does not scale well since if you have 1000 times more students you need 1000 times more teachers. Another serious drawback in a traditional setup is that students in different schools in different cities will not communicate with each other, even if they are struggling exactly the same mathematical problems and learning processes.

By using a flat non-hierarchical structure and modern social networking tools, we can make it possible for thousands of students together to share their learning processes, independently of were they are living. This is a big difference compared to the traditional way of organizing courses.


A new educational paradigm


MATH.SE are taking care of infrastructure, support, formalities, etc. Activities for students to successfully complete the course on the Internet.

Call center solution for tutoring – the students can call or email their personal mentor.

Students belongs to different universities but study together in one virtual classroom.

Universities get administrative tools and can monitor ”their” students.

MATH.SE – a model proved to work nationally in Sweden and now expanding in Europe

10 000 students in the same classroom. High quality resource allocation.

Large community.


11

Ten thousand studentsstudy maths on the summer holidays

”It gives me such confidence”

From the newspaper:


An investigation shows that students who completed the summer course on the Internet (Bridge course) is equally successful in the first year at KTH as students with the highest marks in all courses from school.


School A B C D E

UniversityRepetitionBridging

Bridge courses and Foundation courses

From School to University: Bridges the gap and gives a flying start at the first part of university math.

Since 1997 the department of Mathematics at the Royal Institute of Mathematics has evaluated the discrepancies between freshmen actual knowledge of mathematics and the KTH faculty expectations. The course curriculum in the bridging courses is based on these discrepancies.

The same type of bridging courses are now offered in different subjects suchas physics, computer science and chemistry.


Clear advantage to do at least one chapter on the Internet based bridge course

Pass rate on first exam at KTH (Calculus course):

Civil engineering students (Master of Science in Engineering):

Passrate 86% (prepared using the Bridge course)

Passrate 68% (did not follow the Bridge course)

Students on Bachelor of Science in Engingeering programme:Passrate 78% (prepared using the Bridge course)

Passrate 63% (did not follow the Bridge course)

(Year 2008)



Do you think you benefit you benefit from the bridge course during your further studies in the first semester?

Civil engineering students (Master of Science in Engineering):

"Yes", or "Yes, much" : 75%

No: : 25%

(Year 2003)



Students at the end of 1st semester: “Are you experiencing that you should have rehearsed logarithms?”

19% (prepared using the Bridge course)

45% (did not follow the Bridge course)26% (“A” students)

“Are you experiencing that you should have rehearsed limits, continuity and asymptotes?”

52% (prepared using the Bridge course)

42% (did not follow the Bridge course)44% (“A” students)

(Year 2006)


The way through the system for the student• Application• Computer account• Log in• Study, tutoring,

supervision, feedback, remainders

• Examination


The student log in through a central database• Will then be linked to the specific learning platform and

the net based tools within which activities will be done. • Instead of a solution based on one single LMS we use

different modules and specific net based tools, linked together in one framework.

• Different technical solutions, for example a Wiki for publishing course material, systems for automatically corrected tests, systems for collaboration and groupwork, etc.


Student Lounge

Links to• Course literature• Examination parts and tests• Course forum• Personal tutor• Study guidance, etc• Administrative tools

Personal study plan

After logging in, the student comes to their personal page.


Course material in a wiki solution

• Teachers can develop material in working teams

• Version management• Mathematical signs and other

special signs can be displayed directly in the web browser (LaTeX)

• In discussion forums the students easily can write mathematics by clicking on buttons for special signs or more advanced by using LaTeX.


The examination process is partitioned into levels and performed continuously during the course:

• Basic facts - basic tests after each section

• Problem solving - final tests after each section

• Analysis and synthesis, ability to discuss and evalute – wiki for individual assignment and group assignment

• Final evaluation and feedback of cohesive knowledge, feedback and grading – written or oral exam

The student will be coached by tutors online, who follows and tracks the work by the students with the net based examination, continuously during the course.

Net based examination – framework for courses


Formativ examination

Summativ examination

Low stake tests

High stake tests

Net based examination – framework for courses


23

Automatic and peer assessment in large student groups based on a detailed and level-partitioned examination process.

Assessment is integrated with the students learning activities and is utilized as a structure for instructional scaffolding and follow-up.


Different interfaces extract information from the database, such as tools for mentors for following up and monitoring students based on their progress, tools for sending out remainders by email or SMS, calling up students who seem to be late, etc.

Mentor interface for follow up and support of students


Administrative system Interfaces for tutors, teachers and administrators


Higher generation e-learning course

• Scalability• Flexibility • High quality resource allocation

• Based on detailed learning outcomes in a combination of interactive learning objects, peer learning, peer assessment, and detailed design of activities through the platform• Student torrent - analogous to Bittorrent techniques (larger ”downloads” generates larger bandwidth)

Processes are similar to a ”boiling stew pan” full of activities, independently of the diameter of the pan since the students are learning from each other.The teachers’ roles will be to design, spice up, stimulate and coach the activities.


WWWVirtual

classroom

Textbooks Classroom

StudentINTERNET

Online

examination

Advanced

follow up

What is an Internet based course?


KTH Virtual Campus– now building the future

• Framework of integrated services from different technical solutions

• Coordination and standardization of development and initiatives already going on at KTH

• Makes the “real world” campus clearer and activities more meaningful and effective

• Crowd sourcing


Johan ThorbiörnsonPhD, Director MATH.SE

Director RCN - Resource Centre for Netbased Education

[email protected], +46 703 371100


http://www.math.se

[email protected]