Measuring Business Value of Learning Technology ...€¦ · Measuring Business Value of Learning Technology Implementation in HE Setting Nia Alexandrov, Education and Training Team,

ICCS 2014 – BRIDGE workshop 10 to 12/062014

Nia Alexandrov, BSC 1

www.bsc.es

Measuring Business Value of Learning Technology Implementation in HE

Setting

Nia Alexandrov,

Education and Training Team, BSC

Nia Alexandrov 2

Computational Science and Skills Gap in HE Context

The skills considered hardest to find:

– scientists with HPC capabilities,

– parallel programmers,

– algorithm developers,

– system administrator with high-end computing

experience.



Nia Alexandrov 3

Aims

Intersecting influences: pedagogical approaches, technological affordances, and the management of the processes at programs, departments and University strategy levels

– Decades technological developments have strong impact on teaching particularly when combined with a virtual learning environment putting technology in the position of an enabler and enhancer

– Quality teaching using collaborative learning is the bases for building of a set of research skills

– Management practices affect the success or failure of courses and impact motivation to learn

Intersecting Influences Model [1/2]

Nia Alexandrov 4

Legend:M – motivatorH – hygiene factorFactors standing in overlapping areas are under intersecting influences



Contributing Areas

Nia Alexandrov 5

Area Influence Impact Affected Change

Pedagogy: Ways of teaching Understanding the way we learn informs the design of the format of individual work and subject/unit assessment

Collaborative work & tools bring additional learning outcomes: acquiring of interpersonal skills, social values, and professional attitudes

Technology Tools and their affordances & constrains

Technological advances bring changes in the lecturing format

Management Motivational theory and business value assessment

Staff training, student –lecturer interaction,

Financial and structural implication of ILT on organizational level

PEDAGOGY AND IT IMPLEMENTATION



Integrated Learning Processes Model consists of four distinct stages and conforms to Kolb’s natural learning cycle and the work of James Zull Nia Alexandrov 7

Underlying Pedagogy

Construction • Classify • Associate • Analyze • Recall • Reflection

Abstraction • Visualize • Synthesize • Plan • Conjecture • Hypothesize

Concrete Experience • Navigate • Explore • Sense • Review

Action • Experiment • Discuss • Debate • Create • Practice

Tools Supporting Increased Complexity of Interaction Type of teaching materials

Media forms Learning activities

Complexity of interaction

Elements of Collaborative work

Class of Web 2.0 toolsFacilitating the corresponding level

Level 5: Discursive

Productive Project workNegotiation of learning objectives

Clearly Perceived Positive Interdependence/Individual Accountability and Personal Responsibility/ Group Processing / Small Group Skills/ Interaction

Wikis/ Social Networking/ Syndication and notification/ Collaborative editing

Level 4: Adaptive(by the tutor)

Adaptive

Practical session/Laboratory work

Based on previous communication, teacher may refocus the dialogue and the student would re-conceptualize as result

Group Processing / Small Group Skills/ InteractionInternal and interpersonal dialogue

Syndication and notification/ Collaborative editing/Media Sharing/ Simulations/ Immersive Technologies

Level 3: Interactive(at the level of actions)

CommunicativeSeminar/ team discussion

Teacher functions as moderator supporting students discussions and reflections on their work

Small Group Skills/ InteractionTwo cycles of interaction/ dialogue are possible:Peer-to-peer within the team and Tutor-student(s)

Social Bookmarking/ Media Sharing/ Forums / Quizzes & Tests

Level 2: Reflective

Interactive TutorialThere is a provision of feedback on student´s work

InteractionOne to one, tutor - student interaction

Blogs/ Forums

Level 1: Descriptive

Narrative LectureMakes concepts available

No collaboration needed Mashups / Static pages

Nia Alexandrov 8



Gains of Implementation of ITTeaching Methods and Curricula Design

Management Processes

Subject SpecificU

se o

f IT

Support of team work and individual accountability

Common timetable and progression pathAccess to teaching and administrative resources irrespective of location

Remote access to relevant teaching resources including HPC resources

Gai

n

Development of problem solving, research and collaboration skills through implementation of CCCP activities

Streamlined administrationDecentralised control over material update and student feedback

HPC and domain specific applications;Visualisation and simulation environments;Mathematical Modelling capabilitiesOver multidisciplinary subject area

Nia Alexandrov 9

BUSINESS VALUE OF IT IN HE CONTEXT

Nia Alexandrov10



Measuring the Organisational Level of Performance Using the CMF

Business Value for IT - mainly industry driven emerging area

– Intel Corporation - M. Curley (2005) and D. Sward (2006) and Innovation Value Institute - the National University of Ireland, Maynooth

The approach:

– introducing certain metrics for estimating the Business Value of IT

– Linking the metrics with defining the baseline,

– defining the IT innovations to be introduced,

– defining the expected productivity gain and measuring Return On Investment within the given timeline

Nia Alexandrov 11

Reasoning for using IT-CMF in HE context

Measuring the level of implementation of IT as a teaching tool and in the management of the teaching/ learning process allows us to determine the organisational level of performance

The principles for good practice in HE relate motivation to study context

In management theory Herzberg links dissatisfies (hygiene factors) with job context.

Nia Alexandrov 12



5 stages of maturity of the organizational attitudes towards spending for IT

Ad Hoc – no defined processes and no future-proof investment decisions

Repeatable – focus on total cost ownership - seeking to control cost and quality of services

Defined – shift from TCO to return of investment-new understanding that the business value of IT investment lies outside of the IT department

Managed – portfolio management, sophisticated approach to managing IT investments attempting to optimise investments based on a defined criteria and taking into account the organisational business strategy

Optimised – measurement and tracking of actual and projected benefits based on strong alignment of IT investment objectives with the business objectives of the organisation

Nia Alexandrov 13







Nia Alexandrov 14






Defined – shift from TCO to return on investment-new understanding that the business value of IT investment lies outside of the IT department



Nia Alexandrov 15







Nia Alexandrov 16









Nia Alexandrov 17

Understanding of Business Value in the Higher Education Setting

CMF Level

Description CMF in the Higher Education Context

1Ad hoc

No defined processes

Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2

Cost control and services focus Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

3

Investment governance based on business cases

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks; availability of collaborative tools and digital learning materials; simulation and visualisation tools

4

Portfolio management Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5

Alignment with organisational business objectives

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.Research publications increase based on achieving results impossible without IT investment and so on

Nia Alexandrov 18




CMF Level


1

No defined processes Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2Repeata

ble

Cost control and services focus

Cost effective investments in providing services –centrally supported VLE , central DB, conferencing tools, mail service for all students,

3



4


5


Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.Research publications increase based on achieving results impossible without IT investment and so on Nia Alexandrov 19


CMF Level


1No defined processes Ad Hoc decision resolving particular current needs – isolated cases of use of VLE,

conferencing or web tools by some courses or departments

2Cost control and services focus Cost effective investments in providing services – centrally supported VLE ,

conferencing tools, mail service for all students, central DB

3Defined


Management or educational reasons for investment – centralised administrative systems for submissionand management of work and marks; availability of collaborative tools and digital learning materials; simulation and visualisation tools

4


5



Nia Alexandrov 20




CMF Level




2Cost control and services focus Cost effective investments in providing services – centrally supported VLE ,

conferencing tools, mail service for all students, central DB

3



4Managed

Portfolio management

Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5



Nia Alexandrov 21


CMF Level




2Cost control and services focus Cost effective investments in providing services – centrally supported VLE , conferencing

tools, mail service for all students, central DB

3



4


5Optimised


Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.Research publications increase based on achieving results impossible without IT investment and so on Nia Alexandrov 22



CMF Level

CMF in the Higher Education Context ACET CMF level in the UoR context

1

Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

Developing the E-LANE Integrated Metacognitive Processes Model (IMPM)

2Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

Transition to BlackBoard based course support and non-paper office for ACET MSc programs and creation of student forum

3


Management and teaching using BlackBoard and RISIS at UoR as the VLE for local MSc courses and transnationally for the EM MSc in NeBCC; ACET student applications and alumnae forum integration

4


Integration of ReaCTor (Immersive Virtual System)and IBM Themes Blue JS21 based Supercomputer into multidisciplinary teaching and research under the Computational Science Research Theme

5

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability. Research publications increase based on achieving results impossible without IT investment and so on

Nia Alexandrov 23

CMF Level


1Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments


2

Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB


3



4



5


Nia Alexandrov 24



CMF Level






3



4



5


Nia Alexandrov 25

CMF Level






3



4



5

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability. Research publications increase based on achieving results impossible without IT investment and so on Nia Alexandrov 26







3



4



5

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability. Research publications increase based on achieving results impossible without IT investment and so on… Nia Alexandrov 27

HERZBERG THEORY ANDSTUDENTS’ MOTIVATION



Relation between the principles of good practice in HE and Herzberg´s motivators

Principles of good practice in HE Herzberg´s Motivators in work environment

Student faculty interaction Recognition, Interpersonal relations - superior

Student collaboration Recognition, Interpersonal relations - peers

Encouragement of active learning Work itself

Prompt feedback Recognition, Company policy and administration

Emphasize on time-on-task Company policy and administration

Communication of high expectations Achievement, Possibility of growth

Respect of diverse learning styles Recognition, Work itselfNia Alexandrov 29

Inferring the Motivation Factors in HE

The case study was the MSc students of ACET from 3 consecutive intakes:– 80 overall 61 returned the survey

The questionnaire is divided in three parts: – questions on identifying the satisfying factors using IT,

– questions on identifying dissatisfying factors using IT

– questions on general technology use.

A posteriori approach to study design:– Herzberg’s methodology

– categories were derived from the data and compared with the already derived by Herzberg in the Motivation to Work case

Nia Alexandrov 30



Motivators and Hygiene factors for StudentsMotivators Hygiene factors

Achievements IT Support

Recognition Content and Assessment

Student Involvement and Participation Quality of Teaching

Teaching Methods Teaching and Learning

Resources

Feedback Administrative Support

Technology Features Facilitating the

learning

Organization and Student

Support

Interaction with Convenors and

Professors

Visibility Nia Alexandrov 31


Nia Alexandrov 32

Legend:M – motivatorH – hygiene factorFactors standing in overlapping areas are under intersecting influences



Nia Alexandrov 33


Organising the Motivation Factors into sets with predominant influence of one or several areas – pedagogy, technology and management:– Single area of influence:

• IT support (HF) and Technological features supporting learning (M) - Tech

• Quality of Teaching (HF), Content and Assessment (HF), Interaction with Convenors and Professors (M), and Teaching Methods (M) - Ped

– Dual areas of influences • The administrative support (HF) Man or Tech.

• Teaching and learning resources (HF) Ped or Tech.

– Triple (complex) areas of influences• Achievements (M) and Recognition (M) - interlinked factors in Long-Range

sequences are in the triple intersection and to be successfully facilitated require adequately functioning technology, appropriate pedagogical environment and supportive management processes.

Nia Alexandrov 34

Conclusions:

The capacity created by management practices was identified by determining the Business Value of IT and of the organizational level of performance on IT-CMF in HE setting.

the position of technology as means to resolve or aggravate issues as facilitator was determined by inferring the Motivation and Hygiene Factors.

Pedagogy sets the principles of teaching that create the needed skills set and as such has leading position but is unable to resolve all issues on its own.



References

[1] N.Alexandrov, R. Ramirez-Velarde, and V. Alexandrov, ed., “Technological Advances in Interactive

Collaborative Learning”, Taylor and Francis, 250 pages,Dec, 2012, ISBN-9781466502086.

[2] N. Alexandrova, V. Alexandrov and Raul Ramirez “The Role of Computational Science and

Emerging Technologies in the Natural Sciences Education at University Level”, Procedia Computer

Science, Elsevier, Volume 9, pp. 1789-1798, 2012.

[3] Martin Curley, “Managing IT for Business Value”, Intel Press, 2005.

[4] David Sward, “Measuring Business Value of IT”, Intel Press, 2006.

[5] IVI, Innovation Value Institute, http://ivi.nuim.ie/, Maynooth, Ireland.

[6] F. Herzberg, B. Mausner, B.B. Snyderman, “The Motivation to Work”, Transaction Publishers, Sixth

Edition, 2003.

[7] Chickering, A.W., and Gamson, ZF 1991, Applying the Seven Principles for Good Practice in

Undergraduate Education, New Directions for Teaching and Learning. Number 47, Fall 1991. San

Francisco: Jossey-Bass Inc.

[8]Walker R, J. Voce and J. Ahmed, 2012, UCISA Report: 2012 Survey of Technology Enhanced

Learning for Higher Education in UK, http://www.ucisa.ac.uk/tel

Nia Alexandrov 35

Documents

Measuring Business Value of Learning Technology ...€¦ · Measuring Business Value of Learning Technology Implementation in HE Setting Nia Alexandrov, Education and Training Team,