The Academy Research Observatory – ‘Surveying the Landscape’ & update on developments

Martin Oliver

London Knowledge Lab

& Higher Education Academy

Emails - m.oliver@ioe.ac.uk

- martin.oliver@heacademy.ac.uk

- observatory@heacademy.ac.uk

More information at:http://www.heacademy.ac.uk/ourwork/research/observatory

Piloting site:http://academy-research-observatory.pbwiki.com/

The Observatory

A service promoting and exploring the use of practice- and research-based evidence to influence policy and practice in teaching and learning in Higher Education

Tools to enable access to evidence and syntheses of this evidence (e.g. prototype repository and syndicated search, wiki)

Spaces (real and virtual) to help communities explore evidence-based practice and its implications for students' learning

A work in progress

Initial proposal to HEFCE for an e-Learning Research Observatory

Landscaping studyhttps://mw.brookes.ac.uk/display/hearoc

Judged to have wider relevanceA research observatory for Higher Educatione-Learning, Widening Participation, Employer EngagementOther ‘strands’ may be added - although the strands may not be explicit represented in the final structure

Overview of development• August 07 – July 08

• Scoping the Observatory• Landscaping report for e-learning (exemplar area)• Generation of pilot resources and services• Proof of concept piloted at Academy conference

• July 08 – July 09• Development phase• Pilots focusing on community engagement• Wider consultation• Promoted at Academy conference July 09

Phase 0

Phase 1

Phase 2, 3…

Landscaping evidence use in e-learning

Series of exploratory studies within e-learning Beetham, Sharpe & Benfield

‘Landscaping’ consultationInterviews with key informantsSurvey (116 responses)Follow-up interviewshttps://mw.brookes.ac.uk/display/hearoc/

Yes, but…Keen on single point of access; research reviewed, evaluated and synthesised

Questions – e.g. how does this relate to other Observatories…?

After that, great variability

“The impossibility of categorising respondents as users, producers, policy makers or intermediaries for research is in itself is an important outcome.”

HB: What kind of research or evidence should a research observatory focus on?

I: Evidence that e-learning really works.

HB: What would that evidence look like?

I: It would need to show real improvements to learning outcomes, across a large number of students. It would have to have credibility and rigour.

HB: Can you think of an example of research evidence of that kind?

I: Not off the top of my head, no.

HB: So does this research really exist?

I: No, the observatory would have to fund it. OR Yes, it is out there, the observatory will have to work really hard to find it.

HB: Can you think of a situation when evidence like this has really changed people’s practice or understanding, in your experience?

I: Well, the cynics always ask for evidence that e-learning really works.

HB: Do you think that evidence, if you had it, would lead them to change their minds?

I: No, they would find arguments against it from their own discipline perspective.

HB: So what about people who are actually open to change?

I: They never ask for evidence. They ask for examples, especially from their own subject area, and practical ideas. They are really responsive to other people in their discipline who have tried something and made it work.

Identified strategic choices, approaches and risksStrong persuader or neutral observer? (Setting agendas)Funder?Push or pull communications?Quality assured, selective research or evidence and examples?Audience: researchers, intermediaries, practitioners or policy makers?Building authority or democratic knowledge? (Who can write?)Central or local knowledge management?Face-to-face or technology supported networks?

Building from this: choices and risks

Internal discussions about priorities, style, focusSomething that has to be revisited. Regularly.

Oakleigh evaluation report – the need for engagement with the community; perceived as remote, authoritarian

Personal interests in people and how they make sense of technology in their work

Risk assessmentScoped scenarios; risk document based on report

Focus for discussion within project team meetings

However…Impending ‘launch’ at Academy conference – determined initial priorities

Pilot site for preliminary consultation

Building from this: e-Learning pilot

How do communities produce, share and use evidence?QA/QE in e-learning SIGExpert review about e-portfoliosDiscussion about professional role (M25 Learning Technologists group)Open peer commentary on the national development programmes

Review processes, to identify approaches that may have wider valueFeed these back to inform the development of the whole observatory

Overview of lessons:

Resources provided for people were not necessarily (ever?) taken up

Communities rise and fall

Expert review uninviting to others (useful, but not an invitation to contribute)

Open peer commentary engaged invitees then stopped

Building from this pt2: wider consultation

Inviting contributions from wider groups

Employee learning and widening participation communities

TLRP conference workshops

ELESIG meeting (Today!)

Case studies with communities (Heads of e-Learning Forum)

Input into specifying the Observatory; documented cases of evidence generation and use“Phases” – question of whether this is Phase 1, 2 or 3…

Building from this pt3: e-Learning projects

Small grants for research

Projects awarded, visited once, conclude by generating reports

New model

Projects awarded, brought together, given technical infrastructure (wiki, Ning)

Will be visited, encouraged to use Web2.0, brought together mid-project and again at end

Wiki for Academy-funded projects

And what next?

New round of projects (provisionally)

Smaller-scale reviews

Special Interest Groups

Possibly projects about technology to support evidence-informed practice

Emphasis on engagement (I hope)Wiki-based reviews for open engagement

Social networks – finding people, not just research outputs

However…

Continued visibility of central resource

Yes, it’s a repository… but it’s not just a repository

If it’s just a website it’ll be pointless

Invitation to engage does not guarantee engagement

So…

The “hidden” agenda for this meeting:

ELESIG as a community of research/evidence producers (and potentially, users too…)

An opportunity to support and encourage evidence use

Documenting the process

Identification of some outcomes• Available via the RO wiki (so you can edit it ’til you’re

happy!)

What now?

Working from your experiences to influencing policy and practice

Morning: generating messages from current practice

Afternoon: working with key messages

Task:

Claims you’d like to make based on what you’re currently up to

The format

In tables…

1. Chat about what you’ve been up to

2. Jot down claims you’d like to make

3. Note, under each, evidence to support them

Then…

4. Swap tables! Critique!

5. Return and repair to make claims robust