Research in computer science educationSue Sentance

Structure of this session

Overview of research topics

Highlights from four areas

Key research questions (over to you – 3pm)

Discussion & Questions

Finish 3:15 pm

Research topics in computer science education (some!)

Programming

misconceptions

Peer

instruction

Pair

programming

Block-based

environments

Tracing and

reading code

Assessment

Papert and

constructionism

Sub-goal

modelling

Growth

mindset

Visualisation

and tools

Courses and

curricular

Unplugged

computing

And not forgetting computational thinking, teacher development, diversity inc gender, motivation and outreach …

Research topics in computer science education (some!)

Programming

misconceptions

Peer instruction

Pair

programming

Block-based

environments

Tracing and

reading code

Assessment

Papert and

constructionism

Sub-goal

modelling

Growth mindsetVisualisation and

tools

Courses and

curricular

Unplugged

computing

And not forgetting computational thinking, teacher development, diversity inc gender, motivation and outreach …

Research topics in computer science education (some!)

Programming

misconceptions

Peer instruction

Pair

programming

Block-based

environments

Tracing and

reading code

Assessment

Papert and

constructionism

Sub-goal

modelling

Growth mindsetVisualisation and

tools

Courses and

curricular

Unplugged

computing

And not forgetting computational thinking, teacher development, diversity inc gender, motivation and outreach …

Themes from CS Ed articles 2004-2014

0%

5%

10%

15%

20%

25%

Chart Title

School HE All

Numbers of papers: All – 2225 University - 1285 School -420

Research topics in computer science education (some!)

Programming

misconceptions

Peer instruction

Pair

programming

Block-based

environments

Tracing and

reading code

Assessment

Papert and

constructionism

Sub-goal

modelling

Growth mindsetVisualisation and

tools

Courses and

curricular

Unplugged

computing

And not forgetting computational thinking, teacher development, diversity inc gender, motivation and outreach …

A. Learning programming: tracing and reading code

Some research highlights

2004: Multi-institutional study of reading and tracing skills

shows better performance in programming by those able to

trace code (Lister et al, 2004)

2011: Use of neo-Piagetian framework to establish stages

that novice programmers go through (Lister et al, 2011)

2014: Exemplification of framework through case studies,

using think-aloud to find out more about students’ thought

process while programming (Teague and Lister,, 2014).

Application in school

PLAN C (CAS Scotland) have developed a

model for enabling students to trace through

code called TRACS

Work in this area suggests that

student pass through neo-

Piagetian stages: sensorimotor,

preoperational, and concrete

operational stages, before

eventually reaching programming

competence at the formal

operational stage

B. Learning programming: Common misconceptions

One recent paper: Exploring programming

misconceptions (Sirkia and Sorva, 2014)

Based on analysis of 24,000 log files from

UG students first learning programming:

Research found:

Inverted assignment (first = second) -

wrong way round

If X == Y: (execute then part even if false)

Returning False from function even

though condition does not hold

Not storing return value of function

Etc….

The 1980s

Early work on misconceptions and

novice programmers (Ben du Boulay,

1986, Bayman and Mayer, 1983, Bonar

and Solway, 1985)

Loops are difficult

Differentiating between a string and a

number

Confusion of equality and assignment

Inputting data (from where?)

etc…

Steady stream

of work in this

field, including

Juha Sorva

PhD

C. Learning programming: block-based environments

Increase in numbers of papers about Scratch since 2007

0

2

4

6

8

10

12

2007 2008 2009 2010 2011 2012 2013 2014

Papers about Scratch (2007-2014)

Some key papersLearning computer science concepts with Scratch

(Meerbaum et al, 2013)

• Focus on concepts learned (not skills)

• Tested > 200 Year 9 students taught Scratch

systematically

• Testing showed difficulties with loops, variables and

concurrency

Habits of programming Scratch (Meerbaum-Salant et al,

2011)

- block-based environments increase extremely fine-

grained programming (too bottom-up) – bad habits for

future

Recent developments

Recent developments: comparing understanding in blocks-

based and text-based programs (Weintrop and Wilensky, 2015)

Another paper last year found that although attitudes and

perceived difficulty was the same with block-based and text-

based programming, pupils achieved goals more quickly (less

idle time) with block-based (Price & Barnes, 2015).

PatchAmended version of Scratch developed by Bill Robinson, Norfolk Computing teacher, developed

from discussions held about tracing, pseudocode, transition to text-based languages etc.

Python-like loop structures to aid transition to PythonHints

Ability to trace step-by-step through codePseudocode Scratch blocks

D. Computing pedagogy: Peer instruction

Well-evidenced pedagogical strategy

Combination of:

- Flipped learning

- Collaborative working

- Well-chosen MCQs

Process

- Step 0: Study topic before session

- Step 1: MCQ question presented to class

- Step 2: Individually decide on answer

- Step 3: All vote

- Step 4: Discuss answer as group

- Step 5: Group decides on answer

- Step 6: All groups vote

- Step 7: Discuss as class

For more information see

http://peerinstruction4cs.org

Peer instruction – the evidence

Statistically

significant

effect

Twice as effective

as a good teacher

explanation

Successfully used in

Physics, Maths and

Computing Science

Develops better sense

of self efficacy

particular amongst

girls

SIGCSE 2016 – Best paper award given for a multi-institutional study on peer instruction

Could we do this with the Quantum questions?

Thanks to

PLAN C for

this

summary

Key research questions - discussion time

Here are 10 starter research questions

Are these important?

At what ages should we teach particular

programming concepts?

1

Can everyone learn to program?

2

What impact does learning computing at an

early age have on learning of other subjects,

particularly literacy and numeracy?

3

Is there a link between success in maths and

science and success in computing?

4

To what extent does lack of home access to

computer science tools impact computer science

performance and/or interest in school?

5

Which tools or online environments support

learning in Computing?

6

How can we evaluate the effectiveness of

tools for use in the classroom?

7

To what extent does pair programming support

the development of secure programming skills?

8

Does unplugged-style computing improve

learning outcomes in computing?

9

Is it possible to teach computational

thinking?

10

Key research questions - audience participation

5 minute task

• In small groups decide what it is you think we really need to know (and should invest time and

money in finding out)

• Use handout for inspiration but these are just ideas

• Write (large, with flipchart marker) on a piece of A4 paper

Feedback

Research in the CAS Community

CAS Research group is for …

• Universities engaging in research

• Teachers interested in research

• PhD students in the CAS community

Meetings:

- Networking

- Ideas

- Forum to present

Mailing list

- Information

- Colleagues

Join at: https://groups.google.com/d/forum/cas-research

New!

The Royal Society computing education project

Phase 1: Evidence-gathering

Phase 2: Projects to support recommendationshttps://royalsociety.org/topics-policy/projects/computing-education/

Thanks for listening!

New King’s Computer Science Education Research blog at http://blogs.kcl.ac.uk/cser

References

Bayman, P., & Mayer, R. E. (1983). A diagnosis of beginning programmers' misconceptions of BASIC programming statements. Communications of

the ACM, 26(9), 677-679.

Boulay, B. D. (1986). "Some Difficulties of Learning to Program." Journal of Educational Computing Research 2(1): 57-73.

Lister, Raymond, Elizabeth S. Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney et al. "A multi-national study

of reading and tracing skills in novice programmers." In ACM SIGCSE Bulletin, vol. 36, no. 4, pp. 119-150. ACM, 2004.

Lister, R. (2011, January). Concrete and other neo-Piagetian forms of reasoning in the novice programmer. In Proceedings of the Thirteenth

Australasian Computing Education Conference-Volume 114 (pp. 9-18). Australian Computer Society, Inc..

Meerbaum-Salant O., Armoni M. & Ben-Ari M., 2011. Habits of programming in scratch. In Proceedings of the 16th annual joint conference on

Innovation and technology in computer science education - ITiCSE ’11. ACM Press.

Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (2013). Learning computer science concepts with Scratch. Computer Science Education, 23(3),

239-264.

Porter, L., Guzdial, M., McDowell, C., & Simon, B. (2013). Success in introductory programming: What works?. Communications of the ACM, 56(8),

34-36

Price, T. W., & Barnes, T. (2015, July). Comparing Textual and Block Interfaces in a Novice Programming Environment. In Proceedings of the

eleventh annual International Conference on International Computing Education Research (pp. 91-99). ACM.

Sirkiä, T., & Sorva, J. (2012, November). Exploring programming misconceptions: an analysis of student mistakes in visual program simulation

exercises. In Proceedings of the 12th Koli Calling International Conference on Computing Education Research (pp. 19-28). ACM.

Teague, D., & Lister, R. (2014, June). Blinded by their Plight: Tracing and the Preoperational Programmer. In Proceedings of the Psychology of

Programming Interest Group Annual Conference 2014 (pp. 53-64).

Weintrop, D., & Wilensky, U. (2015, July). Using commutative assessments to compare conceptual understanding in blocks-based and text-based

programs. In Proceedings of the Eleventh Annual International Conference on International Computing Education Research, ICER (Vol. 15, pp. 101-

110).