165
15|51 • 01-09-16 • 1

15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

15|51 • 01-09-16 • 1

Page 2: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Program

14:00 Richard van der Weide, Intergo 14:15 Karel Brookhuis, Universiteit Groningen 14:45 Sarah Wiseman, University of London 15:15 Break 15:45 Chris Jansen, Universiteit Utrecht 16:15 Marc Woods, Mindhacker

Stella Donker, Universiteit Utrecht 17:00 Drinks

15|51 • 01-09-16 • 2

Page 3: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Marit Wilms

Richard vd Weide

Melcher Zeilstra

Diana Went

Gert-Jan Kamps

Kirsten Schreibers Fenneke Blommers

David de Bruijn

Alfred v Wincoop Henk Frieling

Colete Weeda

Page 4: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 5: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 6: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 7: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 8: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 9: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 10: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 11: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 12: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Deconducteurvandetoekomst

Page 13: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 14: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Program

14:00 Richard van der Weide, Intergo 14:15 Karel Brookhuis, Universiteit Groningen 14:45 Sarah Wiseman, University of London 15:15 Break 15:45 Chris Jansen, Universiteit Utrecht 16:15 Marc Woods, Mindhacker

Stella Donker, Universiteit Utrecht 17:00 Drinks

15|51 • 01-09-16 • 14

Page 15: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

1

Karel Brookhuis (Traffic Psychology)

Traffic Psychology Group, Dept. Neuropsychology

Faculty Behavioural & Social Sciences

University of Groningen (Delft University of Technology) Colleagues: Dick de Waard, Janet Veldstra, Anselm Fürmaier, Arjan Stuiver

Human Factors in traffic Rijksuniversiteit Groningen

Faculty of Behavioural & Social Sciences Department of Psychology

Page 16: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Experimental & Work Psychology

Traffic in 1900

1900 Seat belt Patent by Gustave-Desire Levau

Page 17: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Experimental & Work Psychology

The rationale of Traffic Psychology

Accidents at least 95% “Human Factor” The driver makes errors. is not alert, distracted, tired, etc.

Page 18: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Experimental & Work Psychology

Development of fatal accidents in traffic in the Netherlands (SWOV)

Jaar1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 20000

500

1000

1500

2000

2500

3000

3500

0

1

2

3

4

5

6

1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000

mln

Page 19: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 5

Research on in-vehicle ICT 25 years of driver support

• European projects, off 1989 • GIDS (Generic Intelligent Driver Support System) • AWAKE (Drowsiness warning)

• Dutch national initiatives (Ministry) • ISA (Intelligent Speed Assistance) • Belonitor (Rewarding good i.s.o. Punishing bad) • PAYD (Pay As You Drive, with Insurance cies)

• National Research Council (NWO) programs • PITA (Personal Intelligent Travel Assistant) • Amici (Advanced Multi-agent Information and Control for

Integrated traffic networks) • MDPIT (Multidisciplinary pricing policy)

Page 20: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 6

25 years of research on effects of driver support & information supply, EU start: Generic Intelligent Driver Support (European Framework, DRIVE 1989 - 1991)

• Philips (CARIN) & Bosch navigation systems • RDS – TMC (Radio Data System – Traffic

Message Channel)

• Simulator studies (VSC/TRC, Groningen) • Instrumented Vehicle (TNO, Soesterberg)

Page 21: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 7

GIDS: (early, 1990) Carin navigation system

Page 22: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 8

( GIDS in 1989 … simulator 2010)

Page 23: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 9

Results EU studies, GIDS … AutoVeh • Problems: attention, distraction, alertness • Accident likelihood:

• 2 a 2.5 second rule (within car) • 4 second rule (outside car) • 15 second rule (total processing time)

• Combination auditory and visual information • Improved route guidance systems

• GIDS book, Michon et al., T&F 1993 • Guidelines travel / traffic information

• HASTE Special Issue TR / Part F • In-Safety (Book, 2010) • Special Issue TR/F (History Traffic Psychology, 2015)

Page 24: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 10

Intelligent Speed Assistant ISA (1995, in advisory version)

Page 25: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 11

Intelligent Speed Assistant ISA (1995, in advisory version)

Page 26: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 27: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Results of Feedback

C

C

ISA

ISA

Page 28: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Compound opinions on ISA

-3 -2 -1 0 1 2

Pleasurable OLD

Practical OLD

Pleasurable YOUNG

Practical YOUNG

BeforeAfter

Page 29: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 15

ISA effects: 36% less injury accidents, up to 59% less fatal accidents

Longitudinal effects: wears off, but not completely

Page 30: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

12 maart 2008 Symposium Synergie in Mobiliteit, Universiteit Twente

Ministry (NL) funded project

New concept: reward i.s.o. punish

• distance: 1.3 s

• local speed limit

• 62 lease-car drivers

• 4 weeks baseline

• 16 weeks test

• 4 weeks baseline

• subjective and objective measurements

Belonitor – Rewarding safe driving behaviour

Page 31: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

12 maart 2008 Symposium Synergie in Mobiliteit, Universiteit Twente

Earned credits

Belonitor – Rewarding safe driving behaviour

Page 32: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

12 maart 2008 Symposium Synergie in Mobiliteit, Universiteit Twente

Bad behaviour

Belonitor – Rewarding safe driving behaviour

Page 33: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

12 maart 2008 Symposium Synergie in Mobiliteit, Universiteit Twente

Belonitor – Rewarding safe driving behaviour

Good behaviour

Page 34: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

12 maart 2008 Symposium Synergie in Mobiliteit, Universiteit Twente

Belonitor - Conclusions General Attitude: 68% positive about system, 75% improved behaviour

Traffic Safety • Estimated reduction lethal & severely injured 15% • Estimated reduction injured 9% Efficiency and Environmental • reduction accidents 10% – traffic jams 1.2%

• reduction fule use 5.5 %

Bron: MinVenW, 2006

Page 35: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

12 maart 2008 Symposium Synergie in Mobiliteit, Universiteit Twente

USA 2013: statistics indicates a 4% to 5% increase in accident involvement

Side effects of Lane Departure Warning

Page 36: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 22

Page 37: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 23

2006: RouteLint HSM (Railway-Project Het Spoor Meester)

Page 38: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 24

Attention for action → Distraction

Page 39: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Multitasking ↔ Distraction

Page 40: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Multitasking ↔ Distraction

Thanks to Peter Hancock

Page 41: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

27 Driver distraction and inattention in its various forms is thought to play a role in 20-30% of all road

crashes (Wang, Knipling & Goodman, 1996).

Page 42: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

05 September 2016 28

Future ?? (Google stopped !!) =>ICT in traffic

http://www.youtube.com/watch?v=V8ofTlynWPo&feature=player_detailpage

Page 43: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

So, no more human drivers ? Automated Vehicle !

Page 44: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Opleiding Psychologie/Faculteit gedrags- en

maatschappijwetenschappen

Summary of Levels of Driving Automation for On-Road Vehicles This table summarizes SAE International’s levels of driving automation for on-road vehicles. Information Report J3016 provides full definitions for these levels and for the italicized terms used therein. The levels are descriptive rather than no rmative and technical rather than legal. Elements indicate minimum rather than maximum capabilities for each level.

“System" refers to the driver assistance system, combination of driver assistance systems, or automated driving system, as appropriate.

The table also shows how SAE’s levels definitively correspond to those developed by the Germany Federal Highway Research Institute (BASt) and approximately correspond to those described by the US National Highway Traffic Safety Administration (NHTSA) in its “Pre liminary Statement of Policy Concerning Automated Vehicles” of May 30, 2013.

Lev

el

Name Narrative definition

Execution of steering and acceleration/ deceleration

Monitoring of driving

environment

Fallback performance of dynamic driving task

System capability (driving modes) B

AS

t le

vel

NH

TS

A

leve

l

Human driver monitors the driving environment

0 No Automation

the full-time performance by the human driver of all aspects of the dynamic driving task, even when enhanced by warning or intervention systems Human driver Human driver Human driver n/a

Driv

er

on

ly

0

1 Driver Assistance

the driving mode-specific execution by a driver assistance system of either steering or acceleration/decelerati on using information about the driving environment and with the expectation that the human driver perform all remaining aspects of the

dynamic driving task

Human driver and system Human driver Human driver

Some driving modes A

ssis

ted

1

2 Partial Automation

the driving mode-specific execution by one or more driver assistance systems of both steering and acceleration/deceleration using information about the driving environment

and with the expectation that the human driver perform all remaining aspects of the dynamic driving task

System Human driver Human driverSome driving modes P

art

ially

a

uto

mate

d

2

Automated driving system (“system”) monitors the driving environment

3 Conditional Automation

the driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task with the expectation that the human driver will respond

appropriately to a request to intervene System System Human driver

Some driving modes H

ighl

y a

uto

mate

d

3

4 High Automation

the driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task, even if a human driver does not respond appropriately to a

request to intervene System System System

Some driving modes

Fully

a

uto

mat

ed

3/4

5 Full Automation

the full-time performance by an automated driving system of all aspects of the dynamic driving task under all roadway and environmental conditions that can be managed by a

human driver System System System All driving

modes

-

Source: SAE Standard J3016 Report

Summary of Levels of Driving Automation for On-road Vehicles

Page 45: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

› Simulator experiment in Groningen • Drivers of Hermes and Arriva companies • Trajectory Eindhoven CS to Eindhoven Airport

Automating Public Transport SAE level 2

Page 46: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Phileas (APTS bv)

Page 47: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 48: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

› HOV in Eindhoven • Hybrid system and control: (half)automatic and normal • Phileas (Automatic Public Transport System)

› Simulator experiment in Groningen, • Bus drivers of Hermes and Arriva • Track Eindhoven CS to Eindhoven Airport

Automating Public Transport 1996 (Hoogwaardig Openbaar Vervoer)

Page 49: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 50: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Route

Page 51: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 52: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Incident 1: vehicle blocks the road

o Braking in time

o Bus back to non- or semi-automatic

Page 53: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 54: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Incident 2: a cyclist runs red light

o 1st time: 72% okay, but 28 % not !!!

o 2nd time: 100% okay !

Incident 1: vehicle blocks the road

o Braking in time

o Bus back to non- or semi-automatic

Page 55: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Experiment in driving simulator SAE level 3

Page 56: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Autonomous driving

Page 57: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Emergency situation on automated highway

Page 58: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Reactions to the unexpected event

Reactions Time slot Proportion participants No reaction - f 50 % Braked Late > 14 s 15 % Braked Not fast 9-14 s 30 % Braked Early < 9 s 5 %

Page 59: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Attitude towards fully automated system Before After › Usefulness 0.53 0.38 › Satisfying –0.31 –0.36

The road towards fully autonomous vehicles: via SAE levels 2 and 3 ??

Perhaps Chris Janssen will tell us

Page 60: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Mental Model, evidence form another field…. (thanks to Dietrich Manzey) According to Malinge (2011), the reduction of the annual accident rate of fourth generation airlines has stagnated, notwithstanding the inherent effectiveness. Although automated systems have evolved to reduce pilot workload, recent concerns exist about their safety effectiveness. Automation introduces a paradox: providing crews with necessary operational assistance simultaneously dissociates the crew from those operations. Unfortunately, this shift in pilot tasking exhibits itself in many forms of adverse crew behavior such as automation induced complacency (Manzey, Reichenbach & Onnasch, 2012), automation bias (Mosier, Skitka, Heers & Burdick, 1998), decision making errors (Orasanu, Martin & Davison, 1998), lack of (procedural and declarative) system knowledge and/or manual control skills (Potter et al., 2012), which are all in turn aggravated by overconfidence (Wood, 2004) and fatigue (Caldwell, 2012). These behaviors may contribute to the loss of situational awareness (SA).

Page 61: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

18 March 2016 47 www.its.leeds.ac.uk

Survey: Safety and Priority?

Page 62: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

So, automation… ?

When, how, where, who ?

Kyriakidis et al. “A Human Factors perspective on Automated Driving ” (i.e. European perspective), in preparation

[email protected]

Page 63: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Program

14:00 Richard van der Weide, Intergo 14:15 Karel Brookhuis, Universiteit Groningen 14:45 Sarah Wiseman, University of London 15:15 Break 15:45 Chris Jansen, Universiteit Utrecht 16:15 Marc Woods, Mindhacker

Stella Donker, Universiteit Utrecht 17:00 Drinks

15|51 • 01-09-16 • 64

Page 64: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Everything you always wanted to know about

numbers, but were afraid to ask

Dr Sarah WisemanGoldsmiths, University of London

Page 65: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Who am I? •  Currently: !

•  Research and teaching fellow at Goldsmiths!

•  Public speaker on HCI and HF!

•  In the past:!

•  HCI PhD looking at number entry interfaces!

Page 66: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 67: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Who am I? •  Currently: !

•  Research and teaching fellow at Goldsmiths!

•  Public speaker on HCI and HF !

•  In the past:!

•  HCI PhD looking at number entry interfaces!

•  Daily Mail accredited “scientist” !

Page 68: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Why do you care about numbers?

Page 69: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 70: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Newtons Pounds per square inch

Page 71: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Newtons Pounds per square inch

Page 72: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 73: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

x!

Page 74: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

x!

Page 75: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

x!x!

Page 76: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Number entry interfaces in the hospital

Electronic patient !records!

Infusion pump !devices!

Radiography / X-ray !equipment!

Page 77: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Why are number entry interfaces designed

like that?

Page 78: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

7 8 9

4 5 6

1 2 3

0

1 2 3

4 5 6

7 8 9

0

Page 79: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Telephone Calculator

Page 80: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 81: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 82: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 83: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 84: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 85: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 86: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 87: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 88: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Expectations

Page 89: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 90: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 91: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 92: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 93: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 94: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 95: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

all day long

Page 96: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

55%

Page 97: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

8%

Page 98: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

7%

Page 99: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

55%

Page 100: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

But why are they laid out in a grid like

that?

Page 101: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 102: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Reality

Page 103: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 104: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 105: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 106: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 107: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 108: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 109: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 110: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 111: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 112: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 113: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 114: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

“offered certain engineering advantages”

Page 115: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Desires

Page 116: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 117: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 118: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Can we improve the standard number entry interface?

Page 119: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Improving text entry

Page 120: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Improving text entry

URL email

Page 121: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Ok before we improve anything, what’s the most

common number?

Page 122: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Benford’s Law

hUp://www.thecleverest.com/benfords_law.html

Page 123: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

When does Benford apply? “Naturally occurring cumulative numbers”

Number of twitter followers Length of rivers

Electricity bills First 652066 Fibonacci Numbers

Page 124: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

When does Benford apply?

When does it not apply?

“Naturally occurring cumulative numbers”

Number of twitter followers Length of rivers

Electricity bills First 652066 Fibonacci Numbers

Lottery numbers

Page 125: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

When does Benford apply?

When does it not apply?

“Naturally occurring cumulative numbers”

Number of twitter followers Length of rivers

Electricity bills First 652066 Fibonacci Numbers

Lottery numbers

Fraudulent invoices/expenses

Page 126: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 127: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance
Page 128: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

When does Benford apply?

When does it not apply?

“Naturally occurring cumulative numbers”

Number of twitter followers Length of rivers

Electricity bills First 652066 Fibonacci Numbers

Lottery numbers

Fraudulent invoices/expenses

Page 129: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

When does Benford apply?

When does it not apply?

“Naturally occurring cumulative numbers”

Number of twitter followers Length of rivers

Electricity bills First 652066 Fibonacci Numbers

Lottery numbers

Fraudulent invoices/expenses

Drug dosages in hospitals

Page 130: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Numbers used in the hospital

100, 5.5, 300, 1000, 750, 33.33, 999, 12.25, 150, 200, 40, 600…

Page 131: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Numbers used in the hospital

Page 132: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Numbers used in the hospital

Page 133: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Redesigning the interface

Page 134: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Redesigning the interface

Page 135: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Redesigning the interface

Significantly faster

No increase in error rate

Page 136: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Redesigning the interface

Significantly faster

No increase in error rate

Page 137: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Is the interface design all there is to think about?

(You are a very smart audience)

Page 138: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Number entry is a 3-step process

1. Read the number

2. Memorise the number

3. Type the number

Page 139: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Number entry is a 3-step process

1. Read the number

2. Memorise the number

3. Type the number

Page 140: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Sarah Wiseman www.swiseman.co.uk

@oopsohno [email protected]

Page 141: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Program

14:00 Richard van der Weide, Intergo 14:15 Karel Brookhuis, Universiteit Groningen 14:45 Sarah Wiseman, University of London 15:15 Break 15:45 Chris Jansen, Universiteit Utrecht 16:15 Marc Woods, Mindhacker

Stella Donker, Universiteit Utrecht 17:00 Drinks

15|51 • 01-09-16 • 154

Page 142: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

9/5/16&

1&

Automation and HCI: A case study of (semi-) autonomous car

Chris Janssen [email protected] www.cpjanssen.nl

H2020-MSCA-IF-2015, 705010, 'Detect and React'

http://jalopnik.com/teslas-autopilot-system-is-awesome-and-creepy-and-a-sig-1736573089

Page 143: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

9/5/16&

2&

(semi-) Autonomous cars: Challenge? SAE level

Name Description

Human driver monitors driving environment

0 No Automation

1 Driver Assistance

2 Partial Automation

Automatic driving system monitors driving system

3 Conditional Automation

4 High Automation

5 Full Automation

SAE International (2014)

Designing for automated systems

Linda Boyle U Washington

Andrew Kun U New Hampshire

Lewis Chuang Max Planck

Wendy Ju Stanford

Initiated at Dagstuhl 16262

Driving requires varied behavior..

Strategic

Maneuvering

Control

After Michon (1985)

Lane position detection

Traffic detection

Change lanes

Driving requires varied behavior..

Lane position detection

Traffic detection

Change lanes ✔

Page 144: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

9/5/16&

3&

Non-automated car

Change lanes Traffic detection Lane detection

Fully automated car (e.g., “ideal” google car)

Change lanes Traffic detection Lane detection

Change lanes Traffic detection Lane detection

Change lanes Traffic detection Lane detection

Change lanes Traffic detection Lane detection

Partial automated car (e.g. “Tesla”)

Change lanes Traffic detection Lane detection

Change lanes Traffic detection Lane detection

Change lanes Traffic detection Lane detection

Change lanes Traffic detection Lane detection

Potential dangers / issues •  Transitions:

-  How to inform human about transition? -  How does driver know which state they

are in? -  How to inform and enforce a shift in

responsibility?

Page 145: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

9/5/16&

4&

Broader context: automation How do we.. •  Communicate change? •  Design for shared responsibility? •  Assess system state? •  Assess human state?

Example studies: asses human state

1.  ‘Looking at the eyes’

2.  ‘Sensing in the brain’

1. ‘Looking at the eyes’

Hidde van der Meulen MSc AI Utrecht

Andrew Kun U New Hampshire

Van der Meulen, Kun, Janssen (2016) Proceedings ACM AutoUI

Set-up

Page 146: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

9/5/16&

5&

Eye-gaze

Figure 4. Top-figure: The percentage of looking at the road center (PRC) as a function of time. Bottom-figure: the average driving

speed over time. In both figures, the vertical dashed line shows the moment in time where both conditions have the same speed, with the standard deviation displayed as vertical grey bars. The error bars and ribbon show the standard error values of each

metric.

We used a 24 (time after take-over: buckets of 5s) x 2 (situation: parking or autonomous) within-subjects ANOVA to determine the effect of time and condition on the PRC. The time after take-over had a significant influence on the PRC with F(1, 23) = 3.73, p < .001, there was no significant effect for driving type with F(1, 1) = 0.93, p = .542 and no significant interaction effect between time after take-over and the type of driving with F(1, 23) = 0.55, p = .954. As Figure 4 shows, the PRC gradually increases in the first 15 to 25 seconds and then stabilizes

Percent dwell time Figure 5 plots the PDT score for the autonomous driving condition (dark grey bars) and the parking condition (light grey bars) for the 100 seconds before taking over control (left two bars) and 2 minutes after taking over control (right two bars). A 2 (timing: before, after take-over) x 2 (situation: parking or autonomous) within-subjects ANOVA revealed that there was a main effect of timing, such that drivers looked more at the road after taking over, F(1, 15) = 78.75, p < .001. There was also a main effect of driving situation, F(1, 15) = 7.39, p = .016. This was influenced by an interaction effect, F(1, 15) = 7.48, p = .015.

As Figure 5 shows, the interaction was such that before the take-over, the participants looked almost twice as often at the road in the autonomous driving condition (M = 48%, SD = 9.4%) compared to the parking condition (M = 27%, SD = 6.22%), whereas after take-over both groups spent roughly a

similar amount of time gazing at the road (in both conditions M = 99%, SD = 0.5%).

Figure 5. Percentage of time spent looking at the road before

and after the take-over for both autonomous driving and parking conditions.

% g

aze

to tr

affic

sce

nario

2. ‘Sensing the brain’

Remo vd Heiden Promovendus

Leon Kenemans Chantal Merkx Rijkswaterstaat

Stella Donker

H2020-MSCA-IF-2015, 705010, 'Detect and React'

Data: BSc thesis Lotte Hardeman & Keri Mans

Page 147: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

9/5/16&

6&

0 500 1000 1500

-50

510

P3a at FCz (Novel - Standard)

Time (ms)

Am

plitu

de (m

icro

Vol

t)

StationaryAutonomousDriving

(Active)

Preliminary results Do not cite

0 500 1000 1500

-50

510

P3a at FCz (Novel - Standard)

Time (ms)

Am

plitu

de (m

icro

Vol

t)

StationaryAutonomousDriving

(Passive)

Preliminary results Do not cite

Broader context: automation How do we.. •  Communicate change? •  Design for shared responsibility? •  Assess system state? •  Assess human state?

Page 148: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Program

14:00 Richard van der Weide, Intergo 14:15 Karel Brookhuis, Universiteit Groningen 14:45 Sarah Wiseman, University of London 15:15 Break 15:45 Chris Jansen, Universiteit Utrecht 16:15 Marc Woods, Mindhacker

Stella Donker, Universiteit Utrecht 17:00 Drinks

15|51 • 01-09-16 • 188

Page 149: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Mindhacking:factorficFon?

[email protected]

189

Page 150: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

190

Why does “mindhacking” work?

Page 151: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

191

Mindhacking in the “real” world

Page 152: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

192

Mindhacking in the “real” world

Page 153: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Why does mindhacking work?

193

PercepFonislearned:BrainislazyPaUernrecogniFonEfficienttojumptoconclusions

Page 154: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

DirecFngaUenFon

Page 155: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

In/uitcheckenAmstelstaFon

DirecFngaUenFonin the “real” world

Page 156: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

IncheckenamstelstaFon

DirecFngaUenFonin the “real” world

Page 157: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

DirecFngaUenFonin the “real” world

Failureofawareness:theinformaFonistherebut(some)peopledonotseeit

Page 158: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

198

Failure of awareness

Page 159: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

199

Failure of awareness

Page 160: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Lookedbutfailedtosee(SMIDSY)

“Sorry mate, didn’t see you”

Page 161: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Failure of awareness

Page 162: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

TheinformaFonisthere,butsFll(some)peopledonotseeit

Failure of awareness

Page 163: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Mindhackingisafact

203

Mindhacking:FactorFicFon?

Page 164: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

204

Mindhacking:FactorFicFon?

istheresultofthemagnificantpowerofthebrain.

ThefactthatamindhackercanmakeficFonfeelreal,

Page 165: 15|51 - Intergo › public › Publicaties › 9 › bestand... · Elements indicate minimum rather than maximum capabilities for each level. “System" refers to the driver assistance

Program

14:00 Richard van der Weide, Intergo 14:15 Karel Brookhuis, Universiteit Groningen 14:45 Sarah Wiseman, University of London 15:15 Break 15:45 Chris Jansen, Universiteit Utrecht 16:15 Marc Woods, Mindhacker

Stella Donker, Universiteit Utrecht 17:00 Drinks

15|51 • 01-09-16 • 206