View
41
Download
0
Category
Preview:
DESCRIPTION
Analysis of usability of scheduling algorithms: a case study in the Netherlands Railways. R.J. Jorna, W. van Wezel & J. Bos Faculty of Economics and Business, University of Groningen (Groningen, The Netherlands) Contact: r.j.j.m.jorna@rug.nl. Third Rail Human Factors Conference; - PowerPoint PPT Presentation
Citation preview
Analysis of usability of scheduling algorithms: a case study in the
Netherlands Railways
R.J. Jorna, W. van Wezel & J. BosFaculty of Economics and Business, University of Groningen (Groningen, The Netherlands)Contact: r.j.j.m.jorna@rug.nl
Third Rail Human Factors Conference;Lille, France, March 3-5, 2009
2
Structure
Introduction– Netherlands Railways and their planning– Background & Problem statement: effects of planning support
Research:– What does “inadequate use” of algorithms mean?– Results of experiments with 8 planners– Comparison planners and OR-specialist
Algorithms and the function-allocation theory Conclusions, statements and further research
3
The Netherlands Railways and Planning
Netherlands Railways (NS): 2700 railroad carriages; per day 5000 train services; 384 stations.
NS overall: about 350 planners; 4 major kinds of planning:
– A: timetables;– B: planning rolling stock and rolling staff;– C: local and central planning (stock and staff);– D: year plan (long term) & day plan (short term)– Sequence: timetable, rolling material, rolling staff
and then shifts for individual persons.
4
Background and problem statement
In staff planning shift schedules need to be changed frequently (see Figure 1).
Here, study of 8 planners of Day Planning (Central) (re)schedule shifts of train drivers/ticket collectors.
Planners use DSS CREWS (see Figure 2), with a GUI to create plans by dragging and dropping.
Recently, an algorithm to automatically create plans is included. Planners can influence algorithm by setting parameters.
Problem: Use of algorithm did not result in expected increase in performance. What is going on?
5
Example of change within shiftsFigure 1
6
Figure 2Screen shot of CREWS
7
Theoretical assumptions: general
Task performance with DSS is a function of – Planning problem complexity,– Quality of automation (software support level),– Mental load of planners,– Many other aspects (trust, cost, cooperation, etc.)
Use of algorithmic support can be studied in quasi-experiments with planners
8
Cognitive workloadSituation awareness
ComplacencySkill degradation
Quality of automationRisk of automation failure
TrustCost of incorrect decisions
How can we apply thisto planning / scheduling?
Factors influencing the adequate use of planning support
9
Theoretical assumptions: model
Relation between independent variables and performance
Two conditions: manualand algorithm support
Two conditions: simple and complex problem
10
Experiments with planners (1)
Procedure– Planner reads assignments, solves problem with system;– For Mental Load: planner uses NASA TLX questionnaire;– During experiment, number of activities of planner are
counted: a) use of searching functionality in system; b) # backtrackings (withdrawn reasoning steps); # decisions resulting in constraint violations
Simple problem Complex problem Number of shifts 31 94
Number of stations 4 6
11
Experiments with planners: results simple problem (2)
Task performance for simple problem (N = 8)
Indicators (1) Average Manual (2)
Algorithm vs Manual (3)
Average Algorithm (4)
Conflicts 0,1 0 0 # Passenger tasks 24,6 + 14,7 # Taxi rides 0 0 0 >30 minutes exceeding shift length 1 0 0,6 Extra shifts 0 0 0,3 Empty shifts 1,4 + 3,7 Solution time (minutes) 63,3 + 43,1 # Reasoning steps 137,1 + 53 Reasoning Steps withdrawn 5 + 2,3 Wrong steps 8,7 + 3,1 Search / Filter 47,4 + 11,6 > Related to stock 16,9 + 2,7
For simple problem: algorithm better than manual; for complex the reverseQuestion: WHY?
+: algorithm better
12
Experiments with planners: results mental load and complex problem (3)
No difference mental load for simple problem (N=8) nor for complex problem (N=2)
Experiment with complex problem took more than 8 hours (only two planners were analyzed)
Mental load for simple problem Planner Manually With algorithm Average 34,6 36,3 SD 9,8 15,5
13
Experiments with planners: conclusions (4)
Some conclusions: For algorithms, planners are concerned with usability:
– does planner understand outcome of algorithm?– can schedule be explained to the staff that will execute it?– can planner change schedule easily, etc.)
(all of the above are difficult to formalize) If algorithm is not aligned to planner, no increase of performance In using algorithms, significant individual planner differences. Individual planner differences concerning: mental load, time to
solve problem, quality of outcome. Planners spend much time on configuring parameters and
interpreting the outcome (from protocols).
14
Experiment comparing planners, algorithm and developer (5)
(n=2) Planners without algorithm
Planners with algorithm Developer
Time (minutes) 290 185 60Extra shifts 14 12 7Empty shifts 12 8 6Number of steps 709 271 27Passenger tasks 56 48 36Taxi 0 0,5 7Conflicts 0 0 0Exceed max shift length 16 11 3
Planners can not plan as good as OR specialists?
15
What is wrong with planners?
Planners do not use their algorithms correctly and should be trained
or Algorithms are not developed to be usable by planners
(The algorithm was designed by winners of 2008 Informs “Franz Edelman Award for Achievement in OR and the Management Sciences”. Should they, the developers, become planners?)
Is planning/scheduling in railways a matter of control of dynamic situations and should support be adjusted to this kind of task?
16
12-02-2008 | 16
CALCULATEDURING THE
NIGHT
FIX ERRORS
17
Basic assumptions concerning planners in algorithmic support
Interaction with algorithms implies knowledge of the model being used, e.g.:
– Specify parameters; specify weights on goal functions– Steer the backtracking process of the algorithm– Fix the solution manually
How does interaction between parameters lead to constraint violations and goal realization? What goal?
– Planners do not have this knowledge !!– Planners will never have this knowledge !!
We propose that Interaction should be determined a priori, instead of a posteriori
18
The perspective of the control of dynamic situations
“Dynamic situations are situations which are only partly controlled by the human. He/she then has to keep the situation between acceptable limits while managing high risk and/or time pressure”
Examples: Air traffic control, nuclear power plant, aircraft operation, flexible manufacturing system
In the field of Cognitive Ergonomics, Function Allocation is successfully used to design automation
19
Conclusions / statements
The fundamental problems concerning planning support did not change in 30 years
Practice is ahead of academia We don’t fully understand ‘old’ planning, yet
businesses started ‘new’ planning Organization theory (various organizational forms)
related to planning hinders adequate support Algorithms do not pay attention to human problem
solving
20
Conclusions / further research
Balance requirements of tasks, algorithmic possibilities, and human limitations/creativity
Include the user in the development process from the beginning, but:
Not necessarily take the limitations of current humans for granted! Training or even partly replacing human planners is an option as well.
However, many goals of planning/scheduling are often confusing / difficult to formalize!
21
22
Interactive Scheduling Systems;gap between theory and practice
Combinatorial problems are hard to tackle Problems in practice change Interactive systems are rare Operations managers lack understanding of
scheduling systems Cost/benefit ratio hard to prove Human planners don’t have a good grasp of the
implications of their decisions Schedulers are buffered from outside pressure
23
The planner and his tasks
• Macro task division• Organizational setting• Who does what? Order acceptance, stock planning,
scheduling, rescheduling, etc.• Micro task division
• Collecting information• Counting• Checking errors• Evaluating• Problem solving
Recommended