Applied Scientific Research - Egypt Scholars...Civil Engineering (Intelligent Transportation...

Applied Scientific Research:From Lab to Field

Samah El-Tantawy, PhDAssistant Professor

Faculty of Engineering, Cairo University

2/07/2018

Applied Scientific Research: From Lab to Field

My Academic Career Journey

Electronics and Communications

Engineering, B.Sc.

Engineering mathematics,

M.Sc.

Civil Engineering (Intelligent

Transportation Systems), Ph.D.

AI Applications (Transportation,

Education,…)

2

Basics of Scientific Research

Agenda

• I: What is research? The Scientific Research Process

• II: Why is Research?

• III: A Journey of An Applied Scientific Research Example

From Lab to Reality

• IV: When to Start Research?

• V: What do you need to know before starting your

research?

3

Applied Scientific Research: From Lab to Field

I. Scientific Research Process

4

Applied Scientific Research: From Lab to Field

Have you conducted scientific research before?

5

6

Applied Scientific Research: From Lab to Field

Research is a Key Driver of Innovation

7

II. Why Do We Need to Conduct Research?

Why should we conduct Research?

• To develop knowledge for professions.

• To develop effective policies.

• To solve practical problems.

• To make informed decisions.

• To increase the knowledge base of larger society.

Huge amounts of daily life and experience in our society are based on what we have learned using the logic and evidence involved in scientific research.

8

Applied Scientific Research: From Lab to Field

III. Taking Research From Lab to Field

9

Smart Traffic Lights that Learn !

Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers

M A R L I N

Smart Traffic Lights that Learn !

Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers

M A R L I N

Baher Abdulhai, Ph.D., P.Eng.

Director, ITS Centre and Testbed, University of Toronto

Co-founder of Pragmatek Transport Innovations Inc.

Samah El-Tantawy, Ph.D.

Post Doctoral Fellow, University of Toronto

Co-founder of Pragmatek Transport Innovations Inc.

Applied Scientific Research: From Lab to Field 11

1. Make An Observation

Traffic Congestion: Urban CANCER

0

5

10

15

20

25

0

2

4

6

8

10

12

Jakarta Sydney Chicago Area NY-NJ Cairo Los Angeles Toronto

Pp

op

ula

tio

n in

Mill

ion

s

Co

st in

$ B

illio

ns

Cost of Congestion Population

Applied Scientific Research: From Lab to Field

Traffic Congestion Problem

Tahrir Square, 1914

Tahrir Square, 2014

13

2. Define a Problem Statement and

Formulate a Research Question

Applied Scientific Research: From Lab to Field 15

Applied Scientific Research: From Lab to Field

Traffic Congestion Solutions

More Supply

Less Demand

Intelligence

16

Applied Scientific Research: From Lab to Field 17

Intelligent Transportation Systems

Intelligent Transportation Systems (ITS)

Advanced Travelers Information Systems

Advanced Public Transit Systems

Advanced Traffic Management Systems

Ramp Metering Signal Control Route Divergence

Fixed-Time Control Actuated ControlAdaptive Traffic Signal

Control (ATSC)

Applied Scientific Research: From Lab to Field 18

3. Gather InformationLiterature Review

Applied Scientific Research: From Lab to Field 19

Applied Scientific Research: From Lab to Field 20

Applied Scientific Research: From Lab to Field

Research Database

21

Applied Scientific Research: From Lab to Field

ATSC in North America

22

MARLIN-ATSC: Level 4

Evolution of “Adaptive” Traffic Signal Control

Level 0• Fixed-Time

and Actuated Control

• TRANSYT• 1969, UK

Level 1• Centralized

Control, Off-line Optimization

• SCATS• 1979,

Australia • >50

installations worldwide

Level 2• Centralized

Control, On-line Optimization

• SCOOT• 1981, UK • >170

installations worldwide

Level 3 • Distributed

Control, Model-Based

• OPAC, RHODES• 1992, USA• 5 installations in

USA

Level 4• Distributed

Self-Learning Control

• MARLIN-ATSC• 2011, Canada

23

23

Issues with Leading ATSC Technologies

• Expensive• Not scalable • Not robust

Centralized

• Relying on an accurate traffic modelling framework

• the accuracy of which is questionableModel-Based

• Increasing the complexity of the system exponentially with the increase in the number of intersections/controllers

Curse of Dimensionality

• Requiring highly skilled labour to operate due to their complexity.

Human Intervention

Requirements

24

24

4. Hypothesis Construction

Applied Scientific Research: From Lab to Field

In a Nutshell• Grand objective

– Intersections "talk to each other"

– Each is affected by what is happening upstream

– Each affects what is happening downstream –

– Whole network control in one shot from a grand brain is the dream

• Issue

– Intractable theoretically

– Too complex practically

– Requires massive and very expensive communication

• Solution??

– Decentralized

– Self learning: agents learn to control their local intersection

– Game theory based: agents learn to collaborate

26

5. Define Methodology and Do Experiments

Applied Scientific Research: From Lab to Field

Traffic Signal Control ProblemMarkov Decision Process

28

Current State

Next State ??

Applied Scientific Research: From Lab to Field

Markov Decision ProcessDynamic Programming

29

• The Transition Probability Matrices (TPM)

• (-1)* Delay

•Extend Green (a1)•Switch to Minor Street (a2)

• Queue Length• Short (S1)• Long (S2)

State Action

Transition ProbabilityReward

Queue Length

Optimal Control Policy that

maximizes the expected long-term

rewarda1

a2Curse of Modeling Curse of Dimensionality

Applied Scientific Research: From Lab to Field

Smart / Intelligent Machine?

SMA

RT

can

:

Sense

Interpret

Learn

React

Optimize

Adapt

Help

30

Applied Scientific Research: From Lab to Field

The Most Intelligent Machine Known to Man?

31

Artificial Intelligence MethodReinforcement Learning

Reward (savings in delay)

environment

action

observation (queue Lengths)

MARLIN

32

Q a1 a2

s1 -10 -5

s2 -3 -15

Q Table

Model-Free Algorithms

Model-Based Algorithms

Solution Form

Theoretical Framework

Problem Identification

Stochastic Control Problem

Single Agent (Stationary Environment)

Markov Decision Process

Optimal Control Policy

Dynamic Programming Methods

Reinforcement Learning

Multi-Agent (Non-Stationary Environment)

Stochastic Game

Optimal Joint-Policy

Game Theory Methods

Multi-Agent Reinforcement Learning

Single vs. Multiple Agents:MARLIN Independent vs Integrated33

33

Corridor Synchronization Vs. Network-Wide Coordination

Collaboration with each adjacent intersection in the

neighborhood

Collaboration with each adjacent intersection in the

neighborhood

34

Corridor SynchronizationCorridor Synchronization

34

Applied Scientific Research: From Lab to Field

MARLIN-ATSC Framework

Agent(Learning and

Decision Making)

Environment(Observation and Acting)

Environment-Agent Interface

(Synchronized Interaction)

RL

In

pu

t P

ara

me

ters

RL

In

pu

t P

ara

me

ters

Sig

na

l D

ata

P

ara

me

ters

Sig

na

l D

ata

P

ara

me

ters

35

Applied Scientific Research: From Lab to Field 36

6. Data Collection and Analysis

Applications Simulation Testbeds on Toronto

Applications Simulation Testbeds on Toronto

37

Applied Scientific Research: From Lab to Field

Traffic Simulator

38

Applied Scientific Research: From Lab to Field

Testbed Networks and Applications

Isolated Int.

→ Different RL Design Parameters

5- Int. Network

→ Transferabilityand Different CoordinationApproaches

Large-Scale

→ Scalability and Network-wide

Coordination Effect

39

Applied Scientific Research: From Lab to Field

Isolated IntersectionBay and Front (Downtown Toronto)

Zone 1

Zone 2

Zone 3

Zone 4

0%

5%

10%

15%

20%

25%

1 2 3 4 5 6

Dem

an

d A

rriv

al P

erce

nta

ge

Time Interval (10min)

Uniform Profile For All Phases Phase 1 (NBL-SBL) Profile Phase 2 (NB-SB) Profile Phase 3 (EBL-WBL) Profile Phase 4 (EB-WB) Profile

40

Applied Scientific Research: From Lab to Field

5-Intersections Network

4 Bay & Front

2 University/

York & Front

3 Wellington

& Bay

1 Bay & Lake

Shore

5 Yonge &

Front

4 Bay & Front

2 University/

York & Front

3 Wellington

& Bay

1 Bay & Lake

Shore

5 Yonge &

Front

IN

Wellington & BayIE

Yonge & Front

IW

University/

York & FrontIW

Bay & Lake Shore

IC

Bay &

Front

41

Applied Scientific Research: From Lab to Field

Large-Scale Application in Toronto

42

Applied Scientific Research: From Lab to Field 43

7. Make Conclusions

Applied Scientific Research: From Lab to Field

Large-Scale ApplicationAverage Delay % Improvements

MARLIN-IC vs BCMARLIN-IC vs BC

% ImprovementArea 1

Area 2

Area 3

44

Applied Scientific Research: From Lab to Field

Large-Scale ApplicationAverage Route Travel Time for Selected Routes

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9 10 11 12

Av

era

ge T

ra

vel

Tim

e (

min

)

Time Interval (5 min)

Gardiner EB

BC MARL-TI MARLIN-IC

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)

Time Interval (5 min)

LakeShore EB to Spadina NB

BC MARL-TI MARLIN-IC

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)

Time Interval (5 min)

LakeShore EB to Spadina NB

BC MARL-TI MARLIN-IC0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)

Time Interval (5 min)

LakeShore EB to Spadina NB

BC MARL-TI MARLIN-IC0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)

Time Interval (5 min)

LakeShore EB to Spadina NB

BC MARL-TI MARLIN-IC

45

Hardware-in-the-Loop Simulation (HILS)

Hardware-in-the-Loop Simulation (HILS)

Samah El-Tantawy Kasra Rezaee Hossam Abdelgawad Tamer Abdulazim Baher Abdulhai

Applied Scientific Research: From Lab to Field

Controller Interface Device(CID)RS485 to USB

Traffic Signal Controller

RS485 -SDLC protocol

USB -SDLC protocol

Industrial Computer

Ethernet -NTCIP protocol

Paramics Modeller

MARLIN-HILS Architecture

47

Applied Scientific Research: From Lab to Field 48

8. Communicate Results

Applied Scientific Research: From Lab to Field

Writing a Research Proposal/ Publishing a Paper

7/23/2018 49

Applied Scientific Research: From Lab to Field 50

Applied Scientific Research: From Lab to Field

MARLIN RecognitionIEEE ITSS Best Ph.D. Dissertation Award, 2013

Canadian Institute of Transportation Engineers Scholarship, 2010

INFORMS George B. DantzigDissertation Award, 2013

ITS Canada Scholarship, 2011

51

Applied Scientific Research: From Lab to Field

MARLIN Featured in Media!

52

52

Towards System Implementation Towards System Implementation

53

MARLIN Field Implementation

Ethernet

Switch

Industrial Computer

running MARLIN Traffic Controller

54

Lessons Learned for Future Consideration

Applied Scientific Research: From Lab to Field 56

Applied Scientific Research: From Lab to Field

IV. When to Start?

57

Applied Scientific Research: From Lab to Field 58

Applied Scientific Research: From Lab to Field

IV. What do you need to know before starting your research?

7/23/2018 59

Applied Scientific Research: From Lab to Field7/23/2018 60

Applied Scientific Research: From Lab to Field7/23/2018 61

Applied Scientific Research: From Lab to Field

Lessons Learned

62

• Research Never Ends– Stay on Top of the Literature

• Passion and Handwork are the Keys

• Adapting toYour Audience Academia Vs. Industry Vs. Government

“If Opportunity Doesn’t Knock, Build A Door “ Milton Berle

Being Out of Your Comfort Zone

MARLIN is One Piece of the Puzzle

Samah El-Tantawy, PhDsamah.elshafiey@gmail.com

Assistant ProfessorFaculty of Engineering, Cairo University

27/07/2017