Project Report: Road Pothole Classification and Reporting with

Data Quality Estimates

Ayush Rajesh Vora | arv1924@rit.edu

Project Advisor : Prof. Leonid Reznik | lr@cs.rit.edu

Table of Contents:

Sections Description Page No.

1. Abstract 2

2. Introduction 2

3. Background a. Real-time detection using smartphone accelerometers b. Pothole Detection and Warning System using Wireless Sensor

Networks c. Existing Smartphone Apps

3-5

4. Requirement Analysis 6 5. Proposed Solution 7

6. Approach and System Explained a. Literature Review b. Android Smartphone Apllication – Pothole Reporter

i. User Interfaces ii. Output

c. Intelligent System - Pothole Priority Rule Engine. d. Intelligent System – Data Quality Rule Engine. e. Purpose and Implementation of the Database connection

8-17

7. Results 18

8. Observations 19

9. Conclusion 20 10. Future Work 21

11. References 22

. 2

1. Abstract:

This research discusses a solution to report, record and classify potholes based on factors

that influence the destructiveness of the pothole, by using a widely popular platform in the

current generation. Harnessing the power of the mobile computing platform has opened up

newer possibilities of gathering and classifying data by leveraging the use of crowd-

sourcing. In the current generation that is being dominated by the mobile computing

platform namely smartphones, crowd-sourcing can be achieved in a relatively hassle-free

yet effective means of collecting data from a large source-pool, viz, people. Tapping into

this source-pool however has a drawback of data reliability. The approach used in this

project aims to realize the most influential factors related to destructiveness of potholes

that are encountered on roadways, while supplementing the data with quality estimates

derived from the completeness of the data and certain factors of the input device itself that

is involved in the data collection process, thereby addressing the drawback of data

reliability.

2. Introduction

One of the most notable aspects of the development of human civilizations involve the

advent of roadways. Roadways have allowed civilizations to exchange goods and services

and have provided civilizations with the necessary mobility required for survival. The

problem of maintenance of these roadways has persisted to this date and as time has

progressed this problem has only become more complex and significant.

The problem of road-maintenance has particularly gained immense significance with the

advent of vehicular transportation. The roadways have become highly complex road-

networks spanning larger and larger areas. Invention of the internal combustion engine

. 3

boosted human civilization exponentially, but with it the matter of road maintenance

became extremely vital to minimize the threat of vehicular accidents.

One particular problem of road maintenance involves the gradual emergence of deep

cavities on pavements and roads due to natural processes. These deep cavities and

depressions are potholes which have been a recurrent and inevitable problem for road

maintenance. They are the results of general wear and tear and the weather cycle. In the

modern era potholes generally tend to be bowl-shaped openings in the road and their

depths can go up to and above 10 inches in the most severely ignored cases. Modern roads

have a concrete base and a top layer made of asphalt and the top asphalt layer can wear

quickly exposing the concrete base. This aggravates due to rain-water, snow etc

accumulating in these cavities and results into big potholes that may eventually cause

damage and car accidents.

Poor maintenance can lead to numerous damages that range from vehicular damage to loss

of human lives. Since this is an inevitable problem and they are difficult to tackle

preemptively, there is an urgent requirement of having a classification scheme that would

prioritize potholes as per their destructiveness. Once prioritized their repair can take place

in a systematic manner, thereby reducing the damage they can cause.

3. Background:

Using smartphones for reporting general infrastructure problems is recently emerging and

mostly within the stages of its infancy. A wide range of solutions exist that involve Real-

time detection using smartphone accelerometers[1], Pothole Detection and Warning System

using Wireless Sensor Networks[2], smartphone apps which record the geo-location of

potholes along with some physical characteristics[4] etc.. The project research included here

will highlight some of the existing solutions, with the advantages they offer and the

drawbacks that exist for those particular solutions.

. 4

a. Real-time detection using smartphone accelerometers:

This research tries to harness the embedded accelerometers within smartphones to

detect potholes in real time, rather than using external accelerometers and external

gps devices. They aim to provide a solution which utilizes more advanced heuristic

real-time event detection using limited hardware and software resources. They

followed the approach from previous researches like PotholePatrol[3] which

threshold the acceleration amplitudes at Z-axis. It uses values exceeding specific

thresholds that identify the type of pothole( i.e small, medium and large) as features

that classify the measurements. Instead of using virtual re-orientation of the

accelerometer, they follow a simpler approach by fixing the placement of the

accelerometer and eliminating the need for virtual re-orientation. They use various

algorithms like Z-Thresh and Z-diff to detect pothole events.

Advantages

o Potholes are detected in real-time without user-intervention.

o System could detect larger potholes at medium vehicular speeds with a fairly

decent true-positive rate.

o System can run on a popular platform like Android.

Disadvantages

o Continuous use of accelerometers tend to drain a lot of battery power on

smartphone devices.

o Does not provide a detailed description of a pothole other than some loose

physical aspects.

o Lacks the contextual information that can put the significance of the pothole

within a different perspective.

o System does not have a relatively considerable true-positive rate with smaller

potholes and/or at low vehicular speeds.

. 5

b. Pothole Detection and Warning System using Wireless Sensor Networks

This research tries to propose the use of devices like embedded sensors like

accelerometers within the vehicle, embedded communication subsystem and

external Wi-Fi Access Points, to detect and warn vehicular commuters about

impending potholes. It proposes to use a Mobile Node (MN) within the vehicle

which does the job of sensing the pothole, transmitting certain pothole data to a Wi-

Fi Access Point (AP) and receiving localized information on a pothole from an AP.

Advantages

o Dedicated sensors, and communication devices make detection and warning a

lot more accurate.

o Can work well as a good detection and warning system.

o Does not involve user-intervention.

Disadvantages

o Expensive to install dedicated sensors and communication devices

o Does nothing to address the problem of classification of potholes for scheduling

repair.

o Lacks the contextual information that can put the significance of the pothole

within a different perspective.

c. Existing Smartphone Apps

There are existing smartphone apps like Commonwealth Connect and iReport, which

also harnesses crowd-sourcing. They address the problem of reporting potholes to

authorities. However they are reported in the context of general infrastructure

problems and don't do much to provide a detailed description or any contextual

information of the pothole.

. 6

4. Requirement Analysis:

After analyzing some of the drawbacks of existing solutions, we realize the requirement of

having accurate and descriptive data related to potholes. This project aims at realizing some

general and universal policies related to classifying potholes via crowd-sourcing. Some of

the requirements to achieve this were recognized below.

a. Determining the most appropriate influencing factors that would be able to signify the

destructiveness of a pothole so it can be assigned a priority.

b. Providing vehicle commuters or concerned citizens a quick and efficient way to report

pothole data related to factors that influence its priority and at the same time extracting

some contextual information if possible from users so as to assist the effort of

scheduling the repair of said pothole.

c. After extracting the information, using a progressive intelligent system to determine a

calculated priority for the for the pothole.

d. Estimating the accuracy of the data that is generated using some factors which are

derived from the completeness of the data and certain factors of the input device itself

that is involved in the data collection process.

e. Sending the information to a database where, it can be analyzed and used for

scheduling pothole repair.

These requirements address the drawbacks of, lack of descriptive data related to potholes,

provides information on accuracy of the extracted information, provides some contextual

information related to the pothole, ensures that proposed solution does not drain a

significant amount of smartphone resources like battery life etc. It does this, while

sacrificing on the required level of user-intervention, which needs to be high as per nature

of crowd-sourcing applications.

. 7

5. Proposed Solution

The proposed solution follows the approach below to address the requirements.

a. Performing a literature review to gather domain knowledge on roadway maintenance

systems, by going through various roadway maintenance policies, protocols and

recognizing the most significant factors related to the pothole, the patterns and rules to

establish a general rule-base for classifying a pothole based on its destructiveness,

described by the most influential factors.

b. Designing and implementing a smartphone application on the popular platform Android.

Establishing a user-friendly UI which assists the user to recognize the qualities of the

pothole and requesting the user to provide some contextual information using visual

cues and a survey-type activity.

c. Designing and implementing the intelligent system which involves balancing the

significance of various factors that influence the priority of the pothole and calculating

the priority while taking into consideration the contextual information provided by the

user.

d. Calculate the completeness ratio (i.e amount of information that is received through the

user input / amount of total information requested), determine camera quality of the

android smartphone, and the current security characteristics of the android smartphone

to estimate the data quality.

e. Use the Android-PHP-MySQL approach to store and record the data in a MySQL

database. This MySQL database is maintained by a separate project which will provide

additional features like lookup and analysis of potholes and identify trends.

This project is iterative in nature i.e, it is made extensible to incorporate newer factors and

context influencing the potholes priority which is based on its destructiveness. All the above

steps can be repeated to improve the system.

. 8

6. Approach and System Explained:

a. Literature Review: After reading through many online road maintenance policy

and protocol documents from various counties a common pattern was observed.

Factors like the type of street, weather, depth and location were mentioned

within all protocols. The main purpose of the literature review was to gather

information on how different road maintenance authorities and offices react to

various contextual conditions and physical characteristics of potholes. Some of

these factors are taken into consideration for the intelligent system to assign a

priority to the pothole. The research arrived and finalizes the following factors.

. 9

Pothole Data reflecting the significantly influential factors

i. Type Of Street

ii. Current Weather Conditions

iii. Depth of pothole

iv. Traffic Volume

v. Area of pothole

vi. Average Speed

vii. Safety Zone Rating

viii. Type of Junction

ix. Estimated location(PPRRW)

x. Road Material

xi. Existing Repair Conditions

xii. Type of pothole

xiii. GPS location

xiv. Image

xv. Priority

To determine an estimate of the accuracy of the data, the research arrived to the

following factors.

Data reflecting the quality of input

i. Camera Quality

ii. Data Completeness

iii. Security Characteristics

b. Android Smartphone Application:

. 10

i. User Interface :-

The smartphone application was designed keeping in mind the willingness

of users to input the requested information. Since this is based on a crowd-

sourcing approach, this solution works on the assumption that a

considerable amount of users participate. In order to make users

participate in surveying the potholes, the UI needed to be made as hassle-

free as possible.

Figure 2.a. Picture & Location Extraction

Capturing the image and enabling locations services is encouraged and is

the first suggestion made to the user, since this is one of the most

important information for an official that might use this data in the

database to decide on scheduling repair of potholes.

. 11

Figure 2.b. Dimension Extraction

The next important information that needed to be captured is the

dimensions of the potholes. The area, depth is captured using an Android

Number picker, the limits are set to values that are practically observed

and keeping in mind the average width of roads in the US.

. 12

Figure 2.c. Pothole Type Extraction

Another physical characteristic that needed to be captured was the type of

pothole. Visual cues are used to get this information by using a custom UI

component that uses a "SelectOneImage" approach, to help users choose

which type is most applicable to the pothole they want to report. This

information may help officials identify what approach they could follow in

performing repair ( For example, what tools, materials may be necessary

and help to cut costs.)

. 13

Figure 2.c. Survey Activity

The next section of the activities comprises of a survey for gathering

contextual information that helps the intelligent system to assess the

priority of the pothole being reported.

This activity and the models that back it are kept generic so as to

incorporate newer contextual factors.

The survey activity is unique, follows a fixed choice question style survey

and uses a generic framework, wherein new questions and new choices

can be easily defined and gets automatically incorporated into the UI.

The choices are fixed, so as to avoid inaccurate information being recorded.

. 14

ii. Output :-

i. Once the survey activity is complete, the pothole data rule engine

assesses the priority of the pothole being reported and assigns a

priority score and the pothole data quality rule engine assesses

the quality of the input and assigns a quality score.

ii. All the physical characteristics, contextual information that

influence the priority of the pothole along with the pothole

priority score as well as the data quality score is sent to a separate

database project that maintains this data and officials can use this

database for scheduling repair or analyzing the reported data.

iii. The app is divided into different types of activities. The

architecture allows different aspects of pothole related data to be

extracted through different types of UI elements because the

nature of the information may differ, while at the same time be

necessary for arriving at accurate results.

iv. The app does not force the user to input all the information,

respecting different levels of participation. However they are

encouraged to do so. Depending on the level of participation the

data quality rule engine determines data completeness which

plays the biggest role in calculating data quality score.

. 15

c. Intelligent System - Pothole Priority Rule Engine:

Figure 3.a. Pothole Priority Rule Engine

o Priority is scored within 0-100 pts to determine a suitable priority class.

o All fields given a fixed weight and together sum to a 100 pts.

o All values of each field assigned a numeric value between (0-1).

o Rule Engine is a forward chained rule-based hierarchical tree and

evaluates each prioritized field in order and determines if the next field’s

weight needs to be adjusted based on its current value.

o If weight is adjusted then weights of all fields down the hierarchy are re-

adjusted accordingly, so that maximum priority sum does not change.

o It is important to note that, some fields which are partially dependent on

the previous fields require a weight adjustment, while some others don’t.

. 16

d. Intelligent System - Data Quality Rule Engine:

Figure 3.b. Data Quality Rule Engine

Data quality is determined in a similar way as priority, taking into consideration the

data completeness, camera quality and security in that order. No dependencies

between the factors seem to be obvious hence weight adjustments are not

considered in this model.

o Quality scored between 0-100.

o All factors given fixed weights.

o Completeness rating determined based on the number of questions answered by

the user to the number of answers requested by the app.

o Camera quality rating determined through existing android API's

o Security rating determined using the security rule engine from a previously

developed application called Android Security Evaluator.

o After determining rating for each factor it is multiplied with each factor's weight

and final weighted sum gives the calculated data quality.

. 17

e. Purpose and Implementation of the Database connection:

Once all the information is acquired from the user, it needed to be stored into a

database so that it can be used by the officials for scheduling repair of potholes and

even for further analysis of the recorded data.

The intention is to analyze the data and detect patterns such as regions which are

extremely prone to specific types of potholes, or specific sized potholes, or how the

weather conditions affect potholes in a specific region as opposed to how similar

weather conditions affect pothole severity in a different regions.

Once adequate amount of data gets populated in the database it can exponentially

increase the ability to answer many interesting questions and ultimately help in the

effort of pre-emptive action against pothole related maintenance.

An Android-PHP-MySQL approach is used for the implementation. The Android

smartphone application converts all the data related to the pothole into a JSON

object and sends it to the Apache Web Server where a PHP file receives the JSON

object and inserts it into the MySQL database. This database is maintained by a

separate project.

. 18

7. Result:

Table 1. Pothole Instances Table

Factors Instance 1 Instance 2 Instance 3 Instance 4

Weather Type Snowy Sunny Sunny Rainy

Street Type Arterial Commercial Alleyway Commercial

Depth 16 cm 12 cm 4 cm 8 cm

Area 20 sq.ft 12 sq.ft 1 sq.ft 12 sq.ft

Traffic Volume Extremely High Medium Extremely Low Medium

Average Speed Extremely High High Extremely Low High

Safety Zone Type School Zone None None School Zone

Junction Intersection T-Junction None Intersection

Relative Position Center Side-Center Sides Side-Center

Road Material Asphalt Asphalt Concrete Asphalt

Repair Permanent/None Temporary Temporary Permanent/None

Priority Extremely High Medium Extremely Low High

A lot of different sets of inputs have been tested and compared. To better contrast the

results three input instances have been isolated to better help explain the priority score

results and draw interesting observations.

Interpretation of Priority Score :-

Instance 1: The priority score for this set of inputs was 98.3%. The priority class assigned

to this instance evaluates to Extremely High, indicating that this is a one of the most

significant potholes in the region. It must be immediately looked at and either requires a

temporary repair (depending on weather conditions) or a permanent repair (if the

weather permits and if temporary repair may have already been performed). Neglecting

this pothole may even result in the loss of lives.

Instance 2: The priority score for this set of inputs was 64.31%, The priority class

assigned to this instance evaluates to Medium, indicating that although this pothole

. 19

may be a nuisance, it is not likely to cause an enormous amount of damage. Pothole

must be repaired soon to provide commuters a good experience.

Instance 3: The priority score for this set of input was 24.23%. The priority class assigned

to this instance evaluates to Extremely Low, indicating that this pothole is not in any

way an immediate danger, it may be within an Alleyway or parking space, where vehicle

speeds are slow enough to not cause any considerable damage to either people or

vehicles. It must be repaired after all important potholes in the region have been

attended to.

Instance 4: The priority score for this set of inputs was 73.59%. The priority class

assigned to this instance evaluates to High, indicating that this pothole may be a

dangerous pothole owing either to its size or owing to the contextual significance of this

pothole. It should be repaired as soon as possible and shouldn't be neglected and has

the potential to cause some serious damage

8. Observations:

The observations that were made after looking at the results of each pothole instance are

as follows

Instance 1: We see that if weather conditions are snowy and the type of the street where

the pothole was observed is an Arterial street, where a considerably size pothole has

appeared, with extremely high volumes of traffic and extremely high speed traffic, is almost

certainly a very high priority pothole and the system answers this obvious question by

giving an obvious answer based on the significance of its physical characteristics as well as

the contextual information.

Instance 2: We see that if the general weather conditions are sunny and clear, the type of

the street is commercial , with a fairly deep and big pothole, with medium volume of traffic

. 20

and high speed traffic, indicates a fairly common pothole scenario and prioritized to be a

pothole of medium significance.

Instance 3: We see that if general weather conditions are sunny and clear, the type of street

may be an alleyway or parking space, with a very small pothole, with extremely low volume

of traffic and extremely low speed traffic, indicates the least significant kind of potholes but

this shouldn't diminish the effort to report these since they are still a nuisance.

Instance 4: This is the instance where we can clearly see the contextual information at work

in raising the significance of a pothole on characteristics other than physical. We see that

the difference between Instance 2 and Instance 4 is the weather being rainy, safety zone

type being a school zone, and the road being an intersection, which raises its priority score

and evaluates to a High priority pothole even though Instance 4 reflects a smaller pothole

with regards to its physical dimensions like depth.

9. Conclusion:

There were a lot of things to consider for making this smartphone application to be viable.

o Some dependencies of the fields needed to be adjusted so as to reflect a more

accurate model.

o Field Weights were calibrated adequately so that outliers were not produced.

o Tests showed that system prioritized potholes as expected for a lot of instances.

o We see that contextual information can play an important role in determining a

potholes significance, this is important because it has been consistently observed

that context plays an important role in understanding what amount of damage

the pothole in question maybe able to inflict.

o The system manages to weigh contextual information in a fairly balanced

manner.

. 21

o Adaptive and relative weights help to address the issue of devising a fair

and balanced intelligent system where each field is given required

attention

Some other conclusions that are important to note are :-

o The Android Smartphone application UI made it significantly easier to gather

data.

o There is no universally accepted guide or policy existing to prioritize

potholes and this project tries to realize and establish this.

o Baseline research and literature review was extremely beneficial for the

intelligent system as it provided lot of insight in developing a balanced IS.

10. Future Work:

Since this project is based on an iterative process, improving the amount of involved

fields, the intelligent system by tweaking the field weights and updating the database

will improve the contextual accuracy of the system.

Essentially in the future there should be more user types and notifications for users

when the pothole gets repaired because of their report.

One of the major intention of this research project is to analyze the data and detect

patterns such as regions which are extremely prone to specific types of potholes, or

specific sized potholes, or how the weather conditions affect potholes in a specific

region as opposed to how similar weather conditions affect pothole severity in a

different regions.

Also as the database populates, the comparison between the determined priority and

the actually assigned priority should be used for improving the IS.

. 22

Once adequate amount of data gets populated in the database it can exponentially

increase the ability to answer many interesting questions and ultimately help in the

effort of pre-emptive action against pothole related maintenance.

11. References:

[1] Artis Mednis, Girts Strazdins, Reinholds Zviedris, Leo Selavo, Real-time detection using smartphone

accelerometers, Conference: 2011 7th IEEE International Conference on Distributed Computing in Sensor

Systems and Workshops (DCOSS).

[2] Sudarshan S Rode, Shonil Vijay, Prakhar Goyal, Purushottam Kulkarni, Pothole Detection and Warning

System using Wireless Sensor Networks,Kavi Arya Embedded Real-Time Systems Laboratory Indian Institute of

Technology Bombay.

[3] Jakob Eriksson, Lewis Girod, Bret Hull, Ryan Newton, Samuel Madden, Hari Balakrishnan, The Pothole

Patrol: Using a Mobile Sensor Network for Road Surface Monitoring, MobiSys’08, June 17–20, 2008,

Breckenridge, Colorado, USA.

[4] http://commonwealthconnect.io/

[5] http://www.krdo.com/news/local-news/pothole-patrol-how-does-the-city-prioritize-pothole-

repairs_20160715115016223/35612995

[6] http://stewartvillemn.com/media/Policy-Pothole-Repair.pdf

[7] http://www.johnsoncitytn.com/publicworks/street/pothole/

[8] http://www.canoncity.org/Departments/Engineering/StreetsProjectList/2016_Street_Info/Pothole%2

0Repair%20Policy.pdf

[9] https://www.southampton.gov.uk/roads-parking/road-maintenance/

[10] http://blog.nola.com/potholepatrol/2008/07/know_your_potholes.html

. 23