View
223
Download
2
Category
Tags:
Preview:
Citation preview
Specification-Based Error Localization
Brian DemskyMartin Rinard
Laboratory for Computer ScienceMassachusetts Institute of Technology
Problem
Error Introduced
Execution with
Broken Data
Structure
Crash or Unexpected
Result
• Have to trace symptom back to cause• Corruption may not cause visible
error in test suite
Problem
Error Introduced
Execution with
Broken Data
Structure
Crash or Unexpected
Result
• Solution: discover bugs when • they corrupt data• not when effect becomes visible
• Perform frequent consistency checks• Bug localized between
• first unsuccessful check and• last successful check
Architecture
next
value
value
o1
o1
o1
o1
values
nodes
Concrete Data Structure Abstract Model
Model DefinitionRules
Consistency Constraints
Architecture RationaleWhy use the abstract model?
• Model construction separates objects into sets• Reachability properties• Field values
• Different constraints for objects in different sets• Appropriate division of complexity
• Data structure representation complexity encapsulated in model construction rules
• Consistency property complexity encapsulated in (clean, uniform) model constraint language
Simplified Freeciv Example
tile grid[EDGE][EDGE];structure tile {
int terrain;city *city;
} structure city {
int population;}
PO MM
OO MP
PO MM
PP MP
Terrain Grid
City Structures
O = OceanP = PlainM = Mountain
Sets and Relations in Model
• Sets of objectsset TILE of tilegrid;set CITY of city;
• Relations between objects – values of object fields, referencing relationships between objectsrelation CITYMAP : TILE -> CITY;relation TERRAIN : TILE -> integer;
Model TranslationBits translated to sets and relations in abstract
model using statements of the form:
Quantifiers, Condition Inclusion Constraint
for x in 0..EDGE*EDGE, true grid[x] in TILE
for t in TILE, true t,t.terrain in TERRAIN
for t in TILE, !t.city = NULL t,t.city in CITYMAPfor t in TILE, !t.city=NULL t.city in CITY
Model in Examplegrid[0]
grid[1]
grid[2]
grid[3]
terrain:
1 2 3 4
city: NULL NULLpopulation:
10,000
grid[0]
grid[1]
grid[2]
grid[3]
Tiles
1234
city
Cities
Consistency PropertiesQuantifiers, Body
• Body is first-order property of basic propositions• Inequality constraints on numeric fields • Cardinality constraints on sizes of sets• Referencing relationships for each object• Set and relation inclusion constraints
• Example:for t in TILE, MIN <= t.TERRAIN and t.TERRAIN<=MAXfor c in CITY, size(CITYMAP.c)=1for c in CITY, !(CITYMAP.c).TERRAIN=OCEAN
Consistency ViolationsEvaluate consistency properties, find
violationsfor c in CITY, size(CITYMAP.c)=1
grid[0]
grid[1]
grid[2]
grid[3]
Tiles
1234
city
Cities
Slide about checks
• TODO
Optimized Implementation• Compilation (4.7x speedup)• Fixed point elimination (210x speedup)
• Evaluate model definition rules using simple traversal• Relation construction elimination (500x speedup)
• Evaluate uses of relations directly on data structures• Set construction elimination (3900x speedup)
• Evaluate constraints while traversing data structures• Bottom line
• Interpreted version X times slower than uninstrumented
• Optimized version Y times slower than uninstrumented
Freeciv Case Study
• Multiplayer Client/Server based online game
• Available at www.freeciv.org• Case study looked at the server• Server contains 73,000 lines of code• Added 750 instrumented sites• 20,000 consistency checks performed in
our sample execution
Case Study
• Created three buggy version of Freeciv• Two groups of three developers
• One used conventional tools• One used specification-based
consistency checking• Each participant was asked to spend at
least one hour on each version• Both populations given a pre-
instrumented version of Freeciv
Consistency Properties• Map exists
size(MAP)=1• Grid of tiles exists
size(GRID)=1• Tiles have valid terrain values
for t in TILE, MIN <= t.TERRAIN and t.TERRAIN<=MAX
• Cities are not in the oceanfor c in CITY, !(CITYMAP.c).TERRAIN=OCEAN
• Each city has exactly one reference from the gridfor c in CITY, size(CITYMAP.c)=1
Bugs Introduced
• Actual errors in buggy versions• First error creates invalid terrain values
(violates valid terrain property)• Second causes two tiles to refer to the
same city (violates single reference property)
• Third causes a city to be placed on ocean (violates cities not in ocean property)
Results
• User study shows benefit from approach• With tool
• All developers found and fixed all bugs• Mean of 11 minutes required
• Without tool• Three developers found total of one
bug (out of nine developer/bug combinations)
• Others spent hour debugging (unsuccessfully)
Repair for Deployed Systems
• Consistency specifications for repair• Input: inconsistent data structure• Output: consistent data structure• Technique enables programs to recover
from data structure corruption• And continue to execute successfully
• OOPSLA ’03 paper describes this technique
Full reference here
Related Work
• Specification languages such as UML or Alloy
• Specification-based testing • Korat (Boyapati et. al. ISSTA 2002)• Testera (Marinov and Khurshid)• Eiffel (Meyer)
• Invariant inference and checking• Daikon (Ernst et. al.)• DIDUCE (Hangal and Lam)• Carrot (Pytlik et. Al.)
Conclusion
• Consistency checking to localize data structure corruption bugs
• Good experimental results• With checker, bugs fixed in minutes• Without checker, bugs not fixed in an
hour• Optimizations for good performance• Data structure repair
Recommended