Adarza TechnologiesAdarza TechnologiesBacteria Classification Bacteria Classification AssistantAssistantENSC 440 Presentation
April 19th, 2005
Members:Edward Goh Linda WuJosna RaoGeeda Ip
© Adarza Technologies 2005
Agenda for TodayAgenda for Today• Introduction
• System Overview
• Data Acquisition
• Image Processing
• Data Filtering
• User Interface
• Testing Procedure
• Cost Analysis
• Future Features
• Conclusion
• Acknowledgements
• Q&A
• Demonstration
© Adarza Technologies 2005
Company ProfileCompany Profile
• Adarza Technologies– Edward Goh (Chief Executive Officer)– Linda Wu (Chief Operations Officer and Chief
Financial Officer)– Josna Rao (Chief Technology Officer)– Geeda Ip (Chief Information Officer)
© Adarza Technologies 2005
IntroductionIntroduction
• The Bacteria Classification Assistant– Helps user identify bacteria– Assists with training new technicians
• Technologically Relevant– microbiology field– Applications – ID microscopic objects
© Adarza Technologies 2005
System OverviewSystem Overview
• .NET framework with Open Computer Vision Library
• Microsoft Access - Database
• Windows Forms - GUI
• Accepts JPEG images from hardware
• Returns subset of database
• Classification Criteria:– Colour– Size– Boundary Shape– Colony Shape
© Adarza Technologies 2005
Data AcquisitionData Acquisition
• Digital Camera Microscope
• Camera - Canon Powershot G5 with Scopetronix MaxView Plus accessories for the microscope
• Microscope - Olympus VMT fitted with 1X, 2X objectices
© Adarza Technologies 2005
Image Processing AgendaImage Processing Agenda
• Preprocessing Modules– Edge– Edge sequence– Centre Finding
• CC Processing Modules– Size– Colour– Boundary Shape– Colony Shape
© Adarza Technologies 2005
Pre-processingPre-processing
• Features that are not specifically classification criteria
• Edge detection
• Edge sequencing
• Centre detection
© Adarza Technologies 2005
Edge ProcessingEdge Processing
• Returns image of colony in white, on black background
• Method:– Split image into red, green, and blue planes– Remove noise– Find edges in each plane by calculating
gradient and setting threshold– Adjust threshold until contour closes– Floodfill colony white and background black
© Adarza Technologies 2005
Edge ExamplesEdge ExamplesNeisseria gonorrhoeae
• Transparent Edge
• No distinct colour difference
© Adarza Technologies 2005
Edge SequencingEdge Sequencing
• Detects contours
• Sorts and orders the points from contours
• Returns an OpenCV sequence object
• Used to determine boundary shape and colony shape
© Adarza Technologies 2005
Centre Finding ModuleCentre Finding Module
• Centroid equations
• Returns cvPoint
• Relies on edge image
• Centre used for boundary and colony shape
© Adarza Technologies 2005
Size ModuleSize Module
• Goal: 4 Categories– Tiny, Small, Medium, Large
• Comparison based on 2D area
• Uses library function to count non-zero pixels
• Requires monochrome version of sample and calibration images
• Ratio of calibration and sample area taken
© Adarza Technologies 2005
Colour ModuleColour Module
• Goal: 7 Categories– Non-pigmented: White, Gray, White-Yellow-Gray, Red– Pigmented: Blue, Green, Yellow, Purple
• Returns two values: primary and secondary– Primary: most likely– Secondary: for borderline cases
• Method: analyze image in Hue-Saturation-Brightness channels– Search for local maxima
• Some examples
© Adarza Technologies 2005
Hue-Saturation-BrightnessHue-Saturation-Brightness
• Hue: 0° ~ 360°
• Saturation: 0% ~ 100%
• Brightness: 0% ~ 100%
© Adarza Technologies 2005
Colour Module MethodColour Module Method
• Image separated into 3 planes
Hue Plane Saturation Plane Value Plane
© Adarza Technologies 2005
Colour Module MethodColour Module Method
• Each plane analyzed with mask
Hue Plane Saturation Plane Value Plane
100 184239
© Adarza Technologies 2005
Colour ModuleColour Module
• Non-pigmented Colonies
Red - Transparent
White-Yellow-Gray
GrayWhite
© Adarza Technologies 2005
Colour ModuleColour Module
• Test Cases1. Strongly pigmented colonies
– Ex: Chryseomonas luteola
2. Very white colonies– Ex: Stomatococcus mucilaginosis
3. Borderline cases– Relies on secondary colour for identification– Relax contraints
© Adarza Technologies 2005
Colony Shape ModuleColony Shape Module
• Goal: Classify colony as Circular or Irregular
• Method:– Histogram of Radial Lengths
• Circular:– Single large peak
• Irregular:– Multiple peaks of lower height
© Adarza Technologies 2005
Colony Shape ModuleColony Shape Module
• Histogram Analysis:– Locate max peak in histogram– Evaluate average height of surrounding columns– Use threshold to determine if height great enough to
be circular
Length of radial vector
Number of edge points
© Adarza Technologies 2005
Colony Shape ModuleColony Shape Module
• Test Cases:1. Colonies of different sizes:
– Small number of edge points: Passed(correct classifier returned)
– Large number of edge points: Passed
2. Different edge cases:– Washout edges: Passed (always irregular)
© Adarza Technologies 2005
Boundary Shape ModuleBoundary Shape Module
• Goal: 4 Categories– Smooth, Undulating, or Finger-like
• Method of edge angles– Histogram of Edge Angles– Difference between tangent angle
and normal angle
• Smooth– 0°
• Undulating– Some at 0°
• Fingerlike– Widely varying from 0° to 90°
© Adarza Technologies 2005
Boundary Shape ModuleBoundary Shape Module
• Histogram Analysis:– Evaluate average height of bins around 0°– Evaluate max peak height and location– Evaluate average height of bins around 90°– Combination of 4 major thresholds
Edge Angle0° 90 °
Number of edge points
10%
© Adarza Technologies 2005
Boundary Shape ModuleBoundary Shape Module
• Reliability:– Borderline cases:
• Certain colonies have characteristics of more than one shape type
• Edge may also be unreliable, perhaps due to blurry colony
– Solution:• Peaks closer to thresholds in number and angle
are given two shape classifiers• Both classifiers used to filter results from database
© Adarza Technologies 2005
Boundary Shape ModuleBoundary Shape Module
• Test Cases:1. Colonies of different sizes: Passed2. Different edge cases:
– Washout edges: Passed (always finger-like)– Blurry edges: Two shape classifiers returned in
85% of tests
3. Colonies of different shapes:– Circular: Passed– Irregular: Passed– Highly irregular (length more than twice width):
– Leads to some unreliability, often classified as finger-like
© Adarza Technologies 2005
DatabaseDatabase
• Introduction
• Technology – Microsoft Access 2000– ADO.NET
• Considerations in design:– Normalization– Size– Portability– Adaptability
© Adarza Technologies 2005
Database Structure OverviewDatabase Structure Overview
PhotoTable
PK PHOTOINDEX
PHOTOFILEPATH CALIBRATIONINDEX
SizeTable
PK SIZEINDEX
SIZENAME
BoundaryShapeTable
PK BOUNDARYSHAPEINDEX
BOUNDARYSHAPENAME1 m
LightingCondTable
PK LIGHTCONDINDEX
LIGHTCONDDESC
1
m
ColourTable
PK COLOURINDEX
COLOURNAME ISPIGMENTATION
BacteriaPictureTable
PK BPINDEX
NAME PHOTOINDEX ISREFERENCEDATA BCAVERSION BOUNDARYSHAPEINDEX COLONYSHAPEINDEX COLOURINDEX SIZEINDEX ISGRAMPOSITIVE AGARMEDIUM SMELL TEXTURE CONTAMINATION INCUBTEMP INCUBTIME INCUBATMOSCOND IDCODE COMMENTS UPPERLEFT LOWERRIGHT INSIDEUPPERLEFT INSIDELOWERRIGHT
m
m
m
1
1
1
ColonyShapeTable
PK COLONYSHAPEINDEX
COLONYSHAPENAME
m 1
CalibrationTable
PK CALIBRATIONINDEX
CALIBFILEPATH LIGHTCONDINDEX
m
1
© Adarza Technologies 2005
Bacteria Picture TableBacteria Picture TableBacteriaPictureTable
PK BPINDEX
NAME PHOTOINDEX ISREFERENCEDATA BCAVERSION BOUNDARYSHAPEINDEX COLONYSHAPEINDEX COLOURINDEX SIZEINDEX ISGRAMPOSITIVE AGARMEDIUM SMELL TEXTURE CONTAMINATION INCUBTEMP INCUBTIME INCUBATMOSCOND IDCODE COMMENTS UPPERLEFT LOWERRIGHT INSIDEUPPERLEFT INSIDELOWERRIGHT
• Where the main data for each record is stored.
• Can store reference or sample entries
• BCAVERSION field
© Adarza Technologies 2005
Boundary Shape TableBoundary Shape Table
BoundaryShapeTable
PK BOUNDARYSHAPEINDEX
BOUNDARYSHAPENAME
• Structure:
• Sample Data:
BOUNDARYSHAPEINDEX BOUNDARYSHAPENAME1 Smooth2 Undulating3 Finger-like
© Adarza Technologies 2005
Colony Shape TableColony Shape Table
• Structure:
• Sample Data:
ColonyShapeTable
PK COLONYSHAPEINDEX
COLONYSHAPENAME
COLONYSHAPEINDEX COLONYSHAPENAME1 Circular2 Irregular
© Adarza Technologies 2005
Colour TableColour Table• Structure:
• Sample Data:
ColourTable
PK COLOURINDEX
COLOURNAME ISPIGMENTATION
COLOURINDEX COLOURNAME ISPIGMENTATION1 White N2 Grey N3 Red Y4 Blue Y5 Green Y6 Yellow Y7 White-Yellow-Green N8 Purple Y
© Adarza Technologies 2005
Size TableSize Table
• Structure:
• Sample Data:
SizeTable
PK SIZEINDEX
SIZENAME
SIZEINDEX SIZENAME1 Tiny2 Small3 Medium4 Large
© Adarza Technologies 2005
Photo and Calibration TablesPhoto and Calibration TablesPhotoTable
PK PHOTOINDEX
PHOTOFILEPATH CALIBRATIONINDEX
LightingCondTable
PK LIGHTCONDINDEX
LIGHTCONDDESC
1
m
CalibrationTable
PK CALIBRATIONINDEX
CALIBFILEPATH LIGHTCONDINDEX
m
1
• Photo Table
• Calibration Table
• Lighting Condition Table– Describes the lighting condition under which
the batch of pictures were taken.
– Each entry in the Photo Table points to a bacteria picture file on disk.
– Each entry in the Calibration Table points to a calibration picture file on disk.
– Each entry also points to an entry in the Calibration Table.
– Pictures from the same batch would point to same Calibration entry.
– Each entry also points to an entry in the Lighting Condition Table.
© Adarza Technologies 2005
How Files are Stored on DiskHow Files are Stored on Disk
• Top Level – Where the BCA
program is located.
BCABCA.exe
DatabaseBCA.mdb
DataPicture1.jpg
Picture2.jpg
Etc.
• Database Level– Where the Access database
(BCA.mdb) is located.
• Data Level– Where all the pictures are located.
© Adarza Technologies 2005
Goals of the GUIGoals of the GUI
• Collect all necessary input from user.
• Guide user through sample image analytical process.
• Present information from the database in a user-friendly manner.
• Allow user to easily search, view, and modify database.
© Adarza Technologies 2005
Startup ScreenStartup Screen
• Choice 1: Analyze sample image
• Choice 2: View/Modify database
© Adarza Technologies 2005
Searching and Setting FiltersSearching and Setting Filters
• Values between criteria are AND-ed
• Values within criteria are OR-ed
© Adarza Technologies 2005
Before Filters are SetBefore Filters are Set
• Check marks indicate which filters to set.
• Check box items are in black.
• Once we click “Set Filters”…
© Adarza Technologies 2005
After Filters are SetAfter Filters are Set
• Check box items appear in blue.
• Number of filtered records are reflected in the title bar:
• Can undo filters or add/remove additional filters.
© Adarza Technologies 2005
After Filters are SetAfter Filters are Set
• Selecting/deselecting additional filters will toggle the appropriate filter buttons.
© Adarza Technologies 2005
Analyze Sample Walk-ThroughAnalyze Sample Walk-Through
• Get the calibration image file.
• Get a point inside the calibration image.
• Get the sample image file.
• Get a crop of a single bacterial colony.
• Get a rectangle within the colony.
• Next…
Add to DatabaseAdd to Database
• Automatically fill in calculated values.
• Notifies user if required information is missing.
© Adarza Technologies 2005
Testing ProcedureTesting Procedure
• Individual Module testing– Setting threshold levels– 100% accuracy on non-border cases– At least 90% accuracy on border cases
• Consider both primary and secondary choices
• Integrated testing– Multiple testing of reference set– Searching for consistency– Compromise between computer and human
perception
© Adarza Technologies 2005
Cost AnalysisCost AnalysisItem Estimated Market Price (CDN) Our Estimated Cost Our Real Cost
Research expenses (photocopying, printing, etc.) $100.00 $100.00 $10.25
Textbooks on C++ and Visual Studio $127.98
Developmental tools and software
· MS Visual Studio .NET Academic Edition 2003 $135.00 $0.00 $0.00
· Matlab Student Version with image acquisition, image processing, and fuzzy logic toolboxes. $177.00 $0.00 $0.00
· Paint Shop Pro 9.0 $144.99 $0.00 $0.00
· CVS Advanced $12.00
Sony Cyber-shot 3.2 DSC-P72 mega-pixel digital camera $300.00 $0.00 $0.00
Microscope by C&A Scientific $135.00 $0.00 $0.00
Lab Petri Dishes 100 x 15mm (20/Pk) $13.50 $0.00 $0.00
Travel expense for consultation meetings $15.00 $15.00 $7.75
Website $15.00 $15.00 $18.11
Consultation Lunch $24.04
Contingencies $20.00 $20.00 $0.00
Total: $1,055.49 $150.00 $200.13
© Adarza Technologies 2005
Future FeaturesFuture Features
• Image Processing Features– Active contours– Additional classification criteria
• User Interface Features– Advanced search capabilities– Remove extra input from user– Results ordered by probability
© Adarza Technologies 2005
ConclusionConclusion
• Proof of concept complete
• Further development is possible
© Adarza Technologies 2005
AcknowledgementsAcknowledgements
• Dr. Mirza Faisel Beg, Assistant Professor of the School of Engineering Science
• Mr. David Chan, Senior Lab Technician @ BCCDC
• Dr. Swee Han Goh, Associate Professor @ BCCDC