Embed Size (px)
DESCRIPTION
The tool is in its initial stages. It has to yet tested against a huge repository, say around 300 species. If you have any suggestions please mail to [email protected]
Citation preview
Name: Common Banded Peacock
Scientific Name: Papilio crino
Species ID #: 12324343553354
Family: Papilionidae
Genus: Papilio
Distribution: Central and Southern
India, and Srilanka
Status: Locally Common
Wing Span: 100 – 116mm
Conservation Status: Not
threatened
ID-Me : Tool for identifying
species from an image
Kishen Das Kondabagilu Rajanna
UT Arlington, Texas
Planet Earth has huge diversity
Credits: http://www.currentresults.com/Environment-Facts/Plants-Animals/number-species.php
• There is a considerable reduction in number of taxonomists owing to the popularity of Genetics and Microbiology
• Taxonomy is highly challenging and sometimes highly irritating
• Collections are prohibited in many countries
• Photography has become very popular
• Collection of few endangered species has pushed them on the verge of extinction
•Ex: Edith's Checkerspot in southern Sierra Nevada, California and Miami Blue in Bahia Honda State Park, FloridaHonda State Park, Florida
•Global warming and environmental pollution triggering human induced mass extinction ( Every 20 minutes we are losing a species)
•There is an immediate need for documenting biodiversity and taking measures to conserve it
Computer Aided Taxonomy
Using computers for identification of a species in
an image with the help of image processing
techniques and taxonomic keys
Asian Elephant
Map butterfly
Indian Cobra -
Snake
Query Image [keypoints1,
descriptors1] =
vl_sift();
Training Image
Set 1
Fine tune SIFT
Parameters ( Peak
Thresh, Edge Thresh,
and Match Thresh)
Peak Thresh=1.0
Edge Thresh=2000
Match Thresh=1.5
[keypoints2,
descriptors2] =
vl_sift();
Training Image
Set 2
[keypointsQuery,
Algorithm
[keypointsQuery,
descriptorsQuery] =
vl_sift();
. . .
Keypoint ->
4 X 1 matrix
(Scale, Orientation,
Translation along x,
Translation along y)
Descriptor ->
128 X 1 matrix
[keypointsN,
descriptorsN] =
vl_sift();
Training Image
Set N
[matches1,scores1] =vl_ubmatch(descriptors1, descriptorsQuery, match_thresh)
[matches2,scores2] =vl_ubmatch(descriptors2, descriptorsQuery, match_thresh)
[matchesN,scoresN] =vl_ubmatch(descriptorsN, descriptorsQuery, match_thresh)
Filter Descriptors based on scores, so that each matching keypoint of query image has
unique match with that of keypoints of training images
Run Hough Transform on filtered matches. This is a modified hough transform which is
performed in 4D space i.e., Scale, Orientation and two Translations
Algorithm ( Continued …)
Select the Cluster that has voted for the configuration with Maximum votes i.e.,
[Scaling Bin with max votes, Orientation Bin with max votes, Translation Bins with max votes ]
Perform RANSAC in 4D space on cluster of matches obtained in the previous step
Esimate Homography Matrix and further filter bad matches
Select the best possible match from training sets based on remaining matchings and plot the
final matchings
Scale Invariant Feature Transform(SIFT)ConsiderationsVL_FEAT
David Lowe’s SIFT
ASIFT
SIFT in a nutshell•Keypoint detection and localization
( Gaussian Filtering)
•Orientation Assignment
Peak Thresh=1.0
Edge Thresh=2000
Match Thresh=1.5
Keypoint ->
4 X 1 matrix
(Scale,
Orientation,
Translation along x,
Translation along y)
Descriptor ->
128 X 1 matrix
•Orientation Assignment
•Keypoint descriptor matching
( Nearest neighbough indexing)
1. Extract descriptors for dt and dq
2. Extract first closest descriptor d1
3. Extract second closest descriptor d2
4. Accept d1, if dist(dq,d2) > dist(dq,, d1)
Edge threshold eliminates peaks of the DoG scale space whose curvature is too small
Peak threshold filters peaks of the DoG scale space that are too small (in absolute value)Peak threshold filters peaks of the DoG scale space that are too small (in absolute value)
Credits: http://www.vlfeat.org/overview/sift.html
199
81
Matches
51
Matches
199
Matches
Hough Transform in 4D space
Step 1) Prepare the bins based on David Lowe [3]
Step 2) Each matching votes for 4 bins ( In David Lowe [3],each matching votes for 16 bins, to avoid the boundary effects. I am considering only 4 bins for simplicity)
Step 3) Find out the configuration [ Scale Bin with maximum votes, Orientation Bin Step 3) Find out the configuration [ Scale Bin with maximum votes, Orientation Bin with maximum votes, X Translation Bin with maximum votes, Y Translation Bin with maximum votes]
Step 4) Choose the cluster of keypoints that has voted for the above configuration and discard rest of the clusters.( Ideally you should consider all the other clusters as well before discarding the keypoints, again for simplicity I am considering only one configuration with maximum votes) .
25
10
Matches
6
Matches
25
Matches
Modifed RANSAC in 4D space for
Affine TransformationStep 1) Group the remaining matches into all possible combinations of groups of 3
Step 2) For each group of 3-matches, find the differences of affine transformations between each member of this group and rest of the matches. Check whether the differences are within the margin of "OriginalBinSize/3". If yes, that's an inlier.yes, that's an inlier.
Step 3) Select group of 3-matches, if there are at least 10 inliers wrt that group.
Step 4) From the groups of 3-matches selected in previous step, pick the final individual matches, such that more than 50% of the groups have been picked in step 3 where this match belongs to.
Modified Homography for
uncalibrated camers
Step 1 ) Take 4 matches , estimate Homography matrix using the equation
∑ [ x training_image]X H [ x_query_image] < t, where t is close to zero.
If 't' cannot be closer to zero, then those set of matches don't belong to the same plane and hence discard them.
Here the above equation is converted into the form Ax = O ( Zero vector) and then Homography matrix is estimated using SVD.Here the above equation is converted into the form Ax = O ( Zero vector) and then Homography matrix is estimated using SVD.
Step 2) Keep repeating Step 1) till there are maximum number of inliers such that
∑ [ x training_image]X H [ x_query_image] < t, where t -> 0
Step 3) Discard outliers that will not fit into Homography constraint.
0
Matches
23
Matches
0
Matches
Matches
Related Work
http://www.ifpindia.org/biotik/index.php
In this tool, one can try to identify trees of ever green forest by building a query based on different tree parts (Leaf, branch, flower, etc
http://ippcweb.science.oregonstate.edu/LepID/
This software comes somewhat close to ID-Me. The major drawback is that the user has to know which part of the wing is important in identifying that species.
http://www.elec.york.ac.uk/research/projects/Automated_Identification_of_Insects_using_Image_Processing.html age_Processing.html
In this approach tool will automatically extract the wing venation and later identification will be done using 2 different types of artificial neural networks , multi-layer perceptron and learning vector quantisation
http://ipmnet.org/bugwing/
Its a simple tool to assist the amateur ecologists with identification. Here veins and their basic sequence of branching have been made use of in distinguishing the insects to the family or subfamily level.
Future Work
•Using Color descriptors
•Removing current limitations
•Using database to store keypoints and descriptors
•More images for each Training Image
•Fine tuning existing algorithms
•Introducing confidence levels
•Collaborations with taxonomists and computer scientists
0
Matches
11
Matches
0
Matches
0
Matches
18
Matches
0
Matches
0
Matches
7
MatchesMatches
0
Matches
