23
1 ECE 661 - HW 9 Abhiram Gnanasambandam [email protected] I. THEORY A. Fundamental matrix Fundamental matrix has a 7DoFs. We have to get at least 8 correspondences between the two images to estimate the Fundamental matrix. We have to normalize the pixels tuch thae mean is zero and the avergae distance from the center is 2 so the the Hartley and Zisserman algorithm gives better results. Denote the transformation which does this as T 1 and T 2 . Then using the normalized 8 points,we get an estimate of the F matix, given two sets of corrspondences x i and x 0 i / The theory goes as follows. We have to solve the equation Af =0, where A = x 0 i x i x 0 i y i x 0 i y 0 i x i y 0 i y i x i y i 1 The matrix gets populated by 8 such rows given by 8 such correspondences. F = F 11 F 12 F 13 F 21 F 22 F 23 F 31 F 32 F 33 T We can esaily solve for F by using SVD and finding the eigen-vector which corrresponds to the minimum eigen value. We have to make sure that rank of such a F matrix is always 2. We can ensure it by forcing the third and least eigen value to go to 0. Then we can denormalize the matrix F, by using F de = T T 2 FT 1 . Now, we can use the LM optimization to get better estimates. Now, we have to estimate P and P’ the Projection matrices to construct the world coordinated. As the cameras are not calibrated, we find the canonocal form. P =[I |0] and P 0 = [[e 0 ] x F |e 0 ] The world co-orrdinates are obtained by finding the solution to AX world =0 given the constraint ||X world = 1||. Here A is given by A = x i P T 3 - P T 1 y i P T 3 - P T 2 x 0 i P 0T 3 - P 0T 1 y 0 i P 0T 3 - P 0T 2 Now, we generate the error function that will be used to optimize the F. The error function is projecting the world coordinate using P and P’ to the respective image coordinates and finding the square of euclidean distance with the already existing coordinates. Now, we have to use LM to optimize the estimate of F again. B. Image Rectification We first rotate the image 2 by T 1 matrix. We then find the angle the eppole makes with x-axis and rotate the image so that the epipole beecomes parallel to x-axis. e 0 = f 0 1 The next step os tp send the epipole to infinity. For that we use the following G matrix. G = 1 0 0 0 1 0 -1/f 0 1 Now, we multiply with another transform T 2 to make image back to the center. So, the overall Homography is H 2 = T 2 GRT 1 . As we have the homography for Image2, we now have to find the homography for image 1. We use least squares mnimization to find it. min H1 X d (H 1 x i ,H 2 x 0 i )

ECE 661 - HW 9...ECE 661 - HW 9 1 Abhiram Gnanasambandam [email protected] I. THEORY A. Fundamental matrix Fundamental matrix has a 7DoFs. We have to get at least 8 correspondences

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

  • 1ECE 661 - HW 9Abhiram Gnanasambandam

    [email protected]

    I. THEORY

    A. Fundamental matrix

    Fundamental matrix has a 7DoFs. We have to get at least 8 correspondences between the two images toestimate the Fundamental matrix. We have to normalize the pixels tuch thae mean is zero and the avergaedistance from the center is

    √2 so the the Hartley and Zisserman algorithm gives better results. Denote the

    transformation which does this as T1 and T2.Then using the normalized 8 points,we get an estimate of the F matix, given two sets of corrspondences xi

    and x′i/ The theory goes as follows. We have to solve the equation Af = 0, where

    A =[x′ixi x

    ′iyi x

    ′i y

    ′ixi y

    ′iyi xi yi 1

    ]The matrix gets populated by 8 such rows given by 8 such correspondences.

    F =[F11 F12 F13 F21 F22 F23 F31 F32 F33

    ]TWe can esaily solve for F by using SVD and finding the eigen-vector which corrresponds to the minimum

    eigen value. We have to make sure that rank of such a F matrix is always 2. We can ensure it by forcing thethird and least eigen value to go to 0. Then we can denormalize the matrix F, by using Fde = TT2 FT1. Now,we can use the LM optimization to get better estimates.

    Now, we have to estimate P and P’ the Projection matrices to construct the world coordinated. As the camerasare not calibrated, we find the canonocal form. P = [I|0] and P ′ = [[e′]xF |e′]

    The world co-orrdinates are obtained by finding the solution to AXworld = 0 given the constraint ||Xworld =1||. Here A is given by

    A =

    xiP

    T3 − PT1

    yiPT3 − PT2

    x′iP′T3 − P ′T1

    y′iP′T3 − P ′T2

    Now, we generate the error function that will be used to optimize the F. The error function is projecting

    the world coordinate using P and P’ to the respective image coordinates and finding the square of euclideandistance with the already existing coordinates. Now, we have to use LM to optimize the estimate of F again.

    B. Image Rectification

    We first rotate the image 2 by T1 matrix. We then find the angle the eppole makes with x-axis and rotatethe image so that the epipole beecomes parallel to x-axis.

    e′ =

    f01

    The next step os tp send the epipole to infinity. For that we use the following G matrix.

    G =

    1 0 00 1 0−1/f 0 1

    Now, we multiply with another transform T2 to make image back to the center. So, the overall Homography

    is H2 = T2GRT1.As we have the homography for Image2, we now have to find the homography for image 1. We use least

    squares mnimization to find it.minH1

    ∑d

    (H1xi, H2x′i)

  • The details of the extended method can be found in Multiple View Geometry by Richard Hartley and AndrewZisserman.

    Once, the homographies are found, we can use them to get the rectified images.

    C. Using SIFT and canny edge detectors

    As we have already done HWs on Canny and SIFT, I am excluding the discussions on them here. We findthe corresponding points between the two rectified images using one of two methods. And once we have tocorresponding methods, we learnt in last two sections how to triangulate and find the world coordinates.

    II. RESULTS

    Figure 1: View 1

    Figure 2: View 2

    2

  • Figure 3: Rectified Images and the SIFT correspondences

    Figure 4: Result of canny on rectified image 1

    3

  • Figure 5: Result of canny on rectified image 2

    4

  • Figure 6: The projected 3D points with SIFT on the rectified images

    5

  • Figure 7: The projected 3D points with SIFT on the rectified images , with details for understanding the scenestructure

    Figure 8: The projected 3D points with SIFT on the rectified images , before (red lines and black circles) andafter(blue lines and red dots) using LM optimization.

    6

  • Figure 9: The projected 3D points with Canny on the rectified images

    7

  • III. CODE

    # −*− co d i ng : u t f −8 −*−" " "C r e a t e d on Sun Nov 18 1 2 : 4 9 : 4 6 2018

    @author : a b h i r" " "

    i m p o r t cv2 as cvi m p o r t numpy as npfrom s c i p y . o p t i m i z e i m p o r t l e a s t _ s q u a r e s # LM a l g o r i t h m from h e r ei m p o r t mathi m p o r t m a t p l o t l i b . p y p l o t a s p l tfrom m p l _ t o o l k i t s . mplot3d i m p o r t Axes3D

    d e f r e t u r n _ c o r r e s p o n d e n c e s ( edges1 , edges2 ) :p1 = [ ]p2 = [ ]f o r i i n r a n g e ( 3 , l e n ( edges1 )−3) :

    f o r j i n r a n g e ( 3 , l e n ( edges1 [ 0 ] ) −3 ) :i f ( edges1 [ i , j ] == 2 5 5 ) :

    d i s t =1 e10i 1 = if o r j 1 i n r a n g e ( 3 , l e n ( edges2 [0])− 3 ) :

    i f ( np . l i n a l g . norm ( im1 [ i −2: i +3 , j −2: j +3] − im2 [ i1 −2: i 1 +3 , j1 −2: j 1 + 3 ] ) < d i s t ) :t = j 1d i s t = np . l i n a l g . norm ( im1 [ i −2: i +3 , j −2: j +3] − im2 [ i1 −2: i 1 +3 , j1 −2: j 1 + 3 ] )

    p1 . append ( [ i , j ] )p2 . append ( [ i1 , t ] )

    r e t u r n p1 , p2

    d e f d r a w _ l i n e s ( image1 , image2 , p o i n t s 1 , p o i n t s 2 , p o i n t s 1 1 , p o i n t s 2 1 , number ) :s1 = np . shape ( image1 )s2 = np . shape ( image2 )

    nimage = np . z e r o s ( ( s1 [ 0 ] , s1 [ 1 ] + s2 [ 1 ] , 3 ) ) ;nimage [ : , 0 : s1 [ 1 ] ] = image1nimage [ : , s1 [ 1 ] : ] = image2

    # nimage = nimage . a s t y p e ( ’ u i n t 8 ’ )

    i f ( number == 0 ) :f o r i i n r a n g e ( 0 , l e n ( p o i n t s 1 ) ) :

    cv . l i n e ( nimage , ( p o i n t s 1 [ i ] [ 1 ] , p o i n t s 1 [ i ] [ 0 ] ) , ( s1 [ 1 ] + p o i n t s 2 [ i ] [ 1 ] , p o i n t s 2 [ i ] [ 0 ] ) , ( 0 , 0 , 2 5 5 ) , 1 )

    i f ( number == 1 ) :f o r i i n r a n g e ( 0 , l e n ( p o i n t s 1 1 ) ) :

    cv . l i n e ( nimage , ( p o i n t s 1 1 [ i ] [ 1 ] , p o i n t s 1 1 [ i ] [ 0 ] ) , ( s1 [ 1 ] + p o i n t s 2 1 [ i ] [ 1 ] , p o i n t s 2 1 [ i ] [ 0 ] ) , ( 0 , 2 5 5 , 0 ) , 1 )

    r e t u r n nimage

    8

  • d e f r e t u r n _ c o r r e s _ p o i n t s ( SSd , h a r r i s 1 , h a r r i s 2 , SSDthresh , NCCthresh ) :l 1 = l e n ( SSd )

    points1_NCC = [ ]points2_NCC = [ ]

    poin ts1_SSD = [ ]poin ts2_SSD = [ ]

    p o i n t 2 = [ ]

    f o r i i n r a n g e ( 0 , l 1 ) :i f ( np . sum ( SSd [ i ] < SSDthresh ) > 0) :

    j 1 = np . argmin ( SSd [ i ] )i f ( j 1 n o t i n p o i n t 2 ) :

    i f ( np . abs ( h a r r i s 1 [ i ] [ 0 ] − h a r r i s 2 [ j 1 ] [ 0 ] ) < 3 0 ) :p o i n t 2 . append ( j 1 )poin ts1_SSD . append ( [ h a r r i s 1 [ i ] [ 0 ] , h a r r i s 1 [ i ] [ 1 ] ] )poin ts2_SSD . append ( [ h a r r i s 2 [ j 1 ] [ 0 ] , h a r r i s 2 [ j 1 ] [ 1 ] ] )

    r e t u r n points1_SSD , points2_SSD , points1_NCC , points2_NCCd e f p 2 p _ c o r r e s ( des1 , des2 ) :

    l 1 = l e n ( des1 )l 2 = l e n ( des2 )

    SSD = np . z e r o s ( ( l1 , l 2 ) )

    f o r i i n r a n g e ( 0 , l 1 ) :f o r j i n r a n g e ( 0 , l 2 ) :

    SSD[ i , j ] = np . l i n a l g . norm ( des1 [ i ] − des2 [ j ] )

    r e t u r n SSD

    d e f r e c t i f y i m a g e ( im1 , H1 , fn , kkkk ) :Hinv = H1# F i n d i n g t h e s i z e o f t h e o u t p u t imaget l = np . matmul ( Hinv , [ 0 . 0 , 0 . 0 , 1 . 0 ] ) ; #Map f o r Top l e f t c o r n e rt r = np . matmul ( Hinv , [ l e n ( im1 [ 0 ] ) , 0 . 0 , 1 . 0 ] ) ; #Map f o r Top r i g h t c o r n e rb r = np . matmul ( Hinv , [ l e n ( im1 [ 0 ] ) , l e n ( im1 ) , 1 . 0 ] ) ; #Map f o r bot tom r i g h t c o r n e rb l = np . matmul ( Hinv , [ 0 . 0 , l e n ( im1 ) , 1 . 0 ] ) ; #Map f o r bot tom l e f t

    t l = t l / t l [ 2 ] ;t r = t r / t r [ 2 ] ;b r = br / b r [ 2 ] ;b l = b l / b l [ 2 ] ;

    p r i n t t l , t r , br , b l

    9

  • # Find x l e n g t h o f t h e o u t p u t imageminx = np . min ( [ t l [ 0 ] , t r [ 0 ] , b r [ 0 ] , b l [ 0 ] ] ) ;maxx = np . max ( [ t l [ 0 ] , t r [ 0 ] , b r [ 0 ] , b l [ 0 ] ] ) ;x l e n = np . round ( maxx − minx ) . a s t y p e ( ’ i n t ’ ) ;

    # Find y l e n g t h o f t h e o u t p u t imageminy = np . min ( [ t l [ 1 ] , t r [ 1 ] , b r [ 1 ] , b l [ 1 ] ] ) ;maxy = np . max ( [ t l [ 1 ] , t r [ 1 ] , b r [ 1 ] , b l [ 1 ] ] ) ;y l e n = np . round ( maxy − miny ) . a s t y p e ( ’ i n t ’ ) ;

    # p r i n t y len , x l e n#### I n i t i a l i s e t h e oupu t image

    H _ s c a l i n g = np . eye ( 3 ) ;p r i n t ’ asd ’p r i n t minx , miny , maxx , maxy , x len , y l e n

    # minx = −15.5947476171# miny = −138.06632525# maxx = 623.459010284# maxy = 585.306281335# x l e n = 639# y l e n = 723

    H _ s c a l i n g [ 0 , 0 ] = f l o a t ( np . shape ( im1 ) [ 1 ] ) / x l e n / 2H _ s c a l i n g [ 1 , 1 ] = f l o a t ( np . shape ( im1 ) [ 0 ] ) / y l e n / 2

    Hinv = np . matmul ( H_sca l ing , Hinv )

    # p r i n t H _ s c a l i n g / H _ s c a l i n g [ 2 , 2 ]

    t l = np . matmul ( Hinv , [ 0 . 0 , 0 . 0 , 1 . 0 ] ) ; #Map f o r Top l e f t c o r n e rt r = np . matmul ( Hinv , [ l e n ( im1 [ 0 ] ) , 0 . 0 , 1 . 0 ] ) ; #Map f o r Top r i g h t c o r n e rb r = np . matmul ( Hinv , [ l e n ( im1 [ 0 ] ) , l e n ( im1 ) , 1 . 0 ] ) ; #Map f o r bot tom r i g h t c o r n e rb l = np . matmul ( Hinv , [ 0 . 0 , l e n ( im1 ) , 1 . 0 ] ) ; #Map f o r bot tom l e f t

    t l = t l / t l [ 2 ] ;t r = t r / t r [ 2 ] ;b r = br / b r [ 2 ] ;b l = b l / b l [ 2 ] ;

    # p r i n t t l , t r , br , b l# Find x l e n g t h o f t h e o u t p u t imageminx = np . min ( [ t l [ 0 ] , t r [ 0 ] , b r [ 0 ] , b l [ 0 ] ] ) ;maxx = np . max ( [ t l [ 0 ] , t r [ 0 ] , b r [ 0 ] , b l [ 0 ] ] ) ;x l e n = np . round ( maxx − minx ) . a s t y p e ( ’ i n t ’ ) ;

    # Find y l e n g t h o f t h e o u t p u t imageminy = np . min ( [ t l [ 1 ] , t r [ 1 ] , b r [ 1 ] , b l [ 1 ] ] ) ;maxy = np . max ( [ t l [ 1 ] , t r [ 1 ] , b r [ 1 ] , b l [ 1 ] ] ) ;y l e n = np . round ( maxy − miny ) . a s t y p e ( ’ i n t ’ ) ;

    10

  • im3 = np . z e r o s ( ( y len , x len , 3 ) ) . a s t y p e ( ’ u i n t 8 ’ ) ;

    H1 = np . l i n a l g . i n v ( Hinv )

    # p r i n t H1 / H1 [ 2 ] [ 2 ]# Get t h e imagef o r i i n r a n g e ( 0 , l e n ( im3 ) ) :

    f o r j i n r a n g e ( 0 , l e n ( im3 [ 0 ] ) ) :newindex = np . matmul ( H1 , [ j + minx , i +miny , 1 . 0 ] )newindex = newindex / newindex [ 2 ] ;

    #Use b i l i n e a r a p p r o x i m a t i o n t o g e t t h e o u t p u t p i x e l v a l u ei f ( newindex [ 1 ] > 0 and newindex [ 1 ] < l e n ( im1)−1 and newindex [ 0 ] > 0 and newindex [ 0 ] < l e n ( im1 [ 0 ] ) −1 ) :

    f l o o r x = np . f l o o r ( newindex [ 1 ] ) ;f l o o r x i = f l o o r x . a s t y p e ( ’ i n t ’ ) ;f l o o r y = np . f l o o r ( newindex [ 0 ] ) ;f l o o r y i = f l o o r y . a s t y p e ( ’ i n t ’ ) ;

    c e i l x = np . c e i l ( newindex [ 1 ] ) ;c e i l x i = c e i l x . a s t y p e ( ’ i n t ’ ) ;c e i l y = np . c e i l ( newindex [ 0 ] ) ;c e i l y i = c e i l y . a s t y p e ( ’ i n t ’ ) ;

    p i x e l _ v a l u e = ( c e i l x − newindex [ 1 ] ) * ( c e i l y − newindex [ 0 ] )* im1 [ f l o o r x i ] [ f l o o r y i ] + ( c e i l x − newindex [ 1 ] ) * ( newindex [ 0 ] − f l o o r y )* im1 [ f l o o r x i ] [ c e i l y i ] + ( newindex [ 1 ] − f l o o r x ) * ( newindex [ 0 ] − f l o o r y )* im1 [ c e i l x i ] [ c e i l y i ] + ( newindex [ 1 ] − f l o o r x ) * ( c e i l y − newindex [ 0 ] )* im1 [ c e i l x i ] [ f l o o r y i ] ;

    im3 [ i ] [ j ] = p i x e l _ v a l u e . a s t y p e ( ’ u i n t 8 ’ )

    # Wr i t e t h e imagecv . i m w r i t e ( fn , im3 )

    kkk=np . z e r o s ( ( l e n ( kkkk ) , 3 ) )

    f o r i i n r a n g e ( 0 , l e n ( kkkk ) ) :kkk [ i ] = np . matmul ( Hinv , [ kkkk [ i , 0 ] , kkkk [ i , 1 ] , 1 . 0 ] )kkk [ i ] = kkk [ i ] / kkk [ i , 2 ]

    kkk [ i , 0 ] = kkk [ i , 0 ] − minxkkk [ i , 1 ] = kkk [ i , 1 ] − miny

    r e t u r n im3 , kkk

    # d e f r e c t i f y _ i m a g e s ( image , H, p o i n t s ) :

    # Find homography f o r r e c t i f i c a t i o nd e f f ind_homo ( e1 , e2 , P1 , P2 , cp1 , cp2 , F1 ) :

    x1 = cp1 ;x2 = cp2 ;s s = np . shape ( im1 ) ;

    a n g l e = math . a t a n (−( e2 [1]− s s [ 0 ] / 2 ) / ( e2 [0]− s s [ 1 ] / 2 ) )

    11

  • p r i n t a n g l e# p r i n t a n g l e

    f = math . cos ( a n g l e ) * ( e2 [0]− s s [ 1 ] / 2 ) −math . s i n ( a n g l e ) * ( e2 [1]− s s [ 0 ] / 2 ) ;# p r i n t f

    G = np . z e r o s ( ( 3 , 3 ) )G[ 0 , 0 ] = 1G[ 1 , 1 ] = 1G[ 2 , 0 ] = −1/ fG[ 2 , 2 ] = 1 ;

    R = np . z e r o s ( ( 3 , 3 ) )R [ 0 ] [ 0 ] = math . cos ( a n g l e )R[ 0 , 1 ] = −math . s i n ( a n g l e )R[ 1 , 0 ] = math . s i n ( a n g l e )R[ 1 , 1 ] = math . cos ( a n g l e )R[ 2 , 2 ] = 1

    T = np . z e r o s ( ( 3 , 3 ) )

    T [ 0 , 0 ] = 1 ;T [ 0 , 2 ] = −s s [ 1 ] / 2T [ 1 , 1 ] = 1T [ 1 , 2 ] = −s s [ 0 ] / 2T [ 2 , 2 ] = 1 ;

    H2 = np . matmul (G, np . matmul (R , T ) ) ;

    im_c = np . z e r o s ( 3 )

    im_c [ 0 ] = s s [ 1 ] / 2 ;im_c [ 1 ] = s s [ 0 ] / 2im_c [ 2 ] = 1 ;

    c _ r e c = np . matmul ( H2 , im_c )c _ r e c = c _ r e c / c _ r e c [ 2 ]

    T2 = np . eye ( 3 )T2 [ 0 , 2 ] = s s [1] /2− c _ r e c [ 0 ]T2 [ 1 , 2 ] = s s [0] /2− c _ r e c [ 1 ]

    H2 = np . matmul ( T2 , H2 ) ##########

    # F i r s t e p i p o l e# M = np . matmul ( P2 , np . l i n a l g . p inv ( P1 ) )

    sec_m = np . ones ( ( 3 , 3 ) )

    sec_m [ : , 0 ] = e2 ;sec_m [ : , 1 ] = e2 ;sec_m [ : , 2 ] = e2 ;

    12

  • F1 = F1 / F1 [ 2 , 2 ]

    # p r i n t a_x ( e2 ) , e2 , F1M = np . matmul ( a_x ( e2 ) , F1 ) + sec_m# M = exF + e v t

    # p r i n t M/M[ 2 , 2 ]

    # M = np . matmul ( P2 , np . l i n a l g . p inv ( P1 ) )H0 = np . matmul ( H2 ,M)

    # p r i n t H0l = l e n ( cp1 )

    x1_dash = np . z e r o s ( ( l , 3 ) ) ;x2_dash = np . z e r o s ( ( l , 3 ) ) ;

    f o r i i n r a n g e ( 0 , l ) :x1_dash [ i ] = np . matmul ( H0 , x1 [ i ] )x1_dash [ i ] = x1_dash [ i ] / x1_dash [ i , 2 ]

    x2_dash [ i ] = np . matmul ( H2 , x2 [ i ] )x2_dash [ i ] = x2_dash [ i ] / x2_dash [ i , 2 ]

    A = x1_dashb = x2_dash [ : , 0 ]

    # p r i n t A, b

    x = np . matmul ( np . l i n a l g . p inv (A) , b )

    Ha = np . eye ( 3 )Ha [ 0 , 0 ] = x [ 0 ]Ha [ 0 , 1 ] = x [ 1 ]Ha [ 0 , 2 ] = x [ 2 ]

    H1 = np . matmul ( Ha , H0 )

    c _ r e c = np . matmul ( H1 , im_c )c _ r e c = c _ r e c / c _ r e c [ 2 ]

    T1 = np . eye ( 3 )T1 [ 0 , 2 ] = s s [1] /2− c _ r e c [ 0 ]T1 [ 1 , 2 ] = s s [0] /2− c _ r e c [ 1 ]

    H1 = np . matmul ( T1 , H1 )

    F_rec = np . matmul ( np . l i n a l g . p inv ( np . t r a n s p o s e ( H2 ) ) , np . matmul ( F1 , np . l i n a l g . p inv ( H1 ) ) ) ;

    x 1_ re c = np . z e r o s ( ( l , 3 ) ) ;x 2_ re c = np . z e r o s ( ( l , 3 ) ) ;

    13

  • f o r i i n r a n g e ( 0 , l ) :x 1_ re c [ i ] = np . matmul ( H1 , x1 [ i ] )x 1_ re c [ i ] = x 1_ re c [ i ] / x 1_ r ec [ i , 2 ]

    x 2_ re c [ i ] = np . matmul ( H2 , x2 [ i ] )x 2_ re c [ i ] = x 2_ re c [ i ] / x 2_ r ec [ i , 2 ]

    u , s , vh = np . l i n a l g . svd ( F_rec )

    e 1 _ r e c = np . t r a n s p o s e ( vh [ 2 ] )e 1 _ r e c = e 1 _ r e c / e 1 _ r e c [ 2 ]e 2 _ r e c = u [ : , 2 ]e 2 _ r e c = e 2 _ r e c / e 2 _ r e c [ 2 ]

    r e t u r n x1_rec , x2_rec , H1 , H2 , F_rec , e1_rec , e 2 _ r e c

    # LM c o s t f o r F m a t r i xd e f c o s t f ( f ) :

    F = np . z e r o s ( ( 3 , 3 ) )F [ 0 , 0 ] = f [ 0 ]F [ 0 , 1 ] = f [ 1 ]F [ 0 , 2 ] = f [ 2 ]F [ 1 , 0 ] = f [ 3 ]F [ 1 , 1 ] = f [ 4 ]F [ 1 , 2 ] = f [ 5 ]F [ 2 , 0 ] = f [ 6 ]F [ 2 , 1 ] = f [ 7 ]F [ 2 , 2 ] = f [ 8 ]

    u , s , vh = np . l i n a l g . svd ( F )

    # s [ 2 ] = 0## F = np . matmul ( u , np . matmul ( np . d i a g ( s ) , vh ) )# u , s , vh = np . l i n a l g . svd ( F )

    e1 = np . t r a n s p o s e ( vh [ 2 ] )

    e2 = u [ : , 2 ]

    # p r i n t e1 , e2

    P = np . z e r o s ( ( 3 , 4 ) )

    P [ 0 : 3 , 0 : 3 ] = np . eye ( 3 )

    e2x = a_x ( e2 )

    P_dash = np . z e r o s ( ( 3 , 4 ) )

    P_dash [ 0 : 3 , 0 : 3 ] = np . matmul ( e2x , F )P_dash [ 0 : 3 , 3 ] = e2

    # p r i n t e2x# p r i n t ’ \ n \ n \ n ’

    14

  • # p r i n t P , P_dashC = [ ]l = l e n ( cp1 )f o r i i n r a n g e ( 0 , l ) :

    A = np . z e r o s ( ( 4 , 4 ) ) ;A[ 0 ] = cp1 [ i ] [ 0 ] * P [ 2 , : ] − P [ 0 , : ]A[ 1 ] = cp1 [ i ] [ 1 ] * P [ 2 , : ] − P [ 1 , : ]A[ 2 ] = cp2 [ i ] [ 0 ] * P_dash [ 2 , : ] − P_dash [ 0 , : ]A[ 3 ] = cp2 [ i ] [ 1 ] * P_dash [ 2 , : ] − P_dash [ 1 , : ]

    u , s , vh = np . l i n a l g . svd (A)

    xw = np . t r a n s p o s e ( vh [ 3 ] )

    xw = xw / np . l i n a l g . norm ( xw )

    x1 = np . matmul ( P , xw )x2 = np . matmul ( P_dash , xw )

    x1 = x1 / x1 [ 2 ]x2 = x2 / x2 [ 2 ]

    # p r i n t x1 , x2# p r i n t np . l i n a l g . norm ( x1 − cp1 ) * * 2 , np . l i n a l g . norm ( x2 − cp2 )**2# p r i n t cp1 , cp2

    C . append ( np . l i n a l g . norm ( x1 − cp1 [ i ] ) * * 2 ) ;C . append ( np . l i n a l g . norm ( x2 − cp2 [ i ] ) * * 2 )

    # p r i n t np . l i n a l g . norm (C)

    r e t u r n np . a s a r r a y (C)

    d e f a_x (w ) :wx = w[ 0 ]wy = w[ 1 ]wz = w[ 2 ]Wx = np . z e r o s ( ( 3 , 3 ) )Wx[ 0 ] [ 1 ] = −wzWx[ 0 ] [ 2 ] = wyWx[ 1 ] [ 0 ] = wzWx[ 1 ] [ 2 ] = −wxWx[ 2 , 0 ] = −wyWx[ 2 , 1 ] = wx

    r e t u r n Wx# Find t h e Fundamenta l Ma t r i xd e f f i n d F ( np1 , np2 ) :

    # S e t he m a t r i c e s f o r s o l v i n g f o r Fl = l e n ( cp1 )A = np . z e r o s ( ( l , 9 ) ) ;

    f o r i i n r a n g e ( 0 , l ) :A[ i , 0 ] = np2 [ i ] [ 0 ] * np1 [ i ] [ 0 ]A[ i , 1 ] = np2 [ i ] [ 0 ] * np1 [ i ] [ 1 ]A[ i , 2 ] = np2 [ i ] [ 0 ]A[ i , 3 ] = np2 [ i ] [ 1 ] * np1 [ i ] [ 0 ]

    15

  • A[ i , 4 ] = np2 [ i ] [ 1 ] * np1 [ i ] [ 1 ]A[ i , 5 ] = np2 [ i ] [ 1 ]A[ i , 6 ] = np1 [ i ] [ 0 ]A[ i , 7 ] = np1 [ i ] [ 1 ]A[ i , 8 ] = 1

    p r i n t A[ : , 6 ] , np1 [ : , 0 ]u , s , vh = np . l i n a l g . svd (A ) ;

    f = np . t r a n s p o s e ( vh [ 8 ] )

    F = np . z e r o s ( ( 3 , 3 ) )F [ 0 , 0 ] = f [ 0 ]F [ 0 , 1 ] = f [ 1 ]F [ 0 , 2 ] = f [ 2 ]F [ 1 , 0 ] = f [ 3 ]F [ 1 , 1 ] = f [ 4 ]F [ 1 , 2 ] = f [ 5 ]F [ 2 , 0 ] = f [ 6 ]F [ 2 , 1 ] = f [ 7 ]F [ 2 , 2 ] = f [ 8 ]

    u , s , vh = np . l i n a l g . svd ( F ) ;

    s [ 2 ] = 0F = np . matmul ( u , np . matmul ( np . d i a g ( s ) , vh ) )

    r e t u r n F

    # Th i s f u n c t i o n n o r m a l i z e s t h e p o i n t sd e f n o r m a l i z e _ p o i n t s ( p o i n t s ) :

    m1 = np . mean ( p o i n t s [ : , 0 ] ) ;m2 = np . mean ( p o i n t s [ : , 1 ] ) ;

    p r i n t np . shape ( p o i n t s ) , np . shape (m1)

    d i s t 1 = ( p o i n t s [ : , 0 ] −m1 ) * ( p o i n t s [ : , 0 ] −m1)d i s t 2 = ( p o i n t s [ : , 1 ] −m2 ) * ( p o i n t s [ : , 1 ] −m2)

    d i s t = np . z e r o s ( l e n ( p o i n t s ) )

    f o r i i n r a n g e ( 0 , l e n ( p o i n t s ) ) :d i s t [ i ] = np . s q r t ( d i s t 1 [ i ] + d i s t 2 [ i ] )

    md = np . sum ( d i s t ) / l e n ( p o i n t s )

    s c a l e = np . s q r t ( 2 ) / md ;

    n e w _ p o i n t s = np . ones ( np . shape ( p o i n t s ) )

    T = np . z e r o s ( ( 3 , 3 ) )

    T [ 0 ] [ 0 ] = s c a l eT [ 0 ] [ 2 ] = −s c a l e * m1T [ 1 ] [ 1 ] = s c a l e ;

    16

  • T [ 1 ] [ 2 ] = −s c a l e * m2 ;T [ 2 ] [ 2 ] = 1 ;

    f o r i i n r a n g e ( 0 , l e n ( p o i n t s ) ) :n e w _ p o i n t s [ i ] [ 0 ] = ( p o i n t s [ i ] [ 0 ] − m1)* s c a l en e w _ p o i n t s [ i ] [ 1 ] = ( p o i n t s [ i ] [ 1 ] − m2)* s c a l e

    # new_po in t s1 = np . ones ( np . shape ( p o i n t s ) )## f o r i i n r a n g e ( 0 , 8 ) :# new_po in t s1 [ i ] = np . matmul ( T , p o i n t s [ i ] ) ;

    r e t u r n new_poin t s , T

    im1 = cv . imread ( ’ 1 1 . png ’ )im2 = cv . imread ( ’ 1 2 . png ’ )

    cp1 = np . z e r o s ( ( 8 , 3 ) )cp2 = np . z e r o s ( ( 8 , 3 ) )

    # cp1 [ 0 ] = [ 1 2 8 6 , 2 1 5 8 , 1 ]# cp1 [ 1 ] = [ 1 7 3 1 , 2 6 2 1 , 1 ]# cp1 [ 2 ] = [ 2 2 6 1 , 2 5 0 3 , 1 ]# cp1 [ 3 ] = [ 2 3 1 7 , 1 3 2 0 , 1 ]# cp1 [ 4 ] = [ 1 7 8 5 , 1 3 2 6 , 1 ]# cp1 [ 5 ] = [ 1 3 0 3 , 1 2 3 7 , 1 ]# cp1 [ 6 ] = [ 1 4 9 2 , 1 6 2 6 , 1 ]# cp1 [ 7 ] = [ 1 4 9 7 , 1 3 8 9 , 1 ]#### cp2 [ 0 ] = [ 1 3 6 9 , 2 1 2 0 , 1 ]# cp2 [ 1 ] = [ 1 7 8 6 , 2 5 8 4 , 1 ]# cp2 [ 2 ] = [ 2 3 2 2 , 2 4 6 9 , 1 ]# cp2 [ 3 ] = [ 2 3 6 8 , 1 2 8 0 , 1 ]# cp2 [ 4 ] = [ 1 8 2 5 , 1 2 9 1 , 1 ]# cp2 [ 5 ] = [ 1 3 7 4 , 1 2 0 3 , 1 ]# cp2 [ 6 ] = [ 1 5 5 4 , 1 5 9 2 , 1 ]# cp2 [ 7 ] = [ 1 5 5 8 , 1 3 5 8 , 1 ]

    cp1 [ 0 ] = [ 9 4 , 2 3 1 , 1 ]cp1 [ 1 ] = [ 3 8 4 , 4 1 5 , 1 ]cp1 [ 2 ] = [ 4 2 , 1 2 3 , 1 ]cp1 [ 3 ] = [ 3 8 4 , 3 1 1 , 1 ]cp1 [ 4 ] = [ 2 0 5 , 1 3 , 1 ]cp1 [ 5 ] = [ 5 3 5 , 1 2 1 , 1 ]cp1 [ 6 ] = [ 5 1 0 , 2 4 4 , 1 ]cp1 [ 7 ] = [ 1 8 6 , 1 4 7 , 1 ]

    cp2 [ 0 ] = [ 1 0 0 , 2 1 2 , 1 ]cp2 [ 1 ] = [ 3 3 1 , 4 1 9 , 1 ]cp2 [ 2 ] = [ 6 0 , 1 0 7 , 1 ]

    17

  • cp2 [ 3 ] = [ 3 1 8 , 3 1 6 , 1 ]cp2 [ 4 ] = [ 2 2 8 , 2 2 , 1 ]cp2 [ 5 ] = [ 5 1 0 , 1 5 4 , 1 ]cp2 [ 6 ] = [ 4 9 0 , 2 6 8 , 1 ]cp2 [ 7 ] = [ 1 7 6 , 1 4 1 , 1 ]

    #Img1 = [ ]#Img2 = [ ]#Img1 . append ( [ 1 8 5 9 . 0 , 4 4 7 . 0 , 1 . 0 ] )#Img1 . append ( [ 3 9 0 5 . 0 , 1 0 1 1 . 0 , 1 . 0 ] )#Img1 . append ( [ 4 8 1 . 0 , 8 9 7 . 0 , 1 . 0 ] )#Img1 . append ( [ 3 0 0 1 . 0 , 2 0 3 3 . 0 , 1 . 0 ] )#Img1 . append ( [ 7 3 7 . 0 , 2 0 9 3 . 0 , 1 . 0 ] )#Img1 . append ( [ 2 7 0 1 . 0 , 3 3 4 1 . 0 , 1 . 0 ] )#Img1 . append ( [ 2 0 7 3 . 0 , 1 9 3 7 . 0 , 1 . 0 ] )#Img1 . append ( [ 2 7 7 7 . 0 , 2 7 8 5 . 0 , 1 . 0 ] )#####Img2 . append ( [ 1 9 2 1 . 0 , 4 5 9 . 0 , 1 . 0 ] )#Img2 . append ( [ 3 9 7 7 . 0 , 9 8 5 . 0 , 1 . 0 ] )#Img2 . append ( [ 5 5 7 . 0 , 8 5 3 . 0 , 1 . 0 ] )#Img2 . append ( [ 3 0 9 3 . 0 , 1 9 2 9 . 0 , 1 . 0 ] )#Img2 . append ( [ 7 9 3 . 0 , 2 0 7 7 . 0 , 1 . 0 ] )#Img2 . append ( [ 2 7 6 9 . 0 , 3 3 2 9 . 0 , 1 . 0 ] )#Img2 . append ( [ 2 1 5 3 . 0 , 1 8 6 9 . 0 , 1 . 0 ] )#Img2 . append ( [ 2 8 5 7 . 0 , 2 7 2 5 . 0 , 1 . 0 ] )## cp1 = np . a s a r r a y ( Img1 )# cp2 = np . a s a r r a y ( Img2 )

    kkkk1 = cp1 [ 0 : 7 ] ;kkkk2 = cp2 [ 0 : 7 ] ;

    f o r i i n r a n g e ( 0 , 8 ) :cv . c i r c l e ( im1 , ( i n t ( cp1 [ i , 0 ] ) , i n t ( cp1 [ i , 1 ] ) ) , 1 , ( 0 , 0 , 2 5 5 ) , −1)cv . c i r c l e ( im2 , ( i n t ( cp2 [ i , 0 ] ) , i n t ( cp2 [ i , 1 ] ) ) , 1 , ( 0 , 0 , 2 5 5 ) , −1)

    # n o r m a l i z e d p o i n t snp1 , T1 = n o r m a l i z e _ p o i n t s ( cp1 )np2 , T2 = n o r m a l i z e _ p o i n t s ( cp2 )

    Fde = f i n d F ( np1 , np2 )

    F = np . matmul ( np . t r a n s p o s e ( T2 ) , np . matmul ( Fde , T1 ) ) ;

    F = F / F [ 2 , 2 ]u , s , vh = np . l i n a l g . svd ( F )

    e1 = np . t r a n s p o s e ( vh [ 2 ] )e2 = u [ : , 2 ]

    P = np . z e r o s ( ( 3 , 4 ) )

    18

  • P [ 0 : 3 , 0 : 3 ] = np . eye ( 3 )

    e2x = a_x ( e2 )

    P_dash = np . z e r o s ( ( 3 , 4 ) )

    P_dash [ 0 : 3 , 0 : 3 ] = np . matmul ( e2x , F )P_dash [ 0 : 3 , 3 ] = e2F = F / F [ 2 , 2 ]f = np . z e r o s ( 9 )f [ 0 ] = F [ 0 ] [ 0 ]f [ 1 ] = F [ 0 ] [ 1 ]f [ 2 ] = F [ 0 ] [ 2 ]f [ 3 ] = F [ 1 ] [ 0 ]f [ 4 ] = F [ 1 ] [ 1 ]f [ 5 ] = F [ 1 ] [ 2 ]f [ 6 ] = F [ 2 ] [ 0 ]f [ 7 ] = F [ 2 ] [ 1 ]f [ 8 ] = F [ 2 ] [ 2 ]

    # r e s = l e a s t _ s q u a r e s ( c o s t f , f , method = ’ lm ’ )# f o r i i n r a n g e ( 0 , 8 )f1 = f

    F1 = np . z e r o s ( ( 3 , 3 ) )F1 [ 0 , 0 ] = f1 [ 0 ]F1 [ 0 , 1 ] = f1 [ 1 ]F1 [ 0 , 2 ] = f1 [ 2 ]F1 [ 1 , 0 ] = f1 [ 3 ]F1 [ 1 , 1 ] = f1 [ 4 ]F1 [ 1 , 2 ] = f1 [ 5 ]F1 [ 2 , 0 ] = f1 [ 6 ]F1 [ 2 , 1 ] = f1 [ 7 ]F1 [ 2 , 2 ] = f1 [ 8 ]

    F1 = F1 / F1 [ 2 , 2 ]

    #u , s , vh = np . l i n a l g . svd ( F1 )## s [ 2 ] = 0##F1 = np . matmul ( u , np . matmul ( np . d i a g ( s ) , vh ) )#u , s , vh = np . l i n a l g . svd ( F1 )e1 = np . t r a n s p o s e ( vh [ 2 ] )e2 = u [ : , 2 ]

    P = np . z e r o s ( ( 3 , 4 ) )

    P [ 0 : 3 , 0 : 3 ] = np . eye ( 3 )

    e1 = e1 / e1 [ 2 ]e2 = e2 / e2 [ 2 ]

    19

  • e2x = a_x ( e2 )

    P_dash = np . z e r o s ( ( 3 , 4 ) )P_dash [ 0 : 3 , 0 : 3 ] = np . matmul ( e2x , F1 )P_dash [ 0 : 3 , 3 ] = e2

    x1_rec , x2_rec , H1 , H2 , F_rec , e1_rec , e 2 _ r e c = find_homo ( e1 , e2 , P , P_dash , cp1 , cp2 , F1 )# x1_rec , x2_rec , H1 , H2 , F_rec , e1_rec , e 2 _ r e c = find_homo ( e1 , e2 , P , P_dash , cp1 , cp2 )

    i m a g e 1 c l r , kkk1 = r e c t i f y i m a g e ( im1 , H1 , ’ r e c 1 . png ’ , kkkk1 )i m a g e 2 c l r , kkk2 = r e c t i f y i m a g e ( im2 , H2 , ’ r e c 2 . png ’ , kkkk2 )

    kkk1 = kkk1 . a s t y p e ( ’ i n t ’ )kkk2 = kkk2 . a s t y p e ( ’ i n t ’ )#SIFT# s i f t = cv . x f e a t u r e s 2 d . S I F T _ c r e a t e ( )# i m a g e 1 c l r = cv . r e s i z e ( i m a g e 1 c l r , ( 0 , 0 ) , fx = 0 . 2 , fy = 0 . 2 )# image1gray = cv . c v t C o l o r ( i m a g e 1 c l r , cv .COLOR_BGR2GRAY)##kp1 , des1 = s i f t . de tec tAndCompute ( image1gray , None )## h a r r i s 1 = [ ]## f o r i i n kp1 :# h a r r i s 1 . append ( [ i . p t [ 1 ] , i . p t [ 0 ] ] )#### i m a g e 2 c l r = cv . imread ( ’m4 . jpg ’ )## i m a g e 2 c l r = cv . r e s i z e ( i m a g e 2 c l r , ( 0 , 0 ) , fx = 0 . 2 , fy = 0 . 2 )# image2gray = cv . c v t C o l o r ( i m a g e 2 c l r , cv .COLOR_BGR2GRAY)#kp2 , des2 = s i f t . de tec tAndCompute ( image2gray , None )## h a r r i s 2 = [ ]## f o r i i n kp2 :# h a r r i s 2 . append ( [ i . p t [ 1 ] , i . p t [ 0 ] ] )##SSD = p 2 p _ c o r r e s ( des1 , des2 )### f o r i i n r a n g e ( 1 , 1 0 0 0 ) :## i f ( np . sum (SSD < i ) >= 1 0 0 0 ) :## l l = i## b r e a k ;## points1_SSD , points2_SSD , points1_NCC , points2_NCC = r e t u r n _ c o r r e s _ p o i n t s ( SSD , h a r r i s 1 , h a r r i s 2 , 2 0 0 , 0 . 9 9 )## poin ts1_SSD = np . a s a r r a y ( poin ts1_SSD ) . a s t y p e ( ’ i n t ’ )# poin ts2_SSD = np . a s a r r a y ( poin ts2_SSD ) . a s t y p e ( ’ i n t ’ )# points1_NCC = np . a s a r r a y ( poin ts1_SSD ) . a s t y p e ( ’ i n t ’ )# points2_NCC = np . a s a r r a y ( poin ts2_SSD ) . a s t y p e ( ’ i n t ’ )#### nimage1 = d r a w _ l i n e s ( i m a g e 1 c l r , i m a g e 2 c l r , points1_SSD , points2_SSD , points1_NCC , points2_NCC , 0 )## s t r i = ’ s i f t . jpg ’

    20

  • # cv . i m w r i t e ( s t r i , nimage1 )

    #Cannyimage1gray = cv . c v t C o l o r ( i m a g e 1 c l r , cv .COLOR_BGR2GRAY)edges1 = cv . Canny ( image1gray , 2 5 5 * 1 . 5 , 2 5 5 )

    image2gray = cv . c v t C o l o r ( i m a g e 2 c l r , cv .COLOR_BGR2GRAY)edges2 = cv . Canny ( image2gray , 2 5 5 * 1 . 5 , 2 5 5 )

    points1_SSD , poin ts2_SSD = r e t u r n _ c o r r e s p o n d e n c e s ( edges1 , edges2 ) ;

    poin ts1_SSD = np . a s a r r a y ( poin ts1_SSD ) . a s t y p e ( ’ i n t ’ )poin ts2_SSD = np . a s a r r a y ( poin ts2_SSD ) . a s t y p e ( ’ i n t ’ )points1_NCC = np . a s a r r a y ( poin ts1_SSD ) . a s t y p e ( ’ i n t ’ )points2_NCC = np . a s a r r a y ( poin ts2_SSD ) . a s t y p e ( ’ i n t ’ )

    nimage1 = d r a w _ l i n e s ( i m a g e 1 c l r , i m a g e 2 c l r , points1_SSD , points2_SSD , points1_NCC , points2_NCC , 0 )

    s t r i = ’ s i f t . jpg ’cv . i m w r i t e ( s t r i , nimage1 )

    #############Img1 = [ ]Img2 = [ ]f o r i i n r a n g e ( l e n ( poin ts1_SSD ) ) :

    Img1 . append ( [ poin ts1_SSD [ i ] [ 0 ] , po in ts1_SSD [ i ] [ 1 ] , 1 . 0 ] )Img2 . append ( [ poin ts2_SSD [ i ] [ 0 ] , po in ts2_SSD [ i ] [ 1 ] , 1 . 0 ] )

    f o r j i n r a n g e ( l e n ( kkk1 ) ) :Img1 . append ( [ kkk1 [ j ] [ 1 ] , kkk1 [ j ] [ 0 ] , 1 . 0 ] )Img2 . append ( [ kkk2 [ j ] [ 1 ] , kkk2 [ j ] [ 0 ] , 1 . 0 ] )

    cp1 = np . a s a r r a y ( Img1 )cp2 = np . a s a r r a y ( Img2 )

    # f o r i i n r a n g e ( 0 , 8 ) :# cv . c i r c l e ( im1 , ( i n t ( cp1 [ i , 0 ] ) , i n t ( cp1 [ i , 1 ] ) ) , 1 , ( 0 , 0 , 2 5 5 ) , −1)# cv . c i r c l e ( im2 , ( i n t ( cp2 [ i , 0 ] ) , i n t ( cp2 [ i , 1 ] ) ) , 1 , ( 0 , 0 , 2 5 5 ) , −1)

    # n o r m a l i z e d p o i n t snp1 , T1 = n o r m a l i z e _ p o i n t s ( cp1 )np2 , T2 = n o r m a l i z e _ p o i n t s ( cp2 )

    Fde = f i n d F ( np1 , np2 )

    F = np . matmul ( np . t r a n s p o s e ( T2 ) , np . matmul ( Fde , T1 ) ) ;

    F = F / F [ 2 , 2 ]u , s , vh = np . l i n a l g . svd ( F )

    e1 = np . t r a n s p o s e ( vh [ 2 ] )e2 = u [ : , 2 ]

    21

  • P = np . z e r o s ( ( 3 , 4 ) )

    P [ 0 : 3 , 0 : 3 ] = np . eye ( 3 )

    e2x = a_x ( e2 )

    P_dash = np . z e r o s ( ( 3 , 4 ) )

    P_dash [ 0 : 3 , 0 : 3 ] = np . matmul ( e2x , F )P_dash [ 0 : 3 , 3 ] = e2F = F / F [ 2 , 2 ]f = np . z e r o s ( 9 )f [ 0 ] = F [ 0 ] [ 0 ]f [ 1 ] = F [ 0 ] [ 1 ]f [ 2 ] = F [ 0 ] [ 2 ]f [ 3 ] = F [ 1 ] [ 0 ]f [ 4 ] = F [ 1 ] [ 1 ]f [ 5 ] = F [ 1 ] [ 2 ]f [ 6 ] = F [ 2 ] [ 0 ]f [ 7 ] = F [ 2 ] [ 1 ]f [ 8 ] = F [ 2 ] [ 2 ]

    # r e s = l e a s t _ s q u a r e s ( c o s t f , f , method = ’ lm ’ )# f o r i i n r a n g e ( 0 , 8 )f1 = f

    F1 = np . z e r o s ( ( 3 , 3 ) )F1 [ 0 , 0 ] = f1 [ 0 ]F1 [ 0 , 1 ] = f1 [ 1 ]F1 [ 0 , 2 ] = f1 [ 2 ]F1 [ 1 , 0 ] = f1 [ 3 ]F1 [ 1 , 1 ] = f1 [ 4 ]F1 [ 1 , 2 ] = f1 [ 5 ]F1 [ 2 , 0 ] = f1 [ 6 ]F1 [ 2 , 1 ] = f1 [ 7 ]F1 [ 2 , 2 ] = f1 [ 8 ]

    F1 = F1 / F1 [ 2 , 2 ]

    #u , s , vh = np . l i n a l g . svd ( F1 )## s [ 2 ] = 0##F1 = np . matmul ( u , np . matmul ( np . d i a g ( s ) , vh ) )#u , s , vh = np . l i n a l g . svd ( F1 )e1 = np . t r a n s p o s e ( vh [ 2 ] )e2 = u [ : , 2 ]

    P = np . z e r o s ( ( 3 , 4 ) )

    P [ 0 : 3 , 0 : 3 ] = np . eye ( 3 )

    22

  • e1 = e1 / e1 [ 2 ]e2 = e2 / e2 [ 2 ]e2x = a_x ( e2 )

    P_dash = np . z e r o s ( ( 3 , 4 ) )P_dash [ 0 : 3 , 0 : 3 ] = np . matmul ( e2x , F1 )P_dash [ 0 : 3 , 3 ] = e2

    xw = np . z e r o s ( ( l e n ( cp1 ) , 3 ) )f o r i i n r a n g e ( 0 , l e n ( cp1 ) ) :

    A = np . z e r o s ( ( 4 , 4 ) ) ;A[ 0 ] = cp1 [ i ] [ 0 ] * P [ 2 , : ] − P [ 0 , : ]A[ 1 ] = cp1 [ i ] [ 1 ] * P [ 2 , : ] − P [ 1 , : ]A[ 2 ] = cp2 [ i ] [ 0 ] * P_dash [ 2 , : ] − P_dash [ 0 , : ]A[ 3 ] = cp2 [ i ] [ 1 ] * P_dash [ 2 , : ] − P_dash [ 1 , : ]

    u , s , vh = np . l i n a l g . svd (A)

    xw1 = np . t r a n s p o s e ( vh [ 3 ] )xw1 = xw1 / xw1 [ 3 ]

    xw [ i ] = xw1 [ 0 : 3 ]

    f i g = p l t . f i g u r e ( )ax = f i g . a d d _ s u b p l o t ( 1 1 1 , p r o j e c t i o n = ’3d ’ )ax . s c a t t e r ( xw [ : , 0 ] , xw [ : , 1 ] , xw [ : , 2 ] )# ax . p l o t ( xw [ 5 0 : 5 2 , 0 ] , xw [ 5 0 : 5 2 , 1 ] , xw [ 5 0 : 5 2 , 2 ] )# ax . p l o t ( xw [ 5 2 : 5 4 , 0 ] , xw [ 5 2 : 5 4 , 1 ] , xw [ 5 2 : 5 4 , 2 ] )# ax . p l o t ( xw [ 5 4 : 5 6 , 0 ] , xw [ 5 4 : 5 6 , 1 ] , xw [ 5 4 : 5 6 , 2 ] )# ax . p l o t ( [ xw [ 5 0 , 0 ] , xw [ 5 2 , 0 ] ] , [ xw [ 5 0 , 1 ] , xw [ 5 2 , 1 ] ] , [ xw [ 5 0 , 2 ] , xw [ 5 2 , 2 ] ] )# ax . p l o t ( [ xw [ 5 2 , 0 ] , xw [ 5 4 , 0 ] ] , [ xw [ 5 2 , 1 ] , xw [ 5 4 , 1 ] ] , [ xw [ 5 2 , 2 ] , xw [ 5 4 , 2 ] ] )# ax . p l o t ( [ xw [ 5 1 , 0 ] , xw [ 5 3 , 0 ] ] , [ xw [ 5 1 , 1 ] , xw [ 5 3 , 1 ] ] , [ xw [ 5 1 , 2 ] , xw [ 5 3 , 2 ] ] )# ax . p l o t ( [ xw [ 5 3 , 0 ] , xw [ 5 5 , 0 ] ] , [ xw [ 5 3 , 1 ] , xw [ 5 5 , 1 ] ] , [ xw [ 5 3 , 2 ] , xw [ 5 5 , 2 ] ] )

    23

    TheoryFundamental matrix Image RectificationUsing SIFT and canny edge detectors

    Resultscode