1
www.postersession.com Feasibility of Structure from Motion over Planetary Bodies using Small Satellites Mapping and Ocean Color Imaging Satellite (MOCI Sat) – University Nanosat Program 9 – AFRL Caleb Adams *1 , David L. Cotten ^1,2 , Hollis Neel 2 , Nirav Ilango 2 , Graham Grable 2 , Deepak Mishra 1,2 * [email protected], ^ [email protected], 1 Small Satellite Research Laboratory, University of Georgia, 2 Center for Geospatial Research, University of Georgia, A total of 22 initial rendered simulations were performed using the workflow described in Figure 2. In general, the following was used to derive the results of this study: Graphical simulations of our mission were conducted with Python, Blender, VSFM, and MeshLab to evaluate the feasibility of SfM 3D reconstructions. This study simulates a satellite taking images at optimal angles and its ability to perform SfM. 3D reconstructions of a satellite, planet, and objects that serve as final targets were made to scale. Various tests were conducted with differing image counts for each test (varying from 2 - 100), with each image serving as a view of the target objects from different angles. These images were then piped into an open-source SfM software and an open-source mesh generation (which provides the texture to geographic models) to produce 3D models. On tests with high image counts and sufficient noise reduction, the Blender simulations were able to render relatively accurate 3D models. Dense point clouds were generated with over 100,000 points, which indicates a high quality 3D model and meets one full mission requirement. Overview Funding for this research provided by the Air Force Research Laboratory ninth cycle of the University Nanosatellite Program. For more information about CubeSats, the Small Satellite Research Laboratory (SSRL), and our missions, visit www.smallsat.uga.edu. Planetary Structure from Motion with Existing Data Sets Figure 2 The MOCI satellite currently utilizes the GomSpace Nanocam (NC1U) for gathering imaging data. It is possible for the MOCI satellite to utilize the existing system seen in Figure 2 which compares a simulated workflow workflow to the actual workflow. It works as follows: 1. Produce an image set, using the blender software package, equivalent to what the NC1U would produce in terms of spatial resolution and size. This software is capable of reproducing any camera position and specification. 2. Run SfM on the image set with Visual Structure from Motion (VSFM) to produce sparse and dense point clouds with CMVS and PVMS. 3. MeshLab is used to perform Poisson surface reconstruction and provide a final textured surface model. Simulating the Orbital Imaging Process The fundamental concept of the Mapping and Ocean Color Imager (MOCI) relies on the feasibility of Structure from Motion (SfM) from Low Earth Orbit. If this concept does not prove feasible, then the MOCI mission cannot be successful. Fortunately, it seems that this concept can be achieved by building on the foundations of some existing technologies. This poster is based on an internal study that sought to quantify the feasibility of SfM with these technologies. It also seeks to find key areas were new technologies are needed and to suggest a path forward in those areas, specifically for the University of Georgia’s Small Satellite Research Laboratory (SSRL). Simulating Planetary Structure from Motion Figure 3 Figure 4 Figure 5 Figure 6 To start, the Earth was modeled to scale in blender and several test objects were placed onto the surface, see Figure 3. In this example, 60 images were taken of the target area. A sparse point cloud was generated using Clustering Views for Multi-view Stereo (CVMS), see Figure 4. Next, a dense point cloud was generated using Patch-based Multi-view Stereo (PMVS), as seen in Figure 5. The final result is the mesh seen in Figure 6. This is the product manually removing “noise” points in the dense cloud and producing a surface using Poisson surface reconstruction. Significant techniques are needed so that the noise on the surface can be minimized. The resulting dense point cloud, from which the mesh was generated, consisted of 107,852 vertices. The Mesh only consisted of 45,312 vertices after the final computation. The raw images need to compute the point could come to a total of 5.39MB while the mesh is a total of 3.9MB. Such a reduction in data size is ideal for space based mapping applications. Customizing Structure from Motion Figure 7 Figure 8 Figure 9 Figure 10 A typical implementation of the Structure from Motion algorithm can be seen in Figure 1. Starting with an image set, a Scale Invariant Feature Transform (SIFT) is performed. A package, known as bundler, produces a sparse point cloud. A Multi-view Stereo (CVMS) and Patch-based Multi-view Stereo (PMVS) are used to produce a dense point cloud. SIFT was identified as a primary target for optimization as it is the most time intensive step in the SfM process. Future studies will include SIFT implements in a Field Programmable Gate Array (FPGA) to reduce computation time and power consumption. Some tests were performed with existing space based data to determine what the current capabilities of SfM could achieve. Figure 7, for example, is a dense point cloud of the earth’s upper atmosphere generated from a crew operated high definition camera on the International Space Station (ISS). This data sample consisted of 30 images and a total data size of 50.1MB. Figure 8 shows the resulting mesh from this data set and dense cloud. The resulting mesh is a total size of 12.6MB, an improvement for a space based system. Acquire Photo Set SIFT Bundler/Sparse Cloud CMVS PMVS/Dense Cloud Figure 1 Figure 9 shows a dense point cloud of Pluto generated from the New Horizon data set. Using only 15 images from the fly by, a dense point cloud of 39,804 vertices was computed. This was used to build a mesh of 42,866 vertices, Figure 10. This is impressive considering the ISS data, from 400km, generated 146,138 vertices. In the future, these surface models of Pluto will be compared to existing Digital Elevation Models (DEM) from the fly by. Given that large cumulonimbus clouds clouds have a maximum height of 15 km, we can at least confirm that unmodified SfM can distinguish a height differential of at least 15 km from LEO.

Feasibility of Structure from Motion over Planetary Bodies

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Feasibility of Structure from Motion over Planetary Bodies

www.postersession.com

FeasibilityofStructurefromMotionoverPlanetaryBodiesusingSmallSatellitesMappingandOceanColorImagingSatellite(MOCISat)– UniversityNanosatProgram9– AFRL

CalebAdams*1,DavidL.Cotten^1,2,HollisNeel2,Nirav Ilango2,GrahamGrable2,DeepakMishra1,2*[email protected],^[email protected],1SmallSatelliteResearchLaboratory,UniversityofGeorgia,

2CenterforGeospatialResearch,UniversityofGeorgia,

A total of 22 initial rendered simulations were performed using the workflowdescribed in Figure 2. In general, the following was used to derive the resultsof this study:• Graphical simulations of our mission were conducted with Python, Blender,

VSFM, and MeshLab to evaluate the feasibility of SfM 3D reconstructions.• This study simulates a satellite taking images at optimal angles and its

ability to perform SfM.• 3D reconstructions of a satellite, planet, and objects that serve as final

targets were made to scale.• Various tests were conducted with differing image counts for each test

(varying from 2 - 100), with each image serving as a view of the targetobjects from different angles.

• These images were then piped into an open-source SfM software and anopen-source mesh generation (which provides the texture to geographicmodels) to produce 3D models.

• On tests with high image counts and sufficient noise reduction, the Blendersimulations were able to render relatively accurate 3D models.

• Dense point clouds were generated with over 100,000 points, whichindicates a high quality 3D model and meets one full mission requirement.

Overview

Funding for this research provided by the Air Force Research Laboratory ninth cycle of the University Nanosatellite Program. For more information about CubeSats, the Small Satellite Research Laboratory (SSRL), and our missions, visit www.smallsat.uga.edu.

Planetary Structure from Motion with Existing Data Sets

Figure 2The MOCI satellite currently utilizes the GomSpace Nanocam (NC1U) forgathering imaging data. It is possible for the MOCI satellite to utilize theexisting system seen in Figure 2 which compares a simulated workflowworkflow to the actual workflow. It works as follows:1. Produce an image set, using the blender software package, equivalent towhat the NC1U would produce in terms of spatial resolution and size. Thissoftware is capable of reproducing any camera position and specification.

2. Run SfM on the image set with Visual Structure from Motion (VSFM) toproduce sparse and dense point clouds with CMVS and PVMS.

3. MeshLab is used to perform Poisson surface reconstruction and provide afinal textured surface model.

Simulating the Orbital Imaging Process

The fundamental concept of the Mapping and Ocean Color Imager (MOCI)relies on the feasibility of Structure from Motion (SfM) from Low Earth Orbit. Ifthis concept does not prove feasible, then the MOCI mission cannot besuccessful. Fortunately, it seems that this concept can be achieved bybuilding on the foundations of some existing technologies. This poster isbased on an internal study that sought to quantify the feasibility of SfM withthese technologies. It also seeks to find key areas were new technologies areneeded and to suggest a path forward in those areas, specifically for theUniversity of Georgia’s Small Satellite Research Laboratory (SSRL).

Simulating Planetary Structure from Motion

Figure 3 Figure 4

Figure 5 Figure 6

To start, the Earth was modeled to scale in blender and several test objectswere placed onto the surface, see Figure 3. In this example, 60 imageswere taken of the target area. A sparse point cloud was generated usingClustering Views for Multi-view Stereo (CVMS), see Figure 4. Next, adense point cloud was generated using Patch-based Multi-view Stereo(PMVS), as seen in Figure 5. The final result is the mesh seen in Figure 6.This is the product manually removing “noise” points in the dense cloudand producing a surface using Poisson surface reconstruction. Significanttechniques are needed so that the noise on the surface can be minimized.The resulting dense point cloud, from which the mesh was generated,consisted of 107,852 vertices.

The Mesh only consisted of 45,312 vertices after the finalcomputation. The raw images need to compute the point could cometo a total of 5.39MB while the mesh is a total of 3.9MB. Such areduction in data size is ideal for space based mapping applications.

Customizing Structure from Motion

Figure 7

Figure 8

Figure 9 Figure 10

A typical implementation of the Structure from Motionalgorithm can be seen in Figure 1. Starting with animage set, a Scale Invariant Feature Transform (SIFT)is performed. A package, known as bundler, producesa sparse point cloud. A Multi-view Stereo (CVMS) andPatch-based Multi-view Stereo (PMVS) are used toproduce a dense point cloud. SIFT was identified as aprimary target for optimization as it is the most timeintensive step in the SfM process. Future studies willinclude SIFT implements in a Field ProgrammableGate Array (FPGA) to reduce computation time andpower consumption.

Some tests were performedwith existing space based datato determine what the currentcapabilities of SfM couldachieve. Figure 7, forexample, is a dense pointcloud of the earth’s upperatmosphere generated from acrew operated high definitioncamera on the InternationalSpace Station (ISS). This datasample consisted of 30images and a total data size of50.1MB. Figure 8 shows theresulting mesh from this dataset and dense cloud. Theresulting mesh is a total size of12.6MB, an improvement for aspace based system.

Acquire Photo Set

SIFT

Bundler/Sparse Cloud

CMVS

PMVS/Dense Cloud

Figure 1

Figure 9 shows a dense point cloud of Pluto generated from the NewHorizon data set. Using only 15 images from the fly by, a dense pointcloud of 39,804 vertices was computed. This was used to build amesh of 42,866 vertices, Figure 10. This is impressive consideringthe ISS data, from 400km, generated 146,138 vertices. In the future,these surface models of Pluto will be compared to existing DigitalElevation Models (DEM) from the fly by.

Given that large cumulonimbus clouds clouds have a maximumheight of 15 km, we can at least confirm that unmodified SfM candistinguish a height differential of at least 15 km from LEO.