GOOGLE STREET VIEW: Capturing The World At Street Level-Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh,

Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver, Google

2010

 

Ramika Hegderamikahe@usc.edu

Introduction

Friday, April 21, 2023 Introduction 2

• SummaryStreet View serves millions of Google users daily with panoramic imagery captured in hundreds of cities in 20 countries across four continents. A team of Google researchers describes the technical challenges involved in capturing, processing, and serving street-level imagery on a global scale.

• Inspiration ?Larry Page drove around the San Francisco Bay Area and recorded several hours of video footage using a camcorder pointed at building facades. Larry’s idea-this kind of imagery useful at a larger scaleResearch collaboration with Stanford University called CityBlock that soon thereafter became Google Street View .

• Concern The World has 50 million miles of roads across 219 countries. Driving through these roads is

equivalent to circumnavigating the globe 1250 times!

Generation of Vehicles

Wednesday, April 24, 2013 Generation of Vehicles 3

• 1st Generation Garage Phase

Cameras, Lasers, and a GPS on the roof & several computers in its trunk. Disadvantage- Drive through multiple times to capture images

• 2nd Generation Chevy Van

Laser scanners 2 high-speed video cameras, 8 high-resolution cameras (rosette (R) configuration) Computers recording data (20 HD at 500 Mbps.) Disadvantage- could not be built and operated at scale.

• 3rd Generation Lite Cars Focus-Reliability . Recorded wheel encoded messages from the antilock brake system. Advantage-recorded a vast amount of Imagery enabling expansion to Austrailia , New Zealand,

Japan. Disadvantage-Low Image Resolution.

Generation of Vehicles – contd…

4

• Fourth Generation ‘R5’ : Street View’s Panoramic camera system.

Custom hinged mast that allows the camera to be retracted when the vehicle passed under low bridges. 3 Laser scanners on the mast enabling the capture of 3D data alongside the imagery. Captured the majority of imagery live in Street View today. Fifth Generation vehicle design in works.

• Links: The White House Tour.

Data Collection Platforms

• In Parallel with the road vehicles, several other data collection platforms were developed-Trike , Trolley , Snowmobile ,Trekker.

Snowmobile

Trike

Trolley Trekker

Wednesday, April 24, 2013 Data Collection Platforms 5

• Hard drives are sensitive to shock, vibration, and temperature extremes, both while the vehicle is in operation and, to a lesser degree, while being shipped.

• Techniques to minimize data loss

•Shock-mounted disk enclosures

•Custom-shipping packaging with extra-thick foam

•Solid-state disk drives

Wednesday, April 24, 2013 Confrontations 6

Confrontations

Street View Cameras

Wednesday, April 24, 2013 Street View Cameras 7

R7 Street View camera system.

R2 System• a ring of eight 11-megapixel, interline-transfer, charge-coupled device (CCD)

sensors with commercial photographic wide-angle lenses.

R5 System • ring of eight cameras plus a fish-eye lens on top to capture upper levels of

buildings.

R7 System• 15 of these same sensors and lenses, but no fish-eye, to get high-resolution

images over an increased field of view.360 degree view

Pose Optimization

• Accurate position estimates of Street View vehicles are essential for associating our high-resolution panoramas with a street map and for enabling an intuitive navigation experience.

• A batch algorithm open sourced by Google to achieve a smoother and locally accurate solution for the pose trajectory. This trajectory is computed at a resolution of 100 Hz.

• An online Kalman-filter-based algorithm is deployed on the vehicles to provide real-time navigation information to the drivers.

• A probabilistic graphical model of the network is constructed . The model includes detailed knowledge about one-way streets and turn restrictions, display approximate street address information, and draw blue overlays on the map

Wednesday, April 24, 2013 Pose Optimization 8

Navigating Street View Imagery

• 360 degree panorama is the most popular among the street view surfaces.

• The experience is made richer by combining Street View Imagery with data sources.

• Building on the 3D data that we collect as well as Google Maps data, we can place markers and overlays in the scene, resulting in 3D-annotated Street View images

• Google Street view is open to user contribution- Users can correct the exact locations of points of interest by directly dragging markers in Street View and automatically snapping it to facades-Figure(1)

• Street View also surfaces user-contributed photos from Flickr, Panoramio , and Picasa in Street View-Figure(2)

Wednesday, April 24, 2013 Navigating Street View Imagery 9

Figure(1) Figure(2)

Leveraging 3D data for Smart Navigation

• Click-to-go:3D Navigation Mode

•Lets users click their mouse on a point in the scene and be transported to the image nearest to that point’s 3D location.

•Enabling such a feature requires creation of a DEPTH MAP.

Wednesday, April 24, 2013 Leveraging 3D data for Smart Navigation 10

Depth Map

• It stores the distance and orientation of every point in the scene .It encodes the scene’s dominant surfaces, such as building facades and roads , while ignoring smaller entities such as cars and people.

• Laser range scans or image motion (optical flow) when laser data isn’t available.

Laser Range Scan-which accurately measure the depth of a vertical fan of points on the two sides

and the front of the vehicle.(Figure 4)• In the absence of laser range data, we recover the depth by computing optical flow between

successive images of the street facade on both sides of the vehicle. The optical flow at a given

point depends on the vehicle’s motion and that point’s depth.

Wednesday, April 24, 2013 Depth Map 11

Depth Map-contd…

• Panoramic Depth Map:

• Tracing rays from each panorama position.

• Each pixel in the depth map represents a lookup into a table of 3D plane equations, which enables the client code to reconstruct the real depth values at runtime.

• Lossless Compression: The encoded depth map is just a few KB in size. Turning photos into 360 degree Panorama view.

Wednesday, April 24, 2013 Depth Map-contd 12

Computing 3D Models

• Street View data to create photorealistic 3D models for Google Earth.

• Google Earth created 3D city models using airborne imagery, resulting in low-resolution facades with little detail are Suitable for fly-through an Unsuitable for walk-through experience.

• 3D facade models reconstructed from Street View’s laser scans and imagery are high resolution.

• Final 3D Facade Models : existing airborne models are fused into a single model that has high-resolution facades as well as rooftops and back sides from an airborne view.

Wednesday, April 24, 2013 Computing 3D Models 13

Original 3D models of a New York (airborne data) only. (b) Fused 3D model

with high-resolution facades.

Pros & Cons of the Paper

• Pros

Gives us a surface view of how exactly Google Street View works.

Enlightens us with the evolution of Vehicles used in Google Street View.

Depicts how camera equipment has changed from Street View’s Initial Stage to the

Current Stage.

Describes ‘Click to go’ Feature well.

• Cons

Doesn’t give us a clear picture of Depth Map.

Doesn’t describe how User’s can contribute to Street View.

Hasn’t mentioned additional information provided-traffic and weather updates.

Wednesday, April 24, 2013 Pros & Cons of the Paper 14

Additional

• Wi-Spy- User Privacy.

• Germany fines Street View.

• Apple’s Street View.

• Apple’s Map failure.

• Street View Expansion.

• Google Street View Hyperlase.

Wednesday, April 24, 2013 Additional 15