Gap search system

Chapter 2

LITERATURE SURVEY

A literature survey, or literature review, is a proof essay of sorts. It is a study and

review of relevant literature materials in relation to a topic you have been given. Literature

survey is the documentation of a comprehensive review of the published and unpublished

work from secondary sources data in the areas of specific interest to the researcher.

Google achieves great success by moving its navigation applications, such as Google

Map, to mobile phone. As tens of thousands people could afford mobile phones, how to

locate in outdoor with the help of computer vision and GPS technology becomes a new hot

topic. A few systems have been proposed making use of built-in camera and image retrieval

methods for this problem. In, a hybrid image-and-keyword searching system has been

devised.

First, image is used to search through WebPages, and then keywords on these

WebPages are identified and submitted to existing text search engine, such as Google. A

group from Microsoft Research Asia has conducted an experiment on Photo-to-Search

system, which makes use of image retrieval methods to locate around the world which gives

an image retrieval system based on Content Based Image Retrieval methods.

Different from these approaches, the focus is not just on navigation but also a fresh

way to provide information about the world by pictures.

Let us consider the situation where users are in an unfamiliar environment and they

want to get information about an unknown structure.

The users can take a picture of that structure with their phones, and upload it to the

system. After that, the system recognizes the structure and returns useful information.

2.1 Existing systems

Google achieves great success by moving its navigation applications, such as Google

Map, to mobile phone. As tens of thousands people could afford mobile phones, how to

locate in outdoor with the help of computer vision and GPS technology becomes a new hot

topic. A few systems have been proposed making use of built-in camera and image retrieval

Dept. of CSE, SJBIT Page 1

Gap Search System 2013-2014

methods for this problem. In, a hybrid image-and-keyword searching system has been

devised. First, image is used to search through WebPages, and then keywords on these

WebPages are identified and submitted to existing text search engine, such as Google. In, a

group from Microsoft Research

Asia has conducted an experiment on Photo-to-Search system, which makes use of image

retrieval methods to locate around the world. Gives an image retrieval system based on

Content Based Image Retrieval methods.

Different from these approaches, our focus is not just on navigation but also a fresh

way to provide information about the world by pictures. Let us consider the situation that a

user is in an unfamiliar environment and he/she wants to get information on an unknown

object (building, sculpture). The user can take a photo of that object with their phones, and

upload it to our system. After that, our system recognizes the object and returns useful

information. For experiment purpose, we choose the buildings of Gulou Campus of Nanjing

University as our start point.

2.2 Java

Java is a programming language originally developed by James Gosling at Sun

Microsystems (which is now a subsidiary of Oracle Corporation) and released in 1995 as a

core component of Sun Microsystems Java platform. The language derives much of

its syntax from C and C++ but has a simpler object model and fewer low-level facilities. Java

applications are typically compiled to bytecodes (class file) that can run on any Java Virtual

Machine (JVM) regardless of computer architecture. Java is a general-purpose, concurrent,

class-based, object-oriented language that is specifically designed to have as few

implementation dependencies as possible. It is intended to let application developers "write

once, run anywhere". Java is currently one of the most popular programming languages in

use, and is widely used from application software to web applications.

The original and reference implementation Java compilers, virtual machines,

and class libraries were developed by Sun from 1995. As of May 2007, in compliance with

the specifications of the Java Community Process, Sun relicensed most of its Java

technologies under the GNU General Public License. Others have also developed alternative



implementations of these Sun technologies, such as the GNU Compiler for Java, GNU

Classpath and Dalvik.

James Gosling, Mike Sheridan, and Patrick Naughton initiated the Java language

project in June 1991. Java was originally designed for interactive television, but it was too

advanced for the digital cable television industry at the time. The language was initially

called Oak after an oak tree that stood outside Gosling's office; it went by the

name Green later, and was later renamed Java, from a list of random words. Gosling aimed to

implement a virtual machine and a language that had a familiar C/C++ style of notation.

Sun Microsystems released the first public implementation as Java 1.0 in 1995. It

promised "Write Once, Run Anywhere”, providing no-cost run-times on popular platforms.

Fairly secure and featuring configurable security, it allowed network- and file-access

restrictions. Major web browsers soon incorporated the ability to run Java applets within web

pages, and Java quickly became popular. With the advent of Java 2 (released initially as J2SE

1.2 in December 1998–1999), new versions had multiple configurations built for different

types of platforms. For example, J2EE targeted enterprise applications and the greatly

stripped-down version J2MEfor mobile applications (Mobile Java). J2SE designated the

Standard Edition. In 2006, for marketing purposes, Sun renamed new J2 versions as Java

EE, Java ME, and Java SE, respectively.

On November 13, 2006, Sun released much of Java as open source software under the

terms of the GNU General Public License (GPL). On May 8, 2007, Sun finished the process,

making all of Java's core code available under free software/open-source distribution terms,

aside from a small portion of code to which Sun did not hold the copyright. Sun's vice-

president Rich Green has said that Sun's ideal role with regards to Java is as an "evangelist."

Following Oracle Corporation's acquisition of Sun Microsystems in 2009–2010,

Oracle has described itself as the “steward of Java technology with a relentless commitment

to fostering a community of participation and transparency”.



2.2.1 Java Principles

There were five primary goals in the creation of the Java language

1. It should be "simple, object oriented and familiar".

2. It should be "robust and secure".

3. It should have "an architecture-neutral and portable environment".

4. It should execute with "high performance".

5. It should be "interpreted, threaded, and dynamic".

2.2.2 Java Features

Platform Independent

The concept of Write-once-run-anywhere (known as the Platform independent)

is one of the important key feature of java language that makes java as the most powerful

language. Not even a single language is idle to this feature but java is closer to this

feature. The programs written on one platform can run on any platform provided the

platform must have the JVM.

Simple

There are various features that make the java as a simple language.

Programs are easy to write and debug because java does not use the pointers

explicitly. It is much harder to write the java programs that can crash the system but

we cannot say about the other programming languages. Java provides the bug free

system due to the strong memory management. It also has the automatic memory

allocation and de allocation system.

Object-Oriented

To be an Object Oriented language, any language must follow at least the

four characteristics.



Inheritance: It is the process of creating the new classes and using the

behavior of the existing classes by extending them just to reuse the existing

code and adding the additional features as needed.

Encapsulation: It is the mechanism of combining the information and

providing the abstraction.

Polymorphism: As the name suggest one name multiple form, Polymorphism

is the way of providing the different functionality by the functions having the

same name based on the signatures of the methods.

Dynamic binding: Sometimes we don't have the knowledge of objects about

their specific types while writing our code. It is the way of providing the

maximum functionality to a program about the specific type at runtime.

As the languages like Objective C, C++ fulfills the above four characteristics yet

they are not fully object oriented languages because they are structured as well as object

oriented languages. But in case of java, it is a fully Object Oriented language because object

is at the outer most level of data structure in java. No stand alone methods, constants, and

variables are there in java. Everything in java is object even the primitive data types can also

be converted into object by using the wrapper class.

Robust

Java has the strong memory allocation and automatic garbage collection

mechanism. It provides the powerful exception handling and type checking

mechanism as compare to other programming languages. Compiler checks the

program whether there any error and interpreter checks any run time error and makes

the system secure from crash. All of the above features make the java language

robust.

Dynamic

While executing the java program the user can get the required files

dynamically from a local drive or from a computer thousands of miles away from

the user just by connecting with the Internet.



Distributed

The widely used protocols like HTTP and FTP are developed in java.

Internet programmers can call functions on these protocols and can get access the

files from any remote machine on the internet rather than writing codes on their

local system.

Portable

The feature Write-once-run-anywhere makes the java language portable

provided that the system must have interpreter for the JVM. Java also has the

standard data size irrespective of operating system or the processor. These features

make the java as a portable language.

Secure

Java does not use memory pointers explicitly. All the programs in java are

run under an area known as the sand box. Security manager determines the

accessibility options of a class like reading and writing a file to the local disk. Java

uses the public key encryption system to allow the java applications to transmit over

the internet in the secure encrypted form. The byte code Verifier checks the classes

after loading.

Performance

Java uses native code usage, and lightweight process called threads. In the

beginning interpretation of byte code resulting in the performance slow but the

advance version of JVM uses the adaptive and just in time compilation technique that

improves the performance.

Multithreaded



As we all know several features of Java like Secure, Robust, Portable,

dynamic etc; you will be more delighted to know another feature of Java which

is Multithreaded. Java is also a multithreaded programming language.

Multithreading means a single program having different threads executing

independently at the same time. Multiple threads execute instructions according to

the program code in a process or a program. Multithreading works the similar way

as multiple processes run on one computer. Multithreading programming is a very

interesting concept in Java. In multithreaded programs not even a single thread

disturbs the execution of other thread. Threads are obtained from the pool of

available ready to run threads and they run on the system CPUs. This is how

Multithreading works in Java which you will soon come to know in details in later

chapters.

Interpreted

We all know that Java is an interpreted language as well. With an interpreted

language such as Java, programs run directly from the source code. The interpreter

program reads the source code and translates it on the fly into computations. Thus,

Java as an interpreted language depends on an interpreter program. The versatility of

being platform independent makes java to outshine from other languages. The

source code to be written and distributed is platform independent. Another

advantage of Java as an interpreted language is its error debugging quality. Due to

this any error occurring in the program gets traced. This is how it is different to

work with Java.

2.3 Android

Android is a software stack for mobile devices that includes an operating system

middleware and key applications. Google Inc. purchased the initial developer of the software,

Android Inc., in 2005. Android's mobile operating system is based on a modified version of

the Linux kernel. Google and other members of the Open Handset Alliance collaborated on

Android's development and release. The Android Open Source Project (AOSP) is tasked with



the maintenance and further development of Android. The Android operating system is the

world's best-selling Smartphone platform.

Android has a large community of developers writing applications ("apps") that

extend the functionality of the devices. There are currently over 150,000 apps available for

Android. Android Market is the online app store run by Google, though apps can also be

downloaded from third-party sites. Developers write primarily in the Java language,

controlling the device via Google-developed Java libraries.

The unveiling of the Android distribution on 5 November 2007 was announced with

the founding of the Open Handset Alliance, a consortium of 80hardware, software,

and telecom companies devoted to advancing open standards for mobile devices. Google

released most of the Android code under the Apache License, a free software and open

source license.

The Android open-source software stack consists of Java applications running on a

Java-based, object-oriented application framework on top of Java core libraries running on

a Dalvik virtual machine featuring JIT compilation. Libraries written in C include the

surface manager, Open Core media framework, SQLite relational database management

system, OpenGL ES 2.0 3D graphics API, Web Kit layout engine, SGL graphics

engine, SSL, and Bionic libe. The Android operating system, including the Linux kernel,

consists of roughly 12 million lines of code including 3 million lines of XML, 2.8 million

lines of C, 2.1 million lines of Java, and 1.75 million lines of C++.

2.4 Eclipse

Eclipse is a multi-language software development environment comprising an

integrated development environment (IDE) and an extensible plug-in system. It is written

mostly in Java and can be used to develop applications in Java and, by means of various

plugins, other programming languages including Ada, C, C++, COBOL , Perl, PHP,

Python, Ruby (including Ruby on Rails framework), Scala, Clojure, and Scheme. The IDE is

often called Eclipse ADT for Ada, Eclipse CDT for C/C++, Eclipse JDT for Java, and

Eclipse PDT for PHP.

The initial codebase originated from Visual Age. In its default form it is meant for

Java developers, consisting of the Java Development Tools (JDT). Users can extend its



abilities by installing plug-ins written for the Eclipse software framework, such as

development toolkits for other programming languages, and can write and contribute their

own plug-in modules. Released under the terms of the Eclipse Public License, Eclipse is free

and open source software. It was one of the first IDEs to run under GNU Class path and it

runs without issues under Iced Tea.

2.4.1 History

Eclipse began as an IBM Canada project. It was developed by Object Technology

International (OTI) as a Java-based replacement for the Smalltalk based Visual Age family of

IDE products, which itself had been developed by OTI. In November 2001, a consortium was

formed to further the development of Eclipse as open source. In January 2004, the Eclipse

Foundation was created.

Eclipse 3.0 (released on 21 June 2004) selected the OSGi Service Platform

specifications as the runtime architecture. Eclipse was originally released under the Common

Public License, but was later relicensed under the Eclipse Public License. The Free Software

Foundation has said that both licenses are free software licenses, but are incompatible with

the GNU General Public License (GPL). Mike Milinkovich, of the Eclipse Foundation

commented that moving to the GPL would be considered when version 3 of the GPL was

released.

2.4.2 Architecture

Eclipse employs plug-ins in order to provide all of its functionality on top of (and

including) the runtime system, in contrast to some other applications where functionality is

typically hard coded. The runtime system of Eclipse is based on Equinox, an OSG standard

compliant implementation.

This plug-in mechanism is a light weight software component framework. In addition

to allowing Eclipse to be extended using other programming languages such

as C and Python, the plug-in framework allows Eclipse to work with typesetting languages

like LaTeX, networking applications such as telnet, and database management systems. The

plug-in architecture supports writing any desired extension to the environment, such as for

configuration management. Java and CVS support is provided in the Eclipse SDK,

with Subversion support provided by third-party plug-ins. With the exception of a small run-

time kernel, everything in Eclipse is a plug-in. This means that every plug-in developed



integrates with Eclipse in exactly the same way as other plug-ins; in this respect, all features

are "created equal". Eclipse provides plug-ins for a wide variety of features, some of which

are through third parties using both free and commercial models. Examples of plug-ins

include a UML plug-in for Sequence and other UML diagrams, a plug-in for DB Explorer,

and many others.

The Eclipse SDK includes the Eclipse Java Development Tools (JDT), offering an

IDE with a built-in incremental Java compiler and a full model of the Java source files. This

allows for advanced refactoring techniques and code analysis. The IDE also makes use of

a workspace, in this case a set of metadata over a flat file space allowing external file

modifications as long as the corresponding workspace "resource" is refreshed afterwards.

Eclipse implements widgets through a widget toolkit for Java called SWT, unlike

most Java applications, which use the Java standard Abstract Window Toolkit (AWT)

or Swing. Eclipse's user interface also uses an intermediate GUI layer called JFace, which

simplifies the construction of applications based on SWT. Language packs provide

translations into over a dozen natural languages.

2.4.3 Rich Client Platform

Eclipse provides the Eclipse Rich Client Platform (RCP) for developing general

purpose applications. The following components constitute the rich client platform:

Equinox OSGi – a standard bundling framework

Core platform – boot Eclipse, run plug-ins

Standard Widget Toolkit (SWT) – a portable widget toolkit

JFace – viewer classes to bring model view controller programming to SWT,

file buffers, text handling and text editors.

Eclipse Workbench – views, editors, perspectives, wizards

2.4.4 Eclipse: The IDE for Unisys ClearPath Developers

By design, Eclipse itself is a deliberately skinny foundation with minimal

functionality. It is built to be extended via the concept of plug-ins, which are code bundles

that are loaded and used according to a specified configuration. The basic Eclipse structure

defines extension points, which allow new modules (plug-ins) that extend the menus and

offer more services. Developers working within the Eclipse IDE can pick and choose the

features they want to use from the wide, open-source world of plug-in options. The choices



are nearly endless. Unisys is making Eclipse a major focal point for development of

ClearPath applications. In fact, we plan to eventually replace current, proprietary solutions,

such as Programmer’s Workbench, with this open, industry-standard environment.

We have several objectives with our Eclipse offering. The first is to provide industry-

standard IDE assistance for developing Java applications that use ClearPath assets. To this

end, Unisys provides industry-standard Resource Adapters (RAs) for ClearPath database and

transaction access and the Eclipse IDE offers features that assist with development of these

applications. For both ClearPath platforms, there is an RA that supports Distributed

Transaction Processing Services (DTP RA). For ClearPath OS 2200 environments, we

support access to Business Information Server, DMS and RDMS databases, as well as to

TIP/HVTIP transactions. For ClearPath MCP, there are RAs for Enterprise Database Server

(DMS II) and COMS.

Our second major objective with Eclipse is to provide the ability to develop other

3GL applications for ClearPath environments using the same framework that can be used for

Java and other non-ClearPath hosted applications. For ClearPath OS 2200 environments, we

enable development of COBOL, Java, and PLUS applications, as well as the use of TelNet

and CMplus. For MCP, we support COBOL 74/85 and ALGOL, as well as the use of WFL.

Figure 2.1: Architecture of Eclipse

The Unisys engineering team is fully invested in contributing to the open source

movement in ways that benefit our ClearPath customers. We are directing our resources to



creating ClearPath specific plug-ins that naturally build on the Eclipse IDE foundation and

provide capabilities that make sense for developers on ClearPath systems. To that end,

Unisys has built and released Eclipse plug-ins that enable development for ClearPath OS

2200 and MCP operating environments using the Eclipse IDE, as well as integration of

ClearPath based assets in a service-oriented environment. So, what is Unisys providing

exactly for Eclipse? An All-in-One package comprised of the following: The basic, currently

released Eclipse IDE

A selection of open-source plug-ins that we see as particularly valuable in a

ClearPath development environment, such as the Eclipse Data Tools Project

and Web Tools Project.

Unisys plug-ins, some of which are modified versions of open source plug-ins and

others built by Unisys, which enable the development of applications

specifically for ClearPath and access to ClearPath assets (databases and

transactions).

An Application Development Guide to help you get started equally important, the

components of the Unisys Eclipse All-in-One package are:

Integrated and tested by Unisys engineering

Packaged and released as a part of the OS 2200 and MCP IOEs

Supported via Unisys Support at no additional charge.

For more detailed information about the Clear Path OS 2200 IDE for Eclipse, see

our Tech Corner article in this issue of Clear Path Connection. (And, look for a similarly in-

depth article about the Clear Path MCP IDE for Eclipse in the next issue of this newsletter).

2.4.5 Easy, Integrated, and Innovative

With the Unisys All-in-One packages, it’s never been easier to get started using the

Eclipse IDE. Whether you’re an experienced Clear Path programmer or a new college grad

just starting out, it’s time to start exploring the technical and business benefits of this

powerful toolset. You can take advantage of the many plug-ins available from the open

source community and Unisys own plug-ins that are designed to help you get maximum

value from your current Clear Path investment.



2.5 Proposed System

Our system is composed of three layers:

1. The client

2. The server

3. The image retrieval component.

There are two types of the client: the web-based client and the mobile client which is

based on Android mobile system. As the web-based client, users have to manually enter the

latitude and the longitude where the picture is taken. System administrators are able to sign

into the system and manage all the building information, picture information, and user

uploaded photos. As the mobile client, users only need to upload building photos to perform

a search. Latitude and longitude are directly retrieved from GPS instruments.

The server is a conjunction of the client and the image retrieval component and has a

database of the information and images of the buildings to be retrieved. Given an input

image, the image retrieval component finds its nearest image in the database, which indicates

the building this image belongs to.

The image retrieval component contains the key algorithm of the whole system. We

made use of the bag-of-words method for image retrieval due to its good performance in

many image processing and computer vision tasks. The method consists of four steps:

1) Extraction of SIFT features

2) Clustering the features to visual words

3) Generating the frequency vector

4) Image query.

SIFT feature shows good behavior in efficiency and precision so that we adopt it in

our system. In addition, although the positions obtained by GPS may show some error,

usually from 5 meters to 100 meters, we can use the position information to refine the

candidate images that fastens further processing.

A key problem in our system is how to estimate the similarity between a query image

and those in the database. In our approach, the image is represented by frequency vectors.

Thus the problem can be reduced to calculate the distance between frequency vectors.



Chapter 3

SYSTEM DESIGN Systems design is the process of defining the architecture, components, modules,

interfaces, and data for a system to satisfy specified requirements. Systems design could be

seen as the application of systems theory to product development. There is some overlap with

the disciplines of systems analysis, systems architecture and systems engineering. System

development can generally be thought of having two major components: systems analysis

and systems design. In System Analysis more emphasis is given to understanding the details

of an existing system or a proposed one and then deciding whether the proposed system is

desirable or not and whether the existing system needs improvements.

3.1 Introduction of Android

Android is a software stack for mobile devices that includes an operating system,

middleware and key applications. Android is a software platform and operating system for

mobile devices based on the Linux operating system and developed by Google and the Open

Handset Alliance. It allows developers to write managed code in a Java-like language that

utilizes Google-developed Java libraries, but does not support programs developed in native

code.

The unveiling of the Android platform on 5 November 2007 was announced with the

founding of the Open Handset Alliance, a consortium of 34 hardware, software and telecom

companies devoted to advancing open standards for mobile devices. When released in 2008,

It is used to write applications for Android. Unlike other embedded mobile environments,

Android applications are all equal, for instance, an applications which come with the

phone are no different than those that any developer writes. The framework is

supported by numerous open source libraries such as openssl, SQLite and libc. It is

also supported by the Android core libraries. From the point of security, the framework is

based on UNIX file system permissions that assure applications have only those abilities that

mobile phone owner gave them at install time. Most of the Android platform will be made

available under the Apache free-software and open-source license.



3.1.2 Features

Google made a right choice on choosing WebKit as open source web browser. They

added a two pass layout and frame flattening. Two pass layout loads a page without

waiting for blocking elements, such as external CSS or external JavaScript and after a

while renders again with all resources downloaded to the device. Frame flattening

converts founded frames into single one and loads into the browser. These features

increase speed and usability browsing the internet via mobile phone.

Dalvik Virtual Machine

It is extremely low-memory based virtual machine, which was designed

especially for Android to run on embedded systems and work well in low power

situations. It is also tuned to the CPU attributes. The Dalvik VM creates a special

file format (.DEX) that is created through build time post processing.

Conversion between Java classes and .DEX format is done by included “dx” tool.

Optimized Graphics

As Android has 2D graphics library and 3D graphics based on OpenGL

ES 1.0, possibly we will see great applications like Google Earth and

spectacular games like Second Life, which come on Linux version. At this moment,

the shooting legendary 3D game Doom was presented using Android on the mobile

phone.

SQLite

Extremely small (~500kb) relational database management system, which is

integrated in Android. It is based on function calls and single file, where all

definitions, tables and data are stored. This simple design is more than suitable for a

platform such as Android.

Handset Layouts



The platform is adaptable to both larger, VGA, 2D graphics library, 3D

graphics library based on OpenGL ES 1.0 specifications, traditional smart phone

layouts. An underlying 2D graphics engine is also included. Surface Manager

manages access to the display subsystem and seamlessly composites 2D and 3D

graphic layers from multiple applications.

Data Storage

SQLite is used for structured data storage. SQLite is a powerful and

lightweight relational database engine available to all applications.

Connectivity

Android supports a wide variety of connectivity technologies including GSM,

CDMA, Bluetooth, EDGE, EVDO, 3G and Wi-Fi.

Messaging

SMS, MMS, and XMPP are available forms of messaging including threaded

text messaging.

Web Browser

The web browser available in Android is based on the open-source WebKit

application framework. It includes LibWebCore which is a modern web browser engine

which powers both the Android browser and an embeddable web view.

Java Virtual Machine

Software written in Java can be compiled into Dalvik bytecodes and executed in the

Dalvik virtual machine, which is a specialized VM implementation designed for mobile device use,

although not technically a standard Java Virtual Machine.

Media Support

Android will support advanced audio/video/still media formats such as MPEG-4, H.264,

MP3, and AAC, AMR, JPEG, PNG, GIF.

Additional Hardware Support

Android is fully capable of utilizing video/still cameras, touch screens, GPS,

compasses, accelerometers, and accelerated 3D graphics.



Development Environment

Includes a device emulator, tools for debugging, memory and performance profiling

and a plug-in for the Eclipse IDE. There are a number of hardware dependent features,

for instance, a huge media and connections support, GPS, improved support for Camera and

simply GSM telephony. A great work was done for the developers to start work with

Android using device emulator, tools for debugging and plugin for Eclipse IDE.

3.2 Architecture

As discussed, android is an open platform, a software stack for mobile devices that

includes an operating system, middleware and key applications.

The following diagram shows the major components of the Android operating

system. Each section is described in more detail as shown in the following diagram:



Figure 3.1: Architecture of Android

Linux Kernel

Android Architecture is based on Linux 2.6 kernel. It helps to manage security,

memory management, process management, network stack and other important issues.

Therefore, the user should bring Linux in his mobile device as the main operating system

and install all the drivers required in order to run it. Android provides the support for the

Qualcomm MSM7K chipset family. For instance, the current kernel tree supports.

Qualcomm MSM 7200A chipsets, but in the second half of 2008 we should see mobile

devices with stable version Qualcomm MSM 7200, which includes major features:

1. WCDMA/HSUPA and EGPRS network support

2. Bluetooth 1.2 and Wi-Fi support

3. Digital audio support for mp3 and other formats

4. Support for Linux and other third-party operating systems

5. Java hardware acceleration and support for Java applications

6. Qcamera up to 6.0 megapixels

7. GpsOne – solution for GPS

8. And lots of other.

Libraries

In the next level there are a set of native libraries written in C/C++, which are

responsible for stable performance of various components. For example, Surface Manager is

responsible for composing different drawing surfaces on the mobile screen. It manages the

access for different processes to compose 2D and 3D graphic layers. OpenGL ES and SGL

make a core of graphic libraries and are used accordingly for 3D and 2D hardware

acceleration. Moreover, it is possible to use 2D and 3D graphics in the same application in

Android.

The media framework was provided by Packet Video, one of the members of OHA. It

gives libraries for a playback and recording support for all the major media and static

image files. Free Type libraries are used to render all the bitmap and vector fonts. For data



storage, Android uses SQLite. As mentioned before, it is extra light rational management

system, which locates a single file for all operations related to database. WebKit, the same

browser used by Apples’ Safari, was modified by Android in order to fit better in a small

size screens.

Android Runtime

At the same level there is Android Runtime, where the main component Dalvik

Virtual Machine is located. It was designed specifically for Android running in limited

environment, where the limited battery, CPU, memory and data storage are the main issues.

Android gives an integrated tool “dx”, which converts generated byte code from .jar to .dex

file, after this byte code becomes much more efficient to run on the small processors.

Figure 3.2: Conversion from .java to .dex file

As the result, it is possible to have multiple instances of Dalvik virtual machine

running on the single device at the same time. The Core libraries are written in Java

language and contains of the collection classes, the utilities, IO and other tools.

Application Framework



After that, there is Application Framework, written in Java language. It is a toolkit

that all applications use, ones which come with mobile device like Contacts or SMS box, or

applications written by Google and any Android developer. It has several components.

The Activity Manager manages the life circle of the applications and provides a

common navigation back stack for applications, which are running in different processes.

The Package Manager keeps track of the applications, which are installed in the device. The

Windows Manager is Java programming language abstraction on the top of lower level

services that are provided by the Surface Manager. The Telephony Manager contains of a

set of API necessary for calling applications.

Content Providers was built for Android to share a data with other applications, for

instance, the contacts of people in the address book can be used in other applications too.

The Resource Manager is used to store localized strings, bitmaps, layout file descriptions

and other external parts of the application. The View System generates a set of buttons and

lists used in UI. Other components like Notification manager is used to customize display

alerts and other functions.

Application Layer

At the top of Android Architecture we have all the applications, which are used by

the final user. By installing different applications, the user can turn his mobile phone into

the unique, optimized and smart mobile phone. All applications are written using the Java

programming language.

3.2.1 Application Lifecycle

In Android, every application runs in its own process, which gives better

performance in security, protected memory and other benefits. Therefore, Android is

responsible to run and shut down correctly these processes when it is needed.

It is important that application developers understand how different application

components (in particular Activity, Service, and Broadcast Receiver) impact the lifetime of

the application's process. Not using these components correctly can result in the system

killing the application's process while it is doing important work.



To determine which processes should be killed when low on memory, Android

places each process into an "importance hierarchy" based on the components running in

them and the state of those components.

These process types are (in order of importance).

A foreground process is one that is required for what the user is currently doing.

Various application components can cause its containing process to be considered

foreground in different ways. A process is considered to be in the foreground if any of

the following conditions hold:

It is running an Activity at the top of the screen that the user is interacting with (it’s

on Resume() method has been called).

It has a Broadcast Receiver that is currently running (its Broadcast Receiver.on

Receive() method is executing).

It has a Service that is currently executing code in one of its callbacks

(Service.onCreate(), Service.onStart(), or Service.onDestroy()).

There will only ever be a few such processes in the system, and these will only be

killed as a last resort if memory is so low that not even these processes can continue to run.

Generally, at this point, the device has reached a memory paging state, so this action is

required in order to keep the user interface responsive.

A visible process is one holding an Activity that is visible to the user on-screen but

not in the foreground (it’s on Pause() method has been called). This may occur, for

example, if the foreground Activity is displayed as a dialog that allows the previous

Activity to be seen behind it. Such a process is considered extremely important and

will not be killed unless doing so is required to keep all foreground processes

running.

A service process is one holding a Service that has been started with the startService()

method. Though these processes are not directly visible to the user, they are generally doing

things that the user cares about (such as background mp3 playback or background network



data upload or download), so the system will always keep such processes running unless

there is not enough memory to retain all foreground and visible process.

Figure 3.3: Flowchart Showing the Lifecycle of an Application Activity

A background process is one holding an Activity that is not currently visible to the

user it’s on Stop() method has been called. These processes have no direct impact on the user

experience. Provided they implement their Activity life-cycle correctly (see Activity for more

details), the system can kill such processes at any time to reclaim memory for one of the

three previous processes types. Usually there are many of these processes running, so they

are kept in an LRU list to ensure the process that was most recently seen by the user is the

last to be killed when running low on memory.

An empty process is one that doesn't hold any active application components. The only

reason to keep such a process around is as a cache to improve startup time the next time a

component of its application needs to run. As such, the system will often kill these processes



in order to balance overall system resources between these empty cached processes and the

underlying kernel caches.

In the following example we will display a process flow from the Android System

point of view to get a clear idea how the applications behave. Let assume the

Figure 3.4 Running, Saving and resuming process

Possible scenario: A user talks to his friend via mobile phone and he is asked to

browse the internet (a talk is hold for a moment), find a picture of him in his Picasa Album,

send it via Email back to his friend and resume a talk.

In this situation, there are 4 different applications and 4 different processes running,

but from the user point of view none of them are important, as Android manages CPU work

and memory usage by itself. It means the user can travel through the applications forward

and back without thinking about how much memory is left or which processes are run at the

time. Firstly, as the user is talking to his friend, a specific Talk application is opened, which

contains the activity manager. In the following stack we can see two processes running, the

main system process and Talk application process. Moreover, before going to Web Browser

application, the system saves a Talk state T in order to remember that process:

Figure 3.5 Start of the activity



At this point, as a user holds a talk and opens a web browser, the system creates a

new process and new web browser activity is launched in it. Again, the state of the last

activity is saved (W):

Figure 3.6: Role of the Browser

After that, the user browses the internet, finds his picture in Picasa album and saves

it to particular folder. He does not close a web browser, instead he opens a folder to find

saved picture. The folder activity is launched in particular process:

Figure 3.7: Folder inclusive of Activities

At this point, the user finds his saved picture in the folder and he creates a request to

open an Email application. The last state F is saved. Now assume that the mobile phone is

out of the memory and there is no room to create a new process for Email application.

Therefore, Android looks to kill a process. It cannot destroy Folder process, as it was used

previously and could be reused again, so it kills Web Browser process as it is not useful

anymore and locates a new Email process instead:



Figure 3.8: working with the email application

The user opens Email application and sends a picture to his friend via email. Now he

wants to go back to the Talk application and to resume a talk to his friend. Because of the

previously saved states, this work is done fast and easily. In this example, Email application

is popped out and the user sees a previous Folder application:

Figure 3.9: Management by the Web Browser

Next, the user goes back to Web Browser application. Unfortunately, web browser

process was killed previously so the system has to kill another process (in our case it is

Email application process, which is not used anymore) in order to locate Web Browser

process and manage the stack memory:



Figure 3.10: Storage of information collectively

There is a procedure where the conversation is recorded, there is an availability of a

web browser and a folder. The folder has a collective information about various stuffs. All

the information goes to the web browser. There comes a terminology of a “process” which

is inclusive of many system processes, a talk application, an extra folder, an email and an

additional web browser.

Figure 3.11 Deletion of the Folder

Now the user comes back to the Talk application and resumes his talk with his friend.

Because of the saved states, going back procedure is fast and useful, because it remembers

previous activities and its views.

This example shows, that it does not matter how many applications and processes are

active or how much available memory is left, Android it manages fast and without a user

interaction.



3.3 What makes Android special?

There are already many mobile platforms on the market today, including Symbian,

iPhone, Windows Mobile, BlackBerry, Java Mobile Edition, Linux Mobile (LiMo), and

more. While some of its features have appeared before, Android is the first environment that

combines:

A truly open, free development platform based on Linux and open source. Handset

makers like it because they can use and customize the platform without paying a

royalty. Developers like it because they know that the platform “has legs” and is not

locked into any one vendor that may go under or be acquired.

A component-based architecture inspired by Internet mash-ups. Parts of one

application can be used in another in ways not originally envisioned by the developer.

You can even replace built-in components with your own improved versions. This

will unleash a new round of creativity in the mobile space.

Tons of built-in services out of the box. Location based services use GPS or cell

tower triangulation to let you customize the user experience depending on where they

are. A full-powered SQL database lets you harness the power of local storage for

occasionally connected computing and synchronization. Browser and Map views can

be embedded directly in your applications. All these built-in capabilities help to raise

the bar on functionality while lowering your development costs.

Automatic management of the application life cycle. Programs are isolated from each

other by multiple layers of security, which will provide a level of system stability not

seen before in smart phones. The end user will no longer have to worry about what

applications are active, or close some programs so that others can run. Android is

optimized for low-power, low-memory devices in a fundamental way that no previous

platform has attempted.

High quality graphics and sound. Smooth, anti-aliased 2D vector graphics and

animation inspired by Flash is melded with 3D accelerated OpenGL graphics to

enable new kinds of games and business applications. Codecs for the most common

industry standard audio and video formats are built right in, including H.264 (AVC),

MP3, and AAC.



Portability across a wide range of current and future hardware. All your programs are

written in Java and executed by Android’s Dalvik virtual machine so your code will be

portable across ARMx86, and other architectures. Support for a variety of input methods is

included such as keyboard, touch, tilt, camera, voice, and trackball. User interfaces can be

customized for any screen resolution and orientation. Android is a fresh take on the way

mobile applications interact with users, along with the technical underpinnings to make it

possible. But the best part of Android is the software that you are going to write for it. This

book will help you get off to a great start.

3.4 Block Diagram

Figure 3.12 Internal work representation

There is a browser client, that is the server and there is an android mobile client who

are we users. Via the internet and using either 3G, Wi-Fi or GPS, the photo when taken by

the user, gets saved in the search camera application of the android camera and also is

uploaded to the server. From the web application, that is the server, the image is retrieved.

Along with this, the locations, and the entire descriptions as stored in the database id fetched

and is given back to the clients.



3.5 Working Principle

The System consists of two parts

The Client.

The Server.

The server is implemented using j2ee and the client is implemented using Android.

1. Client:

The client is mainly composed of three modules: Interface, Data handling and

Network connection Interface. This part provides users with simple and convenient ways to

search for buildings and review searching history. To make user experience to the best, UI

design of the client follows the design philosophy and principles of android application,

including a clear dashboard, consistent theme and title bar, etc.

2. Data handling:

This part is responsible for data handling and transferring between Views module

and Network connection module. Its function includes generating search query, dealing with

the response of server (both when searching for a building and posting a new building), and

managing the local records of buildings that user searched before.

3. Network connection:

This part serves to make client connect to server, post search query or new buildings

and receive results. By using Http Client, the network part sends query as formatted entities

to the server, receives the response texted as JSON and passes it to the Data handling

module. The client provides functions as mentioned:

a. Forming a query by taking a photo with camera or selecting a picture file.

b. Uploading the query, including picture from the user and GPS information generated by

the client, and show detailed result both as text contents and markers on the Google Map.

c. Generating formatted date of a new building within formation provided by user and

posting it to the server.



d. Providing history of buildings that user has searched for.

4. Server

The main function of the server is to manage the information of the available

buildings, including the latitude, longitude and the related photos. Some of the photos are

marked as CRITICAL only if the photos are greatly taken and are typical to represent the

looking of the building.

The GPS information is also very useful in our search system. To speed up the search

and increase the accuracy, GPS information is used to filter out the impossible building sand

the related photos even if the GPS information is not so accurate.

Within the area that is filtered by the GPS information, the server is able to perform a

search in a very delightfully fast speed. The server uses all the photos marked as critical to

build an index.

The server also includes the management system with the user interface of the web-

based client, system administrators are allowed to add buildings and the related photos and

mark the critical photos.

But the system administrators do not need to build the clusters and index, it’s

automatically done if there is some change on the critical photo set.

The image retrieval component is running on the same server, but it’s not eventually

deployed. As long as there’s a chance, the server is able to run the image retrieval component

on the different servers or even on the selected servers.



3.6 Sequence Diagram

Figure 3.13: Sequence of the working of the Gap Search

This figure represents the sequence diagrams. There are two main components

involved. They are the “client” and the “server”. The client initially sends the uploaded

picture, and simultaneously gets the latitude and longitude. He then sends the report or the

response back to the client with the image and description of the building or the place.

After that, add the details that is the photos and descriptions and GPS points. When these

works are done, the response is sent back to the client.



3.7 Use Case Diagram

Figure 3.14: The Use Cases

The above figure represents the use case diagram, which pictorially presents the users

work, that is the picture which is clicked with the android phone by the user.

When the picture is taken, it is loaded in the server. The computers acts as the server. After

this, the picture uploaded is given descriptions. The descriptions includes all the possible

information which needs to be added about the photograph.

When these work are done, anyone who wants to know about the monuments or the

buildings, about which the details are already fed, can be fetched, when they take the picture

of that desired monument with their android phone, using the “search camera” application

which is the basic part of the application used in the project.

In that application, an “upload” option is there, which when selected by the user, does the

uploading work, and the information needed are all fetched by that user.


Start

Client

GPS Manager

Camera

Server

Image Comparison

Manager

Database Manager

Database

Longitude & Latitude

Picture


3.8 Data Flow Diagram

Figure 3.15 Procedure of Flow of the data

We have many options in this data flow diagram. To start with, we have a start

option, a GPS manager. The client takes the picture from his android phone and uploads to

the client. The database manager stores and sorts all the pictures along with the latitudes and

longitudes details. There is an image comparison manager whose main work is to compare

the images, that which is taken by the client from his own phone, and the one which are

already uploaded in the database.



CHAPTER 4

IMPLEMENTATION

An implementation is a realization of a technical specification or algorithm as a

program, software component, or other computer system through computer programming and

deployment. Many implementations may exist for a given specification or standard.

4.1 SYSTEM REQUIREMENTS

A software requirements definition is an abstract description of the services, which

the system should provide, and the constraints under which the system must operate. It

should only specify only the external behaviour of the system and is not concerned with

system design characteristics. The softwares that are required for the system are: JAVA (JDK

1.6 or More), Android SDK 1.5 or More, Eclipse IDE, Netbeans IDE, Android Emulator,

Apache Web Server, Mysql Database.

Hardware Requirements Analysis is to define and analyse a complete set of

functional, operational, performance, interface, quality factors, and design, criticality and test

requirements. The hardware that are required for the system are: Processor: P4 or More,

RAM: 256 MB or More, Disk Space: Minimum 800MB, Android Mobile Phone.

4.2 MODULES USED

CameraActivity module:

This is a client side module where in the user clicks the picture, saves the picture and then

uploads the picture into the database. Here the latitude and longitude will be obtained.

onCreate( ): It is Called when the activity is first created. This is where you should do all of

your normal static set up: create views, bind data to lists, etc. This method also provides you

with a bundle containing the activity's previously frozen state, if there was one.

Preference.getmgr( ): It is used to help, create preference hierarchies from activities or

XML. Gets a SharedPreferences instance that points to the default file that is used by the



preference framework in the given context. This function returns a SharedPreferences

instance that can be used to retrieve and listen to values of the preferences.

setContentView( ): An activity is a single, focused thing that the user can do. Almost all

activities interact with the user, so the Activity class takes care of creating a window for you

in which you can place your UI with setContentView().

findViewById( ): It is used to retrieve the widgets in that UI that you need to interact with

programmatically.

setOnClickListener( ): It is called when a view has been clicked.

Toast.makeText( ): A toast is a view containing a quick little message for the user. The toast

class helps you create and show those.

getColumnIndexOrThrow( ): Returns the zero-based index for the given column name, or

throws IllegalArgumentException if the column doesn't exist. If you're not sure if a column

will exist or not use getColumnIndex(String) and check for -1, which is more efficient than

catching the exceptions.

setEntity( ): Hands the entity to the request.

public class CameraActivity extends Activity implements Runnable OnClicklistener

public void onCreate(Bundle b)

super.onCreate(b);

sp=PreferenceManager.getDefaultSharedPreferences(this);

address=sp.getString("address",null);

setContentView(R.layout.upload);

btn_camera=(Button)findViewById(R.id.btn_open_camera);

btn_camera.setOnClickListener(this);

Toast.makeText(this,filename, Toast.LENGTH_SHORT).show();

public static File convertImageUriToFile (Uri imageUri, Activity activity)

int file_ColumnIndex = cursor.getColumnIndexOrThrow

(MediaStore.Images. Media.DATA);



bm = BitmapFactory.decodeFile(convertImageUriToFile(imageUri,this).

getPath());

reqEntity.addPart(filename, bab);

postRequest.setEntity(reqEntity);

GalleryViewer module:


uploads the picture into the database. Whenever the user wants to see the picture taken, the

user can go to gallery to view the picture. Here the picture will have its latitude and

longitude.

onStart( ): protected void onStart() is called when the activity is becoming visible to the

user. This is followed by onResume() if the activity comes to the foreground, or onStop() if it

becomes hidden.

onClick(View v): public abstract void onClick (View v) is called when a view has been

clicked.

getReadableDatabase( ): It is used to Create and/or open a database. This will be the same

object returned by getWritableDatabase() unless some problem, such as a full disk, requires

the database to be opened read-only.

cursor.getString( ): Returns the value of the requested column as a String. The result and

whether this method throws an exception when the column value is null or the column type is

not a string type is implementation-defined.

startManagingCursor(cursor): This method allows the activity to take care of managing

the given Cursor's lifecycle for you based on the activity's lifecycle. That is, when the activity

is stopped it will automatically call deactivate() on the given Cursor, and when it is later

restarted it will call requery() for you. When the activity is destroyed, all managed Cursors

will be closed automatically.

public class GalleryViewer extends Activity implements OnClickListener,Runnable

public void onStart()

super.onStart();



PhotoDatabase dat=new PhotoDatabase(this,PhotoDatabase.

DATABASE_NAME,null,1);

SQLiteDatabase db=dat.getReadableDatabase();

cursor=db.query(PhotoDatabase.TABLE_NAME,newString[]{"latitude,

longitude","filename"},null,null,null,null,null);

startManagingCursor(cursor);

public void onClick(View v)

latitude=cursor.getString(0);

longitude=cursor.getString(1);

filename1=cursor.getString(2);

public void run()

bm = BitmapFactory.decodeFile(convertImageUriToFile(uri,this).getPath());

ByteArrayOutputStream bos = new ByteArrayOutputStream();

LocationUpdate module:


uploads the picture into the database. The latitude and longitude obtained from previous

module will be used while inserting data into database fields. Hence the location will be

updated.

getSystemService( ): Return the handle to a system-level service by name. The class of the

returned object varies by the requested name.

requestLocationUpdates( ): Register for location updates using the named provider, and a

pending intent.

onLocationChanged( ): It is called when the location has changed. There are no restrictions

on the use of the supplied Location object.



onProviderDisabled( ): It is called when the provider is disabled by the user. If

requestLocationUpdates is called on an already disabled provider, this method is called

immediately.

onProviderEnabled( ): It is called when the provider is enabled by the user.

onStatusChanged( ): It is Called when the provider status changes. This method is called

when a provider is unable to fetch a location or if the provider has recently become available

after a period of unavailability.

public class LocationUpdate implements LocationListener

public LocationUpdate(Context cntxt)

this.cntxt=cntxt;

Lmanager=(LocationManager)cntxt.getSystemService(

Context.LOCATION_SERVICE);

Lmanager.requestLocationUpdates(LocationManager.

NETWORK_PROVIDER,0,0,this);

public void onLocationChanged(Location location)

latitude=""+location.getLatitude();

longitude=""+location.getLongitude();

public void onProviderDisabled(String provider)

public void onProviderEnabled(String provider)

public void onStatusChanged(String provider, int status, Bundle extras)

MyCameraActivity module:




uploads the picture into the database. Here the latitude and longitude will be obtained. This is

almost similar to that of CameraActivity module with a few changes.

onCreateOptionsMenu( ): Initialize the contents of the Activity's standard options menu.

You should place your menu items in to menu. This is only called once, the first time the

options menu is displayed.

onOptionsItemSelected( ): This hook is called whenever an item in your options menu is

selected. The default implementation simply returns false to have the normal processing

happen (calling the item's Runnable or sending a message to its Handler as appropriate). You

can use this method for any items for which you would like to do processing without those

other facilities.

onClick( ): It is called when a view has been clicked.

public class MyCameraActivity extends Activity implements OnClickListener

public void onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);

setContentView(R.layout.main);

camera=(ImageView)findViewById(R.id.main_img_camera);

camera.setOnClickListener(this);

public boolean onCreateOptionsMenu(Menu m)

super.onCreateOptionsMenu(m);

MenuInflater minf=getMenuInflater();

minf.inflate(R.menu.menu,m);

return true;

public boolean onOptionsItemSelected(MenuItem itm)



super.onOptionsItemSelected(itm);

Intent itt=new Intent(this,Settings.class);

startActivity(itt);

return true;

MyGalleryActivity module:

This module involves in the gallery activity such as viewing the images, using the gallery

image forward button and gallery image backward buttons. It also displays the image

information in the form of text which includes the image filename, latitude, longitude with

the description of the image.

onClickListener( ): Interface definition for a callback to be invoked when a view is clicked.

onCreate(Bundle b): Used to initialize the activity. Here usually setContentView(int) with a

layout resource defining the user interface is called, and using findViewById(int) to retrieve

the widgets in that user interface that has to interact with programmatically.

public void onStart( ): This is an activity. It comes under android.app.Activity API. It is

called after onCreate(Bundle) — or after onRestart() when the activity had been stopped, but

is now again being displayed to the user. It will be followed by onResume().

Bitmap decodeFile (String pathname): Decode a file path into a bitmap. If the specified

file name is null, or cannot be decoded into a bitmap, the function returns null. Parameter

used is pathname complete path name for the file to be decoded. It returns the resulting

decoded bitmap, or null if it could not be decoded.

public class MyGalleryActivity extends Activity implements OnClickListener


setContentView( );

image= ( ) findViewById( );

image.setOnClickListener( );



public void onStart()

public static Bitmap getImageFromWeb(String imglink)

Photodatabase module:

This module creates a database and a table which includes the information of the image

which is uploaded in text format, i.e., latitude, longitude and filename.

Class overview:

A helper class to manage database creation and version management. We create a subclass

implementing onCreate(SQLiteDatabase), onUpgrade(SQLiteDatabase, int, int) and

optionally onOpen(SQLiteDatabase), and this class takes care of opening the database if it

exists, creating it if it does not, and upgrading it as necessary. Transactions are used to make

sure the database is always in a sensible state.

public void onCreate (SQLiteDatabase db):

Called when the database is created for the first time. This is where the creation of tables and

the initial population of the tables should happen.

public abstract void onUpgrade (SQLiteDatabase db, int oldVersion, int newVersion):

Called when the database needs to be upgraded. The implementation should use this method

to drop tables, add tables, or do anything else it needs to upgrade to the new schema version.

This method executes within a transaction. If an exception is thrown, all changes will

automatically be rolled back. Parameters used are db: The database, oldVersion: The old

database version. newVersion: The new database version.



public class PhotoDatabase extends SQLiteOpenHelper

public void onCreate(SQLiteDatabase db)

public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion)

Settings Module:

This module is used to add preferences using android.preference.PreferenceActivity API.

Class Overview:

This is the base class for an activity to show a hierarchy of preferences to the user. This class

is only allows the display of a single set of preference;

void onCreate(Bundle savedInstanceState): It is called when the activity is starting.

public class Settings extends PreferenceActivity


FetchData module:

This is a server side module. This module deals with fetching the data that is the information

of the picture from the database.

Class.forName( ): This method is used to load JDBC driver.

DriverManager.getConnection( ): This method is used to connect to DBMS. It takes 3

arguments. It returns an interface that is used throughout the process to reference the

database.

setContentType( ): This method allows a JSP output to be viewed and downloaded as an

excel.



response.getWriter( ): This method returns a PrintWriter object that can send character text

to the client.

public class FetchData extends HttpServlet

protected void processRequest(HttpServletRequest request, HttpServletResponse

response)throws ServletException, IOException, SQLException

response.setContentType("text/html;charset=UTF-8");

out = response.getWriter();

uploadedItems = upload.parseRequest(request);

Class.forName(driver);

con=DriverManager.getConnection

JPEGImageFileFilter module:

This is a server side module. This module deals with filtering out the images in the database.

Here the filtering process takes place based on latitude and longitude. This module checks

the type of image.

accept( ): This method is a boolean type method. Returns true or false based on given

conditions.

toLowerCase( ): This method is used to convert the filename to lower case if it is in upper

case.

f.getName( ): This method is used to retrieve the filename of the respective image.

endsWith( ): This method is used to check the type of image, that is jpg or jpeg

getDescription( ): This method is used to retrieve the description or the information of the

filtered out image.

public class JPEGImageFileFilter extends FileFilter implements java.io.FileFilter

public boolean accept(File f)

toLowerCase();



public String getDescription()

return "JPEG files";

RetrieveImages module:

This is a server side module. This module deals with retrieving the filtered out image and

displaying it to the client.

getContentType( ): It returns a string containing the value of the "content type" header in

the request.

getContentLength( ): It returns an integer representing the "content length" header

contained in the request.

indexOf( ): Reports the zero-based index of the first occurrence of the specified string in this

instance. The search starts at a specified character position.

request.getRealPath( ): Gets the real path corresponding to the given virtual path. The real

path returned will be in a form appropriate to the computer and operating system on which

the servlet container is running, including the proper path separators.

request.getSession( ): Gets the Session object through which various properties of the

session tracking cookies created on behalf of this ServletContext may be configured.

rescale( ): establish on a new scale, change the size of; make the size more appropriate,

reduce proportionally or increase proportionally.

doGet( ): It is called by the server (via the service method) to allow a servlet to handle a

GET request.

doPost( ): It is called by the server (via the service method) to allow a servlet to handle a

POST request. The HTTP POST method allows the client to send data of unlimited length to

the Web server a single time and is useful when posting information such as credit card

numbers.

public class RetrieveImages extends HttpServlet




response)

String contentType = request.getContentType();

int formDataLength = request.getContentLength();

int boundaryLocation = file.indexOf(boundary, pos) - 4;

TempFileLoc=request.getRealPath("TempFiles");

HttpSession session=request.getSession();

private RenderedImage rescale(RenderedImage i)

private Color[ ][ ] calcSignature(RenderedImage i)

protected void doGet(HttpServletRequest request, HttpServletResponse response)

protected void doPost(HttpServletRequest request, HttpServletResponse response)

UploadData module:

This is a server side module. This module is used to upload the picture to the gallery once the

picture is taken. This module is used to upload the description and details of the images into

the database.

Class.forName( ): This method is used to load JDBC driver.

DriverManager.getConnection( ): This method is used to connect to DBMS. It takes 3

arguments. It returns an interface that is used throughout the process to reference the

database.

sendRedirect( ): This method is used to redirect the user of the current page to the other

page. In this process the control of user is transferred from the current location to another

location using the specified URL and clears the buffer.

setContentType( ): This method allows a JSP output to be viewed and downloaded as an

excel.



response.getWriter( ): This method returns a PrintWriter object that can send character text

to the client.

setString( ): This method is used to set the specified string in the specified position.

pst.executeUpdate( ): This method is used to execute the SQL statement in the

PreparedStatement object, which must be an SQL INSERT, UPDATE or DELETE

statement; or an SQL statement that returns nothing, such as a DDL statement.

con.close( ): This method is used to close the connection with database.

getServletInfo( ): This method is used to return a short description of the servlet.

public class UploadData extends HttpServlet


response)

response.setContentType("text/html;charset=UTF-8");

PrintWriter out = response.getWriter();

pst.setString(1,Place_Name);

pst.executeUpdate();

pst.close();

con.close();

response.sendRedirect("index.jsp?msg=Data Uploaded Successfully");

public String getServletInfo()



Chapter 5

SNAPSHOTS

Android Software development Kit



Snapshot 1

The first time a snapshot-enabled AVD is launched, there is no snapshot data so it

will boot normally. Pressing the close button will save the state (if that's been selected) and

then quit the emulator. Note that saving the state can be three times slower than loading (8

seconds for us) and the window will freeze while that's happening - this is one of the reasons

this is only an alpha-quality feature. Once the snapshot has been saved, loading is fast - 3

seconds - no booting.

How to create android virtual device:



Snapshot 2

The new emulator in Tools has the ability to save and restore its state to a 'snapshot'

file in each AVD - so we can usually avoid booting when we start the emulator. When

creating (or editing) an AVD, tick the 'Snapshot enabled' box.

How to save an android emulator:



Snapshot 3

Here we are saving the state of the emulator into a big file, and restoring its state from

the file later.

There are a couple of ways to get it. The first way is to set the desired options in the

Launch Options dialog when starting the emulator from the Android SDK and AVD

Manager. The Wipe user data option is still there from before, and that will create a new

system from scratch. The new snapshot options are to Launch from snapshot (if one has been

saved previously), or to Save to snapshot when the emulator shuts down.

Obviously we have to save a snapshot before we can launch from one. The snapshot

file can get rather large, and this adds to how long it takes for the emulator to exit when you

close it.



Snapshot 4

Above shown is an android emulator which is a virtual device, which can be used for

the demonstration purpose instead of the mobile phones. In the emulator an android

application should be installed for further functioning of the system and for fetching the

informations.


An android emulator- a virtual device


Snapshot 5

As mentioned about the emulator, a “search camera” application must be

compulsorily installed in the android phones for the users to use it efficiently. In the

emulator, there is an option called “open camera” which comes when we select the camera

photo given in the snapshot 5. As and when, this is selected, users are facilitated with an

advantage of clicking the photographs with their phones.



Snapshot 6

Next, after a picture has been taken, users get an option of “saving” the picture, after

they select the “save” button. As and when they save the picture, the picture gets stored in the

gallery which is shown in snapshot 5, beside the photo of the camera.



Emulator with the gallery

Snapshot 7

Lastly, to check the photos saved in the gallery, users need to press in the “arrow”

buttons shown in the figure above. All the photos which users have “clicked and “saved” are

stored inside the gallery. All the photos along with their latitude and longitude details are fed

in the gallery by the users. All of these are updated to the server along with the descriptions.

When a person clicks the photo of a building, whose photos are being taken and

stored in the server, as the person selects the “upload” button, shown above, all the details of

the picture about which they wanted to know, will be displayed to them.



Images in the database with descriptions

Snapshot 8

These are few photographs taken in all different angles possible using an android

phone with the “search camera” option. There are saved in the gallery along with the

latitudes and longitudes. All of which are stored in the server with descriptions. When user

clicks a photo then the distance between query image and the candidate images selected from

GPS information, there is a score for each candidate image.

The next task is to rank the images according to the scores. The complexity of

Quicksortis O(nlogn), which is more satisfactory compared to other sortalgorithms. So, the

results are ranked by Quicksort.



The search result page

Snapshot 9

The test images are compared with the database images. And above shown is the

picture of how the search result page looks like. The result page is to be made, and this is the

server side working of the system. In this server, photos are uploaded along with the

longitude and latitude descriptions.

Whenever user clicks a photo, and presses the upload button in their phone, they get

all details, about that particular building, exactly what is fed in the database.



CONCLUSION A system using mobile camera, GPS information and PC server is introduced to

search and recognize buildings without typing any words. With quick development of

mobile techniques, a large number of people already own smart phones. The system provides

an attracting and easy way to know about the world using images captured by mobile

cameras. The method is simple but effective.

Images are resized to 320 x 240 or 240 x 320 pixels, extract SIFT feature descriptor

to describe each image in database, and assign each picture with a 300- dimension frequency

vector. For a query image, the system calculates the frequency vector just as the images in

database, selects candidate images by GPS, estimates the scores for each candidate image,

ranks and lists the results. The information is usually in the form of image and

latitude/longitude.

However, in many cases, users may want meta information such as the names and

introduction of the buildings around them. A system that combines network technologies and

image retrieval algorithms is proposed to address this problem. A user uploads a building

photo, and then our system can return its name and other introduction. System is composed

of three layers: the client, the server, and the image retrieval component.

There are two types of the client: the web-based client and the mobile client which is

based on Android mobile system. As the web-based client, users have to manually enter the

latitude and the longitude where the picture is taken. System administrators are able to sign

into the system and manage all the building information, picture information, and user



uploaded photos. As the mobile client, users only need to upload building photos to perform

a search.

Use of eclipse and net beans is necessary for the search camera application to run.

Options such as upload, save are given different modules for different functions are given.

FUTURE ENHANCEMENTThe project proposed has lots of future enhancement chances. In the future, we are

going to add more information about the buildings in our system, such as the courses

undergoing in the teaching building, the kinds of books in the different shelves of the library

building and the historical value of some old but distinguished building.

On the other hand, we will try to expand our system to include more buildings or

other objects, such as statues, etc. The idea of how to expand the system is try to collect, train

and retrieve images according to latitude and longitude. The world can be divided into

several large parts for us to deal with one at a time. The effectiveness and efficiency of the

system under more diverse cases shall be tested and analyzed.

The project might also be used to analyze the human faces and to detect their

characteristics. For this, new technologies which are advanced and sophisticated can be used.



BIBLIOGRAPHY

[1] Jiemin Wang, Yuanhai He, Yujie Zhou, Yu Qiao “iGAPSearch: Using Phone Cameras to

Search around the World” 1. Shenzhen Institutes of Advanced Technology, Chinese Academy

of Science, Shenzhen 518055, China

[2] Smith, J.R., “Image Retrieval Evaluation,” IEEE Workshop on

Content-Based Access of Image and Video Libraries, pp.12-113, Jun. 1998

[3] Java-The Complete Reference – Herbert Schildt, 7th edition, Tata McGraw

Hill, 2007

[4] Jianguo Zhang, Marszalek, M., Lazebnik, S., Schmid, C., “Local features and kernels for

classification of texture and object categories: A comprehensive study,” CVPR, June 2006.

[5] Jégou, H.; Douze, M.; Schmid, C.; Pérez, P., “Aggregating local descriptors into a

compact image representation,” CVPR, June 2010.

[6] G. Csurka, C. Bray, C. Dance, and L. Fan, “Visual Categorization with Bags of

Keypoints,” European Conference on Computer Vision, pp. 1-22, 2004.

[7] Kim-Hui Yap, Tao Chen, Zhen Li, Kui Wu, “A Comparative Study of Mobile-Based

Landmark Recognition Techniques,” IEEE Intelligent Systems, vol. 25, pp. 48-57, Feb. 2010.

[8] Smith, J.R., “Image Retrieval Evaluation,” IEEE Workshop on Content-Based Access of

Image and Video Libraries, pp.12- 113, Jun. 1998.



[9] Menglei Jia, Xin Fan, Xing Xie, Mingjing Li, Wei-Ying Ma, “Photo-to-Search: Using

Camera Phones to Inquire of the Surrounding World,” Mobile Data Management, pp. 46 -

46, May 2006.

[10] J2EE - The Complete Reference - Jim Keogh, Tata McGraw Hill, 2007

[11] Google, Inc., What is Android?

http://code.google.com/android/what-

is android.html.

[12] http://code.google.com/android/ - Official Android Google Code Webpage.

[13] http://www.openhandsetalliance.com/ - Open Handset Alliance Webpage.

[14] http://www.androidwiki.com – Android Wikipedia.

[15] www.ieeexplore.ieee.org

Chapter 1

INTRODUCTION

According to the introduction of smart phone on wiki website, over 45.5 million

people own smart phones out of 234 million total subscribers in United States and it is the

fastest growing segment of the mobile phone market. Many phones have cameras and GPSs,

which provide useful information for users to discover and navigate their environments. The

information is usually in the form of image and latitude/longitude. However, in many cases,

users may want meta information such as the names and introduction of the buildings around

them. In this paper, we propose a system that combines network technologies and image

retrieval algorithms to address this problem. A user uploads a building photo, and then our

system can return its name and other information. Our system is composed of three layers:

the client, the server, and the image retrieval component. There are two types of the client:

the web-based client and the mobile client which is based on Android mobile system. As the

web-based client, users have to manually enter the latitude and the longitude where the

picture is taken. System administrators are able to sign into the system and manage all the

building information, picture information, and user uploaded photos. As the mobile client,


http://www.androidwiki.com/

http://www.openhandsetalliance.com/

http://code.google.com/android/

http://code.google.com/android/what-%20is%20android

http://code.google.com/android/what-%20is%20android


users only need to upload building photos to perform a search. Latitude and longitude are

directly retrieved from GPS instruments. The server is a conjunction of the client and the

image retrieval component and has a database of the information and images of the buildings

to be retrieved. Given an input image, the image retrieval component finds its nearest image

in the database, which indicates the building this image belongs to. The image retrieval

component contains the key algorithm of the whole system. It is made use of the bag-of-

words method for image retrieval due to its good performance in many image processing and

computer vision tasks.

The method consists of four steps: 1) Extraction of SIFT features, 2) Clustering the

features to visual words, 3) Generating the frequency vector according to the visual words, 4)

Image query. SIFT feature shows good behavior in efficiency and precision so that we adopt

it in our system. In addition, although the positions obtained by GPS may show some error,

usually from 5 meters to 100 meters, we can use the position information to refine the

candidate images that fastens further processing. A key problem in this system is how to

estimate the similarity between a query image and those in the database. In this approach, the

image is represented by frequency vectors. Thus the problem can be reduced to calculate the

distance between frequency vectors. In this paper, we introduce four different methods for

distance calculation and compare them in experiments. Our final system makes use of the

one with the best performance. The remainder of this paper is organized as follows: we give

an overview of our system. We describe the image retrieval methods which is the key part of

our system. Most large scale search engines such as Google, baidu, make use of text as

query. In addition, these applications have already been put in mobiles to let more people use

them without the limitation of place and time. With the improvement of image retrieval

algorithms, people start to pay more attention on image search engine. Josef and Andrew put

forward applying text retrieval approach to object matching in videos and got good

performance. The similarity between text and image retrieval problems is that they could

both search by the key words, which are called visual words in image case. Then some

researches focused on finding out the better way to generate the visual words.

Bag of words method has shown good performance in creating visual words. Google

achieves great success by moving its navigation applications, such as Google Map, to mobile

phone. As tens of thousands people could afford mobile phones, how to locate in outdoor



with the help of computer vision and GPS technology becomes a new hot topic. A few

systems have been proposed making use of built-in camera and image retrieval methods for

this problem. In a hybrid image-and-keyword searching system has been devised. First,

image is used to search through web pages, and then keywords on these web pages are

identified and submitted to existing text search engine, such as Google. In a group from

Microsoft Research Asia has conducted an experiment on Photo-to-Search system, which

makes use of image retrieval methods to locate around the world.

There is an image retrieval system based on Content Based Image Retrieval methods.

Different from these approaches, our focus is not just on navigation but also a fresh way to

provide information about the world by pictures. Let us consider the situation that a user is in

an unfamiliar environment and he/she wants to get information on an unknown object

(building, sculpture). The user can take a photo of that object with their phones, and upload it

to our system. After that, our system recognizes the object and returns useful information.

A GPS tracking unit is a device that uses the Global Positioning System to determine

the precise location of a vehicle, person, or other asset to which it is attached and to record

the position of the asset at regular intervals. The recorded location data can be stored within

the tracking unit, or it may be transmitted to a central location data base, or internet-

connected computer, using a cellular (GPRS or SMS), radio, or satellite modem embedded in

the unit. This allows the asset's location to be displayed against a map backdrop either in real

time or when analyzing the track later, using GPS tracking software.

A GPS navigation device and a mobile phone sit side-by-side in the same box,

powered by the same battery. At regular intervals, the phone sends a text message via SMS

or GPRS, containing the data from the GPS receiver. Newer GPS-integrated smartphones

running GPS tracking software can turn the phone into a data pusher (or logger) device; as of

2009 open source and proprietary applications are available for common Java ME enabled

phones, iPhone , Android, Windows Mobile, and Symbian.

Most 21st century GPS trackers provide data "push" technology, enabling

sophisticated GPS tracking in business environments, specifically organizations that employ

a mobile workforce, such as a commercial fleet. Typical GPS tracking systems used in

commercial fleet management have two core parts: location hardware (or tracking device)



and tracking software. This combination is often referred to as an Automatic Vehicle

Location system. The tracking device is most often hardwire installed in the vehicle;

connected to the CAN-bus, Ignition system switch, battery. It allows collection of extra data,

which later get transferred to the GPS tracking server, where it is available for viewing, in

most cases via a website accessed over the internet, where fleet activity can be viewed live or

historically using digital maps and reports.

GPS tracking systems used in commercial fleets are often configured to transmit

location and telemetry input data at a set update rate or when an event (door open/close,

auxiliary equipment on/off, geofence border cross) triggers the unit to transmit data. Live

GPS Tracking used in commercial fleets, generally refers to systems which update regularly

at 1 minute, 2 minute or 5 minute intervals, whilst the ignition status is on. Some tracking

systems combine timed updates with heading change triggered updates. The applications of

these kind of trackers include: Personal tracking Law enforcement. An arrested criminal out

on bail may have to wear a GPS tracker, usually on the ankle, as a bail condition. Race

control. In some sports, such as gliding, participants are required to carry a tracker. This

allows, among other applications, for race officials to know if the participants are cheating,

taking unexpected shortcuts or how far apart they are. This use has been featured in the

movie Rat Race Espionage/surveillance. When put on a person, or on his personal vehicle, it

allows the person monitoring the tracking to know his/her habits. This application is used by

private investigators. These devices are also used by some parents to track their children. The

supporters claim that if cleverly used, this actually allows children more independence.

GPS personal tracking devices are being used increasingly to assist in the care of the

elderly and vulnerable. Devices allow users to call for assistance and optionally allow

designated careers to locate the user's position, typically within 5 to 10 meters. Their use

helps promote independent living and social inclusion for the elderly.

Devices often incorporate either 1-way or 2-way voice communication which is

activated by pressing a button or sliding a switch. Some devices also allow the user to call

several phone numbers using pre-programmed speed dial buttons. Internet Fun. Some Web

2.0 pioneers have created their own personal web pages that show their position constantly,

and in real-time, on a map within their website. These usually use data push from a GPS

enabled cell phone or a personal GPS tracker. In this paper, we introduce a system using



mobile camera, GPS information and PC server to search and recognize buildings without

typing any words. With quick development of mobile techniques, a large number of people

already own smart phones. Our system provides an attracting and easy way to know about

the world using images captured by mobile cameras. We have achieved good performance in

the range of Gulou Campus of Nanjing University.


Engineering

Gap search system