Mobile Device Development Camera and Sensors Dr.YingLiang Ma

Mobile Device Development

Mobile Device Development

Camera and Sensors

Dr.YingLiang Ma

UFCFX5-15-3Mobile Device Development

Agenda

Android Camera Overview

Simple android app using camera

Using Camera API

Using


Camera in android mobile


Camera Specifications 1. Camera resolutions (image sensors)

2. Optical Zoom or Digital Zoom

3. Xenon flash or LED flash

4. Video capture resolutions (4K??)

5. Face recognition

6. Low-light performance

7. Anti-shake performance


Camera Quality variation 1. Camera resolutions

2. With or without flash

3. No face-forward camera

4. No advance features


Simple Camera App

Using existing android camera application in our application

Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE);

1 ACTION_IMAGE_CAPTURE_SECURE

It returns the image captured from the camera , when the device is secured

2 ACTION_VIDEO_CAPTURE

It calls the existing video application in android to capture video

3 EXTRA_SCREEN_ORIENTATION

It is used to set the orientation of the screen to vertical or landscape

4 EXTRA_FULL_SCREEN

It is used to control the user interface of the ViewImage


UI Design<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"

xmlns:tools="http://schemas.android.com/tools"

android:layout_width="match_parent"

android:layout_height="match_parent"

tools:context=".MainActivity">

<ImageView

android:id="@+id/imageView1"



android:contentDescription="@string/hello_world"

android:src="@drawable/ic_launcher" />

<TextView

android:id="@+id/textView1"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_alignParentTop="true"

android:layout_alignRight="@+id/imageView1"

android:text="@string/tap"

android:textAppearance="?android:attr/textAppearanceLarge" />

</RelativeLayout>


String definition

<?xml version="1.0" encoding="utf-8"?>

<resources>

<string name="app_name">Camera</string>

<string name="action_settings">Settings</string>

<string name="hello_world">Hello world!</string>

<string name="tap">Tap the image to open the camera!!</string>

</resources>


Main Activitypublic class MainActivity extends Activity {

ImageView imgFavorite;

@Override

protected void onCreate(Bundle savedInstanceState) {

imgFavorite.setOnClickListener(new OnClickListener() {

@Override

public void onClick(View v) {open(); }

});

}

public void open(){

Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE);

startActivityForResult(intent, 0);

}

@Override

protected void onActivityResult(int requestCode, int resultCode, Intent data) {

// TODO Auto-generated method stub

super.onActivityResult(requestCode, resultCode, data);

Bitmap bp = (Bitmap) data.getExtras().get("data");

imgFavorite.setImageBitmap(bp);

}

}


Result


Camera API app

We will be using the camera API to integrate the camera in our application

Class Description

Camera It is used to control the camera and take images or capture video from the camera

SurfaceView This class is used to present a live camera preview to the user.


UI Design<?xml version="1.0" encoding="utf-8"?>

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"

android:layout_width="fill_parent"

android:layout_height="fill_parent"

android:orientation="horizontal" >

<LinearLayout

android:layout_width="wrap_content"


android:layout_weight="0.30"

android:orientation="vertical" >

<FrameLayout

android:id="@+id/camera_preview"

android:layout_width="fill_parent"

android:layout_height="400dp" />

<Button

android:id="@+id/button_capture"


android:layout_height="wrap_content"

android:onClick="snapIt"

android:text="@string/Capture" />

<ImageView android:id="@+id/imageView1" android:layout_width="fill_parent" android:layout_height="fill_parent" android:scaleType="fitXY" android:src="@drawable/ic_launcher" />

</LinearLayout>

</LinearLayout>


UI


Android Manifest file<?xml version="1.0" encoding="utf-8"?>

<manifest xmlns:android="http://schemas.android.com/apk/res/android"

package="com.example.camera1"

android:versionCode="1"

android:versionName="1.0" >

<uses-sdk

android:minSdkVersion="8"

android:targetSdkVersion="17" />

<uses-permission android:name="android.permission.CAMERA"/>

<uses-feature android:name="android.hardware.camera" />

<uses-feature android:name="android.hardware.camera.autofocus" />

<application

android:icon="@drawable/ic_launcher"

android:label="@string/app_name" >

<activity

android:name="com.example.camera1.MainActivity"

android:label="@string/app_name"

android:configChanges="orientation"

android:screenOrientation="portrait">


Structure

MainActivity ShowCamer

a

Preview windows

Main UIFrameLayout AddView

PictureCallbackTake photo


Show Camera Activitypublic class ShowCamera extends SurfaceView implements SurfaceHolder.Callback {

private SurfaceHolder holdMe;

private Camera theCamera;

public ShowCamera(Context context,Camera camera) {

super(context);

theCamera = camera;

holdMe = getHolder();

holdMe.addCallback(this);

}

@Override

public void surfaceCreated(SurfaceHolder holder) {

try {

theCamera.setPreviewDisplay(holder);

theCamera.startPreview();

} catch (IOException e) {

}

}

@Override

public void surfaceDestroyed(SurfaceHolder arg0) {

}

}


Main Activity

public class MainActivity extends Activity {

private Camera cameraObject;

private ShowCamera showCamera;

private ImageView pic;

@Override


super.onCreate(savedInstanceState);

setContentView(R.layout.activity_main);

pic = (ImageView)findViewById(R.id.imageView1);

cameraObject = isCameraAvailiable();

showCamera = new ShowCamera(this, cameraObject);

FrameLayout preview = (FrameLayout) findViewById(R.id.camera_preview);

preview.addView(showCamera);

}


Camera thread

private PictureCallback capturedIt = new PictureCallback() {

@Override

public void onPictureTaken(byte[] data, Camera camera) {

Bitmap bitmap = BitmapFactory.decodeByteArray(data , 0, data .length);

if(bitmap==null){

Toast.makeText(getApplicationContext(), "not taken", Toast.LENGTH_SHORT).show();

}

else

{

Toast.makeText(getApplicationContext(), "taken", Toast.LENGTH_SHORT).show();

pic.setImageBitmap(bitmap);

}

cameraObject.startPreview();

}

};


Save the picture Put SD card write permission in android Manifest file<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

In PictureCallback function Toast.makeText(getApplicationContext(), "taken", Toast.LENGTH_SHORT).show();

pic.setImageBitmap(bitmap);

String path = Environment.getExternalStorageDirectory().toString();

String filename = path + "/SpyApp"+String.format("%d.jpg", System.currentTimeMillis());

FileOutputStream outStream = null;

try {

outStream = new FileOutputStream(filename);

outStream.write(data);

outStream.close();

} catch (FileNotFoundException e) {

e.printStackTrace();

} catch (IOException e) {

e.printStackTrace();

} finally {

}


See the picture files Download File Manage from Google Play Store


See the picture files


See the picture files


Record audio

If you want to have more control over audio recording you could use the MediaRecorder.


Example codes

Set audio permission in android Manifest file<uses-permission android:name="android.permission.RECORD_AUDIO"/>

MediaRecorder recorder = new MediaRecorder();

recorder.setAudioSource(MediaRecorder.AudioSource.MIC);

recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);

recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);

recorder.setOutputFile(PATH_NAME);

recorder.prepare();

recorder.start(); // Recording is now started

...

recorder.stop();

recorder.reset(); // You can reuse the object by going back to setAudioSource() step

recorder.release(); // Now the object cannot be reused


Add them to MainActivity

Set audio permission in android Manifest file<uses-permission android:name="android.permission.RECORD_AUDIO"/>


…..

///record audio

String path = Environment.getExternalStorageDirectory().toString();

String filename = path + "/SpyAudio"+String.format("%d.3gp", System.currentTimeMillis());

recorder = new MediaRecorder();

recorder.setAudioSource(MediaRecorder.AudioSource.MIC);

recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);

recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);

recorder.setOutputFile(filename);

try {

recorder.prepare();

} catch (IllegalStateException e) {e.printStackTrace();}

recorder.start(); // Recording is now started


Stop recording@Override

public boolean onKeyDown(int keyCode, KeyEvent event) {

//Handle the back button

if(keyCode == KeyEvent.KEYCODE_BACK) {

recorder.stop();

recorder.reset(); // You can reuse the object by going back to setAudioSource() step

recorder.release(); // Now the object cannot be reused

return true;

}

else

return super.onKeyDown(keyCode, event);

}


Use Sensor in Android App

Motion sensors

These sensors measure acceleration forces and rotational forces along three axes. This category includes accelerometers, gravity sensors, gyroscopes, and rotational vector sensors.

Environmental sensors

These sensors measure various environmental parameters, such as ambient air temperature and pressure, illumination, and humidity. This category includes barometers, photometers, and thermometers.

Position sensors

These sensors measure the physical position of a device. This category includes orientation sensors and magnetometers.


Use Sensor in Android App

You can access sensors available on the device and acquire raw sensor data by using the Android sensor framework.

Determine which sensors are available on a device.Determine an individual sensor's capabilities, such as its maximum range, manufacturer, power requirements, and resolution.Acquire raw sensor data and define the minimum rate at which you acquire sensor data.Register and unregister sensor event listeners that monitor sensor changes.


Sensor availablity

Few Android-powered devices have every type of sensor.

For example, most handset devices and tablets have an accelerometer and a magnetometer, but fewer devices have barometers or thermometers.

Also, a device can have more than one sensor of a given type. For example, a device can have two gravity sensors, each one having a different range.

Identifying Sensors and Sensor Capabilities

private SensorManager mSensorManager;

mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);List<Sensor> deviceSensors = mSensorManager.getSensorList(Sensor.TYPE_ALL);


Monitoring Sensor Eventspublic class SensorActivity extends Activity implements SensorEventListener {


private Sensor mLight;

@Override

public final void onSensorChanged(SensorEvent event) {

// The light sensor returns a single value.

// Many sensors return 3 values, one for each axis.

float lux = event.values[0];

// Do something with this sensor value.

}

}


Handling Different Sensor Configurations


...

mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);

if (mSensorManager.getDefaultSensor(Sensor.TYPE_PRESSURE) != null){

// Success! There's a pressure sensor.

}

else {

// Failure! No pressure sensor.

}

Using Google Play filters to target specific sensor configurations

<uses-feature android:name="android.hardware.sensor.accelerometer"

android:required="true" />


Best Practices for Accessing and Using Sensors

Unregister sensor listeners

Don't test your code on the emulator

Don't block the onSensorChanged() method (do as little as possible within the onSensorChanged(SensorEvent) method as sensor data can change at a high rate)

Avoid using deprecated methods or sensor types

Verify sensors before you use them

Choose sensor delays carefully


Using the Proximity Sensorprivate SensorManager mSensorManager;

private Sensor mSensor;

...


mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_PROXIMITY);

@Override

public final void onSensorChanged(SensorEvent event) {

float distance = event.values[0];

// Do something with this sensor data.

}


Using the Orientation Sensor

The orientation sensor lets you monitor the position of a device relative to the earth's frame of reference (specifically, magnetic north).



...


mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION);

@Override

public void onSensorChanged(SensorEvent event) {

float azimuth_angle = event.values[0];

float pitch_angle = event.values[1];

float roll_angle = event.values[2];

// Do something with these orientation angles.

}


Using the Step Counter Sensor

The step counter sensor provides the number of steps taken by the user since the last reboot while the sensor was activated. The step counter has more latency (up to 10 seconds) but more accuracy than the step detector sensor.



...


mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER);


Environment SensorsUnlike most motion sensors and position sensors, which return a multi-dimensional array of sensor values for each SensorEvent, environment sensors return a single sensor value for each data event.

TYPE_AMBIENT_TEMPERATURE event.values[0] °C Ambient air temperature.

TYPE_LIGHT event.values[0] lx Illuminance.

TYPE_PRESSURE event.values[0 hPa or mbar Ambient air pressure.

TYPE_RELATIVE_HUMIDITY event.values[0] % Ambient relative humidity.

TYPE_TEMPERATURE event.values[0] °C Device temperature.

Documents

Mobile Device Development Camera and Sensors Dr.YingLiang Ma