How to Build & Use OpenCL on OpenCV & Android NDK

©SIProp Project, 2006-2008 1

How to Build & Use OpenCLon OpenCV & Android NDK

Noritsuna Imamura

[email protected]


!!!!!!ATTENTION!!!!!!

This manual is “How to Build” manual.

If you want to “Use OpenCL on OpenCV” ONLY,

Please see

http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSample

http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDK


How to Build OpenCL on OpenCV

for Android NDK


Download OpenCV & NDK tool

› wget https://github.com/Itseez/opencv/archive/2.4.10.1tar.gz

› tar zxfv 2.4.10.1.tar.gz

› mkdir ~/android/› cd ~/android/› wget http://dl.google.com/android/android-

sdk_r23.0.2-linux.tgz› wget

http://dl.google.com/android/ndk/android-ndk-r10c-linux-x86_64.bin

› tar zxfv android-sdk_r23.0.2-linux.tgz› ./android-ndk-r10c-linux-x86_64.bin


Edit Setup File

./opencv-2.4.10.1/platforms/scripts/cmake_android_arm.sh

Add WITH_EIGEN=off option.Currently version has some bugs.

› cmake -DWITH_EIGEN=OFF -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DCMAKE_TOOLCHAIN_FILE=../android/android.toolchain.cmake $@ ../..


Set Path to SDK/NDK

› export ANDROID_NDK=~/android/android-ndk-r10c

› export ANDROID_SDK=~/android/android-sdk-linux


Add “SO loading“ func for Android 1/3

Change “SO loading“ func for Linux to this funcwithout Android.

~/opencv-2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp

› #if defined(__linux__)↓

› #if defined(__linux__) && !defined(__ANDROID__)



Add “SO loading“ func for Android. (continue to next page)

~/opencv-2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp

› #if defined(__ANDROID__)› #include <dlfcn.h>› #include <sys/stat.h>

› #if defined(__ARM_ARCH_8A__) || defined(_X64_)› static const char *default_so_paths[] = {› "/system/lib64/libOpenCL.so",› "/system/vendor/lib64/libOpenCL.so",› "/system/vendor/lib64/egl/libGLES_mali.so"› };› #else› static const char *default_so_paths[] = {› "/system/lib/libOpenCL.so",› "/system/vendor/lib/libOpenCL.so",› "/system/vendor/lib/egl/libGLES_mali.so"› };› #endif

›



› static int access_file(const char *filename)› {› struct stat buffer;› return (stat(filename, &buffer) == 0);› }

› static void* GetProcAddress (const char* name)› {› static void* h = NULL;› unsigned int i;› if (!h)› {› const char* name;› for(i=0; i<(sizeof(default_so_paths)/sizeof(char*)); i++)› {› if(access_file(default_so_paths[i])) {› name = (char *)default_so_paths[i];› h = dlopen(name, RTLD_LAZY);› if (h) break;› }› }› if (!h)› return NULL;› }

› return dlsym(h, name);› }› #define CV_CL_GET_PROC_ADDRESS(name) GetProcAddress(name)› #endif


Execute Auto Config & make

This Script make Makefile for OpenCV.

In “~/opencv-2.4.10.1/platforms/scripts/build_android_arm.sh”

Make !!!

› cd ~/opencv-2.4.10.1/platforms/scripts/› ./cmake_android_arm.sh

› cd ~/opencv-2.4.10.1/platforms/scripts/build_android_arm/

› make


Where are Made Files?

OpenCV Libs.

In “~/opencv-2.4.10.1/platforms/build_android_arm/lib”

highgui

core

etc...

3rd Party Libs for OpenCV.

In “~/opencv-2.4.10.1/platforms/build_android_arm/3rdparty/lib”

libIlmImf

libjasper

libjpeg

libpng

libtiff


Copy libs to sample program

OpenCV Libs.

cp ~/opencv-2.4.10.1/platforms/build_android_arm/lib/armeabi-v7a/* ~/opencv-2.4.10.1/samples/android/native-activity/libs_opencv/

3rd Party Libs for OpenCV.

cp ~/opencv-2.4.10.1/platforms/build_android_arm/3rdparty/lib/armeabi-v7a/* ~/opencv-2.4.10.1/samples/android/native-activity/libs_opencv/


How to Use OpenCL on OpenCV

for Android NDK(Native App)


Download Sample Source Code

http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSample

Based OpenCV-2.4.10.1 & Android 4.4

Made Android Native Activity.

Convert RGB Color Image to Black&White Image.

http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDK


Native Application

NDK wo/ADT

Standard Android Application for C/C++

Only C/C++ on LimitedLibrary Layer

Advantage

Only C/C++

DirectCall C/C++ API

Dis-Advantage

Use a few Android Tools

A few Docs from Google Developer Site & Blogs

APK File(Your Application)(C/C++)

Library Layer (C/C++)

Kernel/Driver Layer(C/ASM)

Call as C/C++ APIs

Call as SysCall(C/ASM)

Call Stack


How to Make this Sample Source Code from OpenCV’s Sample Project


Use NativeActivity of OpenCV

This Sample uses Only C/C++ Code.

~/opencv-2.4.10.1/samples/android/native-activity

Structure of NativeActivity

AndroidManifest.xml <- Property File for Android

res <- Directory that has Resource file(Image,Voice)

jni <- Directory that has files for NDKAndroid.mk <- make file of NDK

Application.mk <- property file of NDK

native.cpp <- Sample App

Other Files/Dir, don’t use.


Edit AndroidManifest.xml

Replace “application” Section

1. <application android:label="@string/app_name" android:hasCode="false">

2. <activity android:name="android.app.NativeActivity"3. android:label="@string/app_name"4.

android:configChanges="orientation|keyboardHidden">5. 6. <meta-data android:name="android.app.lib_name"7. android:value="native_activity" />8. <intent-filter>9. <action

android:name="android.intent.action.MAIN" />10. <category

android:name="android.intent.category.LAUNCHER" />11. </intent-filter>12. </activity>13. </application>


About AndroidManifest.xml

API Level

Permission

1. 

2. <uses-sdk android:minSdkVersion=“9" />

1. <uses-permission android:name="android.permission.CAMERA"/>


Create jni/Android.mk 1/9

1. Delete jni/Android.mk & Create the new jni/Android.mk

2. Set LOCAL_PATH & include

1. LOCAL_PATH := $(call my-dir)

2. include $(CLEAR_VARS)



1. Set OpenCV’s Modules & Libs

1. OPENCV_MODULES:=contrib legacy ml stitching superres objdetect ts videostabvideo photo calib3d features2d highguiimgproc flann ocl core androidcamera

2. OPENCV_3RDPARTY_COMPONENTS:=assetmanager tbb libjpeg libpng libtiff libjasperIlmImf

3. OPENCV_CAMERA_MODULES:= native_camera_r2.2.0 native_camera_r2.3.3 native_camera_r3.0.1 native_camera_r4.0.0 native_camera_r4.0.3 native_camera_r4.1.1 native_camera_r4.2.0 native_camera_r4.3.0 native_camera_r4.4.0



Set Loading OpenCV Libs Function

1. $(foreachmodule,$(OPENCV_MODULES),$(eval $(call add_opencv_module,$(module))))

1. define add_opencv_module2. include $(CLEAR_VARS)3. LOCAL_MODULE:=opencv_$14.

LOCAL_SRC_FILES:=../libs_opencv/libopencv_$1.a

5. include $(PREBUILT_STATIC_LIBRARY)6. endef



Set Loading OpenCV Libs Function

1. $(foreachmodule,$(OPENCV_3RDPARTY_COMPONENTS),$(eval $(call add_opencv_3rdparty_component,$(module))))

1. define add_opencv_3rdparty_component2. include $(CLEAR_VARS)3. LOCAL_MODULE:=$14.

LOCAL_SRC_FILES:=../libs_opencv/lib$1.a5. include $(PREBUILT_STATIC_LIBRARY)6. endef



Set Loading AndroidCamera Libs Function

1. $(foreachmodule,$(OPENCV_CAMERA_MODULES),$(eval $(call add_opencv_camera_module,$(module))))

1. define add_opencv_camera_module2. include $(CLEAR_VARS)3. LOCAL_MODULE:=$14.

LOCAL_SRC_FILES:=../libs_opencv/lib$1.so5. include $(PREBUILT_SHARED_LIBRARY)6. endef



Set loading info lib

1. include $(CLEAR_VARS)2. LOCAL_MODULE:=opencv_info3. LOCAL_SRC_FILES:=../libs_opencv/libopenc

v_info.so4. include $(PREBUILT_SHARED_LIBRARY)



Set Include Dir

Set Module name & Source Code

OpenCVwithOpenCL4AndroidNDKNativeActivity.cpp

1. include $(CLEAR_VARS)

2. LOCAL_C_INCLUDES+=./jni/include

3. LOCAL_MODULE := OpenCV_CL4. LOCAL_SRC_FILES :=

OpenCVwithOpenCL4AndroidNDKNativeActivity.cpp



Set Loading Lib & OpenCV Lib

LOCAL_LDLIBS-lm -llog -lc -ldl -lz -landroid

LOCAL_STATIC_LIBRARIES$(foreach mod, $(OPENCV_MODULES), opencv_$(mod)) $(OPENCV_3RDPARTY_COMPONENTS)

1. LOCAL_LDLIBS += -lm -llog -lc -ldl -lz -landroid

2. LOCAL_STATIC_LIBRARIES := android_native_app_glue $(foreach mod, $(OPENCV_MODULES), opencv_$(mod)) $(OPENCV_3RDPARTY_COMPONENTS)



Set Library Type

Make SO libBUILD_SHARED_LIBRARY

Set Loading Android module

Use “native_app_glue” module$(call import-module,android/native_app_glue)

1. include $(BUILD_SHARED_LIBRARY)

2. $(call import-module,android/native_app_glue)


Setup/Make Build.xml 1/2

Making Setting File for Android using ANT

Today, Development on CUI Only!!!Don’t use Eclipse!

› android list target› Available Android targets:› ----------› id: 1 or "android-9"› Name: Android 2.3.1› Type: Platform› API level: 9› Revision: 2› Skins: WQVGA400, QVGA, HVGA, WVGA800 (default), WVGA854› ABIs : armeabi› ----------› id: 2 or "android-18"› Name: Android 4.3› Type: Platform› API level: 18› Revision: 2› Skins: WXGA800, WQVGA400, QVGA, WQVGA432, WXGA800-7in, WVGA800

(default), HVGA, WXGA720, WSVGA, WVGA854› ABIs : armeabi-v7a› ----------


Setup/Make Build.xml 2/2

Make Build.xml

› android update project --path . --target 2


Build NativeActivity

Build C/C++ Source Codes

Make APK File

› ndk-build [clean all]

› ant [debug|release]


Install NativeActivity.apk

Install APK File

UnInstall APK File

Show Log

› adb install bin/NativeActivity-debug.apk

› adb logcat

› adb uninstall com.example.native_activity


adb(Android Debug Bridge)

Functions

Shell on Android Deviceadb shell

Show Connected Android Devicesadb devices

Install/Uninstall APK Fileadb install/uninstall [App Name]/[Package Name]

Send/Get File to/from Android Deviceadb push/pull [from File Path] [to File Path]

Show Log (DDMS)adb logcat


Quick Explanation to OpenCL

1. void android_main(android_app* app) {2. （snip）3. cv::Mat drawing_frame;4. (snip)

5. colorSampleImage = imread(file_path);6. (snip)

7. // Create Mat Object for OpenCL8. cv::ocl::oclMat oclIn(colorSampleImage), oclOut;9. // Change Color RGB->Black&White on OpenCL10. cv::ocl::cvtColor(oclIn, oclOut, cv::COLOR_BGR2GRAY);11. // Download Mat Object for OpenCL to Mat for CPU12. oclOut.download(drawing_frame); 13.14. engine_draw_frame(&engine, drawing_frame);


How to Set up OpenCL Driver

adreno-3xx: Qualcomm's Adreno 3xx GPUs

mali-t6xx: ARM's Mali-T6xx GPUs


Download OpenCL Drivers

OpenCL Drivers was deleted since 4.2

adreno-3xx: Qualcomm's Adreno 3xx GPUs

mali-t6xx: ARM's Mali-T6xx GPUs

Download from:

https://github.com/madeye/opencl-android-proprietary

https://github.com/madeye/opencl-android-proprietary


Set up Drivers to your Android

Ex. adreno-3xx: Qualcomm's Adreno 3xx GPUs

1. $ adb push kitkat/system/vendor/lib /sdcard/2. $ adb shell3. $ su4. # mount -o rw,remount -t yaffs2 /dev/block/mtdblock3

/system5. # cp /sdcard/libOpenCL.so /system/vendor/lib/6. # cp /sdcard/libllvm-a3xx.so /system/vendor/lib/7. # chmod 644 /system/vendor/lib/libOpenCL.so8. # chmod 644 /system/vendor/lib/libllvm-a3xx.so

Technology

How to Build & Use OpenCL on OpenCV & Android NDK