Mobile App development course project By Vijaya Varigonda

Table Of Contents Introduction Face Substitution Facial LandMark Detection JNI Classes Dlib Libraries/ HaarCascade files Face Morphing Difficulties

Introduction We used face detection untill now. What I worked on: Saving the captured image for further usage Facial landmark detection, Using dlib libraries,jni classes, Haarcascade xml files

Face Substitution Use camera to capture an image/face Save the captured image Then use an existing image of ears or a puppy nose etc to replace the source/captured image. Different methods to try: addWeighted() copyTo() Using the Dlib library, use shape_predictor_68_face_landmarks.dat model to get 68 facial points.

addWeighted() alpha = 0.5 beta = ( 1.0 - alpha ); addWeighted( src1, alpha, src2, beta, 0.0, dst); addWeighted() would give: ‘Dst’ is the linear blend of both images.

copyTo() Save captured image Mat img1 = Highgui.imread(path_of_our_capturedimage); Mat subareaToReplace = img1.submat() replaceImage.copyTo(subareaToReplace);

Facial Landmark Detections

My idea.. After we find the landmarks, we can replace those pixels with our ears, nose etc.

JNI Classes To use c++ classes in android studio. Terminal -> javah -d jni –classpath ../../build/intermediates/classes/debug com.example.praneethavarigonda.thisistest.

Android.mk Application.mk

Face Morphism First capture an image from the camera and save it to an external source. We can then use a previously stored image and morph it with the captured image. Find the facial landmarks in both images using dlib library. Perform Delaunay triangulation We use ‘alpha’ as a parameter. When it is set to 0.5 we get the morphed image. Using OpenCv we use the getAffineTransform method to generate the morphed image.

For each triangle in image 1, use the affine transform calculated in the previous step to transform all pixels inside the triangle to the morphed image. warpAffine in open cv can perform this.

Face Morphing

Haarcasscade haarcascade_eye_tree_eyeglasses.xml haarcascade_frontalface_alt.xml They were much more useful in detecting the facial features.

Dlib An open source toolkit for image processing. Has useful functions for face detection, object detection, facial landmarks detection etc. Has good documentation. Works well with opencv

Difficulties Ndk errors, Opencv file not recognized. Figuring out the input parameters to give Not much documentation. Only a few people who have done the coding and only that is available. Any complexity in the code would make the application crash If we have to create a new project for any sort of testing, preparation of the project would take a long time.(Importing opencv, making sure open cv is loaded successfully, creating of native files etc)

Thank You !