Napredno prepoznavanje ljudi koristeći Microsoft Azure Cognitive Services SLAVEN MIŠAK, Span d.o.o. IVAN MARKOVIĆ, Span d.o.o.

Agenda Uvod u Azure Cognitive Services Face API Emotion API Demo

UVOD U AZURE COGNITIVE SERVICES

Machine learning Big data Large dataset Machine learning algorithms Extensive testing and validation Target example

Cognitve services origins Microsoft Research Project Oxford (https://projectoxford.ai) Microsoft Azure SaaS How-old.net, TwinsOrNot.net, what-dog.net

Available services Vision Speech Language Knowledge Search Computer vision Emotion Face Video Speech Bing Speech Custom recognition Speaker recognition Language Bing spell check Language understanding Linguistic analysis Text analysis WebLM Knowledge Academic Entity linking Knowledge exploration Recomendations Search Bing web search Bing image search Bing video search Bing news search Bing autosuggest

FACE API

Face API Two main functionalities: Face Detection Face Recognition

Face Detection Detect up to 64 human faces in an image. Face detection can be done: By uploading image file Specifying a URL of an existing image on the web

Face API- Detection Steps Subscribe for Face API and get subscription key Configure the Face API client library (or use REST API) Upload images to detect faces Mark faces in the image

Face API – Get Started faceServiceClient = new FaceServiceClient("Your subscription key"); private async Task<FaceRectangle[]> UploadAndDetectFaces(string imageFilePath) { try using (Stream imageFileStream = File.OpenRead(imageFilePath)) var faces = await faceServiceClient.DetectAsync(imageFileStream); var faceRects = faces.Select(face => face.FaceRectangle); return faceRects.ToArray(); } catch (Exception) return new FaceRectangle[0];

Face API – Get Started var requiedFaceAttributes = new FaceAttributeType[] { FaceAttributeType.Age, FaceAttributeType.Gender, FaceAttributeType.Smile, FaceAttributeType.FacialHair, FaceAttributeType.HeadPose, FaceAttributeType.Glasses }; var faces = await faceServiceClient.DetectAsync(imageUrl, returnFaceLandmarks: true, returnFaceAttributes: requiedFaceAttributes); foreach (var face in faces) { var id = face.FaceId; var attributes = face.FaceAttributes; var age = attributes.Age; var gender = attributes.Gender; var smile = attributes.Smile; var facialHair = attributes.FacialHair; var headPose = attributes.HeadPose; var glasses = attributes.Glasses; }

Face API - Face Attributes Age Gender Smile Facial Hair Head Pose Glasses

Face API – face landmarks 27 predefined landmark points

Face API – Response example JSON: [ { "faceId": "d585cde5-6e2e-4fb9-b46a-2e941a4efe0a", "faceRectangle": { "width": 289, "height": 289, "left": 266, "top": 292 }, "faceLandmarks": { "pupilLeft": { "x": 337.8, "y": 372.4 "pupilRight": { "x": 478.5, "y": 378.1

Face API – Face Recognition Four recognition functionalities: Face Verification Similar Face Searching Automatic Face Grouping Person Identification

Face Verification Request Response { "faceId1":"c5c24a82-6845-4031-9d5d-978df9175426", "faceId2":"015839fb-fbd9-4f79-ace9-7675fc2f1dd9" } { "isIdentical":true, "confidence":0.9 }

Similar Face Searching Request Response { "faceId":"c5c24a82-6845-4031-9d5d-978df9175426", "faceListId":"sample_list", "maxNumOfCandidatesReturned":10 } [ { "persistedFaceId" : "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "confidence" : 0.82 }, ... ]

Face Grouping Request Response { "faceIds":[ "c5c24a82-6845-4031-9d5d-978df9175426", "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "65d083d4-9447-47d1-af30-b626144bf0fb", "fce92aed-d578-4d2e-8114-068f8af4492e", "30ea1073-cc9e-4652-b1e3-d08fb7b95315", "be386ab3-af91-4104-9e6d-4dae4c9fddb7", "fbd2a038-dbff-452c-8e79-2ee81b1aa84e", "b64d5e15-8257-4af2-b20a-5a750f8940e7" ] } { "groups": [ [ "c5c24a82-6845-4031-9d5d-978df9175426", "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "fce92aed-d578-4d2e-8114-068f8af4492e", "b64d5e15-8257-4af2-b20a-5a750f8940e7" ],[ "65d083d4-9447-47d1-af30-b626144bf0fb", "30ea1073-cc9e-4652-b1e3-d08fb7b95315" ] ], "messyGroup": [ "be386ab3-af91-4104-9e6d-4dae4c9fddb7" }

Face Identification Authorize the API call Create the person group Define people for the person group Detect faces and register them to correct person Train the person group Identify a face against a defined person group

Face Identification – result example

Face API – Technical info JPEG, PNG, GIF and BMP are supported. Max. File size is 4MB The detectable face size is between 36x36 and 4096x4096 pixels. A maximum of 64 faces could be detected

EMOTION API

Emotion API Returns the confidence across a set of emotions for each face in the image Happiness Sadness Surprise Anger Fear Contempt Disgust Neutral

Emotion API – Get Started EmotionServiceClient emotionServiceClient = new EmotionServiceClient(subscriptionKey); //Optional parameter FaceRectangles[] Emotion[] emotionResult = await emotionServiceClient.RecognizeAsync(url);

DEMO

Q&A