1. Napredno prepoznavanje ljudi koristeći Microsoft Azure Cognitive Services SLAVEN MIŠAK, Span d.o.o.
IVAN MARKOVIĆ, Span d.o.o.

3. Agenda
Uvod u Azure Cognitive Services
Face API
Emotion API
Demo

4. UVOD U AZURE COGNITIVE SERVICES

5. Machine learning Big data
Large dataset
Machine learning algorithms
Extensive testing and validation
Target example

6. Cognitve services origins
Microsoft Research
Project Oxford (
Microsoft Azure SaaS
How-old.net, TwinsOrNot.net, what-dog.net

7. Available services Vision Speech Language Knowledge Search
Computer vision
Emotion
Face
Video
Speech
Bing Speech
Custom recognition
Speaker recognition
Language
Bing spell check
Language understanding
Linguistic analysis
Text analysis
WebLM
Knowledge
Academic
Entity linking
Knowledge exploration
Recomendations
Search
Bing web search
Bing image search
Bing video search
Bing news search
Bing autosuggest

8. FACE API

9. Face API
Two main functionalities:
Face Detection
Face Recognition

10. Face Detection Detect up to 64 human faces in an image.
Face detection can be done:
By uploading image file
Specifying a URL of an existing image on the web

11. Face API- Detection Steps
Subscribe for Face API and get subscription key
Configure the Face API client library (or use REST API)
Upload images to detect faces
Mark faces in the image

12. Face API – Get Started
faceServiceClient = new FaceServiceClient("Your subscription key");
private async Task<FaceRectangle[]> UploadAndDetectFaces(string imageFilePath) { try using (Stream imageFileStream = File.OpenRead(imageFilePath)) var faces = await faceServiceClient.DetectAsync(imageFileStream); var faceRects = faces.Select(face => face.FaceRectangle); return faceRects.ToArray(); } catch (Exception) return new FaceRectangle[0];

13. Face API – Get Started
var requiedFaceAttributes = new FaceAttributeType[] {
FaceAttributeType.Age,
FaceAttributeType.Gender,
FaceAttributeType.Smile,
FaceAttributeType.FacialHair,
FaceAttributeType.HeadPose,
FaceAttributeType.Glasses };
var faces = await faceServiceClient.DetectAsync(imageUrl, returnFaceLandmarks: true, returnFaceAttributes: requiedFaceAttributes);
foreach (var face in faces) {
var id = face.FaceId;
var attributes = face.FaceAttributes;
var age = attributes.Age;
var gender = attributes.Gender;
var smile = attributes.Smile;
var facialHair = attributes.FacialHair;
var headPose = attributes.HeadPose;
var glasses = attributes.Glasses; }

14. Face API - Face Attributes
Age
Gender
Smile
Facial Hair
Head Pose
Glasses

15. Face API – face landmarks
27 predefined landmark points

16. Face API – Response example
JSON: [ { "faceId": "d585cde5-6e2e-4fb9-b46a-2e941a4efe0a", "faceRectangle": { "width": 289, "height": 289, "left": 266, "top": 292 }, "faceLandmarks": { "pupilLeft": { "x": 337.8, "y": "pupilRight": { "x": 478.5, "y": 378.1

17. Face API – Face Recognition
Four recognition functionalities:
Face Verification
Similar Face Searching
Automatic Face Grouping
Person Identification

18. Face Verification Request Response
{ "faceId1":"c5c24a d5d-978df ", "faceId2":"015839fb-fbd9-4f79-ace9-7675fc2f1dd9" }
{ "isIdentical":true, "confidence":0.9 }

19. Similar Face Searching
Request
Response
{ "faceId":"c5c24a d5d-978df ", "faceListId":"sample_list", "maxNumOfCandidatesReturned":10 }
[ { "persistedFaceId" : "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "confidence" : 0.82 }, ... ]

20. Face Grouping Request Response
{ "faceIds":[ "c5c24a d5d-978df ", "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "65d083d d1-af30-b626144bf0fb", "fce92aed-d578-4d2e f8af4492e", "30ea1073-cc9e-4652-b1e3-d08fb7b95315", "be386ab3-af e6d-4dae4c9fddb7", "fbd2a038-dbff-452c-8e79-2ee81b1aa84e", "b64d5e af2-b20a-5a750f8940e7" ] }
{ "groups": [ [ "c5c24a d5d-978df ", "015839fb-fbd9-4f79-ace9-7675fc2f1dd9", "fce92aed-d578-4d2e f8af4492e", "b64d5e af2-b20a-5a750f8940e7" ],[ "65d083d d1-af30-b626144bf0fb", "30ea1073-cc9e-4652-b1e3-d08fb7b95315" ] ], "messyGroup": [ "be386ab3-af e6d-4dae4c9fddb7" }

21. Face Identification Authorize the API call Create the person group
Define people for the person group
Detect faces and register them to correct person
Train the person group
Identify a face against a defined person group

22. Face Identification – result example

23. Face API – Technical info
JPEG, PNG, GIF and BMP are supported. Max. File size is 4MB
The detectable face size is between 36x36 and 4096x4096 pixels.
A maximum of 64 faces could be detected

24. EMOTION API

25. Emotion API
Returns the confidence across a set of emotions for each face in the image
Happiness
Sadness
Surprise
Anger
Fear
Contempt
Disgust
Neutral

26. Emotion API – Get Started
EmotionServiceClient emotionServiceClient = new EmotionServiceClient(subscriptionKey);
//Optional parameter FaceRectangles[]
Emotion[] emotionResult = await emotionServiceClient.RecognizeAsync(url);

27. DEMO

28. Q&A