The Extended Cohn-Kanade Dataset(CK+):A complete dataset for action unit and emotion-specified expression Author：Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar Conference on Computer Vision and Pattern Recognition 2010 Speaker：Liu, Yi-Hsien

Outline Introduction The CK+ Dataset Emotion Labels Baseline System Experiments Conclusion

Introduction In 2000, the Cohn-Kanade (CK) database was released Automatically detecting facial expressions has become an increasingly important research area 1.目的是為了促進對於自動偵測個人臉部表情的研究，自發表之後CK資料庫就成為對於演算法的發展與評估最被廣泛使用的資料庫之一 2.自動偵測臉部表情涉及了computer vision, machine learning and behavioral sciences 而且可被用在像是 security, human-computer-interaction, driver safety, and health-care 等等應用上

Introduction(Cont.) The CK database contains 486 sequences across 97 subjects. Each of the sequences contains images from onset (neutral frame) to peak expression (last frame). The peak frame was reliably FACS(Facial Action Coding System ) coded for facial action units (AUs). Facial Action Coding System (FACS)

Introduction(Cont.) Facial Action Coding System (FACS) is a system to taxonomize human facial movements by their appearance on the face 臉部行為編碼系統，是藉由對臉部的外觀進行人類臉部動作分類的系統

Introduction(Cont.) While AU codes are well validated, emotion labels are not The lack of a common performance metric against which to evaluate new algorithms Standard protocols for common databases have not emerged 在CK發表後的十年間，CK資料庫常常被拿來使用，但也因此出現了三個問題 所以emotion labels常常被用來指定成某個另外的情緒，而不是他原本代表的情緒 缺少跟新演算法評估的基準

The CK+ Dataset Participants were 18 to 50 years of age, 69% female, 81% Euro-American, 13% Afro-American, and 6% other groups Image sequences for frontal views and 30-degree views were digitized into either 640x490 or 640x480 pixel arrays with 8- bit gray-scale or 24-bit color values.

The CK+ Dataset(Cont.) For the CK+ distribution, they have augmented the dataset further to include 593 sequences from 123 subjects (an additional 107 (22%) sequences and 26 (27%) subjects). For the 593 posed sequences, full FACS coding of peak frames is provided.

Emotion Labels They included all image data from the pool of 593 sequences that had a nominal emotion label based on the subject’s impression of each of the 7 basic emotion categories: Anger, Contempt, Disgust, Fear, Happy, Sadness and Surprise. 如果就這樣把這些標籤當成ground truth是很不可靠的 ，直接這樣進行訓練之類的動作可能會發生錯誤

Emotion Labels(Cont.) Compared the FACS codes with the Emotion Prediction Table from the FACS After the first pass, a more loose comparison was performed The third step involved perceptual judgment of whether or not the expression resembled the target emotion category. 所以，他們根據FACS來為這些情緒標記，實行步驟就是這三個步驟 情緒預測表格列出除了輕視(Contempt)以外每種情緒的原型和主要變化的臉部配置，也就是AU的組合，如果某片段滿足某情緒原型和主要變化，就暫時將這片段分類到那情緒 如果有個片段包含某個不屬於此情緒原型和變型的AU，就要判斷這片段到底屬不屬於這情緒，判斷標準就像這表格，下面的圖是例子 3. 需要第三個步驟是因為FACS碼只敘述最後一張的表情而沒有顧及到在形成最後一張的表情時所需要的臉部變化，也就是說，要決定某個表情是否為某情緒的表現是需要直接從頭看到尾

Emotion Labels(Cont.)

Emotion Labels(Cont.) As a result of this multistep selection process, 327 of the 593 sequences were found to meet criteria for one of seven discrete emotions.

Baseline System 這篇論文採用了基於active appearance models的系統來擷取特徵，接著使用support vector machines來分類表情跟情緒，流程圖就是上面這張圖

Baseline System(Cont.) Active Appearance Models (AAMs) The shape s of an AAM is described by a 2D triangulated mesh. In particular, the coordinates of the mesh vertices define the shape s = [x1; y1; x2; y2; …. ; xn; yn]

Baseline System(Cont.) SPTS：The similarity normalized shape, refers to the 68 vertex points for both the x- and y- coordinates, resulting in a raw 136 dimensional feature vector CAPP： The canonical normalized appearance, refers to where all the shape variation has been normalized with respect to the base shape 上面是SPTS下面是CAPP

Baseline System(Cont.) SVMs(Support Vector Machines) attempt to find the hyper plane that maximizes the margin between positive and negative observations for a specified class. 實線Support Hyper-planes，這篇論文使用的是二分法，例如在分類時分為Angry 跟 not Angry，或Happy 跟 Not Happy

Experiments Emotion detection. To maximize the amount of training and testing data, they believe the use of a leave-one-subject-out cross-validation configuration should be used.

Experiments(Cont.) SPTS Happy跟Surprise的偵測率比起其他情緒好很多，因為這兩種情緒在臉上造成很多變形

Experiments(Cont.) CAPP 可以發現Disgust(厭惡)的偵測率大幅提高，因為Disgust情緒會造成鼻子附近的紋理大量變化

Experiments(Cont.) SPTS+CAPP 鄙視(Contempt)大幅提高，從20幾%上升至80幾%，可能的解釋是，鄙視是個很細微的情緒，所以很容易跟其他情緒搞混，但同時使用形狀跟外觀兩種特徵來偵測之後，就比較不容易被搞混

Conclusion In this paper, they try to address those three issues by presenting the Extended Cohn-Kanade (CK+) database Added another 107 sequences as well as another 26 subjects. The peak expression for each sequence is fully FACS coded and emotion labels have been revised and validated

Conclusion(Cont.) Propose the use of a leave-one-out subject cross-validation strategy for evaluating performance Present baseline results on this using our Active Appearance Model (AAM)/support vector machine (SVM) system.