1     Definition 2     Functional Requirements 3     Syntax
4     Semantics 5    Conformance Testing 6     Performance Assessment

1    Definition

Face Descriptors is a Data Type representing the features of the Face of an Entity resulting from the motion of the muscles of its face.

2     Functional Requirements

Face Descriptors are represented by the Actions Units of the Facial Action Coding System (FACS) originally developed by Carl-Herman Hjortsjö, adopted by Paul Ekman and Wallace V. Friesen (1978) and updated by Ekman, Friesen, and Joseph C. Hager (2002). Each Action Unit is represented by an Action Unit ID.

The eye motion Action Units (Eyes turn left/right and Eyes up/down) have an additional parameter representing the degrees of the motion of a line perpendicular to the eye with respect to the 0° of the resting position when the eye looks horizontally and perpendicular to the body. The value of the angle is expressed with 1 Byte where 0= 0° degrees for the resting position (0°) and 255 for 90°.

3     Syntax

https://schemas.mpai.community/PAF/V1.5/data/FaceDescriptors.json

4     Semantics

Header N1 Bytes Face Descriptors Header.
– Standard-FaceDescriptors 9 Bytes The characters “OSD-FCD-V”
– Version N2 Bytes Major version – 1 or 2 characters
– Dot-separator 1 Byte The character “.”
– Subversion N3 Bytes Minor version – 1 or 2 characters
FaceDescriptorsID N4 Bytes Identifier of Face Descriptors.
AU Description N5 Bytes Facial muscle generating the Action
1 Inner Brow Raiser 1 Byte Frontalis, pars medialis
2 Outer Brow Raiser 1 Byte Frontalis, pars lateralis
4 Brow Lowerer 1 Byte Corrugator supercilii, Depressor supercilii
5 Upper Lid Raiser 1 Byte Levator palpebrae superioris
6 Cheek Raiser 1 Byte Orbicularis oculi, pars orbitalis
7 Lid Tightener 1 Byte Orbicularis oculi, pars palpebralis
9 Nose Wrinkler 1 Byte Levator labii superioris alaquae nasi
10 Upper Lip Raiser 1 Byte Levator labii superioris
11 Nasolabial Deepener 1 Byte Zygomaticus minor
12 Lip Corner Puller 1 Byte Zygomaticus major
13 Cheek Puffer 1 Byte Levator anguli oris (a.k.a. Caninus)
14 Dimpler 1 Byte Buccinator
15 Lip Corner Depressor 1 Byte Depressor anguli oris (a.k.a. Triangularis)
16 Lower Lip Depressor 1 Byte Depressor labii inferioris
17 Chin Raiser 1 Byte Mentalis
18 Lip Puckerer 1 Byte Incisivii labii superioris and Incisivii labii inferioris
20 Lip stretcher 1 Byte Risorius with platysma
22 Lip Funneler 1 Byte Orbicularis oris
23 Lip Tightener 1 Byte Orbicularis oris
24 Lip Pressor 1 Byte Orbicularis oris
25 Lips part 1 Byte Depressor labii inferioris or relaxation of Mentalis, or Orbicularis oris
26 Jaw Drop 1 Byte Masseter, relaxed Temporalis and internal Pterygoid
27 Mouth Stretch 1 Byte Pterygoids, Digastric
28 Lip Suck 1 Byte Orbicularis oris
41 Lid droop 1 Byte Relaxation of Levator palpebrae superioris
42 Slit 1 Byte Orbicularis oculi
43 Eyes Closed 1 Byte Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis
44 Squint 1 Byte Orbicularis oculi, pars palpebralis
45 Blink 1 Byte Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis
46 Wink 1 Bytes Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis
61 Eyes turn left 2 Bytes Lateral rectus, medial rectus
62 Eyes turn right 2 Bytes Lateral rectus, medial rectus
63 Eyes up 2 Bytes Superior rectus, Inferior oblique
64 Eyes down 2 Bytes Inferior rectus, Superior oblique

5     Conformance Testing

A Data instance Conforms with Face Descriptors (PAF-FCD) V1.4 if t

  1. The Data validates against the Face Descriptors’ JSON Schema.
  2. All Data in the  Face Descriptors’ JSON Schema
    1. Have the specified type.
    2. Validate against their JSON Schemas.

6     Performance Assessment

7     Mapping of AUs to Personal Status (Informative)

MPAI has defined a set of Cognitive States, Emotions, and Social Attitudes included in Personal Status. The Table below offers an informative mapping of some elements of Personal Status to Action Units (from 1).

Personal Status Cognitive State Emotion Prototypical (and variant AUs)
Happy 17 12, 25 [6 (51%)]
Sad 32 4, 15 [1 (60%), 6 (50%), 11 (26%), 17 (67%)]
Fearful 13 1, 4, 20, 25 [2 (57%), 5 (63%), 26 (33%)]
Angry 2 4, 7, 24 [10 (26%), 17 (52%), 23 (29%)]
Surprised 18 1, 2, 25, 26 [5 (66%)]
Disgusted 11 9, 10, 17 [4 (31%), 24 (26%)]

This Table was obtained through a series of experiments with human subjects. AUs used by a subset of the subjects are shown in brackets with the percentage of the subjects using this less common AU in parentheses.

[1] Compound facial expressions of emotion | PNAS