Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment
1 Definition
Face Descriptors Object is a Data Type comprising:
- Face Descriptors Data representing the features of the Face of an Entity.
- Face Descriptors Data Qualifier specified by MPAI-TFA providing information on Sub-Types, Formats, and Attributes of Face Descriptors Data.
2 Functional Requirements
The features depend on the motion of the muscles of the Face of an Entity.
3 Syntax
https://schemas.mpai.community/PAF/V1.6/data/FaceDescriptorsObject.json
4 Semantics
| Header | Face Descriptors Object Header – Standard “PAF‑FDO‑Vx.y”. |
| MInstanceID | Identifier of the M-Instance (Virtual Space) where this Face is or is intended to be located. |
| UEnvironmentID | Identifier of the U-Environment (Real Space) where this Face is or is intended to be located. |
| FaceDescriptorsObjectID | Unique identifier of this Face Descriptors Object instance. |
| FaceDescriptorsObjectTime | Time this Face Descriptors Object was produced. |
| FaceDescriptorsObjectSpaceTime | Space/Time where/when this Face is located. |
| FaceDescriptorsData[] | Face Descriptors Data representing facial features of an Entity. Each item provides the data as inline content, a URI reference, or an external ID. |
| FaceDescriptorsQualifier | Qualifier providing information on Sub-Types, Formats, and Attributes of Face Descriptors Data. |
| DataXMData | MPAI-PTF Data Exchange Metadata providing provenance, authorisation, legal, security, and confidence information associated with this Face Descriptors Object in a trusted data exchange context. |
| DescrMetadata | Human-readable descriptive metadata of the Face Descriptors Object (plain text, max 2048 characters). |
5 Conformance Testing
A Data instance conforms with Face Descriptors Object (PAF‑FDO) if:
- The Data validates against the Face Descriptors Object’s JSON Schema.
- All Data in the Face Descriptors Object’s JSON Schema:
- Have the specified type.
- Validate against their JSON Schemas.
- Conform with their Data Qualifiers.
6 Performance Assessment
Not part of this specification.
7 Face Action Units (Informative)
The Face Action Units of the Facial Action Coding System (FACS) were originally developed by Carl-Herman Hjortsjö, adopted by Paul Ekman and Wallace V. Friesen (1978) and updated by Ekman, Friesen, and Joseph C. Hager (2002). Each Action Unit is represented by an Action Unit ID. They provide a set of Face Descriptors.
| AU | Description | Facial muscle generating the Action |
|---|---|---|
| 1 | Inner Brow Raiser | Frontalis, pars medialis |
| 2 | Outer Brow Raiser | Frontalis, pars lateralis |
| 4 | Brow Lowerer | Corrugator supercilii, Depressor supercilii |
| 5 | Upper Lid Raiser | Levator palpebrae superioris |
| 6 | Cheek Raiser | Orbicularis oculi, pars orbitalis |
| 7 | Lid Tightener | Orbicularis oculi, pars palpebralis |
| 9 | Nose Wrinkler | Levator labii superioris alaquae nasi |
| 10 | Upper Lip Raiser | Levator labii superioris |
| 11 | Nasolabial Deepener | Zygomaticus minor |
| 12 | Lip Corner Puller | Zygomaticus major |
| 13 | Cheek Puffer | Levator anguli oris (a.k.a. Caninus) |
| 14 | Dimpler | Buccinator |
| 15 | Lip Corner Depressor | Depressor anguli oris (a.k.a. Triangularis) |
| 16 | Lower Lip Depressor | Depressor labii inferioris |
| 17 | Chin Raiser | Mentalis |
| 18 | Lip Puckerer | Incisivii labii superioris and Incisivii labii inferioris |
| 20 | Lip Stretcher | Risorius with platysma |
| 22 | Lip Funneler | Orbicularis oris |
| 23 | Lip Tightener | Orbicularis oris |
| 24 | Lip Pressor | Orbicularis oris |
| 25 | Lips Part | Depressor labii inferioris or relaxation of Mentalis, or Orbicularis oris |
| 26 | Jaw Drop | Masseter, relaxed Temporalis and internal Pterygoid |
| 27 | Mouth Stretch | Pterygoids, Digastric |
| 28 | Lip Suck | Orbicularis oris |
| 41 | Lid Droop | Relaxation of Levator palpebrae superioris |
| 42 | Slit | Orbicularis oculi |
| 43 | Eyes Closed | Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis |
| 44 | Squint | Orbicularis oculi, pars palpebralis |
| 45 | Blink | Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis |
| 46 | Wink | Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis |
| 61 | Eyes Turn Left | Lateral rectus, medial rectus |
| 62 | Eyes Turn Right | Lateral rectus, medial rectus |
| 63 | Eyes Up | Superior rectus, Inferior oblique |
| 64 | Eyes Down | Inferior rectus, Superior oblique |
The eye motion Action Units (Eyes turn left/right and Eyes up/down) have an additional parameter representing the degrees of the motion of a line perpendicular to the eye with respect to the 0° of the resting position when the eye looks horizontally and perpendicular to the body. The value of the angle is expressed with 1 Byte where 0 = 0° for the resting position and 255 = 90°.
8 Mapping Face Action Units to Personal Status (Informative)
MPAI has defined a set of Cognitive States, Emotions, and Social Attitudes included in Personal Status. The table below offers an informative mapping of some elements of Personal Status to Action Units (from 1).
| Personal Status | Cognitive State | Emotion | Prototypical (and variant AUs) |
|---|---|---|---|
| Happy | 17 | 12, 25 [6 (51%)] | |
| Sad | 32 | 4, 15 [1 (60%), 6 (50%), 11 (26%), 17 (67%)] | |
| Fearful | 13 | 1, 4, 20, 25 [2 (57%), 5 (63%), 26 (33%)] | |
| Angry | 2 | 4, 7, 24 [10 (26%), 17 (52%), 23 (29%)] | |
| Surprised | 18 | 1, 2, 25, 26 [5 (66%)] | |
| Disgusted | 11 | 9, 10, 17 [4 (31%), 24 (26%)] |
This Table was obtained through a series of experiments with human subjects. AUs used by a subset of the subjects are shown in brackets with the percentage of the subjects using this less common AU in parentheses.
[1] Compound facial expressions of emotion | PNAS
