1 Definition
The Face Descriptors are the Descriptors of the face of:
- A human having a multimodal conversation with the HCI when it estimates the human’s Personal Status.
- The Avatar embodying the HCI in the Portable Avatar generated by the Personal Status Display AIM that is rendered by the Audio-Visual Scene Rendering
- A human for the purpose of finding their Instance ID.
2 Functional Requirements
MPAI has specified the format of Face Descriptors based on with the Actions Units of the Facial Action Coding System (FACS) originally developed by Carl-Herman Hjortsjö, adopted by Paul Ekman and Wallace V. Friesen (1978) and updated by Ekman, Friesen, and Joseph C. Hager (2002).
3 Syntax
https://schemas.mpai.community/PAF/V1.1/PortableAvatarFormat.json
4 Semantics
Label | Size | Description |
Header | N1 Bytes | |
· Standard | 9 Bytes | The characters “PAF-FAD-V” |
· Version | N2 Bytes | Major version – 1 or 2 characters |
· Dot-separator | 1 Byte | The character “.” |
· Subversion | N3 Bytes | Minor version – 1 or 2 characters |
MInstanceID | N4 Bytes | Identifier of M-Instance. |
FaceDescriptorsID | N5 Bytes | Identifier of the AV Object. |
FaceDescriptors | N6 Bytes | |
Action Units | Description | Facial muscle generating the Action |
1 | Inner Brow Raiser | Frontalis, pars medialis |
2 | Outer Brow Raiser | Frontalis, pars lateralis |
4 | Brow Lowerer | Corrugator supercilii, Depressor supercilii |
5 | Upper Lid Raiser | Levator palpebrae superioris |
6 | Cheek Raiser | Orbicularis oculi, pars orbitalis |
7 | Lid Tightener | Orbicularis oculi, pars palpebralis |
9 | Nose Wrinkler | Levator labii superioris alaquae nasi |
10 | Upper Lip Raiser | Levator labii superioris |
11 | Nasolabial Deepener | Zygomaticus minor |
12 | Lip Corner Puller | Zygomaticus major |
13 | Cheek Puffer | Levator anguli oris (a.k.a. Caninus) |
14 | Dimpler | Buccinator |
15 | Lip Corner Depressor | Depressor anguli oris (a.k.a. Triangularis) |
16 | Lower Lip Depressor | Depressor labii inferioris |
17 | Chin Raiser | Mentalis |
18 | Lip Puckerer | Incisivii labii superioris and Incisivii labii inferioris |
20 | Lip stretcher | Risorius with platysma |
22 | Lip Funneler | Orbicularis oris |
23 | Lip Tightener | Orbicularis oris |
24 | Lip Pressor | Orbicularis oris |
25 | Lips part | Depressor labii inferioris or relaxation of Mentalis, or Orbicularis oris |
26 | Jaw Drop | Masseter, relaxed Temporalis and internal Pterygoid |
27 | Mouth Stretch | Pterygoids, Digastric |
28 | Lip Suck | Orbicularis oris |
41 | Lid droop | Relaxation of Levator palpebrae superioris |
42 | Slit | Orbicularis oculi |
43 | Eyes Closed | Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis |
44 | Squint | Orbicularis oculi, pars palpebralis |
45 | Blink | Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis |
46 | Wink | Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis |
61 | Eyes turn left | Lateral rectus, medial rectus |
62 | Eyes turn right | Lateral rectus, medial rectus |
63 | Eyes up | Superior rectus, Inferior oblique |
64 | Eyes down | Inferior rectus, Superior oblique |
5 Data Formats
Action Units are a Data Format.
6 To Respondents
Respondents are invited to:
- Comment or elaborate on the MPAI Face Descriptors.
- Propose extensions to the identified technologies or new ones.