1   Definition

The Face Descriptors are the Descriptors of the face of:

  1. A human having a multimodal conversation with the HCI when it estimates the human’s Personal Status.
  2. The Avatar embodying the HCI in the Portable Avatar generated by the Personal Status Display AIM that is rendered by the Audio-Visual Scene Rendering
  3. A human for the purpose of finding their Instance ID.

2   Functional Requirements

MPAI has specified the format of Face Descriptors based on with the Actions Units of the Facial Action Coding System (FACS) originally developed by Carl-Herman Hjortsjö, adopted by Paul Ekman and Wallace V. Friesen (1978) and updated by Ekman, Friesen, and Joseph C. Hager (2002).

3   Syntax

https://schemas.mpai.community/PAF/V1.1/PortableAvatarFormat.json

4   Semantics

Label Size Description
Header N1 Bytes
·         Standard 9 Bytes The characters “PAF-FAD-V”
·         Version N2 Bytes Major version – 1 or 2 characters
·         Dot-separator 1 Byte The character “.”
·         Subversion N3 Bytes Minor version – 1 or 2 characters
MInstanceID N4 Bytes Identifier of M-Instance.
FaceDescriptorsID N5 Bytes Identifier of the AV Object.
FaceDescriptors N6 Bytes
Action Units Description Facial muscle generating the Action
1 Inner Brow Raiser Frontalis, pars medialis
2 Outer Brow Raiser Frontalis, pars lateralis
4 Brow Lowerer Corrugator supercilii, Depressor supercilii
5 Upper Lid Raiser Levator palpebrae superioris
6 Cheek Raiser Orbicularis oculi, pars orbitalis
7 Lid Tightener Orbicularis oculi, pars palpebralis
9 Nose Wrinkler Levator labii superioris alaquae nasi
10 Upper Lip Raiser Levator labii superioris
11 Nasolabial Deepener Zygomaticus minor
12 Lip Corner Puller Zygomaticus major
13 Cheek Puffer Levator anguli oris (a.k.a. Caninus)
14 Dimpler Buccinator
15 Lip Corner Depressor Depressor anguli oris (a.k.a. Triangularis)
16 Lower Lip Depressor Depressor labii inferioris
17 Chin Raiser Mentalis
18 Lip Puckerer Incisivii labii superioris and Incisivii labii inferioris
20 Lip stretcher Risorius with platysma
22 Lip Funneler Orbicularis oris
23 Lip Tightener Orbicularis oris
24 Lip Pressor Orbicularis oris
25 Lips part Depressor labii inferioris or relaxation of Mentalis, or Orbicularis oris
26 Jaw Drop Masseter, relaxed Temporalis and internal Pterygoid
27 Mouth Stretch Pterygoids, Digastric
28 Lip Suck Orbicularis oris
41 Lid droop Relaxation of Levator palpebrae superioris
42 Slit Orbicularis oculi
43 Eyes Closed Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis
44 Squint Orbicularis oculi, pars palpebralis
45 Blink Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis
46 Wink Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis
61 Eyes turn left Lateral rectus, medial rectus
62 Eyes turn right Lateral rectus, medial rectus
63 Eyes up Superior rectus, Inferior oblique
64 Eyes down Inferior rectus, Superior oblique

5   Data Formats

Action Units are a Data Format.

6   To Respondents

Respondents are invited to:

  1. Comment or elaborate on the MPAI Face Descriptors.
  2. Propose extensions to the identified technologies or new ones.