HMC-CEC uses six groups of technologies to process a Communication Item:
Receives | Receives a Communication Item directly or creates it from an Audio-Visual Scene. |
Extracts | Personal Status from the Modalities (Text, Speech, Face, or Gesture) in the Communication Item received. |
Understands | The Communication Item from the Modalities and the extracted Personal Status with or without use of the spatial information embedded in the Communication Item. |
Translates | Using the set of Modalities available to the Machine. |
Generates | Response. |
Displays | the response using available Modalities. |
The Attributes and Sub-Attributes of the HMC-CEC Profiles are defined in Table 1 where the Sub-Attributes are expressed with 3 characters.
The first two characters of the Sub-Attributes (followed by O representing Object) of:
- The Audio-Visual Scene represent Text, Speech, Audio, and Visual Sub-Attributes, respectively.
- The Personal Status, Understanding, Translation, and Display Response represent Text, Speech, Face, and Gesture, respectively.
The SPC Sub-Attribute of Understanding represents Spatial Information (SPaCe), i.e., the capability of an HMC-CEC implementation to use also Spatial Information to understand a Communication Item.
Table 1 – Attribute and Sub-Attribute Codes of HMC-CEC.
Attributes | Codes | Sub-Attribute Codes | ||||
Audio-Visual Scene | AVS | TXO | SPO | AUO | VIO | |
Personal Status | EPS | TXO | SPO | FCO | GSO | |
Understanding | UND | TXO | SPO | FCO | GSO | SPC |
Translation | TRN | TXO | SPO | FCO | GSO | |
Display Response | RES | TXO | SPO | FCO | GSO |
The following JSON syntax specifies the MPAI-HMC Profiles.
https://schemas.mpai.community/HMC/V1.1/data/CECProfiles.json
“Header”: “HMC-PRF-V1.1”,
“Profile”: “NUL+(AVS|UND|EPS|DIS)|@AVS#(TXO-AUO-VIO)|@UND#(TXO|SPO)@EPS#(TXO|PSO)@DIS#(TXO|PSO|FCO)+|@TXO(#(ENG))+)+$”
}