| CAE | CAV | HMC | MMC | OSD | PAF |
| AI Workflows | AI Modules |
1 AI Workflows
1.1 Communicating Entities in Context
HMC-CEC is a PAAI that communicates with a human in a real environment and/or an HMC-CEC who may be rendered as a Persona in an M-Instance.
It composed of a set of the following collaborating PAAIs:
| Audio-Visual Scene Description | Describes the real scene as Audio-Visual Scene Descriptors |
| Audio-Visual Scene Integration and Description | Extracts the components of the Communication Item (Personal Avatar) produced by another HMC-CEC PAAI. |
| Entity and Context Understanding | Extracts the components of the Communication Item providing: – Entity ID – Text Object – Meaning of Text Object – Personal Status of Entity – AV Scene Geometry – Instance ID of Audio/Visual Object. |
| Entity Dialogue Processing | Reacts to the six inputs above by producing: – Its own ID – A Text Object – Meaning of its own Text Object – Its own Personal Status. |
| Text-to-Text-Translation | Translates its own Text |
| Personal Status Display | Produces a Communication Item containing its reaction. |
| Audio-Visual Scene Rendering | Renders its Communication Item for use in the real world. |
Figure 12 – Reference Model of HMC-CEC
The following links analyse the AI Modules:
AV Scene Integration and Description
Entity and Context Understanding
2 AI Modules
2.2 Entity and Context Understanding
HMC-ECU is a PAAI composed of the following collaborating PAAIs:
| Audio-Visual Scene Demultiplexing | Provides the Descriptors of the Scene. |
| Automatic Speech Recognition | Converts Speech Object into Text Object. |
| Visual Object Identification | Provides the ID of a Visual Object with reference to a Taxonomy. |
| Audio Object Identification | Provides the ID of a Visual Object with reference to a Taxonomy. |
| Natural Language Understanding | Provides Refined Text and Meaning. |
| Personal Status Extraction | Provides the Personal Status of the Entity. |
| Text-to-Text Translation | Provides a translation of the Text Object. |
Figure 13 – Reference Model of CUI-ECU
The implementation of HMC-ECU depends on the implementation of its components:
- Speech Recognition – Neural Network
- Visual and Audio Object ID Recognition – Neural Network
- Natural Language Understanding enhanced by Scene Geometry
- Personal Status Extraction – partly Neural Networks,
HMC-ECU performs Interpretation Level Operations.
