<- AI Workflows     Go to ToC      Data Types ->

1. Technical Specification 2. Reference Software 3. Conformance Testing 4. Performance Assessment

1. Technical Specifications

Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Human and Machine Communication (MPAI-HMC) V2.0. All previously specified MPAI-HMC AI-Modules are superseded by those specified by V2.0 but may be used by explicitly signalling their version. Bold characters are used to indicate that an AIM is Composite.

Table 1 – Specifications and JSON syntax of AIMs used by MPAI-HMC V2.0

HMC-CEC specifies one Composite and one Basic AI Module. The Composite AIM is in bold.

Table 1 – Basic and Composite AI Modules

Acronym Specification JSON
HMC-ECU Entity and Context Understanding X
HMC-SID AV Scene Integration and Description X

Table 1 provides the full list of with web links to the AI Modules utilised by HMC-CEC organised according to the Technical Specifications specifying them.

Table 1 – AI Modules utilised by HMC-CEC organised by Technical Specifications

CAE

 MMC OSD PAF
Audio Analysis Transform X Automatic Speech Recognition X Audio-Visual Alignment X Audio-Visual Scene Rendering X
Audio Descriptors Multiplexing X Entity Dialogue Processing X Audio-Visual Event Description X Face Identity Recognition X
Audio Object Identification X Entity Speech Description X Audio-Visual Scene Demultiplexing X Entity Body Description X
Audio Scene Description X Entity Text Description X Audio-Visual Scene Description X Entity Face Description X
Audio Separation and Enhancement X Natural Language Understanding X Visual Direction Identification X Portable Avatar Demultiplexing X
Audio Source Localisation X Personal Status Extraction X Visual Instance Identification X PS-Face Interpretation X
Audio Synthesis Transform X Personal Status Multiplexing X Visual Object Extraction X PS-Gesture Interpretation X

HMC

 PS-Speech Interpretation X Visual Object Identification X Personal Status Display X
AV Scene Integration and Description X PS-Text Interpretation X Visual Scene Description X Portable Avatar Multiplexing X
Entity and Context Understanding X Speaker Identity Recognition X
Text and Speech Translation X
Text-To-Speech X
Text-to-Text Translation X

2. Reference Software

As a rule, MPAI provides Reference Software implementing the AI Modules released with the BSD-3-Clause licence and the following disclaimers:

  1. The purpose of the Reference Software is to provide a working Implementation of an AIM, not a ready-to-use product.
  2. MPAI disclaims the suitability of the Reference Software for any other purposes than those of the MPAI-HMC Standard, and does not guarantee that it offers the best performance and that it is secure.
  3. Users shall verify that they have the right to use any third-party software required by the Reference Software, e.g., by accepting the  licences from third-party repositories.

Note that at this stage only part of the MPAI-HMC AIMs have a Reference Software Implementation.

3. Conformance Testing

An implementation of an AI Module conforms with MPAI-HMC if it accepts as input and produces as output Data and/or Data Objects (combination of Data of a certain Data Type and its Qualifier) conforming with those specified by all relevant MPAI Technical Specifications.

The Conformance of an instance of a Data is to be expressed by a sentence like “Data validates against the Data Type Schema”. This means that:

  • Any Data has the specified type.
  • Any Data Sub-Type is as indicated in the Qualifier.
  • The Data Format is indicated by the Qualifier.
  • Any File and/or Stream have the Formats indicated by the Qualifier.
  • Any Attribute of the Data is of the type or validates against the Schema specified in the Qualifier.

The method to Test the Conformance of a Data or Data Object instance is specified in the Data Types chapter.

4. Performance Assessment

Performance Assessment provides methods of assessing the performance of an AIM. Performance may have various connotations, such as:

  1. Quality: Performance Assessment measures how well an AIM performs its function, using a metric that depends on the nature of the function, e.g., the word error rate (WER) of an Automatic Speech Recognition (ASR) AIM.
  2. Bias: Performance Assessment measures how well an AIM performs its function, using a metric that depends on a bias related to certain attributes of the AIM. For instance, an ASR AIM tends to have a higher WER when the speaker is from a particular geographic area.
  3. Legal compliance: Performance Assessment measures how well an AIM performs its function, using a metric that assesses its accordance with a certain legal standard.
  4. Ethical compliance: the Performance Assessment of an AIM can measure the compliance of an AIM to a target ethical standard.

The current MPAI-HMC V2.0 Standard does not provide AIM Performance Assessment methods.

<- AI Workflows     Go to ToC      Data Types ->