<- AI Workflows     Go to ToC      Data Types ->

1. Technical Specification 2. Reference Software 3. Conformance Testing 4. Performance Assessment

1. Technical Specifications

Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Human and Machine Communication (MPAI-HMC) V2.0. The MPAI-HMC V2.0 AI-Modules supersede those specified by earlier versions than V2.0. They may still be used if their version is explicitly signalled. The Composite AIMs are in bold.

Table 1 – Basic and Composite AI Modules

Acronym Specification JSON
HMC-ECU Entity and Context Understanding X
HMC-SID AV Scene Integration and Description X

Table 2 provides the full list of with web links to the AI Modules utilised by HMC-CEC organised according to the Technical Specifications specifying them.

Table 2 – AI Modules utilised by HMC-CEC organised by Technical Specifications

CAE MMC OSD PAF
Audio Analysis Transform Automatic Speech Recognition Audio-Visual Alignment Audio-Visual Scene Rendering
Audio Descriptors Multiplexing Entity Dialogue Processing Audio-Visual Event Description Face Identity Recognition
Audio Object Identification Entity Speech Description Audio-Visual Scene Demultiplexing Entity Body Description
Audio Scene Description Entity Text Description Audio-Visual Scene Description Entity Face Description
Audio Separation and Enhancement Natural Language Understanding Visual Direction Identification Portable Avatar Demultiplexing
Audio Source Localisation Personal Status Extraction Visual Instance Identification PS-Face Interpretation
Audio Synthesis Transform Personal Status Multiplexing Visual Object Extraction PS-Gesture Interpretation
HMC PS-Speech Interpretation Visual Object Identification Personal Status Display
AV Scene Integration and Description PS-Text Interpretation Visual Scene Description Portable Avatar Multiplexing
Entity and Context Understanding Speaker Identity Recognition
Text and Speech Translation
Text-To-Speech
Text-to-Text Translation

2. Reference Software

As a rule, MPAI provides Reference Software implementing the AI Modules released with the BSD-3-Clause licence and the following disclaimers:

  1. The purpose of the Reference Software is to provide a working Implementation of an AIM, not a ready-to-use product.
  2. MPAI disclaims the suitability of the Reference Software for any other purposes than those of the MPAI-HMC Standard, and does not guarantee that it offers the best performance and that it is secure.
  3. Users shall verify that they have the right to use any third-party software required by the Reference Software, e.g., by accepting the  licences from third-party repositories.

Note that at this stage only part of the MPAI-HMC AIMs have a Reference Software Implementation.

3. Conformance Testing

An implementation of an AI Module conforms with MPAI-HMC if it accepts as input and produces as output Data and/or Data Objects (combination of Data of a certain Data Type and its Qualifier) conforming with those specified by all relevant MPAI Technical Specifications.

The Conformance of an instance of a Data is to be expressed by a sentence like “Data validates against the Data Type Schema”. This means that:

  • Any Data has the specified type.
  • Any Data Sub-Type is as indicated in the Qualifier.
  • The Data Format is indicated by the Qualifier.
  • Any File and/or Stream have the Formats indicated by the Qualifier.
  • Any Attribute of the Data is of the type or validates against the Schema specified in the Qualifier.

The method to Test the Conformance of a Data or Data Object instance is specified in the Data Types chapter.

4. Performance Assessment

Performance is a multidimensional entity because it can have various connotations. Therefore, the Performance Assessment Specification should provide methods to measure how well an AIM performs its function, using a metric that depends on the nature of the function, such as:

  1. Quality: Performance Assessment measures how well an AIM performs its function, using a metric that depends on the nature of the function, e.g., the word error rate (WER) of an Automatic Speech Recognition (ASR) AIM.
  2. Bias: Performance Assessment measures how well an AIM performs its function, using a metric that depends on a bias related to certain attributes of the AIM. For instance, an ASR AIM tends to have a higher WER when the speaker is from a particular geographic area.
  3. Legal compliance: Performance Assessment measures how well an AIM performs its function, using a metric that assesses its accordance with a certain legal standard.
  4. Ethical compliance: the Performance Assessment of an AIM can measure the compliance of an AIM to a target ethical standard.

Note that the current MPAI-HMC V2.0 Technical Specification provides AIM Performance Assessment methods for a limited number of AIMs.

<- AI Workflows     Go to ToC      Data Types ->