<-AI Workflows      Go to ToC      Data Types ->

1. Technical Specification 2. Reference Software 3. Conformance Testing 4. Performance Assessment

1. Technical Specifications

Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Multimodal Conversation (MPAI-MMC) V2.3. All previously specified MPAI-MMC AI-Modules are superseded by those specified by V2.3 but may be used by explicitly signaling their version. Bold characters are used to indicate that an AIM is Composite.

Table 1 – Specifications and JSON syntax of AIMs used by MPAI-MMC V2.3

AIMs Name JSON AIMs Name JSON
MMC-AQM Answer to Question Module X MMC-QAM Question Analysis Module X
MMC-ASR Automatic Speech Recognition X MMC-SSD Speech Scene Description X
MMC-AUS Audio Segmentation X MMC-SBS Speech Basic Scene Description X
MMC-EDP Entity Dialogue Processing X MMC-SCM Summary Creation Module X
MMC-ESD Entity Speech Description X MMC-SIR Speaker Identity Recognition X
MMC-ETD Entity Text Description X MMC-SPE Speech Personal Status Extraction X
MMC-MEF Multimodal Emotion Fusion X MMC-STD Speech Translation with Descriptors X
MMC-NLU Natural Language Understanding X MMC-TSD Text-to-Speech with Descriptors X
MMC-PDX Personal Status Demultiplexing X MMC-TST Text and Speech Translation X
MMC-PMX Personal Status Multiplexing X MMC-TIQ Text and Image Query X
MMC-PSE Personal Status Extraction X MMC-TTS Text-To-Speech X
MMC-PSI PS-Speech Interpretation X MMC-TTT Text-to-Text Translation X
MMC-PTI PS-Text Interpretation X MMC-VLA Video Lip Animation X

2. Reference Software

As a rule, MPAI provides Reference Software implementing the AI Modules released with the BSD-3-Clause licence and the following disclaimers:

  1. The purpose of the Reference Software is to provide a working Implementation of an AIM, not a ready-to-use product.
  2. MPAI disclaims the suitability of the Reference Software for any other purposes than those of the MPAI-MMC Standard, and does not guarantee that it offers the best performance and that it is secure.
  3. Users shall verify that they have the right to use any third-party software required by the Reference Software, e.g., by accepting the  licences from third-party repositories.

Note that at this stage only part of the MPAI-MMC AIMs have a Reference Software Implementation.

3. Conformance Testing

An implementation of an AI Module conforms with MPAI-MMC if it accepts as input and produces as output Data and/or Data Objects (combination of Data of a certain Data Type and its Qualifier) conforming with those specified by MPAI-MMC.

The Conformance of an instance of a Data is to be expressed by a sentence like “Data validates against the Data Type Schema”. This means that:

  • Any Data Sub-Type is as indicated in the Qualifier.
  • The Data Format is indicated by the Qualifier.
  • Any File and/or Stream have the Formats indicated by the Qualifier.
  • Any Attribute of the Data is of the type or validates against the Schema specified in the Qualifier.

The method to Test the Conformance of a Data or Data Object instance is specified in the Data Types chapter.

4. Performance Assessment

Performance Assessment provides methods of assessing the performance of an AIM. Performance may have various connotations, such as:

  1. Quality: Performance Assessment measures how well an AIM performs its function, using a metric that depends on the nature of the function, e.g., the word error rate (WER) of an Automatic Speech Recognition (ASR) AIM.
  2. Bias: Performance Assessment measures how well an AIM performs its function, using a metric that depends on a bias related to certain attributes of the AIM. For instance, an ASR AIM tends to have a higher WER when the speaker is from a particular geographic area.
  3. Legal compliance: Performance Assessment measures how well an AIM performs its function, using a metric that assesses its accordance with a certain legal standard
  4. Ethical compliance: the Performance Assessment of an AIM can measure the compliance of an AIM to a target ethical standard.

The current MPAI-MMC V2.3 Standard does not provide AIM Performance Assessment methods.

<-AI Workflows      Go to ToC      Data Types ->