<-AI Workflows      Go to ToC      Data Types ->

1. Technical Specification 2. Reference Software 3. Conformance Testing 4. Performance Assessment

1. Technical Specifications

Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Multimodal Conversation (MPAI-MMC) V2.3. All previously specified MPAI-MMC AI-Modules are superseded by those specified by V2.3 but may be used by explicitly signaling their version. Bold characters are used to indicate that an AIM is Composite.

Table 1 – Specifications and JSON syntax of AIMs used by MPAI-MMC V2.3

AIMs Name JSON AIMs Name JSON
MMC-AQM Answer to Question Module X MMC-QAM Question Analysis Module X
MMC-ASR Automatic Speech Recognition X MMC-SSD Speech Scene Description X
MMC-AUS Audio Segmentation X MMC-SBS Speech Basic Scene Description X
MMC-EDP Entity Dialogue Processing X MMC-SCM Summary Creation Module X
MMC-ESD Entity Speech Description X MMC-SIR Speaker Identity Recognition X
MMC-ETD Entity Text Description X MMC-SPE Speech Personal Status Extraction X
MMC-MEF Multimodal Emotion Fusion X MMC-STD Speech Translation with Descriptors X
MMC-NLU Natural Language Understanding X MMC-TSD Text-to-Speech with Descriptors X
MMC-PDX Personal Status Demultiplexing X MMC-TST Text and Speech Translation X
MMC-PMX Personal Status Multiplexing X MMC-TIQ Text and Image Query X
MMC-PSE Personal Status Extraction X MMC-TTS Text-To-Speech X
MMC-PSI PS-Speech Interpretation X MMC-TTT Text-to-Text Translation X
MMC-PTI PS-Text Interpretation X MMC-VLA Video Lip Animation X

2. Reference Software

As a rule, MPAI provides Reference Software implementing the AI Modules released with the BSD-3-Clause licence and the following disclaimers:

  1. The purpose of the Reference Software is to provide a working Implementation of an AIM, not a ready-to-use product.
  2. MPAI disclaims the suitability of the Reference Software for any other purposes than those of the MPAI-MMC Standard, and does not guarantee that it offers the best performance and that it is secure.
  3. Users shall verify that they have the right to use any third-party software required by the Reference Software, e.g., by accepting the  licences from third-party repositories.

Note that at this stage only part of the MPAI-MMC AIMs have a Reference Software Implementation.

3. Conformance Testing

An implementation of an AI Module conforms with MPAI-MMC if it accepts as input and produces as output Data and/or Data Objects (combination of Data of a certain Data Type and its Qualifier) conforming with those specified by MPAI-MMC.

The Conformance of an instance of a Data is to be expressed by a sentence like “Data validates against the Data Type Schema”. This means that:

  • Any Data Sub-Type is as indicated in the Qualifier.
  • The Data Format is indicated by the Qualifier.
  • Any File and/or Stream have the Formats indicated by the Qualifier.
  • Any Attribute of the Data is of the type or validates against the Schema specified in the Qualifier.

The method to Test the Conformance of a Data or Data Object instance is specified in the Data Types chapter.

4. Performance Assessment

Performance is a multidimensional entity because it can have various connotations. Therefore, the Performance Assessment Specification should provide methods to measure how well an AIM performs its function, using a metric that depends on the nature of the function, such as:

  1. Quality: Performance Assessment measures how well an AIM performs its function, using a metric that depends on the nature of the function, e.g., the word error rate (WER) of an Automatic Speech Recognition (ASR) AIM.
  2. Bias: Performance Assessment measures how well an AIM performs its function, using a metric that depends on a bias related to certain attributes of the AIM. For instance, an ASR AIM tends to have a higher WER when the speaker is from a particular geographic area.
  3. Legal compliance: Performance Assessment measures how well an AIM performs its function, using a metric that assesses its accordance with a certain legal standard
  4. Ethical compliance: the Performance Assessment of an AIM can measure the compliance of an AIM to a target ethical standard.

The current MPAI-MMC V2.3 Standard does not provide AIM Performance Assessment methods.

<-AI Workflows      Go to ToC      Data Types ->