<-AI Workflows      Go to ToC      Data Types ->

1. Technical Specification 2. Conformance Testing 3. Performance Assessment

1. Technical Specifications

Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Multimodal Conversation (MPAI-MMC) V2.3. All previously specified MPAI-MMC AI-Modules are superseded by those specified by V2.3 but may be used by explicitly signaling their version. Bold characters are used to indicate that an AIM is Composite.

Table 1 – Specifications and JSON syntax of AIMs used by MPAI-MMC V2.3

AIMs Name JSON AIMs Name JSON
MMC-AQM Answer to Question Module X MMC-QAM Question Analysis Module X
MMC-ASR Automatic Speech Recognition X MMC-SCM Summary Creation Module X
MMC-AUS Audio Segmentation X MMC-SIR Speaker Identity Recognition X
MMC-EDP Entity Dialogue Processing X MMC-SPE Speech Personal Status Extraction X
MMC-ESD Entity Speech Description X MMC-STD Speech Translation with Descriptors X
MMC-ETD Entity Text Description X MMC-TSD Text-to-Speech with Descriptors X
MMC-MEF Multimodal Emotion Fusion X MMC-TST Text and Speech Translation X
MMC-NLU Natural Language Understanding X MMC-TIQ Text and Image Query X
MMC-PMX Personal Status Multiplexing X MMC-TTS Text-To-Speech X
MMC-PSE Personal Status Extraction X MMC-TTT Text-to-Text Translation X
MMC-PSI PS-Speech Interpretation X MMC-VLA Video Lip Animation X
MMC-PTI PS-Text Interpretation X

2. Conformance Testing

An implementation of an AI Module conforms with MPAI-MMC if it accepts as input and produces as output Data and/or Data Objects conforming with those specified by MPAI-MMC. Note that Data Object is defined as the combination of Data of a certain Data Type and its Qualifier.

The Conformance of an instance of a Data is to be expressed by a sentence like “Data validates against the Data Type Schema”. This means that:

  • Any Data Sub-Type is as indicated in the Qualifier.
  • The Data Format is indicated by the Qualifier.
  • Any File and/or Stream have the Formats indicated by the Qualifier.
  • Any Attribute of the Data is of the type or validates against the Schema specified in the Qualifier.

The method to Test the Conformance of a Data or Data Object instance is specified in the Data Types chapter.

3. Performance Assessment

Performance Assessment provides methods of assessing the performance of an AIM. Performance may have various connotations, such as:

  1. Quality: Performance Assessment measures how well an AIM performs its function, using a metric that depends on the nature of the function, e.g., the word error rate (WER) of an Automatic Speech Recognition (ASR) AIM.
  2. Bias: Performance Assessment measures how well an AIM performs its function, using a metric that depends on a bias related to certain attributes of the AIM. For instance, an ASR AIM tends to have a higher WER when the speaker is from a particular geographic area.
  3. Legal compliance: Performance Assessment measures how well an AIM performs its function, using a metric that assesses its accordance with a certain legal standard.

<-AI Workflows      Go to ToC      Data Types ->