<-AI Workflows Go to ToC Data Types ->
1. Technical Specification | 2. Conformance Testing | 3. Performance Assessment |
1. Technical Specifications
Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Multimodal Conversation (MPAI-MMC) V2.3. All previously specified MPAI-MMC AI-Modules are superseded by those specified by V2.3 but may be used by explicitly signaling their version. Bold characters are used to indicate that an AIM is Composite.
Table 1 – Specifications and JSON syntax of AIMs used by MPAI-MMC V2.3
AIMs | Name | JSON | AIMs | Name | JSON |
MMC-AQM | Answer to Question Module | X | MMC-QAM | Question Analysis Module | X |
MMC-ASR | Automatic Speech Recognition | X | MMC-SCM | Summary Creation Module | X |
MMC-AUS | Audio Segmentation | X | MMC-SIR | Speaker Identity Recognition | X |
MMC-EDP | Entity Dialogue Processing | X | MMC-SPE | Speech Personal Status Extraction | X |
MMC-ESD | Entity Speech Description | X | MMC-STD | Speech Translation with Descriptors | X |
MMC-ETD | Entity Text Description | X | MMC-TSD | Text-to-Speech with Descriptors | X |
MMC-MEF | Multimodal Emotion Fusion | X | MMC-TST | Text and Speech Translation | X |
MMC-NLU | Natural Language Understanding | X | MMC-TIQ | Text and Image Query | X |
MMC-PMX | Personal Status Multiplexing | X | MMC-TTS | Text-To-Speech | X |
MMC-PSE | Personal Status Extraction | X | MMC-TTT | Text-to-Text Translation | X |
MMC-PSI | PS-Speech Interpretation | X | MMC-VLA | Video Lip Animation | X |
MMC-PTI | PS-Text Interpretation | X |
2. Conformance Testing
An implementation of an AI Module conforms with MPAI-MMC if it accepts as input and produces as output Data and/or Data Objects conforming with those specified by MPAI-MMC. Note that Data Object is defined as the combination of Data of a certain Data Type and its Qualifier.
The Conformance of an instance of a Data is to be expressed by a sentence like “Data validates against the Data Type Schema”. This means that:
- Any Data Sub-Type is as indicated in the Qualifier.
- The Data Format is indicated by the Qualifier.
- Any File and/or Stream have the Formats indicated by the Qualifier.
- Any Attribute of the Data is of the type or validates against the Schema specified in the Qualifier.
The method to Test the Conformance of a Data or Data Object instance is specified in the Data Types chapter.
3. Performance Assessment
Performance Assessment provides methods of assessing the performance of an AIM. Performance may have various connotations, such as:
- Quality: Performance Assessment measures how well an AIM performs its function, using a metric that depends on the nature of the function, e.g., the word error rate (WER) of an Automatic Speech Recognition (ASR) AIM.
- Bias: Performance Assessment measures how well an AIM performs its function, using a metric that depends on a bias related to certain attributes of the AIM. For instance, an ASR AIM tends to have a higher WER when the speaker is from a particular geographic area.
- Legal compliance: Performance Assessment measures how well an AIM performs its function, using a metric that assesses its accordance with a certain legal standard.