Go to MPAI-MMC V2.5 AI Modules
Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance
1 Functions
The Multimodal Translation Service (MMC‑MTS) AIM translates what a human says or types into the language named by a Language Selector and presents the translation through a Portable Avatar that speaks it. The translated text is also provided. The service is not affective: no Personal Status is supplied, and the human is anonymous.
2 Reference Model
Figure 1 depicts the Reference Model of the Multimodal Translation Service (MMC‑MTS) AIM.

Figure 1 – Reference Model of the Multimodal Translation Service (MMC‑MTS) AIM
The operation of the Multimodal Translation Service (MMC‑MTS) AIM develops in the following way:
- A human provides:
- Speech, or text typed instead of speaking, to be translated.
- A Language Selector naming the target language.
- The User Agent provides the Avatar Model that represents the machine and the Machine ID.
- The machine provides the Translated Text and the Portable Avatar that speaks it.
3 I/O Data
Table 1 specifies the Input and Output Data of the Multimodal Translation Service (MMC‑MTS) AIM.
| Input | Description |
|---|---|
| Input Speech | Speech of the human, if spoken. Optional. |
| Input Text | Text typed by the human instead of speaking. Optional. |
| Language Selector | The language into which the input is translated. |
| Avatar Model | The 3D model of the avatar that represents the machine. |
| Machine ID | Identifier of the machine. |
| Output | Description |
| Translated Text | The translation as Text. |
| Portable Avatar | The avatar that speaks the translation. |
4 SubAIMs
4.1 Reference Model
Figure 2 depicts the Reference Model of the Multimodal Translation Service (MMC‑MTS) Composite AIM.

Figure 2 – Reference Model of the Multimodal Translation Service (MMC‑MTS) Composite AIM
4.2 Operation
The Multimodal Translation Service (MMC‑MTS) AIM receives the human’s speech, which Automatic Speech Recognition converts to Text, or text (Input Text). Text-to-Text Translation translates the Text into the language named by the Language Selector. Text-To-Speech turns the translated Text into Speech. Portable Avatar Multiplexing puts the translated Text, the Speech, the Avatar Model and the Machine ID into the Portable Avatar. The translated Text is also an output. Acquisition and delivery are the User Agent’s, not SubAIMs.
4.3 Functions of SubAIMs
Table 2 provides the functions of the Multimodal Translation Service (MMC‑MTS) SubAIMs.
| SubAIM | Function |
|---|---|
| Automatic Speech Recognition | Recognises as Text an input Speech. |
| Text-to-Text Translation | Receives a Text and a Language Selector and produces its translation into the selected language. |
| Text-To-Speech | Produces the Speech from the translated Text. |
| Portable Avatar Multiplexing | Puts the translated Text, the Speech, the Avatar Model and the Machine ID in the Portable Avatar. |
4.4 I/O Data of SubAIMs
Table 3 gives the Input and Output Data of the Multimodal Translation Service (MMC‑MTS) SubAIMs.
| SubAIM | Input | Output |
|---|---|---|
| Automatic Speech Recognition | Input Speech | Text |
| Text-to-Text Translation | Text (recognised or Input Text) Language Selector |
Translated Text |
| Text-To-Speech | Translated Text | Speech |
| Portable Avatar Multiplexing | Translated Text Speech Avatar Model Machine ID |
Portable Avatar |
4.5 AIMs and JSON Metadata
Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.
| AIM1 | AIM2 | Name | JSON |
|---|---|---|---|
| MMC‑MTS | Multimodal Translation Service | X | |
| MMC‑ASR | Automatic Speech Recognition | X | |
| MMC‑TTT | Text-to-Text Translation | X | |
| MMC‑TTS | Text-To-Speech | X | |
| PAF‑PMX | Portable Avatar Multiplexing | X |
5 JSON Metadata
https://schemas.mpai.community/MMC/V2.5/AIMs/MultimodalTranslationService.json
6 Profiles
No Profiles.
7 Reference Software
Under development.
8 Conformance Testing
Table 5 provides the Conformance Testing Method for the Multimodal Translation Service (MMC‑MTS) AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.
| Receives | Input Speech | Shall validate against Basic Speech Object schema. Speech Data shall conform with Speech Qualifier. |
| Target Language | Shall validate against Selector schema. | |
| Text to Translate | Shall validate against Basic Text Object schema. Text Data shall conform with Text Qualifier. | |
| Avatar Model | Shall validate against Basic 3D Model Object schema. 3D Model Data shall conform with 3D Model Qualifier. | |
| Machine ID | Shall be a string. | |
| Produces | Translated Text | Shall validate against Basic Text Object schema. Text Data shall conform with Text Qualifier. |
| Portable Avatar | Shall validate against Portable Avatar schema. |
9 Performance Assessment
Not part of this specification.