Go to MPAI-MMC V2.5 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

The Multimodal Translation Service (MMC‑MTS) AIM translates what a human says or types into the language named by a Language Selector and presents the translation through a Portable Avatar that speaks it. The translated text is also provided. The service is not affective: no Personal Status is supplied, and the human is anonymous.

2 Reference Model

Figure 1 depicts the Reference Model of the Multimodal Translation Service (MMC‑MTS) AIM.
 
Multimodal Translation Service MMC-MTS AIM

Figure 1 – Reference Model of the Multimodal Translation Service (MMC‑MTS) AIM

The operation of the Multimodal Translation Service (MMC‑MTS) AIM develops in the following way:

  1. A human provides:
    1. Speech, or text typed instead of speaking, to be translated.
    2. A Language Selector naming the target language.
  2. The User Agent provides the Avatar Model that represents the machine and the Machine ID.
  3. The machine provides the Translated Text and the Portable Avatar that speaks it.

3 I/O Data

Table 1 specifies the Input and Output Data of the Multimodal Translation Service (MMC‑MTS) AIM.

Table 1 – I/O Data of the Multimodal Translation Service (MMC‑MTS) AIM

Input Description
Input Speech Speech of the human, if spoken. Optional.
Input Text Text typed by the human instead of speaking. Optional.
Language Selector The language into which the input is translated.
Avatar Model The 3D model of the avatar that represents the machine.
Machine ID Identifier of the machine.
Output Description
Translated Text The translation as Text.
Portable Avatar The avatar that speaks the translation.

4 SubAIMs

4.1 Reference Model

Figure 2 depicts the Reference Model of the Multimodal Translation Service (MMC‑MTS) Composite AIM.
 
Multimodal Translation Service MMC-MTS Composite AIM

Figure 2 – Reference Model of the Multimodal Translation Service (MMC‑MTS) Composite AIM

4.2 Operation

The Multimodal Translation Service (MMC‑MTS) AIM receives the human’s speech, which Automatic Speech Recognition converts to Text, or text (Input Text). Text-to-Text Translation translates the Text into the language named by the Language Selector. Text-To-Speech turns the translated Text into Speech. Portable Avatar Multiplexing puts the translated Text, the Speech, the Avatar Model and the Machine ID into the Portable Avatar. The translated Text is also an output. Acquisition and delivery are the User Agent’s, not SubAIMs.

4.3 Functions of SubAIMs

Table 2 provides the functions of the Multimodal Translation Service (MMC‑MTS) SubAIMs.

Table 2 – Functions of the Multimodal Translation Service (MMC‑MTS) SubAIMs

SubAIM Function
Automatic Speech Recognition Recognises as Text an input Speech.
Text-to-Text Translation Receives a Text and a Language Selector and produces its translation into the selected language.
Text-To-Speech Produces the Speech from the translated Text.
Portable Avatar Multiplexing Puts the translated Text, the Speech, the Avatar Model and the Machine ID in the Portable Avatar.

4.4 I/O Data of SubAIMs

Table 3 gives the Input and Output Data of the Multimodal Translation Service (MMC‑MTS) SubAIMs.

Table 3 – I/O Data of the Multimodal Translation Service (MMC‑MTS) SubAIMs

SubAIM Input Output
Automatic Speech Recognition Input Speech Text
Text-to-Text Translation Text (recognised or Input Text)
Language Selector
Translated Text
Text-To-Speech Translated Text Speech
Portable Avatar Multiplexing Translated Text
Speech
Avatar Model
Machine ID
Portable Avatar

4.5 AIMs and JSON Metadata

Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.

Table 4 – AIMs and JSON Metadata of the Multimodal Translation Service (MMC‑MTS)

AIM1 AIM2 Name JSON
MMC‑MTS Multimodal Translation Service X
MMC‑ASR Automatic Speech Recognition X
MMC‑TTT Text-to-Text Translation X
MMC‑TTS Text-To-Speech X
PAF‑PMX Portable Avatar Multiplexing X

5 JSON Metadata

https://schemas.mpai.community/MMC/V2.5/AIMs/MultimodalTranslationService.json

6 Profiles

No Profiles.

7 Reference Software

Under development.

8 Conformance Testing

Table 5 provides the Conformance Testing Method for the Multimodal Translation Service (MMC‑MTS) AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 5 – Conformance Testing Method for the Multimodal Translation Service (MMC‑MTS) AIM

Receives Input Speech Shall validate against Basic Speech Object schema. Speech Data shall conform with Speech Qualifier.
Target Language Shall validate against Selector schema.
Text to Translate Shall validate against Basic Text Object schema. Text Data shall conform with Text Qualifier.
Avatar Model Shall validate against Basic 3D Model Object schema. 3D Model Data shall conform with 3D Model Qualifier.
Machine ID Shall be a string.
Produces Translated Text Shall validate against Basic Text Object schema. Text Data shall conform with Text Qualifier.
Portable Avatar Shall validate against Portable Avatar schema.

9 Performance Assessment

Not part of this specification.

Go to MPAI-MMC V2.5 AI Modules