Go to MPAI-MMC V2.5 AI Modules
Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance
1 Functions
The Text and Speech Translation (MMC‑TST) AIM receives an input text or an input speech and language preferences informing about the language of the input text or speech and the target language of the output text or speech, and produces — independently of whether the input is text or speech — a text or speech in the language indicated in the language preferences. The different selections are signalled by the input selector:
| Receives | Selector | To choose between:
|
| Language Preferences | Requested input and output language. | |
| Personal Status | Use of Personal Status. | |
| Text | Input Text to be translated. | |
| Speech | Input Speech to be translated. | |
| Performs | A subset of the following: | |
| Conversion of input Speech | Into Text. | |
| Translation of Text | To the target language. | |
| Extraction of Features | From Speech. | |
| Conversion of Text | Into Speech adding the input Speech’s Features. | |
| Produces | Translated Text | Depends on Selector. |
| Translated Speech | Depends on Selector. |
2 Reference Model
Figure 1 depicts the Reference Model of the Text and Speech Translation (MMC‑TST) AIM.
Figure 1 – The Text and Speech Translation (MMC‑TST) AIM Reference Model
3 I/O Data
Table 1 specifies the Input and Output Data of the Text and Speech Translation (MMC‑TST) AIM.
| Input | Description |
|---|---|
| Selector | Signals whether the input is Text or Speech, whether the input Speech features are preserved in the output Speech, and the input and output languages. |
| Speech Object | Speech produced in input language by a human desiring translation into output language. |
| Text Object | Alternative textual source information to be translated into and pronounced in output language depending on the value of Input Selection. |
| Output | Description |
| Translated Speech Object | Speech in input language translated into output language preserving the input Speech features in the output Speech, depending on Selector. |
| Translated Text Object | Text of input Speech or input Text translated into output language, depending on Selector. |
4 SubAIMs
4.1 Reference Model
Figure 2 depicts the Reference Model of the Text and Speech Translation (MMC‑TST) Composite AIM.
Figure 2 – The Text and Speech Translation (MMC‑TST) Composite AIM
4.2 Operation
The MMC‑TST AIM receives input Speech and/or Text together with a Selector and Language Preferences. Automatic Speech Recognition converts input Speech to Text. Text-to-Text Translation translates the Text into the target language. Entity Speech Description extracts Speech Descriptors from the input Speech. Descriptors Text-to-Speech synthesises the translated Speech preserving the input Speech features. The Selector determines which outputs are produced.
4.3 Functions of SubAIMs
Table 2 gives the functions of the Text and Speech Translation (MMC‑TST) SubAIMs.
| SubAIM | Function |
|---|---|
| Automatic Speech Recognition | Converts input Speech into Text. |
| Text-to-Text Translation | Translates input Text into the target language. |
| Entity Speech Description | Extracts Speech Descriptors from input Speech. |
| Text-to-Speech with Descriptors | Synthesises translated Speech preserving the input Speech features. |
4.4 I/O Data of SubAIMs
Table 3 gives the Input and Output Data of the Text and Speech Translation (MMC‑TST) SubAIMs.
| SubAIM | Input | Output |
|---|---|---|
| Automatic Speech Recognition | Speech Object | Recognised Text |
| Text-to-Text Translation | Selector Text Object Recognised Text |
Translated Text |
| Entity Speech Description | Speech Object | Speech Descriptors |
| Text-to-Speech with Descriptors | Translated Text Speech Descriptors Selector |
Translated Speech |
4.5 AIMs and JSON Metadata
Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.
| AIM1 | AIM2 | Name | JSON |
|---|---|---|---|
| MMC‑TST | Text and Speech Translation | X | |
| MMC‑ASR | Automatic Speech Recognition | X | |
| MMC‑TTT | Text-to-Text Translation | X | |
| MMC‑ISD | Entity Speech Description | X | |
| MMC‑TSD | Text-to-Speech with Descriptors | X |
5 JSON Metadata
https://schemas.mpai.community/MMC/V2.5/AIMs/TextAndSpeechTranslation.json
6 Profiles
The Profiles of Text and Speech Translation are specified.
7 Reference Software
Under development.
8 Conformance Testing
Table 5 provides the Conformance Testing Method for the Text and Speech Translation (MMC‑TST) Composite AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.
| Receives | Selector | Shall validate against Selector schema. |
| Text Object | Shall validate against Text Object schema. Text Data shall conform with Text Qualifier. | |
| Speech Object | Shall validate against Speech Object schema. Speech Data shall conform with Speech Qualifier. | |
| Produces | Translated Text Object | Shall validate against Text Object schema. Text Data shall conform with Text Qualifier. |
| Translated Speech Object | Shall validate against Speech Object schema. Speech Data shall conform with Speech Qualifier. |
9 Performance Assessment
Not part of this specification.
