Go to MPAI-MMC V2.5 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

The Text-To-Speech (MMC‑TTS) AIM receives an input text and produces a synthetic speech version of it. The MMC‑TTS AIM may also receive the Personal Status to be used in the synthetic speech and a Speech Model:

Receives Text Object Input Text.
Personal Status To be contained in the Synthesised Speech Object.
Speech Model Used by AIM depending on Profile.
Feeds Text Object and Personal Status To Speech Model.
Produces Synthesised Speech Object Output of AIM.

2 Reference Model

Figure 1 depicts the Reference Model of the Text-To-Speech (MMC‑TTS) AIM.

Text-To-Speech MMC-TTS AIM

Figure 1 – The Text-To-Speech (MMC‑TTS) AIM

The voices of the Text-To-Speech (MMC‑TTS) AIM are data of Access. The Metadata of the AIM declares, for each language it speaks, the voice and its configuration in AccessSources (voice-en and voice-config-en, and so on) by the URI they are published at and their SHA-256; the configuration is valid only with its voice (ValidWith). The voice of one language is required and the others are optional: the AIM speaks the languages whose voices it has. The program that speaks, for instance Piper, is declared in Access too (tts-program): the archive it is published in is identified by its URI and SHA-256, and the Controller unpacks it. It reads the files where Access keeps the voices.

3 I/O Data

Table 1 specifies the Input and Output Data of the Text-To-Speech (MMC‑TTS) AIM.

Table 1 – I/O Data of the Text-To-Speech (MMC‑TTS) AIM

Input Description
Text Object Input Text.
Personal Status Input Personal Status of the Speech Modality.
Speech Model NN Model used to produce Speech from Text and Personal Status.
Output Description
Speech Object Synthesised Speech output of the Text-To-Speech AIM.

4 SubAIMs

No SubAIMs.

5 JSON Metadata

https://schemas.mpai.community/MMC/V2.5/AIMs/TextToSpeech.json

6 Profiles

The Text-To-Speech Profiles are specified.

7 Reference Software

7.1 Disclaimers

  1. This MMC‑TTS Reference Software Implementation is released with the BSD-3-Clause licence.
  2. The purpose of this MMC‑TTS Reference Software is to provide a working Implementation of MMC‑TTS, not to provide a ready-to-use product.
  3. MPAI disclaims the suitability of the Software for any other purposes and does not guarantee that it is secure.
  4. Use of this Reference Software may require acceptance of licences from the respective repositories. Users shall verify that they have the right to use any third-party software required by this Reference Software.

7.2 Guide to the MMC‑TTS code

Use of this AI Module is for developers who are familiar with Python and downloading models from HuggingFace.

A wrapper for the SpeechT5 NN Module:

  1. Manages input files and parameters: Text Object.
  2. Executes the SpeechT5 Module to perform Text-To-Speech synthesis on each input Text Object.
  3. Outputs Speech Object as answer.

The MMC‑TTS Reference Software is found at the MPAI gitlab site. It contains:

  1. The Python code implementing the AIM.
  2. Required libraries are: pytorch, transformers (HuggingFace), datasets (HuggingFace), and soundfile.

7.3 Acknowledgements

This version of the MMC‑TTS Reference Software has been developed by the MPAI Neural Network Watermarking Development Committee (NNW‑DC).

8 Conformance Testing

Table 2 provides the Conformance Testing Method for the Text-To-Speech (MMC‑TTS) AIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 2 – Conformance Testing Method for the Text-To-Speech (MMC‑TTS) AIM

Receives Text Object Shall validate against Text Object schema. Text Data shall conform with Text Qualifier.
Personal Status Shall validate against Personal Status schema.
Speech Model Shall validate against Machine Learning Model schema. Machine Learning Model Data shall conform with Machine Learning Model Qualifier.
Produces Synthesised Speech Object Shall validate against Speech Object schema. Speech Data shall conform with Speech Qualifier.

9 Performance Assessment

Not part of this specification.

Go to MPAI-MMC V2.5 AI Modules