Go To MPAI-OSD AI Modules

1     Function 2     Reference Model 3     Input/Output Data
4     SubAIMs 5     JSON Metadata 6     Profiles
7     Reference Software 8     Conformance Texting 9     Performance Assessment

1     Functions

Television Splitting (OSD-TVS):

Receives Audio-Visual Object As a file.
Auxiliary Text A Text Object.
Demultiplexes Audio-Visual Object
Produces Text Object
Audio Object
Video Object

2     Reference Model

The Reference Model is specified in Figure 1.

Figure 1 – The Television Splitting (OSD-TVS) AIM Reference Model

3    Input/Output Data

Table 1 specifies the Input and Output Data of the Television Splitting (OSD-TVS) AIM. Links are to the Data Type specifications.

Table 1 – I/O Data of the Television Splitting (OSD-TVS) AIM

Input Description
Audio-Video Input audio-video file.
Auxiliary Text Text provided to facilitate operation.
Output Description
Text The Text component of the input Audio-Video-Text file.
Audio The Audio component of the input Audio-Video-Text file.
Video The Video component of the input Audio-Video-Text file.

4     SubAIMs

No SubAIMs

5     JSON Metadata

https://schemas.mpai.community/OSD/V121/AIMs/TelevisionSplitting.json

6     Profiles

No Profiles.

7     Reference Software

7.1    Disclaimers

  1. This OSD-TVS Reference Software Implementation is released with the BSD-3-Clause licence.
  2. The purpose of this Reference Software is to show a working Implementation of OSD-TVS, not to provide a ready-to-use product.
  3. MPAI disclaims the suitability of the Software for any other purposes and does not guarantee that it is secure.
  4. Users shall verify that they have the right to use any third-party software required by this Reference Software.

7.2    Guide to the code

Use of this AI Modules is for developers who are familiar with Python, Docker, RabbitMQ, and downloading models from HuggingFace.

OSD-TVS extracts a WAV (uncompressed audio) from an input video using FFMPEG. Therefore, any input Audio-Visual file that can be demultiplexed by FFMPEG can be used as input. The current OSD-TVS implementation does not support Auxiliary Text.

The OSD-TVS Reference Software is found at the MPAI gitlab site. It contains:

  1. src: a folder with the Python code implementing the AIM
  2. Dockerfile: a Docker file containing only the libraries required to build the Docker image and run the container
  3. requirements.txt: dependencies installed in the Docker image

Library: https://www.ffmpeg.org/

7.3    Acknowledgements

This version of the OSD-TVS Reference Software has been developed by the MPAI AI Framework Development Committee (AIF-DC).

8     Conformance Testing

Table 2 provides the Conformance Testing Method for OSD-TVS AIM.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and conform with the Qualifier, if present.

Table 2 – Conformance Testing Method for OSD-TVS AIM

Receives Audio-Video file Shall validate against Audio-Visual Object schema.
Audio-Visual Data shall conform with Qualifier.
Auxiliary Text Object Shall validate against Text Object schema.
Text Data shall conform with Qualifier.
Produces Text File Shall validate against Text Object schema.
Text Data shall conform with Qualifier.
Audio File Shall validate against Audio Object schema.
Audio Data shall conform with Qualifier.
Video File Shall validate against Visual Object schema.
Visual Data shall conform with Qualifier.

9     Performance Assessment

Go To MPAI-OSD AI Modules