Go To MPAI-OSD AI Modules

1     Function 2     Reference Model 3     Input/Output Data
4     SubAIMs 5     JSON Metadata 6     Profiles
7     Reference Software 8     Conformance Texting 9     Performance Assessment

1     Functions

Audio-Visual Scene Multiplexing (OSD-SMX):

Receives Space-Time
Speech Objects
Audio Objects
Visual Objects
Speech Scene Geometry
Audio Scene Geometry
Visual Scene Geometry
Produces Audio-Visual Scene Descriptors

2     Reference Model

Figure 1 specifies the Reference Model of the Audio-Visual Scene Multiplexing (OSD-SMX) AIM.

Figure 1 – The Audio-Visual Scene Multiplexing (OSD-SMX) AIM Reference Model

3    Input/Output Data

Table 13 specifies the Input and Output Data of the Audio-Visual Scene Multiplexing (OSD-SMX).

Table 13 – I/O Data of the Audio-Visual Scene Multiplexing (OSD-SMX) AIM

Input Description
Speech Object The Speech Objects of the Scene.
Audio Object The Audio Objects of the Scene.
Visual Object The Visual Objects of the Scene.
Speech Scene Geometry The Geometry of the Audio, Visual, Audio-Visual Objects of the Scene.
Audio Scene Geometry The Geometry of the Audio, Visual, Audio-Visual Objects of the Scene.
Visual Scene Geometry The Geometry of the Audio, Visual, Audio-Visual Objects of the Scene.
Output Description
Audio-Visual Scene Descriptors The combination of the Audio, Visual, and Audio-Visual Objects, and the Geometry of the Objects of the Scene.

4     SubAIMs

No SubAIMs.

5     JSON Metadata

https://schemas.mpai.community/OSD/V1.1/AIMs/AudioVisualSceneMultiplexing.json

6     Profiles

No Profiles.

7     Reference Software

8     Conformance Testing

9     Performance Assessment

Go To MPAI-OSD AI Modules