| 1 Function | 2 Reference Model | 3 Input/Output Data |
| 4 SubAIMs | 5 JSON Metadata | 6 Profiles |
| 7 Reference Software | 8 Conformance Texting | 9 Performance Assessment |
1 Functions
The Multimodal Scene Multiplexing (OSD-SMX) produces Multimodal Scene Descriptors by combining the component elements – Audio Scene Geometry, Speech Scene Geometry, Visual Scene Geometry, Audio Objects, Speech Objects, and Visual Objects:
| Receives | Basic Speech Scene Descriptors (OSD-BSS) or Speech Scene Descriptors (OSD-SSD) | Optional. |
| Basic Audio Scene Descriptors (OSD-BAS) or Audio Scene Descriptors (OSD-ASD) | Optional. | |
| Basic LiDAR Scene Descriptors (OSD-BLS) or LiDAR Scene Descriptors (OSD-LSD) | Optional. | |
| Basic Visual Scene Descriptors (OSD-BVS) or Visual Scene Descriptors (OSD-VSD) | Optional. | |
| Basic Text Object (OSD-BTO) or Text Object (OSD-TXO) | Optional. | |
| Instance Identifier (OSD-IID) | Optional. | |
| Produces | Basic Multimodal Scene Descriptors (OSD-BMS) or Multimodal Scene Descriptors (OSD-MSD) | The Descriptors of the Multimodal Scene, composed of the Scene Descriptors of the modalities received, with the Text and the identifiers. |
2 Reference Model
Figure 1 specifies the Reference Model of the Multimodal Scene Multiplexing (OSD-SMX) AIM.
Figure 1 – The Multimodal Scene Multiplexing (OSD-SMX) AIM Reference Model
3 Input/Output Data
Table 13 specifies the Input and Output Data of the Multimodal Scene Multiplexing (OSD-SMX).
Table 13 – I/O Data of the Multimodal Scene Multiplexing (OSD-SMX) AIM
| Input | Description |
| Basic Speech Scene Descriptors (OSD-BSS) or Speech Scene Descriptors (OSD-SSD) | The Speech Scene Descriptors. Optional. |
| Basic Audio Scene Descriptors (OSD-BAS) or Audio Scene Descriptors (OSD-ASD) | The Audio Scene Descriptors. Optional. |
| Basic LiDAR Scene Descriptors (OSD-BLS) or LiDAR Scene Descriptors (OSD-LSD) | The LiDAR Scene Descriptors. Optional. |
| Basic Visual Scene Descriptors (OSD-BVS) or Visual Scene Descriptors (OSD-VSD) | The Visual Scene Descriptors. Optional. |
| Basic Text Object (OSD-BTO) or Text Object (OSD-TXO) | The Text of the Scene. Optional. |
| Instance Identifier (OSD-IID) | The identifiers of the Objects of the Scene. Optional. |
| Basic Multimodal Scene Descriptors or Multimodal Scene Descriptors | The Multimodal Scene Descriptors of a Scene to augment. Optional. |
| Output | Description |
| Basic Multimodal Scene Descriptors (OSD-BMS) or Multimodal Scene Descriptors (OSD-MSD) | The Multimodal Scene Descriptors: basic if the inputs are basic. |
4 SubAIMs
No SubAIMs.
5 JSON Metadata
https://schemas.mpai.community/OSD/V1.5/AIMs/MultimodalSceneMultiplexing.json
6 Profiles
No Profiles.
7 Reference Software
8 Conformance Testing
Table 2 provides the Conformance Testing Method for OSD-SMX AIM.
If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and conform with the Qualifier, if present.
Table 2 – Conformance Testing Method for OSD-SMX AIM
| Receives | Basic Speech Scene Descriptors (OSD-BSS) or Speech Scene Descriptors (OSD-SSD) | Shall validate against the schema of the Data Type. |
| Basic Audio Scene Descriptors (OSD-BAS) or Audio Scene Descriptors (OSD-ASD) | Shall validate against the schema of the Data Type. | |
| Basic LiDAR Scene Descriptors (OSD-BLS) or LiDAR Scene Descriptors (OSD-LSD) | Shall validate against the schema of the Data Type. | |
| Basic Visual Scene Descriptors (OSD-BVS) or Visual Scene Descriptors (OSD-VSD) | Shall validate against the schema of the Data Type. | |
| Basic Text Object (OSD-BTO) or Text Object (OSD-TXO) | Shall validate against the schema of the Data Type. | |
| Instance Identifier (OSD-IID) | Shall validate against the schema of the Data Type. | |
| Produces | Basic Multimodal Scene Descriptors (OSD-BMS) or Multimodal Scene Descriptors (OSD-MSD) | Shall validate against the schema of the Data Type. |
