Go to MPAI-OSD V1.5 Data Types

Definition
Functional Requirements
Syntax
Semantics
Conformance Testing
Performance Assessment

1      Definition

A Multimodal Object is a Data Type whose rendering has both Audio and Visual perceptibility attributes. Multimodal Objects may have a hierarchical structure where Multimodal Objects contain Basic Multimodal Objects and Multimodal Objects.

2      Functional Requirements

A Multimodal Object must include:

  1. The Multimodal Object Header.
  2. The ID of a Virtual Space (M-Instance) where it is or is intended to be located.
  3. The ID of the Multimodal Object.
  4. The Space/Time of the Multimodal Object.

A Multimodal Object may include:

  1. The ID of a U-Environment (Real Space) where it is or is intended to be located.
  2. Time when this Multimodal Object was produced.
  3. Acoustic properties of the Multimodal Object.
  4. Basic Multimodal Objects and child Multimodal Objects, each with their Space/Time.
  5. A mixed array of Basic Multimodal Objects and child Multimodal Objects.
  6. MPAI-PTF Data Exchange Metadata.
  7. Descriptive Metadata.

3      Syntax

https://schemas.mpai.community/OSD/V1.5/data/MultimodalObject.json

4      Semantics

Table 1 – Semantics of the Multimodal Object Data Type

Label
Description
Header Multimodal Object Header – Standard “OSD‑AVO‑Vx.y”.
MInstanceID Identifier of the M-Instance (Virtual Space) where this Multimodal Object is or is intended to be located.
UEnvironmentID Identifier of the U-Environment (Real Space) where this Multimodal Object is or is intended to be located.
AVObjectID Unique identifier of this Multimodal Object.
AVObjectTime Time this Multimodal Object was produced.
AVObjectSpaceTime Space/Time where/when this Multimodal Object is located.
AVObjectProperties Acoustic properties of this Multimodal Object.
BasicAVObjectCount Number of Basic Multimodal Objects contained in this Multimodal Object.
BasicAVObjects[] Set of Basic Multimodal Objects contained in this Multimodal Object.
    – BasicAVObjectSpaceTime Space/Time where/when this Basic Multimodal Object is located within the containing Multimodal Object. If absent, the Space/Time of the containing object applies.
    – BasicAVObjectIDOrBasicAVObject Either the ID of a Basic Multimodal Object or the Basic Multimodal Object itself.
SubAVObjectCount Number of child Multimodal Objects contained in this Multimodal Object.
SubAVObjects[] Set of child Multimodal Objects contained in this Multimodal Object.
    – SubAVObjectSpaceTime Space/Time where/when this child Multimodal Object is located within the containing Multimodal Object. If absent, the Space/Time of the containing object applies.
    – SubAVObjectIDOrSubAVObject Either the ID of a child Multimodal Object or the child Multimodal Object itself.
BasicAVObjectOrAVObject A mixed array of Basic Multimodal Objects and child Multimodal Objects contained in this Multimodal Object.
DataXMData MPAI-PTF Data Exchange Metadata providing provenance, authorisation, legal, security, and confidence information associated with this Multimodal Object in a trusted data exchange context.
DescrMetadata Human-readable descriptive metadata of the Multimodal Object (plain text, max 2048 characters).

5      Conformance Testing

A Data instance conforms with Multimodal Object (OSD‑AVO) if:

  1. The Data validates against the Multimodal Object’s JSON Schema.
  2. All Data in the Multimodal Object’s JSON Schema:
    1. Have the specified type.
    2. Validate against their JSON Schemas.
    3. Conform with their Data Qualifiers.

6      Performance Assessment

Not part of this specification.

Go to MPAI-OSD V1.5 Data Types