| 1. Definition | 2. Functional Requirements | 3. Syntax | 4. Semantics | 5. Conformance Testing | 6. Performance Assessment |
1. Definition
Descriptors that are based on RE Data In from the scene, Performers, Objects, Controllers, and Cameras in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data).
2. Functional Requirements
Performance Description AIM generates descriptors conveying information on:
- Multimodal Scene Descriptors from source including:
- Audio
- Visual
- Lidar
- MoCap.
- Performers
- Performer ID
- Visual Performer object
- Visual attributes
- Position and Orientation
- Face and Gestures
- Performer Audio
- Performer Biometrics:
- Heart rate and Heart rate variability (HRV).
- Brain state from EEG data (delta, theta, alpha, beta, gamma).
- Galvanic Skin Response (Electrodermal Activity).
- Myoelectric intensity per electrode site.
- Skin temperature.
- Objects:
- ObjectID
- Visual Object Data
- Position and Orientation (Spatial Attitude) of set pieces.
- Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.)
- Cameras
- Audio/Visual
- Position and Orientation.
- Field of View.
3. Syntax
https://schemas.mpai.community/XRV1/V1.0/data/REPerformanceDescriptors.json
4. Semantics
| Label | Description |
| Header | RE Performance Descriptors Header. Shall have constant value “XRV-RFD-V1.0”. |
| MInstanceID | Identifier of the M-Instance (Virtual Space) associated with this RE Performance Descriptors instance. |
| UEnvironmentID | ID of U-Environment connect to M-Instance. |
| REPerformanceDescriptorsID | Identifier of RE Performance Descriptors |
| REPerformanceDescriptorsTime | Space-Time info of RE Performance Descriptors |
| MultimodalSceneDescriptors | Multimodal Scene Descriptors as defined in OSD V1.5. |
| REPerformanceDescriptorsData[] | Collection of Performance Descriptions. |
| – REPerformerID | ID of Performer. |
| – FaceDescriptorsObject | Face Descriptors Object as defined in PAF V1.6. |
| – BodyDescriptorsObject | Body Descriptors Object as defined in PAF V1.6. |
| – SpatialAttitude | Spatial Attitude of Performer. |
| – UtteredSpeech | Uttered Speech of Performer. |
| – BiometricData | Biometric data of Performer |
| – Brainwaves | Frequency, in Hz, of each EEG band, per Brainwaves (AIH-BRW). |
| REObjectDescriptors[] | Array of RE Object Descriptors. |
| – REObjectID | oneOf: knob, slider, button. |
| – SpatialAttitude | Spatial Attitude of Object. |
| – ControlObject | Variable Types in RE Venue Specification; signalled by DMX/Midi. |
| RECameraData[] | Array of RE Camera Data. |
| – REVisualObject | Visual Object as defined in OSD V1.5. |
| – FieldOfView | Field of view, in degrees. |
| – REAudioObject | Audio Object as defined in OSD V1.5. |
| – SpatialAttitude | Spatial Attitude of the camera. |
| DataXMData | MPAI-PTF Data Exchange Metadata providing provenance, authorisation, legal, security, and confidence information associated with this RE Performance Descriptors instance in a trusted data exchange context. |
| DescrMetadata | Descriptive Metadata |
5. Conformance Testing
A Data instance conforms with RE Performance Descriptors (XRV-RFD-V1.0) if:
- The Data validates against the JSON Schema of RE Performance Descriptors (REPerformanceDescriptors.json).
- All Data in the JSON Schema of RE Performance Descriptors have the specified types.
6. Performance Assessment
Not part of this specification.