1. Definition | 2. Functional Requirements | 3. Syntax | 4. Semantics |
1. Definition
Descriptors that are based on RE Data In from Performers, Objects, and Controllers in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data). Performers may be captured by MoCap, one or more video cameras, and one or more microphones.
2. Functional Requirements
Performance Description AIM generates descriptors conveying information on:
- The scene as volume or mesh (Volumetric, e.g., point cloud).
- Performers
- Visual Description
- Performers attributes
- Position and Orientation
- Face and Gestures
- Performers attributes
- Audio behaviour
- Performers are captured by one or more microphones (on body/stage).
- Descriptors on a per-performer basis:
- Text uttered by performer.
- Performer’s audio activity:
- Speaking
- Laughing
- Clapping
- Booing
- Shouting
- Singing
- Biometric data:
- Heart rate and Heart rate variability (HRV).
- Brain state from EEG data (delta, theta, alpha, beta, gamma).
- Galvanic Skin Response (Electrodermal Activity).
- Myoelectric intensity per electrode site.
- Skin temperature.
- Visual Description
- Objects:
- Position and Orientation (Spatial Attitude) of set pieces.
- Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).
3. Syntax
https://schemas.mpai.community/XRV1/V1.0/data/REPerformanceDescriptors.json
4. Semantics
Label | Description |
Header | RE Performance Descriptors Header |
– Standard-REPerformanceDescriptors | The characters “XRV-RFD-V” |
– Version | Major version – 1 or 2 characters |
– Dot-separator | The character “.” |
– Subversion | Minor version – 1 or 2 characters |
SpaceTime | Space-Time info of RE Performance Descriptors |
PerformanceDescriptions[] | Collection of Performance Descriptions. |
– AudioVisual Scene | The Audio-Visual Scene of the stage. |
– PerformerDescription[] | Set of Performer Descriptions, |
– PerformerID | ID of Performer. |
– SpatialAttitude | Of Performer. |
– Descriptors | Set of Descriptors |
– FaceDescriptors | Of Performer. |
– BodyDescriptors | Of Performer. |
– UtteredSpeech | Of Performer. |
– VisualActivity | Boolean: 1=moving, 0=not moving |
– AudioActivity | Boolean: 1=making sound, 0=not making sound |
– BiometricObject | Of Performer |
– OtherObjects[] | Set of Descriptors of other Objects. |
– ObjectID | oneOf: knob, slider, button |
– Spatial Attitude | Object Spatial Attitude. |
– Object variables | Type of variable is defined by RE Venue Specification and signaling by DMX/Midi. |
– ControllerData | From Performer |
DescrMetadata | Descriptive Metadata |