1. Definition | 2. Functional Requirements | 3. Syntax | 4. Semantics |
1. Definition
Descriptors that are based on RE Data In from Performers, Objects, and Controllers in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data). Performers may be captured by MoCap, one or more video cameras, and one or more microphones.
2. Functional Requirements
Performance Description AIM generates descriptors conveying information on:
- Visual Description
- Performers attributes
- Position and Orientation
- Face and Gestures
- Scene attributes
- Position and Orientation of set pieces
- The scene as volume (Volumetric, e.g., point cloud)
- Performers attributes
- Audio behaviour
- Performers are captured by one or more microphones (on body/stage).
- Descriptors on a per-performer basis:
- Text uttered by performer.
- Performer’s audio activity:
- Speaking
- Laughing
- Clapping
- Booing
- Shouting
- Singing
- Object data:
- The scene as volume or mesh (Volumetric, e.g., point cloud).
- Position and Orientation (Spatial Attitude) of set pieces.
- Other data (encoders, triggers, measuring devices, motion/proximity sensors, etc.).
- Biometric data:
- Heart rate and Heart rate variability (HRV).
- Brain state from EEG data (delta, theta, alpha, beta, gamma).
- Galvanic Skin Response (Electrodermal Activity).
- Myoelectric intensity per electrode site.
- Skin temperature.
3. Syntax
4. Semantics
Label | Size | Description |
Header | N1 Bytes | RE Participant Descriptors Header |
– Standard-REPerformanceDescriptors | 9 Bytes | The characters “XRV-RFD-V” |
– Version | N2 Bytes | Major version – 1 or 2 characters |
– Dot-separator | 1 Byte | The character “.” |
– Subversion | N3 Bytes | Minor version – 1 or 2 characters |
SpaceTime | N6 Bytes | Space-Time info of RE Performance Descriptors |
PerformanceDescriptions[] | N7 Bytes | Collection of Performance Descriptions. |
– PerformerDescription | N8 Bytes | Set of Performance Descriptions, |
– PerformerID | N9 Bytes | ID of Performer. |
– SpatialAttitude | N10 Bytes | Of Performer. |
– Descriptors | N11 Bytes | Set of Descriptors |
– FaceDescriptors | N12 Bytes | Of Performer. |
– BodyDescriptors | N13 Bytes | Of Performer. |
– UtteredSpeech | N14 Bytes | Of Performer. |
– VisualActivity | 1 bit | Moving/not moving |
– AudioActivity | 1 bit | Making sound/not making sound |
– BiometricData | N15 Bytes | Of Participant |
– OtherObjects[] | N16 Bytes | Set of Descriptors of other Objects. |
– ObjectDescription | N17 Bytes | |
– ObjectID | N18 Bytes | one of: knob, slider, button |
– Spatial Attitude | N19 Bytes | |
– SpatialAttitude | N20 Bytes | |
– ControllerData | N21 Bytes | From Participant |
– AppData | N22 Bytes | From Participant |
DescrMetadata | N23 Bytes | Descriptive Metadata |