1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on RE Data In from Participants and Controllers in the Real Environment and have a form that is suitable for Interpretation (e.g., Position and Orientation, Face and Gestures, Controller Data). The theatre may be split into two or more sections. Participants may be captured by one or more video cameras and one or more microphones. A calibration process is needed to align video images with seating chart and to register the active area of microphones with seating chart.

2. Functional Requirements

  1. Audio-Visual Scene Descriptors
  2. Seat Arrangement
  3. Participant Description
    1. Participant ID
    2. Spatial Attitude
    3. Descriptors (Speech, Face, Body)
  4. Choice per seat (mobile app, web interface, controller)
    1. Voting
    2. Controller
      1. Spatial Attitude
      2. Actuator type (knob, slider, button)
      3. Values of actuator
      4. Text
  5. Biometric descriptors of the Real Environment participant.
    1. Heart rate and Heart rate variability (HRV).
    2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
    3. Galvanic Skin Response (Electrodermal Activity).
    4. Myoelectric intensity per electrode site.
    5. Skin temperature.

3. Syntax

https://schemas.mpai.community/XRV1/V1.0/data/REParticipantDescriptors.json

4. Semantics

Label Description
Header RE Participant Descriptors Header
– Standard-REParticipantDescriptors The characters “XRV-RTD-V”
– Version Major version – 1 or 2 characters
– Dot-separator The character “.”
– Subversion Minor version – 1 or 2 characters
ParticipantDescriptorsTime  Time info of RE Participant Descriptors
ParticipantDescriptorsData Collection of Participant Descriptors
– AudioVisualSceneDescriptors Descriptors of the portion of the theatre that includes Participants.
– SeatOccupation Seat occupation per the Seat Occupation Data Type.
– ParticipantDescriptors[] Set of description data.
  – ParticipantID ID of Participant.
  – SpatialAttitude Spatial Attitude of Participant.
  – FaceDescriptors Face Descriptors of Participant.
  – BodyDescriptors Body Descriptors of Participant.
  – UtteredSpeech UtteredSpeech of Participant.
  – BiometricData BiometricData of Participant.
  – ControlObject ID of of Participant Controller being described.
  – AppData App data of Participant.
GroupDescriptions[] Collection of Participant Group Descriptors.
– GroupID ID of Participant
– GroupDescriptors Set of Descriptors
  – MotionDescriptors Motion Vector Field of Group.
    – ParticipantID ID of Participant
    – SpatialAttitude Spatial Attitude
DescrMetadata Descriptive Metadata