1. Definition 2. Functional Requirements 3. Syntax 4. Semantics

1. Definition

Descriptors that are based on performance data form the Real Environment and have a form that is suitable for interpretation  and immediate usein the Virtual Environment (digital twin) in addition to providing Descriptors to determine cue points.

2. Functional Requirements

Components from the real environment:

  1. Visual behaviour
    1. Room may be split in two or more sections.
    2. Participants are captured by one or more video cameras.
    3. Calibration process to register the video images with seating chart
    4. Extract descriptors on a per seat or per-section basis
      1. Seats occupation
        1. Per seat
        2. Per section
      2. Participants motion
        1. Hands waving left to right/right to left
        2. Participants standing or sitting
        3. Participants clapping
        4. Participants not moving.
  2. Audio reaction
    1. Room may be split in two or more sections.
    2. Participants are captured by one or more microphones.
    3. Calibration process to register the active area of microphones with seating chart.
    4. Descriptors on a per-seat or per-section basis:
      1. Seats occupation (info coming from video)
      2. Audio activity
        1. Text uttered by Participant.
        2. Participant’s audio activity:
          1. Speaking
          2. Laughing
          3. Clapping
          4. Booing
          5. Shouting
          6. Singing
  3. Choice per seat (mobile app, web interface, controller)
    1. Voting
    2. Controller
      1. Spatial motion
        1. Relative/absolute
        2. Value (Spatial Attitude)
      2. Actuator type:
        1. knob, slider, button
        2. values of actuator
      3. Text includes text string, source, and destination.
  4. Biometric descriptors of the Real Environment participant.
    1. Heart rate and Heart rate variability (HRV).
    2. Brain state from EEG data (delta, theta, alpha, beta, gamma).
    3. Galvanic Skin Response (Electrodermal Activity).
    4. Myoelectric intensity per electrode site.

3. Syntax

4. Semantics