1       Functions 2       Reference Model 3       I/O Data
4       Functions of AI Modules 5       I/O Data of AI Modules 6       AIW, AIMs, and JSON
7       Reference Software 8      Conformance Testing 9     Performance Assessment

1        Functions

The Environment Sensing Subsystem (ESS) of a Connected Autonomous Vehicle (CAV):

  1. Acquires electromagnetic and acoustic data from the Environment using its sensors.
  2. Receives Environment Data (e.g., temperature, pressure, humidity, etc.) from the Motion Actuation Subsystem.
  3. Receives an estimate of the Ego CAV’s Spatial Attitude based on information available at the Motion Actuation Subsystem
  4. Produces a sequence of Basic Environment Descriptors (BED) during the journey. Each BED is enhanced Scene Descriptors whose Object may include Annotations regarding specific objects, specifically related to traffic signaling,, e.g.:
    1. Position and Orientation of the traffic signals in the environment:
    2. Traffic Policeman
    3. Road signs (lanes, turn right/left on the road, one way, stop signs, words on the road).
    4. Traffic signs – vertical signalisation (signs above the road, signs on objects, poles with signs).
    5. Traffic lights
    6. Walkways
    7. Traffic sound (siren, whistle, horn).
  5. Passes the BEDs to the Human-CAV Interaction (HCI) and Autonomous Motion (AMS) Subsystems.
  6. Requests elements of the Full Environment Representations (FER) produced by AMS.

2        Reference Model

Figure 1 gives the Reference Model of the Environment Sensing Subsystem.

Figure 1 – Environment Sensing Subsystem Reference Model

The sequence of operations of the Environment Sensing Subsystem unfolds as follows:

  1. The Spatial Attitude Generation AIM computes the CAV’s Spatial Attitude using the initial Motion Actuation Subsystem’s Spatial Attitude and GNSS Object.
  2. All Environment Sensing Technology (EST)-specific Scene Description AIMs present onboard
    1. receive EST-specific Objects, e.g., the RADAR Scene Descriptions AIM receives a RADAR Object provided by the RADAR EST (not shown in Figure 1. The Online Map is considered as an EST.
    2. Produce and send Alerts, if necessary, to the Autonomous Motion Subsystem.
    3. Can access Basic Environment Descriptors of previous times, if needed.
    4. Produce EST-specific Scene Descriptors, e.g., the RADAR Scene Descriptors.
  3. The Basic Environment Description AIM integrate the different EST-specific Scene Descriptors and the Weather Data into the Basic Environment Descriptors and the Road State.

Note that Figure 1 assumes that Environment Sensing Technologies are individually processed. An implementation, however, may combine two or more Scene Descriptors AIMs to handle two or more ESTs, provided the relevant interfaces are preserved.

3        I/O Data

The currently considered Environment Sensing Technologies (EST) are:

  1. GNSS – Global Navigation Satellite System (~1 & 1.5 GHz Radio).
  2. Spatial Attitude of the Ego CAV – Geographical Position and Orientation and their time derivatives up to 2nd
  3. Audio Object in the audible range (16-20,000 Hz)
  4. Visual Object in the visible range, possibly supplemented by depth information (400 to 700 THz).
  5. LiDAR Object (~200 THz infrared).
  6. RADAR Object (~25 & 75 GHz).
  7. Ultrasound Object (> 20 kHz).
  8. Audio Data in the audible range (16 Hz to 20 kHz).
  9. Spatial Attitude (initial version from from the Motion Actuation Subsystem).
  10. Weather Data.

The Offline Map data can be accessed either from stored or online information.

Table 9 gives the input/output data of the Environment Sensing Subsystem.

Table 1 – I/O data of Environment Sensing Subsystem

Input data From Comment
RADAR Object ~25 & 75 GHz Radio Environment Capture with Radar
LiDAR Object ~200 THz infrared Environment Capture with Lidar
Visual Object Video (400-800 THz) Environment Capture with visual cameras
Ultrasound Object Audio (>20 kHz) Environment Capture with Ultrasound
Offline Map Object Local storage or online cm-level data at time of capture
Audio Object Audio (16 Hz-20 kHz) Environment or cabin Capture with Microphone Array
GNSS Object ~1 & 1.5 GHz Radio Get Pose from GNSS
Spatial Attitude Motion Actuation Subsystem To be fused with Pose from GNSS Data
Weather Data Motion Actuation Subsystem Temperature, Humidity, etc.
Output data To Comment
Alert Autonomous Motion Subsystem Critical information from an EST.
Basic Environment Descriptors Autonomous Motion Subsystem ESS-derived Environment Descriptors
Road State Autonomous Motion Subsystem Integrates Weather and Road Data

4        Functions of AI Modules

Table 2 gives the functions of all AIMs of the Environment Sensing Subsystem.

Table 2 – Functions of Environment Sensing Subsystem’s AI Modules

AIM Function
Audio Scene Description Produces Audio Scene Descriptors and Alert.
Basic Environment Description Produces Basic Environment Descriptors and Road State.
LiDAR Scene Description Produces LiDAR Scene Descriptors and Alert.
Offline Map Scene Description Produces Offline Map Scene Descriptors.
RADAR Scene Description Produces RADAR Scene Descriptors and Alert.
Spatial Attitude Generation Computes the CAV Spatial Attitude from CAV Centre using GNSS Object and MAS’s initial Spatial Attitude.
Ultrasound Scene Description Produces Ultrasound Scene Descriptors and Alert.
Visual Scene Description Produces Visual Scene Descriptors and Alert.

5        I/O Data of AI Modules

For each AIM (1st column), Table 3 gives the input (2nd column) and the output data (3rd column) of the Environment Sensing Subsystem. Note that the Basic Environment Representation in column 2 refers to a previously produced BER.

Table 3 – I/O Data of Environment Sensing Subsystem’s AI Modules

AIM Input Output
Audio Scene Description – Audio Object
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Audio Scene Descriptors
Basic Environment Description – Audio Scene Descriptors
– LiDAR Scene Descriptors
– Offline Map Scene Descriptors
– RADAR Scene Descriptors
– Spatial Attitude
– Ultrasound Scene Descriptors
– Visual Scene Descriptors
– Weather Data		 – Basic Environment Descriptors
– Road State
LiDAR Scene Description – LiDAR Object
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– LiDAR Scene Descriptors
Offline Map Scene Description – Offline Map Object
– Spatial Attitude		 – Offline Map Scene Descriptors
RADAR Scene Description – RADAR Object
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– RADAR Scene Descriptors
Spatial Attitude Generation – GNSS Object
– MAS’s Spatial Attitude		 – Spatial Attitude
Ultrasound Scene Description – Ultrasound Object
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Ultrasound Scene Descriptors
Visual Scene Description – Visual Object
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Visual Scene Descriptors

6      AIW, AIMs, and JSON

Table 4 – AIW, AIMs, and JSON Metadata

AIW AIM Name JSON
CAV-ESS Environment Sensing Subsystem X
CAE-ASD Audio Scene Description X
CAV-BED Basic Environment Description X
CAV-LSD LiDAR Scene Description X
CAV-OSD Offline Map Scene Description X
CAV-RSD RADAR Scene Description X
CAV-SAG Spatial Attitude Generation X
CAV-USD Ultrasound Scene Description X
OSD-VSD Visual Scene Description X

7      Reference Software

 

8         Conformance Testing

 

9     Performance Assessment