1       Functions

2       Reference Model

3       I/O Data

4       Functions of AI Modules

5       I/O Data of AI Modules

6       AIW, AIMs, and JSON

7       AI Modules

8      Data Types

1        Functions

The Environment Sensing Subsystem (ESS) of a Connected Autonomous Vehicle (CAV):

  1. Acquires electromagnetic and acoustic data from the Environment using its sensors.
  2. Receives Environment Data (e.g., temperature, pressure, humidity, etc.) from the Motion Actuation Subsystem.
  3. Receives an initial estimate of the Ego CAV’s Spatial Attitude from the Motion Actuation Subsystem
  4. Produces a sequence of Basic Environment Representations (BER) during the journey.
  5. Passes the Basic Environment Representation to the Human-CAV Interaction (HCI) and Autonomous Motion (AMS) Subsystems.
  6. Requests elements of the Full Environment Representations (FER) produced by AMS.

2        Reference Model

Figure 1 gives the Reference Model of the Environment Sensing Subsystem.

Figure 1 – Environment Sensing Subsystem Reference Model

The sequence of operations of the Environment Sensing Subsystem unfolds as follows:

  1. Compute the CAV’s Spatial Attitude using the initial Spatial Attitude provided by the Motion Actuation Subsystem and GNSS Data.
  2. Receive Environment Sensing Technology (EST)-specific Data, e.g., RADAR Data provided by the RADAR EST.
  3. Produce and send Alert, if necessary, to Autonomous Motion Subsystem.
  4. Access Basic Environment Representations of previous times, if necessary.
  5. Produce EST-specific Scene Descriptors, e.g., the RADAR Scene Descriptors.
  6. Integrate the Scene Descriptors from different ESTs into the Basic Environment Representation.

Note that Figure 1 assumes that Environment Sensing Technologies are individually processed. An implementation, however, may combine two or more Scene Descriptors AIMs to handle two or more ESTs, provided the relevant interfaces are preserved.

3        I/O Data

The currently considered Environment Sensing Technologies (EST) are:

  1. GNSS – Global Navigation Satellite System (~1 & 1.5 GHz Radio).
  2. Spatial Attitude of the Ego CAV – Geographical Position and Orientation and their time derivatives up to 2nd
  3. Visual Data in the visible range, possibly supplemented by depth information (400 to 700 THz).
  4. LiDAR Data (~200 THz infrared).
  5. RADAR Data (~25 & 75 GHz).
  6. Ultrasound Data (> 20 kHz).
  7. Audio Data in the audible range (16 Hz to 20 kHz).
  8. Spatial Attitude (from the Motion Actuation Subsystem).
  9. Other environmental data (temperature, humidity, …).

The Offline Map data can be accessed either from stored or online information.

Table 9 gives the input/output data of the Environment Sensing Subsystem.

Table 1 – I/O data of Environment Sensing Subsystem

Input data From Comment
Input RADAR ~25 & 75 GHz Radio Environment Capture with Radar
Input LiDAR ~200 THz infrared Environment Capture with Lidar
Input Visual Video (400-800 THz) Environment Capture with visual cameras
Input Ultrasound Audio (>20 kHz) Environment Capture with Ultrasound
Input OfflineMap Local storage or online cm-level data at time of capture
Input  Audio Audio (16 Hz-20 kHz) Environment or cabin Capture with Microphone Array
GNSS ~1 & 1.5 GHz Radio Get Pose from GNSS
Spatial Attitude Motion Actuation Subsystem To be fused with Pose from GNSS Data
Other Environment Data Motion Actuation Subsystem Temperature, Humidity, etc.
Output data To Comment
Alert Autonomous Motion Subsystem Critical information from an EST.
Basic Environment Descriptors Autonomous Motion Subsystem ESS-derived Environment Descriptors

4        Functions of AI Modules

Table 2 gives the functions of all AIMs of the Environment Sensing Subsystem.

Table 2 – Functions of Environment Sensing Subsystem’s AI Modules

AIM Function
Audio Scene Description Produces Audio Scene Descriptors and Alert.
Visual Scene Description Produces Visual Scene Descriptors and Alert.
RADAR Scene Description Produces RADAR Scene Descriptors and Alert.
LiDAR Scene Description Produces LiDAR Scene Descriptors and Alert.
Ultrasound Scene Description Produces Ultrasound Scene Descriptors and Alert.
OfflineMap Scene Description Produces Online Map Data Scene Descriptors.
Traffic Signalisation Description Produces Traffic Signalisation Descriptors.
Spatial Attitude Generation Computes the CAV Spatial Attitude from CAV Centre using GNSS and Motion Actuation Subsystem information.
Basic Environment Description Receives Scene Descriptors and critical Environment Representation as Alert from the different ESTs.

Produces Alert and BER including all available information from ESS:

1.      Scene Descriptors from EST-specific Scene Description AIMs.
2.      Environment Data.
3.      The Spatial Attitude of the Ego CAV (Figure 6).

Figure 2 – Spatial Attitude in a CAV

5        I/O Data of AI Modules

For each AIM (1st column), Table 3 gives the input (2nd column) and the output data (3rd column) of the Environment Sensing Subsystem. Note that the Basic Environment Representation in column 2 refers to a previously produced BER.

Table 11 – I/O Data of Environment Sensing Subsystem’s AI Modules

AIM Input Output
Audio Scene Description – Audio Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Audio Scene Descriptors
Visual Scene Description – Visual Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Lidar Scene Descriptors
Lidar Scene Description – Lidar Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Lidar Scene Descriptors
Radar Scene Description – Radar Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Radar Scene Descriptors
Ultrasound Scene Description – Ultrasound Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Ultrasound Scene Descriptors
Map Scene Description – Offline Map Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Map Scene Descriptors
Traffic Signalisation Description – Visual Data
– Spatial Attitude
– Basic Environment Descriptors		 – Alert
– Traffic Signalisation Descriptors
Spatial Attitude Generation – GNSS Data
– Spatial Attitude from MAS		 – Spatial Attitude
Basic Environment Description – RADAR Scene Descriptors
– LiDAR Scene Descriptors
– Traffic Signalisation Descriptors
– Lidar Scene Descriptors
– Ultrasound Scene Descriptors
– Map Scene Descriptors
– Audio Scene Descriptors
– Spatial Attitude
– Other Environment Data		 – Alert
– Basic Environment Descriptors

6      AWW, AIMs, and JSON

7      AI Modules

Initial version of ESS AI Modules is available.

8         Data Types

Initial version of ESS  Data Types is available.