Go to CAV-TEC V1.1 AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1      Functions

The Environment Sensing Subsystem (CAV‑ESS) AIM senses the environment’s electromagnetic and acoustic sources, receives weather information from the Motion Actuation Subsystem (MAS), computes its Spatial Attitude using GNSS and spatial data from the MAS, produces and passes to the AMS a sequence of Basic Environment Descriptors by processing data acquired by a variety of sensors (Audio, Visual, LiDAR, RADAR, Ultrasound, and Offline Map).

Receives Audio Object Environment audio.
Visual Object Environment visual.
LiDAR Object LiDAR generated by CAV received from environment.
RADAR Object RADAR generated by CAV received from environment.
Offline Map Object Offline Map data from Autonomous Motion Subsystem.
Ultrasound Object Ultrasound generated by CAV received from environment.
GNSS Object GNSS data.
Spatial Attitude Spatial data from sensors.
Weather Data Weather data from sensors.
Full Environment Descriptors From Autonomous Motion Subsystem.
Produces Audio Object Audio to User/Environment.
LiDAR Object LiDAR generated by CAV.
RADAR Object RADAR generated by CAV.
Ultrasound Object Ultrasound generated by CAV.
Basic Environment Descriptors Main output of CAV‑ESS.
Alert Alert from one Environment Sensing Technology.

2      Reference Model

Figure 1 depicts the Reference Model of the Environment Sensing Subsystem (CAV‑ESS) AIM.

Environment Sensing Subsystem CAV-ESS AIM

Figure 1 – The Environment Sensing Subsystem (CAV‑ESS) AIM

3      I/O Data

Table 1 specifies the Input and Output Data of the Environment Sensing Subsystem (CAV‑ESS) AIM.

Table 1 – I/O Data of the Environment Sensing Subsystem (CAV‑ESS) AIM

Input Description
Audio Object Environment audio.
Visual Object Environment visual.
LiDAR Object LiDAR generated by CAV received from environment.
RADAR Object RADAR generated by CAV received from environment.
Offline Map Object Offline Map data from Autonomous Motion Subsystem.
Ultrasound Object Ultrasound generated by CAV received from environment.
GNSS Object GNSS data.
Spatial Attitude Spatial data from sensors.
Weather Data Weather data from sensors.
Full Environment Descriptors From Autonomous Motion Subsystem.
Output Description
Audio Object Audio to User/Environment.
LiDAR Object LiDAR generated by CAV.
RADAR Object RADAR generated by CAV.
Ultrasound Object Ultrasound generated by CAV.
Basic Environment Descriptors Main output of CAV‑ESS.
Alert Alert from one Environment Sensing Technology.

4      SubAIMs

4.1 Functions of SubAIMs

Table 2 gives the functions of all SubAIMs of the Environment Sensing Subsystem (CAV‑ESS).

Table 2 – Functions of the Environment Sensing Subsystem (CAV‑ESS) SubAIMs

SubAIM Function
Spatial Attitude Generation Computes the CAV Spatial Attitude from CAV Centre using GNSS Object and MAS’s initial Spatial Attitude.
Audio Scene Description Produces Audio Scene Descriptors and Alert.
Visual Scene Description Produces Visual Scene Descriptors and Alert.
LiDAR Scene Description Produces LiDAR Scene Descriptors and Alert.
RADAR Scene Description Produces RADAR Scene Descriptors and Alert.
Ultrasound Scene Description Produces Ultrasound Scene Descriptors and Alert.
Offline Map Scene Description Produces Offline Map Scene Descriptors.
Basic Environment Description Produces Basic Environment Descriptors.

4.2 Operation

The sequence of operations of the Environment Sensing Subsystem unfolds as follows:

  1. The Spatial Attitude Generation AIM computes the CAV’s Spatial Attitude using the initial Motion Actuation Subsystem’s Spatial Attitude and GNSS Object.
  2. All EST‑specific Scene Description AIMs available onboard:
    1. Receive EST‑specific Data Objects, e.g., the RADAR Scene Description AIM receives a RADAR Object provided by the RADAR EST (not shown in Figure 2). The Offline Map is considered as an EST.
    2. Produce and send Alerts, if necessary, to the Autonomous Motion Subsystem.
    3. Access Basic Environment Descriptors of previous times, if needed.
    4. Produce EST‑specific Scene Descriptors, e.g., the RADAR Scene Descriptors.
  3. The Basic Environment Description AIM integrates the different EST‑specific Scene Descriptors, Weather Data, and Road State into the Basic Environment Descriptors.

Note 1: Although Figure 2 shows individually processed ESTs, an implementation may combine two or more Scene Description AIMs to handle two or more ESTs, provided the relevant interfaces are preserved.

Note 2: The Objects in the BEDs may carry Annotations specifically related to traffic signalling, e.g.:

  1. Position and orientation of traffic signals in the environment.
  2. Traffic policemen.
  3. Road signs (lanes, turn right/left on the road, one way, stop signs, words on the road).
  4. Traffic signs – vertical signalisation (signs above the road, signs on objects, poles with signs).
  5. Traffic lights.
  6. Walkways.
  7. Traffic sound (siren, whistle, horn).

4.3 I/O Data of SubAIMs

Table 3 gives, for each SubAIM, the Input and Output Data of the Environment Sensing Subsystem. The Basic Environment Descriptors in the Input column refer to a previously produced BED.

Table 3 – I/O Data of the Environment Sensing Subsystem (CAV‑ESS) SubAIMs

SubAIM Input Output
Spatial Attitude Generation GNSS Object
MAS’s Spatial Attitude		 Spatial Attitude
Audio Scene Description Audio Object
Spatial Attitude
Other Scene Descriptors
Basic Environment Descriptors		 Alert
Audio Scene Descriptors
Visual Scene Description Visual Object
Spatial Attitude
Other Scene Descriptors
Basic Environment Descriptors		 Alert
Visual Scene Descriptors
LiDAR Scene Description LiDAR Object
Spatial Attitude
Other Scene Descriptors
Basic Environment Descriptors		 Alert
LiDAR Scene Descriptors
RADAR Scene Description RADAR Object
Spatial Attitude
Basic Environment Descriptors		 Alert
RADAR Scene Descriptors
Ultrasound Scene Description Ultrasound Object
Spatial Attitude
Other Scene Descriptors
Basic Environment Descriptors		 Alert
Ultrasound Scene Descriptors
Offline Map Scene Description Offline Map Object
Spatial Attitude		 Offline Map Scene Descriptors
Basic Environment Description Audio Scene Descriptors
LiDAR Scene Descriptors
Offline Map Scene Descriptors
RADAR Scene Descriptors
Spatial Attitude
Ultrasound Scene Descriptors
Visual Scene Descriptors
Weather Data
Full Environment Descriptors		 Basic Environment Descriptors

4.4 AIMs and JSON Metadata

Table 4 provides the links to the AIM specifications and JSON schemas. AIM1 indicates the Composite AIM and AIM2 its SubAIMs.

Table 4 – AIMs and JSON Metadata of the Environment Sensing Subsystem (CAV‑ESS)

AIM1 AIM2 Name JSON
CAV‑ESS Environment Sensing Subsystem X
CAV‑SAG Spatial Attitude Generation X
OSD‑ASD Audio Scene Description X
OSD‑VSD Visual Scene Description X
OSD‑LSD LiDAR Scene Description X
OSD‑RSD RADAR Scene Description X
OSD‑USD Ultrasound Scene Description X
OSD‑OMS Offline Map Scene Description X
CAV‑BED Basic Environment Description X

5      JSON Metadata

https://schemas.mpai.community/CAV2/V1.1/AIMs/EnvironmentSensingSubsystem.json

6      Profiles

No Profiles.

7      Reference Software

Not part of this specification.

8      Conformance Testing

Table 5 provides the Conformance Testing Method for the Environment Sensing Subsystem (CAV‑ESS) Composite AIM. Conformance Testing of the individual SubAIMs is given by the individual AIM specifications.

If a schema contains references to other schemas, conformance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present, and conform with the Qualifier, if present.

Table 5 – Conformance Testing Method for the Environment Sensing Subsystem (CAV‑ESS) Composite AIM

Receives Audio Object Shall validate against Audio Object schema.
Visual Object Shall validate against Visual Object schema.
LiDAR Object Shall validate against LiDAR Object schema.
RADAR Object Shall validate against RADAR Object schema.
Offline Map Object Shall validate against Offline Map Object schema.
Ultrasound Object Shall validate against Ultrasound Object schema.
GNSS Object Shall validate against GNSS Object schema.
Spatial Attitude Shall validate against Spatial Attitude schema.
Weather Data Shall validate against Weather Data schema.
Full Environment Descriptors Shall validate against Full Environment Descriptors schema.
Produces Audio Object Shall validate against Audio Object schema.
LiDAR Object Shall validate against LiDAR Object schema.
RADAR Object Shall validate against RADAR Object schema.
Ultrasound Object Shall validate against Ultrasound Object schema.
Basic Environment Descriptors Shall validate against Basic Environment Descriptors schema.
Alert Shall validate against Alert schema.

9      Performance Assessment

Not part of this specification.

Go to CAV-TEC V1.1 AI Modules