<-AI Workflows Go to ToC Data Types->

1. Technical Specification 2. Reference Software 3. Conformance Testing 4. Performance Assessment

1. Technical Specifications

Table 1 provides the links to the specifications and the JSON syntax of all AIMs specified by Technical Specification: Context-based Audio Enhancement (MPAI-CAE) – Six Degrees of Freedom (CAE-6DF) V1.0. The AI Modules specified by CAE-6DF V1.0 supersede those specified by previous versions. These may be used if their Version is explicitly signaled. AIMs in bold are Composite.

Table 1 – Specifications and JSON syntax of AIMs used by CAE-6DF V1.0

Acronym AIM Name JSON Acronym AIM Name JSON
CAE-6DC 6DF Decompression File CAE-OMP Orthogonal Matching Pursuit File
CAE-6DX 6DF Demultiplexing File CAE-PWI Plane Wave Parameter Interpolation File
CAE-6DI 6DF Interpolation File CAE-PWS Plane Wave Synthesis File
CAE-SMX 6DF Multiplexing File CAE-RET Residual Energy Test File
CAE-ARM Audio Rendering Module File CAE-SFT Short-Time Fourier Transform File
CAE-BAT Binaural Transcoding File CAE-SFR Sound Field Rotation File
CAE-DDX Direct/Diffuse Mix File CAE-SST Spatio-Spectro-Temporal Transform File
CAE-ICC Interpolation Coefficients Calculation File CAE-SHD Spherical Harmonic Decomposition File
CAE-ISF Inverse Short-Time Fourier Transform File CAE-SRF Steered Response Function File
CAE-LIP Linear Interpolation File CAE-TPS Triplet Select File
CAE-LST Loudspeaker Transcoding File

2. Reference Software

As a rule, MPAI provides Reference Software implementing the AI Modules released with the BSD-3-Clause licence and the following disclaimers:

  1. The CAE-USC V2.4 Reference Software Implementation, if in source code, is released with the BSD-3-Clause licence.
  2. The purpose of this Reference Software is to provide a working Implementation of CAE-USC V2.4, not to provide a ready-to-use product.
  3. MPAI disclaims the suitability of the Software for any other purposes and does not guarantee that it is secure.
  4. Use of this Reference Software may require acceptance of licences from the respective copyright holders. Users shall verify that they have the right to use any third-party software required by this Reference Software.

Note that at this stage CAE-USC V2.4 specifies Reference Software only for some AIMs.

3. Conformance Testing

An implementation of an AI Module conforms with CAE-USC V2.4 if it accepts as input _and_ produces as output Data and/or Data Objects (the combination of Data of a Data Type and its Qualifier) conforming with those specified by CAE-USC V2.4 .

The Conformance is expressed by one of the two statements

  1. “Data conforms with the relevant (Non-MPAI) standard” – for Data.
  2. “Data validates against the Data Type Schema” – for Data Object.

The latter statement implies that:

  1. Any Sub-Type of the Data conforms with the relevant Sub-Type specification of the applicable Qualifier.
  2. Any Content and Transport Format of the Data conform with the relevant Format specification of the applicable Qualifier.
  3. Any Attribute of the Data
    1. Conforms with the relevant (Non-MPAI) standard – for Data, or
    2. Validates against the Data Type Schema – for Data Object.

The method to Test the Conformance of an instance of Data or Data Object is specified in the Data Types chapter.

Note that at this stage the CAE-USC V2.4 specifies Conformance Testing only for some AIMs.

4. Performance Assessment

Performance is an umbrella term used to describe a variety of attributes – some specific of the application domain the Implementation intends to address. Therefore, Performance Assessment Specifications provide methods and procedures to measure how well an AIW or an AIM performs its function. Performance of an Implementation includes methods and procedures for all or a subset of the following characteristics:

  1. Quality – for instance, how well a Face Identity Recognition AIM recognises faces, how precise or error-free are the changes in a Visual Scene detected by a Visual Change Detection AIM, or how satisfactory are the responses provided by an Answer to Multimodal Question AIW.
  2. Robustness – for instance, how robust is the operation of an implementation with respect to duration of operation, load scaling, etc.
  3. Extensibility – for instance, the degree of confidence a user can have in an Implementation when it deals with data outside of its stated application scope.
  4. Bias: – for instance, how dependent on specific features of the training data is the inference, as in Company Performance Prediction when the accuracy of the prediction may widely change based on the size or the geographic position of a Company; or face recognition in Television Media Analysis.
  5. Legality – for instance, in which jurisdictions the use of an AIM or an AIW complies with a regulation, e.g., the European AI Act.
  6. Ethics: may indicate the conformity of an AIM or AIW to a target ethical standard.

Note that at this stage the CAE-USC V2.4 specifies Performance Assessment only for some AIMs.

<-AI Workflows Go to ToC Data Types->