Go to CAE-USC AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

The Short-Time Fourier Transform (CAE‑SFT) AIM:

Receives Audio Object (time domain) From Spatio-Spectro-Temporal Transform
Performs The SFT AIM takes Audio Objects represented in the time domain as input and outputs Audio Objects in the frequency domain. The SFT uses a Hann window of length N = 2048 with and overlap size of 7/8 N between blocks of data and performs an FFT.
Produces Audio Object (frequency domain) To Spatio-Spectro-Temporal Transform

2 Reference Model

Figure 1 depicts the Reference Architecture of the Audio Analysis Transform (CAE-AAT) AIM.

Figure 1 – Audio Analysis Transform (CAE-AAT) AIM

3 I/O Data

Table 1 – I/O Data of the Audio Analysis Transform (CAE‑AAT) AIM

Input

Description
Audio Object (time domain) Audio Object in the time domain
Output

Description
Audio Object (frequency domain) Audio Object in the frequency domain

4 SubAIMs

No SubAIMs.

5 JSON Metadata

https://schemas.mpai.community/CAE1/V2.5/AIMs/ShortTimeFourierTransform.json

6 Profiles

No Profiles

7 Reference Software

The Audio Analysis Transform Reference Software can be downloaded from the MPAI Git.

8 Conformance Testing

Table 2 – Conformance Testing Method for the Audio Analysis Transform (CAE‑AAT) AIM

Receives Audio Object (time domain) Shall validate against the Audio Object schema.
The Qualifier shall validate against the Audio Qualifier schema.
The values of any Sub-Type, Format, and Attribute of the Qualifier shall correspond with the Sub-Type, Format, and Attributes of the Audio Object Qualifier schema.
Produces Audio Object (frequency domain) Shall validate against the Audio Object schema.
The Qualifier shall validate against the Audio Qualifier schema.
The values of any Sub-Type, Format, and Attribute of the Qualifier shall correspond with the Sub-Type, Format, and Attributes of the Audio Object Qualifier schema.

9 Performance Assessment

The following steps shall be followed when assessing the Performance of  a CAE-AAT AIM instance.

  1. Use the following datasets:
    1. DS1: n Test Audio Object files including Multichannel Audio as Interleaved Multichannel Audio format.
    2. DS2: n Expected Audio Object Output files including data in Transform Interleaved Multichannel Audio format.
  2. Feed the AIM under test with the Test files (DS1).
  3. Perform the following steps to analyse the Audio Object (Transform) with the Expected Audio Objects (DS2):
    1. Check the data format of the Audio Object (Transform) with the format of the given Expected Audio Objects.
    2. Calculate the peak-to-peak Amplitude (A) of each Audio block in the Expected Audio Objects.
    3. Calculate the RMSE of each Audio block by comparing the Audio Object (Transform) (x) with the Expected Audio Objects  (y).
    4. Accept the AIM under test if, for each audio block, these two conditions are satisfied:
      1. Data format of the Audio Object (Transform) is the same as the format  of the Expected Audio Object and
      2. RMSE < A* 0.1%.
  4. The Performance Assessor will provide the following matrix containing a limited number of input records (n) with the corresponding outputs. If an input record fails, the tester would specify the reason why the test case fails.
Input data (DS1) Expected Output Data (DS2) Data Format RMSE
Audio Object (Microphone Array) ID1 Audio Object (Transform) ID1 T/F < A*0.1%
Audio Object (Microphone Array) ID2 Audio Object (Transform) ID2 T/F < A*0.1%
Audio Object (Microphone Array) ID3 Audio Object (Transform) ID3 T/F < A*0.1%
… … … …
Audio Object (Microphone Array) IDn Audio Object (Transform) IDn T/F < A*0.1%
  1. Final evaluation: T/F Denoting with i, the record number in DS1 and DS2, the matrices reflect the results obtained with input records i with the corresponding outputs i.
DS1 DS2 Audio Object (Transform) output value (from AIM under test)
DS1[i] DS2[i] Audio Object (Transform)[i]

Table 2 provides the Performance Assessment Method for the formats of the CAE-AAT AIM output.

Note: If a schema contains references to other schemas, performance of data for the primary schema implies that any data referencing a secondary schema shall also validate against the relevant schema, if present and perform with the Qualifier, if present.

Go to CAE-USC AI Modules