Go to CAE-USC AI Modules

Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance

1 Functions

Audio Scene Object Identification (CAE-ASI) receives the descriptors of an Audio Scene and tells which of its Audio Objects are speech:

Receives Enhanced Audio Scene Descriptors or Basic Audio Scene Descriptors The Audio Objects of the scene, with their positions.
Classifies Each Audio Object Decides whether the sound of the Audio Object is speech or other sound, with an audio classification model.
Requalifies The speech objects Gives the Audio Object that is speech the form of a Basic Speech Object.
Produces Basic Speech Object and Basic Audio Object The Basic Speech Objects, for Speaker Recognition, Automatic Speech Recognition and Personal Status extraction, and the Basic Audio Objects that are not speech, for Audio Instance Identification.

2 Reference Model

The Audio Scene Object Identification AIM has one input and two outputs, as in Table 1. It is used, for instance, by Human-CAV Interaction (CAV-TEC) and by User State Description (PGM-AUA), which gives its Speech Objects to Automatic Speech Recognition.

3 I/O Data

Table 1 – I/O Data of the Audio Scene Object Identification (CAE‑ASI) AIM

Input Description
Enhanced Audio Scene Descriptors or Basic Audio Scene Descriptors The descriptors of the Audio Scene whose Audio Objects are classified. Enhanced Audio Scene Descriptors are used when the scene has been processed by Audio Scene Enhancement; Basic Audio Scene Descriptors otherwise.
Audio-Visual Scene Descriptors (optional) The scene in which the audio and visual objects are aligned. When present, the Audio Object paired with the avatar is the likely speech of the avatar.
Output Description
Basic Speech Object An Audio Object that is speech, requalified. One per speech object.
Basic Audio Object An Audio Object that is not speech, unchanged.

4 SubAIMs

No SubAIMs.

5 JSON Metadata

https://schemas.mpai.community/CAE1/V2.5/AIMs/AudioSceneObjectIdentification.json

6 Profiles

No Profiles.

7 Reference Software

Not part of this specification.

8 Conformance Testing

Table 2 provides the Conformance Testing Method for the Audio Scene Object Identification (CAE‑ASI) AIM.

Table 2 – Conformance Testing Method for the Audio Scene Object Identification (CAE‑ASI) AIM

Receives Enhanced Audio Scene Descriptors or Basic Audio Scene Descriptors Shall validate against the Enhanced Audio Scene Descriptors schema or the Basic Audio Scene Descriptors schema.
Audio-Visual Scene Descriptors Shall validate against the Basic Multimodal Scene Descriptors schema.
Produces Basic Speech Object Shall validate against the Basic Speech Object schema. Speech Data shall conform with the Speech Qualifier.
Basic Audio Object Shall validate against the Basic Audio Object schema. Audio Data shall conform with the Audio Qualifier.

9 Performance Assessment

Not part of this specification.

Go to CAE-USC AI Modules