Function
Ref. Model
I/O Data
SubAIMs
JSON MData
Profiles
Ref. Software
Conformance
Performance
1 Functions
Audio Scene Object Identification (CAE-ASI) receives the descriptors of an Audio Scene and tells which of its Audio Objects are speech:
| Receives | Enhanced Audio Scene Descriptors or Basic Audio Scene Descriptors | The Audio Objects of the scene, with their positions. |
| Classifies | Each Audio Object | Decides whether the sound of the Audio Object is speech or other sound, with an audio classification model. |
| Requalifies | The speech objects | Gives the Audio Object that is speech the form of a Basic Speech Object. |
| Produces | Basic Speech Object and Basic Audio Object | The Basic Speech Objects, for Speaker Recognition, Automatic Speech Recognition and Personal Status extraction, and the Basic Audio Objects that are not speech, for Audio Instance Identification. |
2 Reference Model
The Audio Scene Object Identification AIM has one input and two outputs, as in Table 1. It is used, for instance, by Human-CAV Interaction (CAV-TEC) and by User State Description (PGM-AUA), which gives its Speech Objects to Automatic Speech Recognition.
3 I/O Data
| Input | Description |
|---|---|
| Enhanced Audio Scene Descriptors or Basic Audio Scene Descriptors | The descriptors of the Audio Scene whose Audio Objects are classified. Enhanced Audio Scene Descriptors are used when the scene has been processed by Audio Scene Enhancement; Basic Audio Scene Descriptors otherwise. |
| Audio-Visual Scene Descriptors (optional) | The scene in which the audio and visual objects are aligned. When present, the Audio Object paired with the avatar is the likely speech of the avatar. |
| Output | Description |
| Basic Speech Object | An Audio Object that is speech, requalified. One per speech object. |
| Basic Audio Object | An Audio Object that is not speech, unchanged. |
4 SubAIMs
No SubAIMs.
5 JSON Metadata
https://schemas.mpai.community/CAE1/V2.5/AIMs/AudioSceneObjectIdentification.json
6 Profiles
No Profiles.
7 Reference Software
Not part of this specification.
8 Conformance Testing
Table 2 provides the Conformance Testing Method for the Audio Scene Object Identification (CAE‑ASI) AIM.
| Receives | Enhanced Audio Scene Descriptors or Basic Audio Scene Descriptors | Shall validate against the Enhanced Audio Scene Descriptors schema or the Basic Audio Scene Descriptors schema. |
| Audio-Visual Scene Descriptors | Shall validate against the Basic Multimodal Scene Descriptors schema. | |
| Produces | Basic Speech Object | Shall validate against the Basic Speech Object schema. Speech Data shall conform with the Speech Qualifier. |
| Basic Audio Object | Shall validate against the Basic Audio Object schema. Audio Data shall conform with the Audio Qualifier. |
9 Performance Assessment
Not part of this specification.