<-Introduction Go to ToC Definitions ->

Technical Specification: Context-based Audio Enhancement (MPAI-CAE) – Audio Scene Management (CAE-ASM) V1.0 – in the following also called CAE-ASM V1.0, or simply CAE-ASM – specifies:

  1. the functionality, the inputs and the outputs of the Audio Scene Management Composite AIM (CAE-ASM);
  2. the functionality, the inputs and the outputs of each of its AIMs;
  3. the acquisition of Audio Objects, and, by reference to MPAI-MMC, of Speech Objects;
  4. the delivery of Audio, Speech and Multimodal Objects and Scenes;
  5. the user interface to Audio Scene Management;
  6. the User Command (CAE-UCM) Data Type, with a request and a response.

The functionality of Audio Scene Management is as described in Introduction. Terms beginning with a capital letter are defined in Table 1.

1 The Audio Scene Management Composite AIM

The Audio Scene Management Composite AIM (CAE-ASM) shall:

  1. receive User Commands and edit Audio Objects, Audio Scenes, Speech Objects and Speech Scenes, as each User Command requests;
  2. identify Objects and Scenes by the name the User gave them, and accept as numbers every amount that a User Command carries;
  3. compose Audio Scenes and Speech Scenes into a Multimodal Scene;
  4. answer each User Command with a response that says how each of its actions was executed;
  5. receive and produce, for Objects and Scenes;
  6. render nothing.

Audio Scene Management receives, at the Ports of its boundary: User Command RQ (CAE-UCM) (Port Numbers 1 to 5), Audio Scene Descriptors (OSD-ASD) #1, Speech Scene Descriptors (OSD-SSD), Basic Speech Object (OSD-BSO) #1 (optional: the speech of the person). It produces, at the Ports of its boundary: Audio Object (OSD-AUO), Audio Scene Descriptors (OSD-ASD), Speech Object (OSD-SPO) #1, Speech Scene Descriptors (OSD-SSD), Multimodal Scene Descriptors (OSD-MSD), User Command RS (CAE-UCM) (Port Numbers 1 to 5), Basic Speech Object (OSD-BSO) #1 (optional: the response to the person, spoken), Basic Text Object (OSD-BTO) #2 (optional: the response to the person, as text), Basic Text Object (OSD-BTO) #1 (optional: what the person said, as text). The Ports, with their Port Numbers, are those of Table 1 of the Audio Scene Management page; each editing AIM has its own User Command Port. The User Point of View and the Objects to acquire are carried by the User Command (UserPoV, AcquiredObject).

The Composite AIM shall be made of the AIMs specified in Section 2, connected as the Topology of its JSON metadata states. It shall be run by an AIF Controller, as described in Introduction, and its Ports shall be addressed by Data Type and Port Number.

2 The AIMs of Audio Scene Management

The functionality, the inputs and the outputs of each AIM are specified by its page, to which the following paragraphs refer.

  1. Audio Object Editing (CAE-AOE) shall edit the open Audio Object as a User Command requests, shall change only the features that the User Command schema lists, and shall output the open Audio Object and the response to the User Command.
  2. Audio Scene Editing (CAE-ASE) shall compose and edit the open Audio Scene as a User Command requests, shall add, move, change and remove Audio Objects in it at the positions and orientations the User Command gives as numbers, and shall output the open Audio Scene Descriptors and the response to the User Command.
  3. Speech Object Editing (MMC-SOE), specified by MPAI-MMC V2.5, shall edit the open Speech Object, and shall have it translated by Text and Speech Translation.
  4. Speech Scene Editing (MMC-SSE), specified by MPAI-MMC V2.5, shall compose and edit the open Speech Scene as Audio Scene Editing does the open Audio Scene.
  5. Multimodal Scene Editing (OSD-MSE), specified by MPAI-OSD V1.5, shall compose the Audio Scene Descriptors and the Speech Scene Descriptors into the Multimodal Scene Descriptors.
  6. Voice Command Interface (MMC-VCI) is specified by MPAI-MMC V2.5, not by CAE-ASM: its functionality, inputs and outputs are as MPAI-MMC V2.5 specifies. Audio Scene Management uses it to receive vocal commands, as Section 5 specifies.
  7. Text and Speech Translation (MMC-TST), specified by MPAI-MMC V2.5, shall translate the Speech Object it receives from Speech Object Editing into the language of the Language Selector it receives, and shall return the translation.

Each AIM shall have the Ports that its page specifies, addressed by Data Type and Port Number, and shall accept and produce the Data Types of those Ports with the semantics that the Data Type pages specify.

3 Acquisition

Audio shall be acquired from a disk or from a network. Acquisition is a functionality of the User Agent, performed by an Acquisition Unit of its Physical Layer: a UA Unit. Audio Object Acquisition (CAE-AOA) specifies the UA Unit that acquires Audio Objects. The acquisition of speech is specified by MPAI-MMC: Speech Object Acquisition (MMC-SOA) specifies the UA Unit that acquires Speech Objects. The acquired Objects and Scenes shall be given by the User Agent to the Ports of the boundary of Audio Scene Management.

4 Delivery

Delivery is performed by Delivery Units of the Physical Layer of the User Agent, which are UA Units, and not by the AIMs of Audio Scene Management. The Delivery Units are specified by CAE-ASM, MPAI-MMC and MPAI-OSD:

  1. Audio Objects and Audio Scene Descriptors shall be delivered by the UA Unit Audio Object Delivery (CAE-AOD) of CAE-ASM;
  2. Speech Objects and Speech Scene Descriptors shall be delivered by the UA Unit Speech Object Delivery (MMC-SOD) of MPAI-MMC;
  3. Multimodal Objects and Multimodal Scene Descriptors shall be delivered by the UA Unit Multimodal Object Delivery (OSD-MOD) of MPAI-OSD.

5 The user interface

The user interface to Audio Scene Management shall be implemented by the User Agent. The User Agent shall give User Commands (User Command RQ (CAE-UCM)), which carry the User Point of View and the Objects to acquire (UserPoV, AcquiredObject), to the Ports of the boundary of Audio Scene Management that address the editing AIMs they are for, by Data Type and Port Number, and shall obtain the responses (User Command RS (CAE-UCM)) from the Ports that address the same editing AIMs.

Vocal commands may be issued to Audio Scene Management through the Voice Command Interface (Voice Command Interface, MMC-VCI). Audio Scene Management shall accept the speech of the person at Basic Speech Object (OSD-BSO) #1 and give it to the Voice Command Interface, shall give the response to the person, spoken, at Basic Speech Object (OSD-BSO) #1 and as text at Basic Text Object (OSD-BTO) #2, and what the person said, as text, at Basic Text Object (OSD-BTO) #1; it shall give the User Command request that the Voice Command Interface forms to the five editing AIMs, and shall return their User Command responses to the Voice Command Interface. These Ports are optional.

6 Remote use through MPAI-MAS

The user interface and the User Agent may be remote. Where they are, the User Agent shall be a Remote Client of MPAI as a Service (MPAI-MAS), the Module Audio Scene Management shall be run by a Controller of the Service, and the User Commands, the Objects and the Scenes shall cross the boundary of the Module as MPAI-MAS specifies, addressed by Data Type and Port Number (MPAI as a Service in MPAI-AIF).

7 Reliance on other standards

CAE-ASM V1.0 relies on the following MPAI standards:

  1. Technical Specification: Artificial Intelligence Framework (MPAI-AIF) V3.0 specifying the AI Framework (AIF) where Audio and Speech Data are processed by AI Modules (AIM), the Controller and the User Agent.
  2. Technical Specification: Process Instance Trust Framework (MPAI-PTF) V1.0 specifying the data structures, processes, and protocols enabling AIMs to establish, evaluate, and maintain trust in the AIF.
  3. Technical Specification: Object and Scene Description (MPAI-OSD) V1.5 specifying the Audio, Speech and Multimodal Objects and Scene Descriptors, Multimodal Scene Editing and Multimodal Object Delivery.
  4. Technical Specification: Multimodal Conversation (MPAI-MMC) V2.5 specifying Speech Object Editing, Speech Scene Editing, Text and Speech Translation, and Speech Object Acquisition and Delivery.
  5. Technical Specification: Data Types, Formats, and Qualifiers (MPAI-TFA) V1.5 specifying the Audio and Speech Qualifiers.
  6. Technical Specification: MPAI as a Service (MPAI-MAS) V1.0 specifying how a Remote Client uses a Module.

An Implementation of an AIM conforms with CAE-ASM V1.0 if it accepts as input and produces as output Data and Data Objects conforming with those specified by the page of the AIM. CAE-ASM was developed by the CAE-DC Development Committee. MPAI may develop CAE-ASM extensions or new Technical Specifications in the CAE-ASM work area.

<-Introduction Go to ToC Definitions ->