<-Introduction Go to ToC Definitions ->

Technical Specification: Context-based Audio Enhancement (MPAI-CAE) – Audio Scene Management (CAE-ASM) V1.0 – in the following also called CAE-ASM V1.0, or simply CAE-ASM – specifies:

  1. The functionality, the inputs and the outputs of the Audio Scene Management Composite AIM (CAE-ASM) and of each of its AIMs;
  2. The Acquisition of Audio Objects, and, by reference to MPAI-MMC, of Speech Objects;
  3. The Delivery of Audio, Speech and Multimodal Objects and Scenes;
  4. The user interface to Audio Scene Management;
  5. The User Command (CAE-UCM) Data Type, with a request and a response.

The functionality of Audio Scene Management is as described in Introduction. Terms beginning with a capital letter are defined in Table 1.

1 The Audio Scene Management Composite AIM

The Audio Scene Management Composite AIM (CAE-ASM):

  1. Receives User Commands, optionally via a Text/Speech interface;
  2. Edits Audio Objects, Audio Scenes, Speech Objects and Speech Scenes, as each User Command requests;
  3. Identifies Objects and Scenes by the name the User gave them, and accept as numbers every amount that a User Command carries;
  4. Composes Audio Scenes and Speech Scenes into a Multimodal Scene;
  5. Answer each User Command with a response that says how each of its actions was executed;
  6. Renders Objects and Scenes.

Audio Scene Management receives, at the Ports of its boundary:

  1. User Command RQ (CAE-UCM)
  2. Audio Scene Descriptors (OSD-ASD)
  3. Speech Scene Descriptors (OSD-SSD)
  4. Basic Text Object (OSD-BTO) or Basic Speech Object (OSD-BSO) (optionally the text or speech of the user).

It produces, at the Ports of its boundary:

2 The AIMs of Audio Scene Management

The functionality, the inputs and the outputs of each AIM are specified by its page, to which the following paragraphs refer.

  1. Audio Object Editing (CAE-AOE) edits the open Audio Object as a User Command requests, shall change only the features that the User Command schema lists, and shall output the open Audio Object and the response to the User Command.
  2. Audio Scene Editing (CAE-ASE) composes and edits the open Audio Scene as a User Command requests, shall add, move, change and remove Audio Objects in it at the positions and orientations the User Command gives as numbers, and shall output the open Audio Scene Descriptors and the response to the User Command.
  3. Speech Object Editing (MMC-SOE) edits the open Speech Object, and has it translated by Text and Speech Translation.
  4. Speech Scene Editing (MMC-SSE) composes and edits the open Speech Scene as Audio Scene Editing does the open Audio Scene.
  5. Multimodal Scene Editing (OSD-MSE) composes the Audio Scene Descriptors and the Speech Scene Descriptors into the Multimodal Scene Descriptors.
  6. Voice Command Interface (MMC-VCI) receives text and vocal commands.
  7. Text and Speech Translation (MMC-TST) translates the Speech Object it receives from Speech Object Editing into the language of the Language Selector it receives returning the translation.

3 Acquisition and Delivery

Audio shall be acquired from a disk or from a network. Acquisition is a functionality of the User Agent, performed by an Acquisition Unit of its Physical Layer: a UA Unit. Audio Object Acquisition (CAE-AOA) specifies the UA Unit that acquires Audio Objects. The acquisition of speech is specified by MPAI-MMC: Speech Object Acquisition (MMC-SOA) specifies the UA Unit that acquires Speech Objects. The acquired Objects and Scenes are given by the User Agent to the Ports of the boundary of Audio Scene Management.

Delivery is performed by Delivery Units of the Physical Layer of the User Agent, which are UA Units, and not by the AIMs of Audio Scene Management. The Delivery Units are specified by CAE-ASM, MPAI-MMC and MPAI-OSD:

  1. Audio Objects and Audio Scene Descriptors shall be delivered by the UA Unit Audio Object Delivery (CAE-AOD) of CAE-ASM;
  2. Speech Objects and Speech Scene Descriptors shall be delivered by the UA Unit Speech Object Delivery (MMC-SOD) of MPAI-MMC;
  3. Multimodal Objects and Multimodal Scene Descriptors shall be delivered by the UA Unit Multimodal Object Delivery (OSD-MOD) of MPAI-OSD.

4 The user interface

The user interface to Audio Scene Management is implemented by the User Agent. The User Agent shall give User Commands (User Command Request (CAE-UCM)), which carry the User Point of View and the Objects to acquire (UserPoV, AcquiredObject), to the Ports of the boundary of Audio Scene Management that address the relevant editing AIMs, by Data Type and Port Number, and obtains the responses (User Command Response (CAE-UCM)) from the Ports that address the same editing AIMs.

Text and Vocal commands may be issued to Audio Scene Management through the Voice Command Interface (Voice Command Interface, MMC-VCI). Audio Scene Management

  1. Accepts the text or speech of the user at Basic Text Object (OSD-BTO or Basic Speech Object (OSD-BSO) and give it to the Voice Command Interface.
  2. Gives the response to the user as text or speech at Basic Text Object (OSD-BTO) and Basic Speech Object (OSD-BSO)
  3. Gives the User Command request that the Voice Command Interface forms to the five editing AIMs (optional).
  4. Return their User Command responses to the Voice Command Interface (optional).

5 Reliance on other standards

CAE-ASM V1.0 relies on the following MPAI standards:

  1. Technical Specification: Artificial Intelligence Framework (MPAI-AIF) V3.0 specifying the AI Framework (AIF) where Audio and Speech Data are processed by AI Modules (AIM), the Controller and the User Agent.
  2. Technical Specification: Process Instance Trust Framework (MPAI-PTF) V1.0 specifying the data structures, processes, and protocols enabling AIMs to establish, evaluate, and maintain trust in the AIF.
  3. Technical Specification: Object and Scene Description (MPAI-OSD) V1.5 specifying the Audio, Speech and Multimodal Objects and Scene Descriptors, Multimodal Scene Editing and Multimodal Object Delivery.
  4. Technical Specification: Multimodal Conversation (MPAI-MMC) V2.5 specifying Speech Object Editing, Speech Scene Editing, Text and Speech Translation, and Speech Object Acquisition and Delivery.
  5. Technical Specification: Data Types, Formats, and Qualifiers (MPAI-TFA) V1.5 specifying the Audio and Speech Qualifiers.
  6. Technical Specification: MPAI as a Service (MPAI-MAS) V1.0 specifying how a Remote Client uses a Module.

An Implementation of an AIM conforms with CAE-ASM V1.0 if it accepts as input and produces as output Data and Data Objects conforming with those specified by the page of the AIM. CAE-ASM was developed by the CAE-DC Development Committee. MPAI may develop CAE-ASM extensions or new Technical Specifications in the CAE-ASM work area.

<-Introduction Go to ToC Definitions ->