1     Functions

2     Reference Model

3     I/O data of AI Workflow.

4     Functions of AI Modules

5     I/O Data of AI Modules

6     AIW, AIMs, and JSON Metadata

1      Functions

The Enhanced Audioconference Experience Use Case addresses the situation where one or more speakers are active in a noisy meeting room and are trying to communicate with one or more interlocutors using speech over a network. In this situation, the user experience is very often far from satisfactory due to multiple competing speakers, non-ideal acoustical properties of the physical spaces that the speakers oc­cupy and/or background noise. These can lead to a reduction in intelligibility of speech resulting in participants not fully understanding what their interlocutors are saying, in addition to creating a distraction and eventually leading to what is known as audioconference fatigue. When microphone arrays are used to capture the speakers, most of the described problems can be resolved by appropriate processing of the captured signals. The speech signals from multiple speakers can be separated from each other, the non-ideal acoustics of the space can be reduced, and any background noise can be substantially suppressed.

CAE-EAE is concerned with extracting from microphone array recordings the speech signals from individual speakers as well as reducing the background noise and the reverberation that reduce speech intelligibility. CAE-EAE also extracts the Spatial Attitudes of the speakers with respect to the position of the microphone array to facilitate the spatial representation of the speech signals at the receiver side if necessary. These Spatial Attitudes are represented in the Audio Scene Geometry format and packaged in a format that is amenable to further processing for efficient delivery and further processing. Data reduction of the extracted speech signals as well as their reconstruction/representation at the receiver side are outside the scope of this Use Case.

CAE-EAE aims to provide a complete solution to process speech signals recorded by microphone arrays to provide clear speech signals substantially free from back­ground noise and acoustics-related artefacts to improve the auditory quality of audioconference experience. Thus, CAE-EAE improves auditory experience in an audioconference, thereby reducing the effects of audioconference fatigue.

2      Reference Model

Figure 5 shows the Reference Model for the CAE-EAE.

Figure 5 – Enhanced Audioconference Experience Reference Model

3      I/O data of AI Workflow

Table 14 shows the input and output data for the Enhanced Audioconference Experience workflow.

Table 14 – I/O data of Enhanced Audioconference Experience

Inputs Comments
Microphone Array Geometry A Data Type representing the position of each microphone comprising a Microphone Array and specific characteristics such as microphone type, look directions, and the array type.
Microphone Array Audio A Data Type whose structure contains between 4 and 256 time-aligned interleaved Audio Channels organised in blocks.
Outputs Comments
Multichannel Audio Stream Interleaved Multichannel Audio packaged with Time Code as specified in Multichannel Audio Stream.

The Enhanced Audio Experience AIW:

  1. Receives:
    • Microphone Array Geometry which describes the number, positioning, and configuration of the microphone(s). Using this information, the system can detect the relative directions of the active speakers according to the microphone array and separate relevant audioconference speech sources from each other and from other spurious sounds. Since audio conferencing is a real-time application scenario, the use case operates on Audio Blocks.
    • Microphone Array Audio which is input to EAE as short Multichannel Audio Blocks comprising real valued time domain audio samples where the number of audio samples in each Audio Block is the same for all the microphones.
  2. Produces Multichannel Audio Stream.

4      Functions of AI Modules

The AIMs required by the Enhanced Audioconference Experience are given in Table 15.

Table 15 – AIMs of Enhanced Audioconference Experience

AIM Function
Audio Analysis Transform Represents the input Multichannel Audio in a new form amenable to further processing by the subsequent AIMs in the architecture.
Sound Field Description Produces Spherical Harmonic Decomposition Coefficients of the Transformed Multichannel Audio.
Speech Detection and Separation Separates speech and non-speech signals in the Spherical Harmonic Decomposition producing Transform Speech and Audio Scene Geometry.
Noise Cancellation Module Removes noise and/or suppresses reverberation in the Transform Speech producing Enhanced Transform Audio.
Audio Synthesis Transform Effects inverse transform of Enhanced Transform Audio producing Enhanced Audio Objects ready for packaging.
Audio Description Packaging Multiplexes Enhanced Audio Objects and the Audio Scene Geometry.

The EAE use case receives Microphone Array Audio and Microphone Array Geometry which describes the number, positioning, and configuration of the microphone(s). Using this information, the system can detect the relative directions of the active speakers according to the microphone array and separate relevant audioconference speech sources from each other and from other spurious sounds. Since audio conferencing is a real-time application scenario, the use case operates on Audio Blocks.

The Multichannel Audio is input to EAE as short Multichannel Audio Blocks comprising real valued time domain audio samples where the number of audio samples in each audio block is the same for all the microphones.

The sequence of operations of the EAE use case is the following:

5      I/O Data of AI Modules

Table 16 – CAE-EAE AIMs and their data

AIM Input Data Output Data
Audio Analysis Transform Microphone Array Audio Transform Multichannel Audio
Sound Field Description Transform Multichannel Audio

 

Spherical Harmonic Decomposition Coefficients
Speech Detection and Separation Spherical Harmonic Decomposition Coefficients Transform Audio

Audio Scene Geometry

Noise Cancellation Module Spherical Harmonic Decomposition Coefficients

Transform Audio

Audio Scene Geometry

Enhanced Transform Audio
Audio Synthesis Transform Enhanced Transform Audio Enhanced Audio Objects
Audio Description Packaging Enhanced Audio Objects

Audio Scene Geometry

Multichannel Audio Stream

6      AIW, AIMs, and JSON Metadata

 

Table 17 – AIW, AIMs, and JSON Metadata

AIW AIMs Names JSON
CAE-EAE Enhanced Audioconference Experience File
CAE-AAT Audio Analysis Transform File
CAE-SFD Sound Field Description File
CAE-SDS Speech Detection and Separation File
CAE-NCM Noise Cancellation Module File
CAE-AST Audio Synthesis Transform File
CAE-ADP Audio Description Packaging File