CAE-USC V2.2 AIW Enhanced Audioconference Experience

<-Go to AI Workflows       Go to ToC

1     Functions	2     Reference Model	3     I/O Data
4     Functions of AI Modules	5     I/O Data of AI Modules	6     AIW, AIMs, and JSON Metadata

1      Functions

The Enhanced Audioconference Experience Use Case addresses the situation where one or more speakers are active in a noisy meeting room and are trying to communicate with one or more interlocutors using speech over a network. In this situation, the user experience is very often far from satisfactory due to multiple competing speakers, non-ideal acoustical properties of the physical spaces that the speakers oc­cupy and/or background noise. These can lead to a reduction in intelligibility of speech resulting in participants not fully understanding what their interlocutors are saying, in addition to creating a distraction and eventually leading to what is known as audioconference fatigue. When microphone arrays are used to capture the speakers, most of the described problems can be resolved by appropriate processing of the captured signals. The speech signals from multiple speakers can be separated from each other, the non-ideal acoustics of the space can be reduced, and any background noise can be substantially suppressed.

CAE-EAE is concerned with extracting from microphone array recordings the speech signals from individual speakers as well as reducing the background noise and the reverberation that reduce speech intelligibility. CAE-EAE also extracts the Spatial Attitudes of the speakers with respect to the position of the microphone array to facilitate the spatial representation of the speech signals at the receiver side if necessary. These Spatial Attitudes are represented in the Audio Scene Geometry format and packaged in a format that is amenable to further processing for efficient delivery and further processing. Data reduction of the extracted speech signals as well as their reconstruction/representation at the receiver side are outside the scope of this Use Case.

CAE-EAE aims to provide a complete solution to process speech signals recorded by microphone arrays to provide clear speech signals substantially free from back­ground noise and acoustics-related artefacts to improve the auditory quality of audioconference experience. Thus, CAE-EAE improves auditory experience in an audioconference, thereby reducing the effects of audioconference fatigue.

2      Reference Model

Figure 1 shows the Reference Model for the CAE-EAE.

Figure 1 – Enhanced Audioconference Experience Reference Model

3      I/O data of AI Workflow

Table 1 shows the input and output data for the Enhanced Audioconference Experience workflow.

Table 1 – I/O data of Enhanced Audioconference Experience

Inputs	Comments
Microphone Array Geometry	A Data Type representing the position of each microphone comprising a Microphone Array and specific characteristics such as microphone type, look directions, and the array type.
Microphone Array Audio	A Data Type whose structure contains between 4 and 256 time-aligned interleaved Audio Channels organised in blocks.
Outputs	Comments
Multichannel Audio Stream	Interleaved Multichannel Audio packaged with Time Code as specified in Multichannel Audio Stream.

The Enhanced Audio Experience AIW:

1. Receives:
   • Microphone Array Geometry which describes the number, positioning, and configuration of the microphone(s). Using this information, the system can detect the relative directions of the active speakers according to the microphone array and separate relevant audioconference speech sources from each other and from other spurious sounds. Since audio conferencing is a real-time application scenario, the use case operates on Audio Blocks.
   • Microphone Array Audio which is input to EAE as short Multichannel Audio Blocks comprising real valued time domain audio samples where the number of audio samples in each Audio Block is the same for all the microphones.
2. Produces Multichannel Audio Stream.

4      Functions of AI Modules

The AIMs required by the Enhanced Audioconference Experience are given in Table 2.

Table 2 – AIMs of Enhanced Audioconference Experience

AIM	Function
Audio Analysis Transform	Represents the input Multichannel Audio in a new form amenable to further processing by the subsequent AIMs in the architecture.
Sound Field Description	Produces Spherical Harmonic Decomposition Coefficients of the Transformed Multichannel Audio.
Speech Detection and Separation	Separates speech and non-speech signals in the Spherical Harmonic Decomposition producing Transform Speech and Audio Scene Geometry.
Noise Cancellation Module	Removes noise and/or suppresses reverberation in the Transform Speech producing Enhanced Transform Audio.
Audio Synthesis Transform	Effects inverse transform of Enhanced Transform Audio producing Enhanced Audio Objects ready for packaging.
Audio Description Packaging	Multiplexes Enhanced Audio Objects and the Audio Scene Geometry.

The EAE use case receives Microphone Array Audio and Microphone Array Geometry which describes the number, positioning, and configuration of the microphone(s). Using this information, the system can detect the relative directions of the active speakers according to the microphone array and separate relevant audioconference speech sources from each other and from other spurious sounds. Since audio conferencing is a real-time application scenario, the use case operates on Audio Blocks.

The Multichannel Audio is input to EAE as short Multichannel Audio Blocks comprising real valued time domain audio samples where the number of audio samples in each audio block is the same for all the microphones.

The sequence of operations of the EAE use case is the following:

5      I/O Data of AI Modules

Table 3 specifies the I/O Data of CAE-EAE.

Table 3 – CAE-EAE AIMs and their data

AIM	Input Data	Output Data
Audio Analysis Transform	Microphone Array Audio	Transform Multichannel Audio
Sound Field Description	Transform Multichannel Audio	Spherical Harmonic Decomposition Coefficients
Speech Detection and Separation	Spherical Harmonic Decomposition Coefficients	Transform Audio Audio Scene Geometry
Noise Cancellation Module	Spherical Harmonic Decomposition Coefficients Transform Audio Audio Scene Geometry	Enhanced Transform Audio
Audio Synthesis Transform	Enhanced Transform Audio	Enhanced Audio Objects
Audio Description Packaging	Enhanced Audio Objects Audio Scene Geometry	Multichannel Audio Stream

6      AIW, AIMs, and JSON Metadata

Table 4 provides links to the AI Modules and JSON Metadata.

Table 4 – AIW, AIMs, and JSON Metadata

AIW	AIMs	Names	JSON
CAE-EAE		Enhanced Audioconference Experience	File
	CAE-AAT	Audio Analysis Transform	File
	CAE-SFD	Sound Field Description	File
	CAE-SDS	Speech Detection and Separation	File
	CAE-NCM	Noise Cancellation Module	File
	CAE-AST	Audio Synthesis Transform	File
	CAE-ADP	Audio Description Packaging	File

 

<-Go to AI Workflows       Go to ToC