AI Modules (AIM) organised in AI Workflows (AIW) executed in the AI Framework enabling initialisation, dynamic configuration, and control of AIWs are a key element of the MPAI approach to AI-based Data Coding standards as depicted in Figure 1. AIMs communicate to other AIMs in the AIW the Data obtained by executing specific functions.
The effectiveness of the functions performed by the AIMs is improved if they know more about the capabilities of the AIMs they are connected to and the Data they receive as demonstrated by natural language processing. An instance of the MPAI Natural Language Processing (MMC-NLU) AIM can produce the recognised text and Meaning using three levels of information:
1.Just the input text.
2.Also the object identifiers referenced in the text.
3.Additionally, the object context in a relevant space.
The accuracy of the refined text and Meaning produced by an MPAI-NLU AIM is expected to improve when moving from the first to the third case. The cases correspond with different levels of AIM capabilities.
Technical Specification: AI Module Profiles (MPAI-PRF) enables an AIM instance to signal its Attributes – such as input data, output data, and functionality – and Sub-Attributes – such as languages supported by a Text and Speech Translation AIM – that uniquely characterise the AIM. Currently, MPAI-PRF defines the Attributes of eight AIMs but Profiles for more AIMs are likely to be defined in the future.
The effectiveness of the functions performed by an AIM can also be enabled or enhanced if the AIM knows more about the characteristics of the Data received. Examples of characteristics include:
- The CIE 1931 colour space of an instance of the Visual Data Type.
- The MP3 format of a speech segment.
- The WAV file format of an audio segment.
- The gamma correction applied to the device that produced a video.
- The ID of an object instance in an audio segment.
- The Text conveyed by a speech segment.
Technical Specification: Data Types, Formats, and Attributes (MPAI-TFA) V1.0 specifies a new Data Type called Qualifier, a container that can be used to represent, for instance, that a Visual Data Type instance:
- Uses a given colour space (Sub-Type)
- Was produced by an AVC codec (Format).
- Is described by Dublin Core Metadata (Attribute).
The current versions of MPAI Technical Specifications generally assume that most of the Media Objects exchanged by AIMs are composed of “Content” and “Qualifiers”.
Therefore, Qualifiers are a specialised type of metadata intended to support the operation of AIMs receiving data from other AIMs and conveying information on Sub-Types, Formats, and Attributes related to the Content. Qualifiers are intended to convey information for use by an AIM. They are human-readable but intended only to be used by AIMs.
MPAI also provides a standard method to attach information to a Data Type instance called Annotation defined as Data attached to an Object or a Scene. As opposed to Qualifier that describes intrinsic properties of a Data Type, an Annotation is spatially and temporally local and changeable.
Future versions of MPAI-TFA will likely be published because of the large variety of application needs that will require the specification of Qualifiers for additional Data Types. MPAI-TFA users are invited to communicate their need for extension of existing and specification of additional Data Types in MPAI-TFA to the MPAI Secretariat. Therefore, versioning of Qualifiers is a critical component of MPAI-TFA.
