All posts

At its 43^rd General Assembly of 2024 April17, MPAI approved the publication of the draft AI Module Profiles (MPAI-PRF) standard with a request for Community Comments. The scope of MPAI-PRF is to provide a solution to the problem that MPAI finds more and more often: the AI Modules (AIM) it specifies in different standards have the same basic functionality but may have different features.

First two words about AIMs. MPAI develops application-oriented standards for applications that MPAI calls AI Workflows (AIW) that can be broken down into components called Ai Modules. AIWs are specified by what they do (functions), by the input and output data and by how its AIMs are interconnected (Topology). Similarly, AIMs are specified by what they do (functions) and by the input and output data. AIMs are Composite if they include interconnected AIMs or Basic if its internal structure is unknown.

Let’s look at the Natural Language Understanding (MMC-NLU) AIM of Figure 1.

Figure 1 – The Natural Language Understanding (MMC-NLU) AIM

The NLU AIM’s basic function is to receive a Text Object – directly from a keyboard or through an Automatic Speech Recognition (ASR) AIM (in which case it is called Recognised Text) and produce a Text Object that can be Refined Text in case it is the output of an ASR AIM and the Meaning of the text. The NLU AIM, however, can also receive “spatial information” about the Audio and/or Visual Objects in terms of their position and orientation in the Scene that the machine is processing. Obviously, this additional information helps the machine produce a response that is more attuned to the context.

This a case shows that there is a need to unambiguously name these two functionally equivalent but very different instances of the same NLU AIMs.

The notion of Profiles, originally developed by MPEG in the summer of 1992 for the MPEG-2 standard and then universally adopted in the media domain comes in handy. An AIM Profile is a label that uniquely identifies the set of AIM Attributes of an AIM instance where Attribute is “input data, output data, or functionality that uniquely characterises an AIM instance”. In the case of the NLU AIM, Text Object (TXO), Recognised Text (TXR), Object Instance Identifier (OII), Audio-Visual Scene Geometry (AVG), and Meaning (TXD or Text Descriptors).

The Draft AI Module Profiles (MPAI-PRF) Standard offers two ways to signal the Attributes of an AIM: those that are supported or those that are not supported. Both can be used, but likely the first (list of those that are supported) if it is shorter than the second (list of those that are not supported) and vice versa. The Profile of an NLU AIM instance that does not handle spatial information can thus be labelled in two ways:

V2.1 refers to the version of the Multimodal Conversation MPAI-MMC standards that specifies the NLU AIM. ALL signals that the Profile is expressed in “negative logic” in the sense that the removed Attributes are AVG for Audio-Visual Scene Geometry and OII. NUL signals that the Profile is expressed in “positive logic” in the sense that the added Attributes are TXO for Text Object from a keyboard and TXR for Recognised Text.

The Profile story does not end here. Attributes are not always sufficient to identify the capabilities of an AIM instance. Let’s take the Entity Dialogue Processing (MMC-EDP) of Figure 2 an AIM that uses different information sources derived from the information issued by an Entity, typically a human – but potentially also a machine – with which this machine is communicating.

Figure 2 – The  Entity Dialogue Processing (MPAI-EDP)

The input data is Text Object and Meaning (output of the NLU), Audio or Visual Instance ID and Scene Geometry (already used by the NLU AIM) and Personal Status, a data type that represents the internal state of the Entity in terms of three Factors (Cognitive State, Emotion, and Social Attitude) and four Modalities (Text, Speech, Face, and Body) for each Factor.

The output of the EDP AIM is Text that can be fed to a regular Text-To-Speech AIM, but can additionally be the machine’s Personal Status, obviously pretended by the machine, but of great value for the Personal Status Display (PAF-PSD) AIM depicted in Figure 3.

Figure 3 – The Personal Status Display AIM

This uses the machine’s Text and Personal Status (IPS) to synthesise the machine using an Avatar Model (AVM) as a speaking avatar. An AIM instance of the PSD AIM may support the Personal Status, but only its Speech (PS-Speech, PSS) and Face (PS-Face, PSF) Factors, as in the case of a PSD AIM designed for sign language. This is formally represented by the following two expressions:

@IPS#PSS#PSF in the first expression indicates that the PSD AIM supports all Attributes, but the Personal Status only includes the Speech and Face Factors. In the second expression +TXO+AVM indicates that the PSD AIM supports Text and Avatar Model and @IPS#PSF#PSG that the Personal Status Factors supported are Face (PSF) and Gesture (PSG).

AI Module Profiles is another element of the AI application infrastructure that MPAI is building with its standards. Read the AI Module Profiles standard for an in-depth understanding. Anybody can submit comments to the draft by sending an email to the MPAI secretariat by 2024/05/08T23:59. MPAI will consider each comment received for possible inclusion in the final version of MPAI-PRF.

Geneva, Switzerland – 17 April 2024. MPAI – Moving Picture, Audio and Data Coding by Artificial Intelligence – the international, non-profit, and unaffiliated organisation developing AI-based data coding standards has concluded its 43^rd General Assembly (MPAI-43) approving the publication of the draft AI Module Profile V1.0 Standard with a request for Community Comments.

AI Module Profiles (MPAI-PRF) V1.0 enables the signalling of AI Module Attributes – input data, output data, or functionality – that uniquely characterise an AIM instance. An AIM Profile is thus a label that uniquely identifies the set of AIM Attributes that are either supported or not supported by that AIM instance. Anybody can submit comments to the draft by sending an email to the MPAI secretariat by 2024/05/08T23:59.

MPAI also informs that the code, the presentation file, and the video recording of the V1.1 version of the Neural Network Watermarking (MPAI-NNW) Reference Software Specification presented  of the on the 16^th of April are now publicly available. The software enables a user to make queries that include a text and an image and obtain a watermarked vocal response that enables the issuer of the query to ascertain that the response is from the intended source. The second software can be used to run watermarked AI-based applications on resource-constrained processing platforms without significant performance loss.

MPAI is continuing its work plan that involving the following activities:

1. AI Framework (MPAI-AIF): developing open-source applications based on the AI Framework.
2. AI for Health (MPAI-AIH): developing the specification of a system enabling clients to improve models processing health data and federated learning to share the training.
3. Context-based Audio Enhancement (CAE-DC): preparing new projects.
4. Connected Autonomous Vehicle (MPAI-CAV): Functional Requirements of the data used by the MPIA-CAV – Architecture standard.
5. Compression and Understanding of Industrial Data (MPAI-CUI): preparation for an extension to existing standard that includes support for more corporate risks.
6. End-to-End Video Coding (MPAI-EEV): video coding using AI-based End-to-End Video coding.
7. AI-Enhanced Video Coding (MPAI-EVC). video coding with AI tools added to existing tools.
8. Human and Machine Communication (MPAI-HMC): developing reference software.
9. Multimodal Conversation (MPAI-MMC): developing reference software and conformance testing and exploring new areas.
10. MPAI Metaverse Model (MPAI-MMM): developing reference software specification and identifying metaverse technologies requiring standards.
11. Neural Network Watermarking (MPAI-NNW): reference software for enhanced applications.
12. Portable Avatar Format (MPAI-PAF): reference software, conformance testing and new areas.
13. Server-based Predictive Multiplayer Gaming (MPAI-SPG): technical report on mitigation of data loss and cheating.
14. XR Venues (MPAI-XRV): development of the standard.

Legal entities and representatives of academic departments supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data can become MPAI members.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.

Geneva, Switzerland – 20 March 2024. MPAI – Moving Picture, Audio and Data Coding by Artificial Intelligence – the international, non-profit, and unaffiliated organisation developing AI-based data coding standards has concluded its 42^nd General Assembly (MPAI-42) approving the release of Reference Software using Neural Network Watermarking for Generative AI applications.

The new V1.1 version of the Neural Network Watermarking (MPAI-NNW) Reference Software includes an implementation of the AIF Framework and of an AI Workflow enabling a user to make queries that include a text and an image and obtain a vocal response. This inference is watermarked, to enable the issuer of the query to ascertain that the response they receive is from the intended source. The Software will be presented online on the 16^th of April at 15 UTC. Register at https://us06web.zoom.us/meeting/register/tZ0udeutqT0vHdBh1DLiUxoRr59cUs7iQzzN.

Presentations and video recordings of all MPAI standards are available (ppt= PowerPoint file), YT=YouTube, nYT=WimTV):

MPAI is continuing its work plan that involving the following activities:

1. AI Framework (MPAI-AIF): developing open-source applications based on the AI Framework.
2. AI for Health (MPAI-AIH): developing the specification of a system enabling clients to improve models processing health data and federated learning to share the training.
3. Context-based Audio Enhancement (CAE-DC): preparing new projects.
4. Connected Autonomous Vehicle (MPAI-CAV): Functional Requirements of the data used by the MPIA-CAV – Architecture standard.
5. Compression and Understanding of Industrial Data (MPAI-CUI): preparation for an extension to existing standard that includes support for more corporate risks.
6. End-to-End Video Coding (MPAI-EEV): video coding using AI-based End-to-End Video coding.
7. AI-Enhanced Video Coding (MPAI-EVC). video coding with AI tools added to existing tools.
8. Human and Machine Communication (MPAI-HMC): developing reference software.
9. Multimodal Conversation (MPAI-MMC): developing reference software and conformance testing and exploring new areas.
10. MPAI Metaverse Model (MPAI-MMM): developing reference software specification and identifying metaverse technologies requiring standards.
11. Neural Network Watermarking (MPAI-NNW): reference software for enhanced applications.
12. Portable Avatar Format (MPAI-PAF): reference software, conformance testing and new areas.
13. Server-based Predictive Multiplayer Gaming (MPAI-SPG): technical report on mitigation of data loss and cheating.
14. XR Venues (MPAI-XRV): development of the standard.

Legal entities and representatives of academic departments supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data can become MPAI members.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.

In the last few months, MPAI has published eight new or update MPAI standards. They have been presented online in the 11-15 March 2024 week.

Here are the titles of the standards with links to the presentations and video recording provided by two services. They are a good opportunity to stay abreast of the progress in MPAI

Geneva, Switzerland – 21 February 2024. MPAI – Moving Picture, Audio and Data Coding by Artificial Intelligence – the international, non-profit, and unaffiliated organisation developing AI-based data coding standards has concluded its 41^st General Assembly (MPAI-41) approving the publication of two standards and announcing the availability of all its standards in linked form on the web.

Context-based Audio Enhancement (MPAI-CAE) V2.1 extends the previously published Version 2.0 adding full online references to the specification of all AI Workflows, AI Modules, JSON Metadata, and Data Types used by the standard.

Human and Machine Communication (MPAI-HMC) V1.0 integrates a wide range of technologies from existing MPAI standards to enable new forms of communication between entities, i.e., humans present or represented in a real or virtual space or machines represented in a virtual space as speaking avatars and acting in a context using text, speech, face, gesture, and audio-visual scene in which they are embedded. It.

In the 11-15 March week, MPAI will be presenting its recently published standards at a series of planned 40-min online sessions. The presentations will illustrate the scope, the features, and the technologies of each standard and will be followed by open discussions. The new web-based access to all published MPAI standards will also be presented. All times are UTC

MPAI is continuing its work plan that involving the following activities:

1. AI Framework (MPAI-AIF): developing open-source applications based on the AI Framework.
2. AI for Health (MPAI-AIH): developing the specification of a system enabling clients to improve models processing health data and federated learning to share the training.
3. Context-based Audio Enhancement (CAE-DC): preparing new projects.
4. Connected Autonomous Vehicle (MPAI-CAV): Functional Requirements of the data used by the MPIA-CAV – Architecture standard.
5. Compression and Understanding of Industrial Data (MPAI-CUI): preparation for an extension to existing standard that includes support for more corporate risks.
6. Human and Machine Communication (MPAI-HMC): developing reference software.
7. Multimodal Conversation (MPAI-MMC): developing reference software and conformance testing, and exploring new areas.
8. MPAI Metaverse Model (MPAI-MMM): developing reference software specification and identifying metaverse technologies requiring standards.
9. Neural Network Watermarking (MPAI-NNW): reference software for enhanced applications.
10. Portable Avatar Format (MPAI-PAF): reference software, conformance testing and new areas.
11. End-to-End Video Coding (MPAI-EEV): video coding using AI-based End-to-End Video coding.
12. AI-Enhanced Video Coding (MPAI-EVC). video coding with AI tools added to existing tools.
13. Server-based Predictive Multiplayer Gaming (MPAI-SPG): technical report on mitigation of data loss and cheating.
14. XR Venues (MPAI-XRV): development of the standard.

Legal entities and representatives of academic departments supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data can become MPAI members.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.

At its 40^th General Assembly (MPAI-40), MPAI approved one draft, one new, and three extension standards. For an organisation that has already nine standards in its game bag, this may not look like big news. There are two reasons, though, to consider this a remarkable moment in the MPAI short but intense life.

Geneva, Switzerland – 24 January 2024. MPAI – Moving Picture, Audio and Data Coding by Artificial Intelligence – the international, non-profit, and unaffiliated organisation developing AI-based data coding standards has concluded its 40^th General Assembly (MPAI-40) approving the publication of a range of standards covering disparate technologies and application domains.

Human and Machine Communication (MPAI-HMC) is a draft published for Community Comments, the last step before publication. It includes a wide range technologies available from existing MPAI standards to enable an Entity, i.e., a human or a machine, to hold a communication with Entities as humans do. Comments are due by 2024/02/19T23:59 UTC to secretariat@mpai.community.

The newly-approved Object and Scene Description  (MPAI-OSD) V1.0 standard provides important technologies enabling the digital representation of position and orientation of Audio and Visual Objects and their combinations in Scenes. The MPAI-OSD capabilities enhance usability of the new Multimodal Conversation (MPAI-MMC) V2.1 and Portable Avatar Format (MPAI-PAF) V1.1.

MPAI Metaverse Model – Architecture (MPAI-MMM) V1.1 updates the MMM- Architecture Metadata to streamline communication between the Processes of a Metaverse Instance and uses the new MPAI-MMM Scripting Language (MMM-Script) to represent a wide range of use cases.

MPAI is now offering an innovative way to access to its new standards via the web:

MPAI-HMC: https://mpai.community/standards/mpai-hmc/mpai-hmc-specification/

MPAI-MMC: https://mpai.community/standards/mpai-mmc/mpai-mmc-specification/

MPAI-OSD: https://mpai.community/standards/mpai-osd/mpai-osd-specification/

MPAI-PAF: https://mpai.community/standards/mpai-paf/mpai-paf-specification/

MPAI is continuing its work plan that involving the following activities:

1. AI Framework (MPAI-AIF): reference software, conformance testing, and application areas.
2. AI for Health (MPAI-AIH): reference model and technologies for a system enabling clients to improve models processing health data and federated learning to share the training.
3. Context-based Audio Enhancement (CAE-DC): new projects are brewing.
4. Connected Autonomous Vehicle (MPAI-CAV): Functional Requirements of the data used by the MPIA-CAV – Architecture standard.
5. Compression and Understanding of Industrial Data (MPAI-CUI): preparation for an extension to existing standard that includes support for more corporate risks.
6. Human and Machine Communication (MPAI-HMC): model and technologies enabling a human or a machine to communicate with a machine or a human in a different cultural environment.
7. Multimodal Conversation (MPAI-MMC): drafting reference software and conformance testing, and exploring new areas.
8. MPAI Metaverse Model (MPAI-MMM): reference software and metaverse technologies requiring standards.
9. Neural Network Watermarking (MPAI-NNW): reference software for enhanced applications.
10. Portable Avatar Format (MPAI-PAF): reference software, conformance testing and new areas.
11. End-to-End Video Coding (MPAI-EEV): video coding using AI-based End-to-End Video coding.
12. AI-Enhanced Video Coding (MPAI-EVC). video coding with AI tools added to existing tools.
13. Server-based Predictive Multiplayer Gaming (MPAI-SPG): technical report on mitigation of data loss and cheating.
14. XR Venues (MPAI-XRV): development of the standard.

Legal entities and representatives of academic departments supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data can become MPAI members.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.

Geneva, Switzerland – 20 December 2023. MPAI, Moving Picture, Audio and Data Coding by Artificial Intelligence, the international, non-profit, and unaffiliated organisation developing AI-based data coding standards has concluded its 39^th General Assembly (MPAI-39) approving the publication of the Context-based Audio Enhancement standard and Object and Scene Description standard for Community Comments.

The draft of the Context-based Audio Enhancement (MPAI-CAE) Version 2.1 standard enhances the compatibility of the Audio with the Visual and the Audio-Visual Scene Description specified by the draft Object and Scene Description (MPAI-OSD) standard. Both are published with requests for Community Comments. These are due by 2024/01/23T23:59 UTC and 17T23:58 UTC, respectively, to secretariat@mpai.community.

MPAI is continuing its work plan that involving the following activities:

1. AI Framework (MPAI-AIF): reference software, conformance testing, and application areas.
2. AI for Health (MPAI-AIH): reference model and technologies for a system enabling clients to improve models processing health data and federated learning to share the training.
3. Context-based Audio Enhancement (CAE-DC): new projects are brewing.
4. Connected Autonomous Vehicle (MPAI-CAV): Functional Requirements of the data used by the MPIA-CAV – Architecture standard.
5. Compression and Understanding of Industrial Data (MPAI-CUI): preparation for an extension to existing standard that includes support for more corporate risks.
6. Human and Machine Communication (MPAI-HMC): model and technologies enabling a human or a machine to communicate with a machine or a human in a different cultural environment.
7. Multimodal Conversation (MPAI-MMC): drafting reference software and conformance testing, and exploring new areas.
8. MPAI Metaverse Model (MPAI-MMM): reference software and metaverse technologies requiring standards.
9. Neural Network Watermarking (MPAI-NNW): reference software for enhanced applications.
10. Portable Avatar Format (MPAI-PAF): reference software, conformance testing and new areas.
11. End-to-End Video Coding (MPAI-EEV): video coding using AI-based End-to-End Video coding.
12. AI-Enhanced Video Coding (MPAI-EVC). video coding with AI tools added to existing tools.
13. Server-based Predictive Multiplayer Gaming (MPAI-SPG): technical report on mitigation of data loss and cheating.
14. XR Venues (MPAI-XRV): development of the standard.

Legal entities and representatives of academic departments supporting the MPAI mission and able to contribute to the development of standards for the efficient use of data can become MPAI members.

Please visit the MPAI website, contact the MPAI secretariat for specific information, subscribe to the MPAI Newsletter and follow MPAI on social media: LinkedIn, Twitter, Facebook, Instagram, and YouTube.

In a previous article, we have described the Architecture of the MPAI Connected Autonomous Vehicle (CAV). The CAV’s Environment Sensing Subsystem (ESS) captures data of the environment with a variety of sensors and produces the Basic Environment Representation (BER) that is passed to the Autonomous Motion Subsystem (AMS). This exchanges (subsets of) the BER with other CAVs in range and uses the received information to produce the Full Environment Representation (FER). Then, the AMS can issue commands to the Motion Actuation Subsystem (MAS) to move the CAV toward the destination.

In a previous article, we have described the Architecture of the MPAI Metaverse Model (MPAI-MMM) where a Metaverse Instance (M-Instance) is defined as a set of Processes providing some or all the following functions (terms beginning with small letters are in the Universe and terms beginning witj a large letter are in an M-Instance:

1. To sense data from U-Locations.
2. To process the sensed data and produce Data.
3. To produce one or more M-Environments populated by Objects that can be either digitised or virtual, the latter with or without autonomy.
4. To process Objects from the M-Instance or potentially from other M-Instances to affect U-Locations (in the Universe) and/or M-Locations (in this or other M-Instances) using Object in ways that are:
   • Consistent with the goals set for the M-Instance.
   • Effected within the capabilities of the M-Instance.
   • Complying with the Rules set for the M-Instance and applicable laws.

At a first glance, it looks like the way a CAV’s BER and FER bear a lot of similarities with the M-Instance of the MMM Architecture as we can see from the comparative .

Table 1 – Comparison between M-Instance and CAV

We need to look more in detail into this “similarities”. Before proceeding, let’s recall two assumptions at the basis of MPAI-MMM – Architecture:

1. User is a type of Process that represents and acts on behalf of a human. A human may have more than one User in an M-Instance.
2. Persona is a rendered User.
3. User may have or acquire the Rights to perform an Action, e.g., to authenticate another User.

To do that, let’s consider the simple case of two CAVs: CAV_A and CAV_B respectively owned by human_A and human_B, where human_A is friend to human_B. human_A has two Users: User_A.1 who represents human_A in the Human-CAV Interaction (HCI) Subsystem (or M-Environment_A.1) and User_A.2 who represents human_A in the Autonomous Motion Subsystem (or M-Environment_A.2). Similarly, for human_B.

human_A wants to see the landscape seen by human_B in their CAV_B.

This is a simplified description of the workflow (a fuller workflow is in the MPAI=CAV – Architecture standard)

1. human_A requests User₁ (HCI) to take them to a destination.
2. User₁ requests User_A.2 (AMS) to take CAV_A to destination.
3. User₂
   • Gets the BER from CAV_A’s ESS (or M-Environment₃).
   • Computes the Route to Destination.
   • Issues a series of Commands to the MAS.
   • Authenticates its peer User₂.
   • Gets a subset of the BER from User₂.
   • Produces CAV_A’s FER.
4. User₁
   • Authenticates its peer User₂.
   • Renders their Persona in CAV_B (e.g., using advanced 3D rendering technologies).
   • Converses with human_B.
   • Watches CAV_B’s M-Location corresponding to the environment currently traversed by CAV_B.

This example is a first demonstration of the compatibility of an M-Instance produced by a CAV implementing the MPAI-CAV – Architecture standard with the MPAI-MMM – Architecture standard.