MPAI publishes 2 draft standards and 1 document for comments

MPAI (https://mpai.community/) was established 11 months ago as an international, not-for-profit, unaffiliated standards developing organisation. Its mission is to develop Data Coding standards that primarily use Artificial Intelligence.

MPAI is currently working on 10 standards projects (https://mpai.community/standards/). The publicly available text of the working drafts of two of these standards is close to settled, and final approval is expected in a matter of weeks:

  1. Multimodal Conversation (https://mpai.community/standards/mpai-mmc/) comprising Conversation with Emotion, Multimodal Question Answering and 3 Speech Translation Use Cases, and
  2. Compression and Understanding of Industrial Data (https://mpai.community/standards/mpai-cui/) comprising the AI-based Company Prediction Use Case


Professional in these fields are invited to send comments on the drafts especially on the suitability of the standards in their current form and suggestions for future work.


Comments should be sent to MPAI Secretariat (secretariat@mpai.community) via email by 20 September 2021.

Comments will not be shared outside MPAI but considered by the Development Committees in charge of the standards. An individual response provided to you.

Comments shall not include any Intellectual Property (IP) matters. If any will be received, the Secretariat will return the email and not forward it to any MPAI member or Development Committee.

Those who believe they have IP in the areas where MPAI develops standards, should read the MPAI Statutes (https://mpai.community/statutes/) and consider joining MPAI (https://mpai.community/how-to-join/join/).

MPAI is also publishing for comments its foundational document titled Governance of the MPAI Ecosystem (https://mpai.community/governance/). This lays down the rules that will govern access to the implementations based on MPAI standards and on their attributes of Reliability, Robustness, Replicability and Fairness.

An introductory paper is available at https://mpai.community/2021/08/27/the-governance-of-the-mpai-ecosystem/.


MPAI engages in standards for AI-based Connected Autonomous Vehicles

For several decades, Autonomous Vehicles have been the target of research and experimentations in industry and academia. Since a decade, trials on real roads have been and are being conducted. Connected Vehicles are a reality today.

Standardisation of Connected Autonomous Vehicle (CAV) components will be required because of the size of the future CAV market (one estimate is 1.38 T$ in 2030). More importantly, users and regulators will need to be assured of the safety, reliability and explainability of CAV components.

In a traditional view of standardisation, the CAV state of development may not warrant space for CAV standardisation. However, MPAI heralds a more modern approach, one where a standard is the result of a continuous interaction between research providing results and standardisation building hypotheses to be proved or modified or disproved by subsequent research results.

MPAI has been working on the first steps of such a process. It has first partitioned a CAV in 5 functionally homogeneous subsystems as depicted in depicted in Figure 1.

Each of these subsystems has an architecture that is based on the emerging MPAI AI Framework (MPAI-AIF) and contains several components called AI Modules (AIM).

Figure 1 – The CAV subsystems

Figure 2 depicts the architecture of the 1st subsystem: Human-CAV interaction. A human may issue vocal commands to the CAV which are interpreted and sent to the Autonomous Motion Subsystem for action. A human may also entertain a dialogue with the CAV or with fellow passengers in the compartment or can indicate objects or make gestures that the CAV would understand and act upon accordingly.

Figure 2 – Human-CAV interaction (HCI)

The existence of an established organisation – MPAI – with a peculiar process to develop standard and actually developing them using that approach facilitates the implementation of the proposed plan. Indeed, MPAI fol­lows a rigorous process represented in Figure 3.

Figure 3 – Standard development process

The first 3 stages – Interest Collection, Use Case and Functional Requirements – are open to participation to non-members. Stage 4 – Commercial Requirements – is the prerogative of Principal Members. Stages 4 and 5 – Call for Technologies and Standard Development – are restricted to MPAI members. Stage 6 – MPAI standard – is again the prerogative of Principal Mem­bers. Note that MPAI membership is open to corporate entities and individuals representing academic depart­ments.

The MPAI-CAV project is currently at stage 3. This means that non-members can participate in the development of the functional requirements document which will provide the final CAV partitioning in subsystems; the functions performed by and the functional requirements of the I/O data of each subsystem; the partitioning of subsystems in AIMs, and the functions performed by and the functional requirements of the I/O data of each AIM. Independently produced results will be collectively assessed and used to design experiments executed by different participants in agreed conditions to provide comparable results.


MPAI has been established on 30 September 2020 as a not-for-profit unaffiliated organisation with the mission (https://mpai.community/statutes/)

  1. to dev­elop data coding standards based on Artificial Intelligence and
  2. to bridge the gap between standards and their practical use through Framework Licences.

MPAI develops its standards through a rigorous process depicted in following figure:

The MPAI standard development process

An MPAI standard passes through 6+1 stages. Anybody can contribute to the first 3 stages. The General
Assembly approves the progression of a standard to the next stage. MPAI defines standard interfaces of AI Modules (AIM) combined and executed in an MPAI-specified AI-Framework (AIF). AIMs receive
data with standard formats and produce output data with standard formats.

An MPAI AI Module (AIM) The MPAI AI Framework (AIF)

MPAI is currently developing 10 technical specifications.

In the following, the MPAI name and acronym and the scope of each standard are provided. The first 4 standards will be approved within 2021,

AI Framework – MPAI-AIF (Stage 5)

Specifies 6 elements: Management and Control, AIM, Execution, Communication, Storage and Access to enable creation and automation of mixed ML-AI-DP processing and inference workflows.

Context-based Audio Enhancement – MPAI-CAE (Stage 5)

Improves the user experience in audio applications, e.g., entertainment, communication, teleconferencing, gam­ing, post-production, restoration etc. for different con­texts, e.g., in the home, in the car, on-the-go, in the studio etc.

Multimodal Conversation – MPAI-MMC (Stage 5)

Enables human-machine conversation that emulates human-human conversation in completeness and intensity

Compression and understanding of industrial data – MPAI-CUI (Stage 5)

Enables AI-based filtering and extraction of governance, financial and risk data to predict company performance.

Server-based Predictive Multi­player Gaming – MPAI-SPG (Stage 3)

Minimises the audio-visual discontinuities caused by network disruption during an online real-time game and provides a response to the need to detect who amongst the players is cheating.

Integrative Genomic/Sensor Analysis – MPAI-GSA (Stage 3)

Understands and compresses the result of high-throughput experiments combining genomic/proteomic and other data, e.g., from video, motion, location, weather, medical sensors.

AI-Enhanced Video Coding – MPAI-EVC (Stage 3)

Substantially enhances the performance of a traditional video codec by improving or replacing traditional tools with AI-based tools.

Connected Autonomous Vehicles – MPAI-CAV (Stage 3)

Uses AI to enable a Connected Autonomous Vehicle with 3 subsystems: Human-CAV interaction, Auton­om­ous Motion Subsystem and CAV-Environment interac­tion

Visual object and scene des­crip­tion – MPAI-OSD (Stage 2)

A collection of Use Cases sharing the goal of describing visual object and locating them in the space. Scene descrip­tion includes the usual description of objects and their attrib­utes in a scene and the semantics of the objects.

Mixed-Reality Collaborative Spaces – MPAI-MCS (Stage 1)

Enables mixed-reality collaborative space scenarios where biomedical, scientific, and industrial sensor streams and recordings are to be viewed where AI can be utilised for immersive presence, spatial map ren­der­ing, multiuser synchronisation etc.

Additionally, MPAI is developing a standard titled “Governance of the MPAI ecosystem”. This will specify how:

  1. Implementers can get certification of the adherence of an implementation to an MPAI standard from the technical (Conformance) and ethical (Performance) viewpoint.
  2. End users can reliably execute AI workflows on their devices.


MPAI is currently developing 4 standards

Established less than 8 months ago – on 30 September 2020 – MPAI has promptly produced a process to develop its standards and it immediately put it in action.
In simple words, the MPAI process identifies the need for standards and determines functional requirements. Then it determines the commercial requirements (framework licences). Then it acquires technologies by issuing a public Call for technologies and developes the standard using the technologies proposed and evaluated.
MPAI is currently developing 4 standards, which means that the functional and commercial requirements have been developed, calls have been issued, and responses received in 4 instances:

  1. Artificial Intelligence Framework (MPAI-AIF) enables creation and automation of mixed ML-AI-DP processing and inference workflows. See https://mpai.community/standards/mpai-aif/
  2. Context-based Audio Enhancement (MPAI-CAE) improves the user experience for several audio-related applications including entertainment, communication, teleconferencing, gaming, post-production, restoration etc. in a variety of contexts. See https://mpai.community/standards/mpai-cae/
  3. Multi-modal conversation (MPAI-MMC) aims to enable human-machine conversation that emulates human-human conversation in completeness and intensity by using AI. See https://mpai.community/standards/mpai-mmc/

Compression and understanding of industrial data (MPAI-CUI) aims to predict the performance of a company by applying Artificial Intelligence to governance, financial and risk data. See https://mpai.community/standards/mpai-cui/

MPAI Use Cases being standardised – Emotion enhanced speech.

Imagine that you have a sentence uttered without a particular emphasis or emotion, that you have a sample sentence uttered with a particular intonation, emphasis and emotion and that you would like to have the emotion-less sentence uttered as in the sample sequence.

This is one of the use cases belonging to the Context-based Audio Enhancement standard that MPAI is developing as part of the process described above.
What is being standardised by MPAI in this Emotion-Enhanced Speech (EES) use case? The input and output interfaces of an EES box that takes a speech uttered without emotion (“emotion-less speech”), a segment of that speech between t1 and t2 and a sample speech containing the emotion, timbre etc. with which the segment of speech between t1 and t2 at the output of EES should be pronounced.
The EES does not stop here. It defines the architecture of the box, composed by AI Modules (AIM) of which only the functionality and the input and output data are defines, but not the internals of the AIM.
MPAI believes that this “lightweight” standardisation reaches two, apparently opposite goals: AIMs can be obtained from different sources and replaced with AIMs with more advanced functionalities.
MPAI standards not only do offer interoperability but also build on and further promote AI innovation.


Two weeks left to the Context-based Audio Enhancement and Multimodal Communication Calls for Technologies.

Last 17 February, MPAI issued two Calls for Technologies.

The Context-based Audio Enhancement (MPAI-CAE) https://bit.ly/3rqrvn1 Call comprises 4 Use Cases designed to improve the user exper­ien­ce for several audio applications in contexts such as in the home, in the car, on-the-go, in the studio etc. Usage examples are: adding a desired emotion to a speech without emotion, preserving old audio tapes, improving the audioconference experience, and removing unwanted sounds while keeping the relevant ones to a user walking in the street.
The Multimodal Conversation (MPAI-MMC) https://bit.ly/3tZqF2y Call comprises 3 Use Cases that use Artificial Intelligence (AI) to enable conver­sations between humans and mach­ines that emulate conversations between humans in completeness and intensity. Usage examples are: an audio-visual conversation with an emotion-understanding machine imper­sonated by a synthetic voice and an animated face, requesting  information about an object while displaying it, a human talking to a machine doing the  translation with  a voice that  preserves the speech features of the human.
MPAI has already received a sufficient number of intentions to submit proposals covering all use cases. However,  more competition makes better standards. If you have relevant technologies, please have a look at the Call for Technologies  (https://bit.ly/3ryNsAF) page, read the text of the Call of your interest, study the Use Cases and Functional Requirements document and review the Framework Licence. In case of doubt,  use the Template for submissions.Your proposal should be received by the 12th April 2021.

A new MPAI Call for Technologies tackles AI-based risk analysis

At the 6th General Assembly MPAI has approved the Compression and understanding of industrial data (MPAI-CUI) https://bit.ly/2PCD1hP Call for Technologies. The standard will enable prediction of a company performance by  extracting information from its governance, financial and risk data. MPAI believes that Artificial Intelligence (AI) can achieve that goal.
In its current form, MPAI-CUI uses AI for such purposes as assessing and monitoring a company’s financial and organisational performance, and the impact of vertical risks (e.g., cyber, seismic, etc.); identifying clues to a crisis or bankruptcy years in advance; supporting financial institutions when deciding on a loan to a troubled company…
The Use Cases and and Functional Requirements (https://bit.ly/3dapNkE) document identifies the requirements that proposed technologies should satisfy e.g. a governance data ontology that captures today’s practice at the global level, a digital representation of financial statements, risk assessment technical data having universally valid semantics and tree-like decision models to predict the probability of company crisis.

The Call for Technologies will be introduced at two online conferences that will be held on 31st of March at 15.00 UTC and 7th if April at 15.00 UTC. Interested parties are welcome to attend using the URLs of the first conference call (https://bit.ly/2PdUSMf).
All parties, including non-MPAI members, who believe they have relevant technologies satisfying all or most of the MPAI-CUI Functional Requirements (https://bit.ly/39CzuaP) are invited to submit proposals using a template (https://bit.ly/39mtzpX). They are also invited to inform the secretariat (secretariat@mpai.community) their intention to respond to the Call ) by the 16th of April.
The MPAI-CUI Call for Technologies (https://bit.ly/3rnDl1i) requests that the technologies proposed, if accepted for inclusion in the standard, be released according to the MPAI-CUI Framework Licence (https://bit.ly/2QNWTzv) to facilitate patent holders in their definition of the final licence.

Framework Licences and MPAI standards

MPAI’s Call for Technologies are documents that describe the purpose of the standard (called XYZ in the following), what submitters should do to respond, how submissions will be evaluated. Additionally, a Call contains the following text that should be mandatorily included in a submission:< Company > submits this technical document in response to MPAI Call for Technologies for MPAI project XYZ.
< Company > explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes https://bit.ly/2PxO3Fm (N80), in particular

< Company > declares that < Company > or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of XYZ, alone or jointly with other IPR holders after the approval of the XYZ Technical Specification by the General Assembly and in no event after commercial implementations of the MPAI-CUI Technical Specification become available on the market.

With this declaration a submitter agrees to license their technologies that have been accepted into the XYZ standard in line with the Framework Licence of the XYZ standard. MPAI has already developed four Framework Licences, (https://bit.ly/2P2aCSM) but what is a Framework Licence?
It is the business model, defined and adopted by the MPAI Principal Members who intend to actively contribute to the standard, to monetise their patents. The Framework Licence does not contain values: such as dollars, percentages, dates etc.
Here are 3 examples of clauses contained in the FWLs adopted for the three standards mentioned in this newsletter:

  • The License will be free of charge to the extent it is only used to evaluate or demo solutions or for technical trials.
  • The License may be granted free of charge for particular uses if so decided by the licensors.
  • A preference will be expressed on the entity that should administer the pool of patent holders.

​MPAI is confident that Framework Licences will accelerate the definition of licences benefit­ting industry, consumers and patent holders.


MPAI is barely 5 months old, but its community is expanding. So, we thought that it might be useful to have a slim and effective communication channel to keep our extended community informed of the latest and most relevant news. We plan to have a monthly newsletter.

We are keen to hear from you, so don’t hesitate to give us your feedback.

MPAI has started the development of its first standard: MPAI-AIF.

In December last year, MPAI issue a Call for Technologies for its first standard. The call concerned “AI Framework”, an environment capable to assemble and execute AI Modules (AIMs), components that perform certain functions that achieve certain goals.

The call requested technologies to support the life cycle of single and multiple AIMs, and to manage machine learning and workflows.

The standard is expected to be released in July 2021

MPAI is looking for technologies to develop its Context-based Audio Enhancement standard

In September last year, 3 weeks before MPAI was formally established, the group of people who was developing the MPAI organisation had already identified Context-based Audio Enhancement as an important target of MPAI standardisation. The idea was to improve the user experience in several audio-related applications including entertainment, communication, teleconferencing, gaming, post-production, restoration etc. The intention was promptly announced in a press release.

A lot has happened since then. Finally, in February 2021 the original intention took shape with the publication of a Call for Technologies for the upcoming Context-based Audio Enhancement (MPAI-CAE) standard.

The Call envisages 4 use cases. In Emotion-enhanced speech (left) an emotion-less synthesised or natural speech is enhanced with a specified emotion with specified intensity. In Audio recor­ding preservation (right) sound from an old audio tape is enhanced and a preservation master file produced using a video camera pointing to the magnetic head;

In Enhanced audioconference experience (left) speech captured in an unsuitable (e.g. at home) enviroment is cleaned of unwanted sounds. In Audio on the go (right) the audio experi­enced by a user in an environment preserves the external sounds that are considered relevant.

MPAI needs more technologies

On the same day the MPAI-CAE Call was published, MPAI published another Call for Technologies for the Multimodal Conversation (MPAI-MMC) standard. This broad application area can vastly benefit from AI.

Currently, the standard supports 3 use cases where a human en­ter­tains an audio-vis­ual con­versation with a machine emulating hum­an-to-human conversation in complet­eness and intensity. In Con­ver­sation with em­otion, the human holds a dial­ogue with speech, video and possibly text with a machine that responds with a syn­thes­ised voice and an animated face.
In Multimedia question answering (left), a human requests information about an object while displaying it. The machine responds with synthesised speech. In Personalized Automatic Speech Translation (right), a sentence uttered by a human is translated by a machine using a synthesised voice that preserves the human speech features.