1        Introduction

MPAI’s standards development is based on projects evolving through a workflow extending on 6 + 1 stages.

# Acr Name Description
0 IC Interest Collection Collection and harmonisation of use cases proposed
1 UC Use cases Proposals of use cases, their description and merger of compatible use cases
2 FR Functional Reqs Identification of the functional requirements that the standard should satisfy
3 CR Commercial Reqs Development and approval of the framework licence of the standard
4 CfT Call for Technologies Preparation and publication of a document calling for technologies supporting the requirements
5 SD Standard development Development of the standard in a specific Development Committee (DC)
6 MS MPAI standard The standard has been successfully completed and all Members have made the appropriate declarations

A project progresses from one stage to the next by resolution of the General Assembly.

The stages of currently (MPAI-7) active MPAI projects are graphically represented by Figure 1.

Figure 1 – Snapshot of the MPAI work plan

2        Areas at stage 5 (SD)

2.1       MPAI-AIF

Artificial Intelligence Framework (MPAI-AIF) enables creation and automation of mixed ML-AI-DP processing and inference workflows for the application areas work currently considered at stages 1, 2 and 3 of the MPAI work plan. MPAI-AIF will be extended to support new applications areas if the need will arise.

The said areas of work share the notion of an environment (the Framework) that includes 6 com­ponents – Management and Control, Execution, AI Modules (AIM), Communication, Storage and Access. AIMs are connected in a variety of topologies and executed under the super­vision of Management and Control. AIMs expose standard interfaces that make them re-usable in different applications. Figure 2 shows the general MPAI-AIF Reference Model.

Figure 2 – Reference model of the MPAI AI Framework

MPAI documents supporting the MPAI-AIF project at the current stage are:

  1. MPAI-AIF Use Cases & Functional Requirements, N74 [1]
  2. MPAI-AIF Call for Technologies, N100 [2]
  3. MPAI-AIF Framework Licence, N101 [3]

Stage 6 is expected to be reached in July 2021.

3        Areas at stage 4 (CT)

3.1       MPAI-CAE

Context-based Audio Enhancement (MPAI-CAE) improves the user experience for several audio-related applications including entertainment, communication, teleconferencing, gaming, post-production, restoration etc. in a variety of contexts such as in the home, in the car, on-the-go, in the studio etc. using context information to act on the input audio content using AI, processing such content via AIMs, and may deliver the processed output via the most appropriate protocol.

So far, MPAI-CAE has been found applicable to 11 usage examples, for 4 of which the definition of AIM interfaces is at an advanced stage: Emotion enhanced speech, Audio Recording Preservation, Enhanced Audioconference Experience and Audio-on-the-go. Figure 3 addresses the Emotion enhanced speech Use Case.

Figure 3 – An MPAI-CAE Use Case: Emotion-enhanced speech

MPAI documents supporting the MPAI-CAE project at the current stage are:

  1. MPAI-CAE Use Case and Functional Requirements, N131 [4]
  2. MPAI-CAE Call for Technologies, N132 [5]
  3. MPAI-CAE Framework Licence [14]

Submissions in response to the MPAI-CAE CfT will be considered at MPAI-7 (2021/04/14).

3.2       MPAI-MMC

Multi-modal conversation (MPAI-MMC) aims to enable human-machine conversation that emulates human-human conversation in completeness and intensity by using AI.

So far, 3 Use Cases have been identified for MPAI-MMC: Conversation with emotion, Multimodal Question Answering (QA) and Personalized Automatic Speech Translation.

Figure 4 addresses the Conversation with emotion Use Case.

Figure 4 – An MPAI-MMC Use Case: Conversation with emotion

MPAI documents supporting the MPAI-MMC project at the current stage are:

  1. MPAI-MMC Use Case and Functional Requirements, N153 [7]
  2. MPAI-MMC Call for Technologies, N154 [8]
  3. MPAI-MMC Framework Licence, [N173] [9]

Submissions in response to the MPAI-MMC CfT will be considered at MPAI-7 (2021/04/14).

3.3       MPAI-CUI

Compression and understanding of industrial data (MPAI-CUI) aims to enable AI-based filtering and extraction of key information to predict company performance by applying Artificial Intellig­ence to governance, financial and risk data.

MPAI-CUI requires standardisation of all data formats to be fed into an AI machine to extract information that is relevant to the intended use. Converted data undergo a further conversion and are then fed to specific neural networks. This is depicted in Figure 5.

Figure 5 – The MPAI-CUI Use Case

 MPAI documents supporting the MPAI-CUI project at the current stage are:

  1. MPAI-CUI Use Cases and Functional Requirement, N200 [10]
  2. MPAI-CUI Framework Licence, N201 [11]
  3. MPAI-CUI Call for Technologies N202 [12]

MPAI plans on publishing the MPAI-CUI Use Cases and Requirements, the Call for Technologies and the Framework Licence at MPAI-6 (2020/03/17)

4        Areas at stage 2 (FR)

4.1       MPAI-GSA

Integrative Genomic/Sensor Analysis (MPAI-GSA) uses AI to understand and compress the res­ult of high-throughput experiments combining genomic/proteomic and other data, e.g., from video, motion, location, weather, medical sensors.

So far, MPAI-GSA has been found applicable to 4 Use Areas (collections of compatible Use Cases):

  1. Integrative analysis of ‘omics datasets
  2. Smart Farming
  3. Genomics and phenotypic/spatial data
  4. Genomics and behaviour

Figure 6 addresses the Use Case Smart Farming.

Figure 6 – An MPAI-GSA Use Case: Smart Framing

 MPAI documents supporting the MPAI-GSA project at the current stage are:

  1. Draft MPAI-GSA Use Cases and Functional Requirement, N191 [13]

MPAI plans on publishing the MPAI-GSA Use Cases and Requirements, the Call for Technologies and the Framework Licence at MPAI-7 (2020/04/14).

4.2       MPAI-SPG

Server-based Predictive Multiplayer Gaming (MPAI-SPG) aims to minimise the audio-visual and gameplay discontinuities caused by high latency or packet losses during an online real-time game. In case information from a client is missing, the data collected from the clients involved in a particular game are fed to an AI-based system that predicts the moves of the client whose data are missing.

Figure 7 depicts the MPAI-SPG reference model connected to a cloud gaming server.

Figure 7 – MPAI-SPG standardisation area (left)

 Approved MPAI document supporting the MPAI-EVC work area is

  1. Draft MPAI-SPG Use Cases and Functional Requirement, N193 [14];

MPAI plans on publishing the MPAI-GSA Use Cases and Requirements, the Call for Technologies and the Framework Licence at MPAI-7 (2020/04/14).

4.3       MPAI-EVC

AI-Enhanced Video Coding (MPAI-EVC) is a video compression stan­dard that substantially en­hances the performance of a traditional video codec by improving or replacing traditional tools with AI-based tools. Two approaches – Horizontal Hybrid and Vertical Hybrid – are envisaged. The Horizontal Hybrid approach introduces AI based algorithms combined with trad­itional image/video codec, trying to replace one block of the traditional schema with a machine learn­ing-based one. This case can be described by Figure 8 where green circles represent tools that can be replaced or enhanced with their AI-based equivalent.

Figure 8 – A reference diagram for the Horizontal Hybrid approach

The Vertical Hybrid approach envigaes an AVC/HEVC/EVC/VVC base layer plus an enhanced machine learning-based layer. This case can be represented by Figure 7.

Figure 9 – A reference diagram for the Vertical Hybrid approach

MPAI is engaged in the MPAI-EVC Evidence Project seeking to find evidence that AI-based technologies provide sufficient improvement to the Horizontal Hybrid approach. A second project on the Vertical Hybrid approach is being considered.

Approved MPAI documents supporting the MPAI-EVC work area are:

  1. MPAI Application Note #3 R1 – MPAI-EVC, N61 [15]
  2. MPAI-EVC Use Cases and Requirements, N92 [16]
  3. Collaborative Evidence Conditions for MPAI-EVC Evidence Project Rev.1, N69 [17]
  4. Operational Guidelines for MPAI-EVC Evidence Project, N70 [18]

5        Areas at stage 1 (UC)

5.1       MPAI-OSD

Visual object and scene description is a collection of Use Cases sharing the goal of describe visual object and locate them in the space. Scene description includes the usual des­cription of objects and their attributes in a scene and the semantic description of the objects.

Unlike proprietary solutions that address the needs of the use cases but lack interoperability or force all users to adopt a single technology or application, a standard representation of the ob­jects in a scene allows for better satifaction of the requirements.

Approved MPAI document supporting the MPAI-OSD work area is:

  1. MPAI Application Note #8 – MPAI-OSD, N158 [19]

6        Areas at stage 0 (IC)

6.1 MPAI-CAV

Connected Autonomous Vehicles (CAV) is a candidate use case using AI to perform a variety of function required by an autonomous vehicle in an environment where a plurality of CAVs communicate to achieve their respective goals.

Figure 10 – A reference diagram for Connected Autonomous Vehicles

6.2 Vision-to-Sound Transformation

It is possible to give a spatial representation of an image that visually impaired people can hear with two headphones as a localization and description medium. It is a conversion (compression) technique from one space to a different interpretation space.

7        Other possible areas

Several potential areas for standardisation are likely to emerge from [120].

7.1       Anomalous service access

A machine that has learnt “typical” service access values for a particular service provider can detect attempts beyond “typical” values.

7.2       Anomalous vibrations

A machine learns from the data generated by inertial sensors (accelerometer with gyroscope) to distinguish between regular and anomalous vibrations.

8        References

  1. MPAI-AIF Use Cases & Functional Requirements, N74
  2. MPAI-AIF Call for Technologies, N100
  3. MPAI-AIF Framework Licence, MPAI N171
  4. MPAI-CAE Use Cases & Functional Requirements; MPAI N151
  5. MPAI-CAE Call for Technologies, MPAI N152
  6. MPAI-CAE Framework Licence, MPAI N171
  7. MPAI-MMC Use Cases & Functional Requirements; MPAI N153; https://mpai.community/standards/mpai-mmc/#UCFR
  8. MPAI-MMC Call for Technologies, MPAI N154
  9. MPAI-MMC Framework Licence, N173
  10. MPAI-CUI Use Cases and Functional Requirement, N200
  11. Draft MPAI-GSA Use Cases and Functional Requirements, N191
  12. MPAI-CUI Framework Licence, N201
  13. MPAI-CUI Call for Technologies N202
  14. Draft MPAI-SPG Use Cases and Functional Requirements, N193
  15. MPAI Application Note #3 R1 – MPAI-EVC, N61
  16. MPAI-EVC Use Cases and Requirements, N92
  17. Collaborative Evidence Conditions for MPAI-EVC Evidence Project Rev.1, N69
  18. Operational Guidelines for MPAI-EVC Evidence Project, N70
  19. Initial ideas for MPAI-OSD Use Cases and Functional Requirements, N158
  20. MPAI Use Cases Rev2.0, N46