Artificial Intelligence Framework (MPAI-AIF)

1 Coverage

The MPAI AI Framework (MPAI-AIF) standard as will be defined in document Nxyz of Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI).

MPAI-AIF specifies a generic execution environment possibly integrating Machine Learning, Artificial Intelligence and legacy Data Processing components implementing application areas such as

  1. Context-based Audio Enhancement (MPAI-CAE)
  2. Integrative Genomic/Sensor Analysis (MPAI-GSA)
  3. AI-Enhanced Video Coding (MPAI-EVC)
  4. Server-based Predictive Multiplayer Gaming (MPAI-SPG)
  5. Multi-Modal Conversation (MPAI-MMC)
  6. Compression and Understanding of Industrial data (MPAI-CUI)

The six application areas are expected to become MPAI standards.

2 Definitions

Term Definition
Data Any digital representation of a real or computer-generated entity, such as moving pictures, audio, point cloud, computer graphics, sensor and actu­ator. Data includes, but is not restricted to, media, manufacturing, auto­mot­ive, health and generic data.
Development Rights Licence to use MPAI-AIF Essential IPRs to develop Implementations
Enterprise Any commercial entity that develops or implements the MPAI-AIF standard
Essential IPR Any Proprietary Rights, (such as patents) without which it is not possible on technical (but not commercial) grounds, to make, sell, lease, otherwise dispose of, repair, use or operate Implementations without infringing those Proprietary Rights
Framework Licence A document, developed in compliance with the gener­ally accepted principles of competition law, which contains the conditions of use of the Licence without the values, e.g., currency, percent, dates etc.
Implementation A hardware and/or software reification of the MPAI-AIF standard serving the needs of a professional or consumer user directly or through a service
Implementation Rights Licence to reify the MPAI-AIF standard
Licence This Framework Licence to which values, e.g., currency, percent, dates etc., related to a specific Intellectual Property will be added. In this Framework Licence, the word Licence will be used as singular. However, multiple Licences from different IPR holders may be issued
Profile A particular subset of the technologies that are used in MPAI-AIF standard and, where applicable, the classes, subsets, options and parameters relevant to the subset

3 Conditions of use of the Licence

  1. The Licence will be in compliance with generally accepted principles of competition law and the MPAI Statutes
  2. The Licence will cover all of Licensor’s claims to Essential IPR practiced by a Licencee of the MPAI-AIF standard.
  3. The Licence will cover Development Rights and Implementation Rights
  4. The Licence will apply to a baseline MPAI-AIF profile and to other profiles containing additional technologies
  5. Access to Essential IPRs of the MPAI-AIF standard will be granted in a non-discriminatory fashion.
  6. The scope of the Licence will be subject to legal, bias, ethical and moral limitations
  7. Royalties will apply to Implementations that are based on the MPAI-AIF standard
  8. Royalties will not be based on the computational time nor on the number of API calls
  9. Royalties will apply on a worldwide basis
  10. Royalties will apply to any Implementation
  11. An MPAI-AIF Implementation may use other IPR to extend the MPAI-AIF Implementation or to provide additional functionalities
  12. The Licence may be granted free of charge for particular uses if so decided by the licensors
  13. The Licences will specify
    1. a threshold below which a Licence will be granted free of charge and/or
    2. a grace period during which a Licence will be granted free of charge and/or
    3. an annual in-compliance royalty cap applying to total royalties due on worldwide rev­enues for a single Enterprise
  14. A preference will be expressed on the entity that should administer the patent pool of holders of Patents Essential to the MPAI-AIF standard
  15. The total cost of the Licences issued by IPR holders will be in line with the total cost of the licences for similar technologies standardised in the context of Standard Development Organisations

The total cost of the Licences will take into account the value on the market of the AI Framework

Use Cases and Functional RequirementsApplication NoteFramework LicenceCall for Technologies – Announcement

Use Cases and Functional Requirements

1 Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international association with the mission to develop AI-enabled data coding standards. Artificial Intelligence (AI) technologies have shown they can offer more efficient data coding than existing technol­ogies.

MPAI has analysed six use cases covering applic­ation areas benefiting from AI technol­ogies. Even though use cases are disparate, each of them can be implemented with a combination of processing modules performing functions that concur to achieving the inten­ded result.

MPAI has assessed that, leaving it to the market to develop individual implementations, would multiply costs and delay adoption of AI technologies, while modules with standard interfaces, combined and executed within the MPAI-specified AI Framework, will favour the emergence of horizontal markets where proprietary and competing module implemen­tations exposing standard interfaces will reduce cost, promote adoption and incite progress of AI technologies. MPAI calls these modules AI Modules (AIM).

MPAI calls the planned AI Framework standard as MPAI-AIF. As AI is a fast-moving field, MPAI expects that MPAI-AIF will be extended as new use cases will bring new requirements and new technologies will reach maturity.

To avoid the deadlock experienced in other high-technology fields, before engaging in the development of the the MPAI-AIF standard, MPAI will develop a Frame­work Licence (FWL) associated with the MPAI-AIF Architecture and Functional Requir­ements defined in this document. The FWL, essentially the business model that standard essential patent (SEP) holders will apply to monetise their Intellectual Properties (IP), but without values such as the amount or percentage of royalties or dates due, will act as Commercial Requirements for the standard and provide a clear IPR licensing framework.

This document contains a summary description of the six use cases (Section 2) followed by a section describing the architecture expected to become normative (Section 3). Section 4 lists the normative requirements identified so far.

2        Use Cases

The six use cases considered cover a broad area of application. Therefore, it is expected that the MPAI-AIF architecture can support a wide range of use cases of practical interest.

Each case is identified by its name and the acronym identifying the future MPAI standard. More information about MPAI-AIF can be found in [1].

2.1 Context-based Audio Enhancement (MPAI-CAE)

2.2 Integrative Genomic/Sensor Analysis (MPAI-GSA)

2.3 AI-Enhanced Video Coding (MPAI-EVC)

2.4 Server-based Predictive Multiplayer Gaming (MPAI-SPG)

2.5 Multi-Modal Conversation (MPAI-MMC)

2.6 Compression and Understanding of Industrial data (MPAI-CUI)

3        Architecture

The normative MPAI-AIF architecture enables the creation and automation of mixed ML-AI-DP processing and inference workflows at scale for the use cases considered above. It includes six basic normative elements of the Architecture called Components addressing different modalities of operation – AI, Machine Learning (ML) and Data Processing (DP), data pipelines jungles and computing resource allocations including constrained hardware scenarios of edge AI devices.

The normative reference diagram of MPAI-AIF is given by the following figure where APIs be­tween different Components at different level are shown.

Figure 3 – Proposed normative MPAI-AIF Architecture

  1. Management and Control

Management concerns the activation/disactivation/suspensions of AIMs, while Control supports complex application scenarios.

Management and Control handles simple orchestration tasks (i.e. represented by the execution of a script) and much more complex tasks with a topology of networked AIMs that can be syn­chronised according to a given time base and full ML life cycles.

  1. Execution

The environment where AIMs operate. It is interfaced with Management and Control and with Communication and Storage. It receives external inputs and produces the requested outputs both of which are application specific.

  1. AI Modules (AIM)

AIMs are units comprised of at least the following 3 functions:

  1. The processing element (ML or traditional DP)
  2. Interface to Communication and Storage
  3. Input and output interfaces (function specific)

AIMs can implement auto-configuration or reconfiguration of their ML-based computational models.

  1. Communication

Communication is required in several cases and can be implemented accordingly, e.g. by means of a service bus. Components can communicate among themselves and with outputs and Storage.

The Management and Control API implements one- and two-way signalling for computational workflow initialisation and control.

  1. Storage

Storage encompasses traditional storage and is referred to a variety of data types, e.g.:

  1. Inputs and outputs of the individual AIMs
  2. Data from the AIM’s state, e.g. with respect to traditional and continuous learning
  3. Data from the AIM’s intermediary results
  4. Shared data among AIMs
  5. Information used by Management and Control.
  6. Access

Access represents the access to static or slowly changing data that are required by the application such as domain knowledge data, data models, etc.

4 Requirements

4.1 Component requirements

  1. The MPAI-AIF standard shall include specifications of the interfaces of 6 Components
    1. Management and Control
    2. Execution
    3. AI Modules (AIM)
    4. Communication
    5. Storage
    6. Access
  2. MPAI-AIF shall support configurations where Components are distributed in the cloud and at the edge
  3. Management and Control shall enable operations on the general ML life cycle: the sequence of steps that and/or traditional data processing life cycle of
    1. Single AIMs, e.g. instantiation-configuration-removal, internal state dumping/retrieval, start-suspend-stop, train-retrain-update, enforcement of resource limits
    2. Combinations of AIMs, e.g. initialisation of the overall computational model, instan­tiation-removal-configuration of AIMs, manual, automatic, dynamic and adaptive configuration of interfaces with Components.
  4. Management and Control shall support
    1. Architectures that allow application-scenario dependent hierarchical execution of workflows, i.e. a combination of AIMs into computational graphs
    2. Supervised, unsupervised and reinforcement-based learning paradigms
    3. Computational graphs, such as Direct Acyclic Graph (DAG) as a minimum
    4. Initialisation of signalling patterns, communication and security policies between AIMs
  5. Storage shall support protocols to specify application-dependent requirements such as access time, retention, read/write throughput
  6. Access shall provide
    1. Static or slowly changing data with standard formats
    2. Data with proprietary formats

4.2 Systems requirements

The following requirements are not intended to apply to the MPAI-AIF standard, but should be used for assessing technologies

  1. Management and Control shall support asynchronous and time-based synchronous operation depending on application
  2. The Architecture shall support dynamic update of the ML models with seamless or minimal impact on its operation
  3. ML-based AIMs shall support time sharing operation enabling use of the same ML-based AIM in multiple concurrent applications
  4. AIMs may be aggregations of AIMs exposing new interfaces
  5. Complexity and performance shall be scalable to cope with different scenarios, e.g. from small MCUs to complex distributed systems
  6. The Architecture shall support workflows of a mixture of AI/ML-based and DP technology-based AIMs.

4.3       General requirements

The MPAI-AIF standard may include profiles for specific (sets of) requirements

5 Conclusions

When the definition of the MPAI-AIF Framework Licence will be completed, MPAI will issue a Call for Technologies that support the AI Framework with the requirem­ents given in this document.

Respondents will be requested to state in their submissions their intention to adhere to the Frame­work Licence developed for MPAI-AIF when licensing their technologies if they have been inc­luded in the MPAI-AIF standard.

The MPAI-AIF Framework Licence will be developed, as for all other MPAI Framework Licences, in compliance with the gener­ally accepted principles of competition law.

6        References

[1] MPAI Application Note#4 – MPAI-AIF Artificial Intelligence Framework

[2] MPAI Application Note#1 R1 – MPAI-CAE Context-based Audio Enhancement

[3] MPAI Application Note#2 R1 – MPAI-GSA Integrative Genomic/Sensor Analysis

[4] MPAI Application Note#3 R1 – MPAI-EVC AI-Enhanced Video Coding

[5] MPAI Application Note#5 R1 – MPAI-SPG Server-based Predictive Multiplayer Gaming

[6] MPAI Application Note#6 R1 – MPAI-MMC Multi-Modal Conversation

[7] MPAI-CAE Functional Requirements work programme

[8] MPAI-GSA Functional Requirements work programme

[9] MPAI-MMC Functional Requirements work programme

[10] MPAI-EVC Use Cases and Requirements

[11] Collaborative Evidence Conditions for MPAI-EVC Evidence Project R1

[12] Operational Guidelines for MPAI-EVC Evidence Project