Functional RequirementsApplication NoteFramework LicenceCall for TechnologiesAssessment of MPAI Submissions

Assessment of MPAI Submissions

This document integrates the MPAI-AIF Call for Technologies. It provides information on how proposals submitted in response to the MPAI-AIF CfT will be assessed.

The following process will be adopted for the review of proposal.

1) Presentation (mandatory) / Demonstration (optional)

Goal To assess the submission based on a presentation and possible demonstration that

1.     Demonstrate the appropriateness and disclose the appropriate range of use.

2.     Provide evidence of the functionalities claimed, and of how the submission satisfies the evaluation criteria.

NB1: A respondent may opt to select a particular use case to demonstrate their functionalities. MPAI encourages to select one of the existing Use Cases. A respondent my demonstrate a new use case. However, they should provide complete description of the use case, of the inputs and outputs of the implemented AIMs and the interaction between AIMs and Management and Control.

NB2: Both demo and presentation will each have a time limit (to be determined).

Output Complete proposal evaluation sheet in Annex B.

How it is done

  1. Evaluators are
    1. All AIF-DC members attending
    2. Non-MPAI members who are respondents
    3. Experts who are non respondents non MPAI members in a consulting capacity
  2. If not all a) b) and c) members intend to be evaluators, an evaluation panel will be created that includes all a), b) and c) individuals interested to be evaluators
  3. Respondents will be asked to present their proposals
  4. MPAI-DC and other respondents will ask questions to the respondents

2) Produce a conclusion

Goal To summarise the results. This should enable MPAI to identify

·       The strong points of the proposal.

·       How the proposal might be adapted or combined with other proposals to enter the Working Draft, and/or be further tested.

Output  Proposed evaluation results.

How it is done

  1. Members of the evaluation panel will be asked to fill out Annex 1
  2. Respondents may wish to make a written response to the relevant comments
  3. Evaluators may wish to ask questions regarding point in their evaluations
  4. In case of complex issues, a minimum of two reviewers per issue will be appointed to review and report about specific points in the proposals

Annex 1

(copy of Annex B of CfT)

Title of the Proposal:

Main Functionalities:

Summary of Response: (a few lines)

Comments on Relevance to the CfT (Requirements):

Comments on possible MPAI-AIF profiles[1]

Evaluation table:

Table 1Assessment of submission features

Submission features Evaluation elements Final Assessment
Completeness of description

Understandability

Adaptability

Extensibility

Use of Standard Technology

Efficiency

Test cases

Maturity of reference implementation

Relative complexity

Support of MPAI use cases

Support of non-MPAI use cases

 

Content of the criteria table cells:

Evaluation facts should mention:

  • Not supported / partially supported / fully supported.
  • What supported these facts: submission/presentation/demo.
  • The summary of the facts themselves, e.g., very good in one way, but weak in another.

Final assessment should mention:

  • Possibilities of improving or adding to the proposal, e.g., any missing or weak features.
  • How sure the experts are, i.e., evidence shown, very likely, very hard to tell, etc.
  • Global evaluation (Not Applicable/ –/ – / + / ++)

New Use Cases/Requirements Identified:

Summary of the evaluation:

  • Main strong points, qualitatively:
  • Main weak points, qualitatively:
  • Overall evaluation: (0/1/2/3/4/5)

0: could not be evaluated

1: proposal is not relevant

2: proposal is relevant, but requires much more work

3: proposal is relevant, but with a few changes

4: proposal has some very good points, so it is a good candidate for standard

5: proposal is superior in its category, very strongly recommended for inclusion in standard

Additional remarks: (points of importance not covered above.)

The submission features in Table 1 are explained in the following Table 2.

Table 2 – Explanation of feature assessment

Submission features Criteria
Completeness of description Evaluators should compare the submission with the list of requirem­ents (Annex C of the CfT) and their implied functionalities.

Evaluators should check if respondents have described in sufficient detail to what part of the architecture their proposal refers to.

Completeness is a merit because reviewers can assess that the compon­ents are integrated. However, submissions should be judged for the merit of what is proposed. The completeness will be assessed API-wise and as a whole.

Understandability Evaluators should identify items that are demonstrably unclear (incon­sistencies, sentences with dubious meaning etc.)
Adaptability Adaptability is synonymous of portability to different computational frameworks.

Evaluators should check if respondent specifies an execution envir­on­ment with its scope of applicability. If respondent does not specify it, evaluators shall assess applicability of the proposal to 1) a low resource environ­ment (e.g., an MCU) and 2) a large-scale environment (e.g., cloud).

Extensibility Extensibility is the capability of the proposed solution to support use cases that are not supported by current requirements.

Evaluators should check if respondent has proposed such use cases

Use of standard Technology Evaluators should check if proprietary libraries are used where widely adopted libraries could have been used
Efficiency Evaluators should assess power consumption, computational speed, computational complexity, required TOPS
Test cases Example cases of API usage provided by respondent
Maturity of reference implementation Maturity is measured by the completeness of the submitted reference implementation. Therefore, a demo HW/SW implementation should have all the parts necessary to test it. Optimized parts of the implem­entation that are demonstrated but not disclosed can be taken into ac­count if and only if such components are replicable in terms of input to and output from APIs.
Relative complexity Evaluators should also identify issues that would make it difficult to implement the proposal compared to the state of the art
Support of MPAI use cases Evaluators should check if any of the current MPAI use cases is supported by the submission
Support of non-MPAI use cases Relevant, use cases not currently contemplated in MPAI can be demonstrated in support of the validity of the proposal.

[1] Profile of a standard is a particular subset of the technologies that are used in a standard and, where applicable, the classes, subsets, options and parameters relevan for the subset


Functional RequirementsApplication NoteFramework LicenceCall for TechnologiesAssessment of MPAI Submissions

MPAI-AIF Call for Technologies

1 Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international non-profit organisation with the mission to develop Artificial Intelligence (AI) enabled digital data coding standards, especially using new technologies such as Artificial Intelligence, and of technologies that facilitate integration of data coding components into ICT systems. With the mechanism of Framework Licences, MPAI seeks to attach clear IPR licensing frameworks to its standards.

As a result of the analysis of several use cases, MPAI has identified the need for a common AI Framework that can support the implementation of Use Cases. MPAI expects that most future use cases will benefit from the use of the MPAI AI Framework or extensions thereof. For this reason, MPAI has decided that a standard satisfying the requirements contained in MPAI document N74 available online would benefit use case implementors.

This document is a Call for Technologies (CfT) that 1) satisfy the requirements of N74 and 2) are released according to the Framework Licence of N101, if selected by MPAI for inclusion in the MPAI AI Framework standard called MPAI-AIF. MPAI will select the most suitable technologies on the basis of their technical merits for inclusion in MPAI-AIF.

All parties who believe they have relevant technologies satisfying all or most of the requirements mentioned in MPAI N74 are invited to submit proposals for consideration by MPAI. The parties do not necessarily have to be MPAI members.

MPAI in not obligated, by virtue of this CfT, to select a particular technology or to select any technology if those submitted are found inadequate.

Submissions are due on 2020/02/15T23:59 UTC and will be reviewed according to the schedule that the 5th MPAI General Assembly (MPAI-5) will define at its online meeting on 2021/02/17. Non-MPAI members should contact the MPAI secretariat (secretariat@mpai.community) for further details on how they can attend the said review.

2 How to submit a response

Those planning to respond to this CfT

  1. Are advised that online events will be held on 2020/12/21 and 2021/01/07 to present the MPAI-AIF CfT and respond to questions. Logistic information of these events will be posted on the MPAI web site
  2. Are requested to communicate their intention to respond to this CfT with an initial version of the form of Annex A to the MPAI secretariat (secretariat@mpai.community) by 2021/01/15. A potential submitter making a communication using the said form is not required to actually make a submission. Submission will be accepted even if the submitter did not communicate their intention to submit a response.

Responses to this MPAI-AIF CfT shall/may include:

Item Status
Detailed documentation describing the proposed technologies mandatory
The final version of Annex A mandatory
The text of Annex B duly filled out with the table indicating which requirements identified in MPAI N74 are satisfied. If a requirement is not satisfied, the submission shall indicate the reason mandatory
Comments on the completeness and appropriateness of the MPAI-AIF requirem­ents and any motivated suggestion to extend those requirements optional
A preliminary demonstration, with a detailed document describing it optional
Any other additional relevant information that may help evaluate the submission, such as additional use cases optional
The text of Annex D mandatory

Respondents are invited to review the check list of Annex C before submitting their response and filling out Annex B.

Responses to this MPAI-AIF CfT shall be submitted to secretariat@mpai.community (MPAI secretariat) by 2020/02/15T23:59 UTC. The secretariat will acknowledge receipt of the submission via email.

Respondents to this CfT are requested to present their submission (mandatory). If no presenter will attend the meeting, the proposal will be discarded.

Respondents are advised that, upon acceptance by MPAI for further evaluation of their submission in whole or in part, MPAI will require that

  • A working implementation, including source code, – for use in the development of the MPAI-AIF Reference Software – be made available before the technology is accepted for the MPAI-AIF standard. Software may be written in programming languages that can be compiled or interpreted and in hardware description languages.
  • A non-MPAI member immediately join MPAI. If the non-MPAI member elects not to do so, their submission will be discarded. Direction on how to join MPAI can be found online.

Further information on MPAI can be obtained from the MPAI website.

3 Evaluation Criteria and Procedure

Submissions will be evaluated on the basis of the criteria identified in Annex B and with the following steps:

1) Presentation (mandatory) / Demonstration (optional)

Goal To assess the submission based on a presentation and possible demonstration that

1.     Demonstrate the appropriateness and disclose the appropriate range of use.

2.     Provide evidence of the functionalities claimed, and of how the submission satisfies the evaluation criteria.

NB1: A respondent may opt to select a particular use case to demonstrate their functionalities. MPAI encourages to select one of the existing Use Cases. A respondent my demonstrate a new use case. However, they should provide complete description of the use case, of the inputs and outputs of the implemented AIMs and the interaction between AIMs and Management and Control.

NB2: Both demo and presentation will each have a time limit (to be determined).

Output Complete proposal evaluation sheet in Annex B.

2) Produce a conclusion

Goal To summarise the results. This should enable MPAI to identify

·       The strong points of the proposal.

·       How the proposal might be adapted or combined with other proposals to enter the Working Draft, and/or be further tested.

Output  Proposed evaluation results.

4 Expected development timeline

Timeline of the call, deadlines and evaluation of the answers:

Call for Technologies 2020/12/16
Conference Calls 2020/12/21 and 2021/01/07
Notification of intention to submit a proposal 2021/01/15
Submission deadline 2021/02/15T23.59 UTC
Evaluation of responses Calendar determined at MPAI-5 2021/02/17

Evaluation to be carried out during 2-hour sessions according to the calendar agrees at MPAI-5

5 References

[1] Use Cases & Functional Requirements of MPAI-AIF, MPAI N74; https://mpai.community/standards/mpai-aif/

[2] Use Case-Requirements-candidate technologies for MPAI-CAE CfT, MPAI N96

[3] Use Case-Requirements-candidate technologies for MPAI-MMC CfT, MPAI N97

[4] MPAI-CUI Use Cases and Functional Requirements, MPAI N95

Annex A: Information Form

This information form is to be filled in by a respondent to the MPAI-AIF CfT

  1. Title of the proposal
  1. Organisation: company name, position, e-mail of contact person
  1. What is the main functionalities of your proposal?
  1. Does your proposal provide or describe a formal specification and APIs?
  1. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

Annex B: Evaluation Sheet

This evaluation sheet is to be used for self-evaluation in the submission and to be filled out during evaluation phase.

Title of the Proposal:

Main Functionalities:

 Summary of Response: (a few lines)

Comments on Relevance to the CfT (Requirements):

Evaluation table:

Submission features Evaluation elements Final Assement
Completeness of description

Understandability

Adaptability

Extensibility

Use of Standard Technology

Efficiency

Test cases

Maturity of reference implementation

Relative complexity

Support of MPAI use cases

Support of non-MPAI use cases

Content of the criteria table cells:

Evaluation facts should mention:

  • Not supported / partially supported / fully supported.
  • What supported these facts: submission/presentation/demo.
  • The summary of the facts themselves, e.g., very good in one way, but weak in another.

Final assessment should mention:

  • Possibilities of improving or adding to the proposal, e.g., any missing or weak features.
  • How sure the experts are, i.e., evidence shown, very likely, very hard to tell, etc.
  • Global evaluation (Not Applicable/ –/ – / + / ++)

 New Use Cases/Requirements Identified:

Summary of the evaluation:

  • Main strong points, qualitatively: 
  • Main weak points, qualitatively:
  • Overall evaluation: (0/1/2/3/4/5)

0: could not be evaluated

1: proposal is not relevant

2: proposal is relevant, but requires much more work

3: proposal is relevant, but with a few changes

4: proposal has some very good points, so it is a good candidate for standard

5: proposal is superior in its category, very strongly recommended for inclusion in standard

Additional remarks: (points of importance not covered above.)

Annex C: Requirements check list

This list has been derived from the Requirements of N74. It is not intended to be a replacement of those Requirements.

The submission shall support the following requirements

  1. General Machine Learning and/or Data Processing life cycles with the possibility to
    1. instantiate-configure-remove
    2. dump/retrieve internal state
    3. start-suspend-stop
    4. train-retrain-update
    5. enforce resource limits
    6. implement auto-configuration/reconfiguration of ML-based computational models of

single AIMs and

  1. initialise the overall computational model
  2. instantiate-remove-configure AIMs
  3. manually, automatically, dynamically and adaptively configure interfaces with Com­ponents
  4. one- and two-way signal for computational workflow initialisation and control of

combinations of AIMs

  1. Application-scenario dependent hierarchical execution of workflows
  2. Topology of networked AIMs that can be synchronised according to a given time base and full ML life cycles
  3. Supervised, unsupervised and reinforcement-based learning paradigms
  4. Computational graphs, such as Direct Acyclic Graph (DAG) as a minimum
  5. Initialisation of signalling patterns, communication and security policies between AIMs
  6. Protocols to specify storage access time, retention, read/write throughput etc.
  7. Storage of Components’ data
  8. Access to
    1. Static or slowly changing data with standard formats
    2. Data with proprietary formats

The submission shall support the implementation of AI Frameworks featuring

  1. Asynchronous and time-based synchronous operation depending on application
  2. Dynamic update of the ML models with seamless or minimal impact on its operation
  3. Time-sharing operation of ML-based AIMs shall to enabl use of the same ML-based AIM in multiple concurrent applications
  4. AIMs which are aggregations of AIMs exposing new interfaces
  5. Workflows that are a mixture of AI/ML-based and DP technology-based AIMs.
  6. Scalability of complexity and performance to cope with different scenarios, e.g. from small MCUs to complex distributed systems

The submission shall not inhibit the creation of MPAI-AIF profiles.

Annex D: Mandatory text in responses

A response to this MPAI-AIF CfT shall mandatorily include the following text

<Company/Member> submits this technical document in response to MPAI Call for Technologies for MPAI project MPAI-AIF (MPAI document N100).

 <Company/Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes, in particular <Company/Member> declares that  <Com­pany/Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-AIF (MPAI document N101), alone or jointly with other IPR holders after the approval of the MPAI-AIF Technical Specif­ication by the General Assembly and in no event after commercial implementations of the MPAI-AIF Technical Specification become available on the market.

In case the respondent is a non-MPAI member, the submission shall mandatorily include the following text

If (a part of) this submission is identified for inclusion in a specification, <Company>  understands that  <Company> will be requested to immediately join MPAI and that, if  <Company> elects not to join MPAI, this submission will be discarded.

Subsequent technical contribution shall mandatorily include this text

<Member> submits this document to MPAI Development Committee AIF as a contribution to the development of the MPAI-AIF Technical Specification.

 <Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes, in particular  <Company> declares that <Company> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-AIF (MPAI document N101), alone or jointly with other IPR holders after the approval of the MPAI-AIF Technical Specification by the General Assembly and in no event after commercial implementations of the MPAI-AIF Technical Specification become available on the market.


Functional RequirementsApplication NoteFramework LicenceCall for TechnologiesAssessment of MPAI Submissions

MPAI-AIF Framework Licence

1 Coverage

The MPAI AI Framework (MPAI-AIF) standard as will be defined in document Nxyz of Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI).

MPAI-AIF specifies a generic execution environment possibly integrating Machine Learning, Artificial Intelligence and legacy Data Processing components implementing application areas such as

  1. Context-based Audio Enhancement (MPAI-CAE)
  2. Integrative Genomic/Sensor Analysis (MPAI-GSA)
  3. AI-Enhanced Video Coding (MPAI-EVC)
  4. Server-based Predictive Multiplayer Gaming (MPAI-SPG)
  5. Multi-Modal Conversation (MPAI-MMC)
  6. Compression and Understanding of Industrial data (MPAI-CUI)

The six application areas are expected to become MPAI standards.

2 Definitions

Term Definition
Data Any digital representation of a real or computer-generated entity, such as moving pictures, audio, point cloud, computer graphics, sensor and actu­ator. Data includes, but is not restricted to, media, manufacturing, auto­mot­ive, health and generic data.
Development Rights Licence to use MPAI-AIF Essential IPRs to develop Implementations
Enterprise Any commercial entity that develops or implements the MPAI-AIF standard
Essential IPR Any Proprietary Rights, (such as patents) without which it is not possible on technical (but not commercial) grounds, to make, sell, lease, otherwise dispose of, repair, use or operate Implementations without infringing those Proprietary Rights
Framework Licence A document, developed in compliance with the gener­ally accepted principles of competition law, which contains the conditions of use of the Licence without the values, e.g., currency, percent, dates etc.
Implementation A hardware and/or software reification of the MPAI-AIF standard serving the needs of a professional or consumer user directly or through a service
Implementation Rights Licence to reify the MPAI-AIF standard
Licence This Framework Licence to which values, e.g., currency, percent, dates etc., related to a specific Intellectual Property will be added. In this Framework Licence, the word Licence will be used as singular. However, multiple Licences from different IPR holders may be issued
Profile A particular subset of the technologies that are used in MPAI-AIF standard and, where applicable, the classes, subsets, options and parameters relevant to the subset

3 Conditions of use of the Licence

  1. The Licence will be in compliance with generally accepted principles of competition law and the MPAI Statutes
  2. The Licence will cover all of Licensor’s claims to Essential IPR practiced by a Licencee of the MPAI-AIF standard.
  3. The Licence will cover Development Rights and Implementation Rights
  4. The Licence will apply to a baseline MPAI-AIF profile and to other profiles containing additional technologies
  5. Access to Essential IPRs of the MPAI-AIF standard will be granted in a non-discriminatory fashion.
  6. The scope of the Licence will be subject to legal, bias, ethical and moral limitations
  7. Royalties will apply to Implementations that are based on the MPAI-AIF standard
  8. Royalties will not be based on the computational time nor on the number of API calls
  9. Royalties will apply on a worldwide basis
  10. Royalties will apply to any Implementation
  11. An MPAI-AIF Implementation may use other IPR to extend the MPAI-AIF Implementation or to provide additional functionalities
  12. The Licence may be granted free of charge for particular uses if so decided by the licensors
  13. The Licences will specify
    1. a threshold below which a Licence will be granted free of charge and/or
    2. a grace period during which a Licence will be granted free of charge and/or
    3. an annual in-compliance royalty cap applying to total royalties due on worldwide rev­enues for a single Enterprise
  14. A preference will be expressed on the entity that should administer the patent pool of holders of Patents Essential to the MPAI-AIF standard
  15. The total cost of the Licences issued by IPR holders will be in line with the total cost of the licences for similar technologies standardised in the context of Standard Development Organisations

The total cost of the Licences will take into account the value on the market of the AI Framework


Functional RequirementsApplication NoteFramework LicenceCall for TechnologiesAssessment of MPAI Submissions

Use Cases and Functional Requirements of MPAI AI Framework (MPAI-AIF)

1 Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international association with the mission to develop AI-enabled data coding standards. Artificial Intelligence (AI) technologies have shown they can offer more efficient data coding than existing technol­ogies.

MPAI has analysed six use cases covering applic­ation areas benefiting from AI technol­ogies. Even though use cases are disparate, each of them can be implemented with a combination of processing modules performing functions that concur to achieving the inten­ded result.

MPAI has assessed that, leaving it to the market to develop individual implementations, would multiply costs and delay adoption of AI technologies, while modules with standard interfaces, combined and executed within the MPAI-specified AI Framework, will favour the emergence of horizontal markets where proprietary and competing module implemen­tations exposing standard interfaces will reduce cost, promote adoption and incite progress of AI technologies. MPAI calls these modules AI Modules (AIM).

MPAI calls the planned AI Framework standard as MPAI-AIF. As AI is a fast-moving field, MPAI expects that MPAI-AIF will be extended as new use cases will bring new requirements and new technologies will reach maturity.

To avoid the deadlock experienced in other high-technology fields, before engaging in the development of the the MPAI-AIF standard, MPAI will develop a Frame­work Licence (FWL) associated with the MPAI-AIF Architecture and Functional Requir­ements defined in this document. The FWL, essentially the business model that standard essential patent (SEP) holders will apply to monetise their Intellectual Properties (IP), but without values such as the amount or percentage of royalties or dates due, will act as Commercial Requirements for the standard and provide a clear IPR licensing framework.

This document contains a summary description of the six use cases (Section 2) followed by a section describing the architecture expected to become normative (Section 3). Section 4 lists the normative requirements identified so far.

2        Use Cases

The six use cases considered cover a broad area of application. Therefore, it is expected that the MPAI-AIF architecture can support a wide range of use cases of practical interest.

Each case is identified by its name and the acronym identifying the future MPAI standard. More information about MPAI-AIF can be found in [1].

2.1 Context-based Audio Enhancement (MPAI-CAE)

2.2 Integrative Genomic/Sensor Analysis (MPAI-GSA)

2.3 AI-Enhanced Video Coding (MPAI-EVC)

2.4 Server-based Predictive Multiplayer Gaming (MPAI-SPG)

2.5 Multi-Modal Conversation (MPAI-MMC)

2.6 Compression and Understanding of Industrial data (MPAI-CUI)

3        Architecture

The normative MPAI-AIF architecture enables the creation and automation of mixed ML-AI-DP processing and inference workflows at scale for the use cases considered above. It includes six basic normative elements of the Architecture called Components addressing different modalities of operation – AI, Machine Learning (ML) and Data Processing (DP), data pipelines jungles and computing resource allocations including constrained hardware scenarios of edge AI devices.

The normative reference diagram of MPAI-AIF is given by the following figure where APIs be­tween different Components at different level are shown.

Figure 3 – Proposed normative MPAI-AIF Architecture

  1. Management and Control

Management concerns the activation/disactivation/suspensions of AIMs, while Control supports complex application scenarios.

Management and Control handles simple orchestration tasks (i.e. represented by the execution of a script) and much more complex tasks with a topology of networked AIMs that can be syn­chronised according to a given time base and full ML life cycles.

  1. Execution

The environment where AIMs operate. It is interfaced with Management and Control and with Communication and Storage. It receives external inputs and produces the requested outputs both of which are application specific.

  1. AI Modules (AIM)

AIMs are units comprised of at least the following 3 functions:

  1. The processing element (ML or traditional DP)
  2. Interface to Communication and Storage
  3. Input and output interfaces (function specific)

AIMs can implement auto-configuration or reconfiguration of their ML-based computational models.

  1. Communication

Communication is required in several cases and can be implemented accordingly, e.g. by means of a service bus. Components can communicate among themselves and with outputs and Storage.

The Management and Control API implements one- and two-way signalling for computational workflow initialisation and control.

  1. Storage

Storage encompasses traditional storage and is referred to a variety of data types, e.g.:

  1. Inputs and outputs of the individual AIMs
  2. Data from the AIM’s state, e.g. with respect to traditional and continuous learning
  3. Data from the AIM’s intermediary results
  4. Shared data among AIMs
  5. Information used by Management and Control.
  6. Access

Access represents the access to static or slowly changing data that are required by the application such as domain knowledge data, data models, etc.

4 Requirements

4.1 Component requirements

  1. The MPAI-AIF standard shall include specifications of the interfaces of 6 Components
    1. Management and Control
    2. Execution
    3. AI Modules (AIM)
    4. Communication
    5. Storage
    6. Access
  2. MPAI-AIF shall support configurations where Components are distributed in the cloud and at the edge
  3. Management and Control shall enable operations on the general ML life cycle: the sequence of steps that and/or traditional data processing life cycle of
    1. Single AIMs, e.g. instantiation-configuration-removal, internal state dumping/retrieval, start-suspend-stop, train-retrain-update, enforcement of resource limits
    2. Combinations of AIMs, e.g. initialisation of the overall computational model, instan­tiation-removal-configuration of AIMs, manual, automatic, dynamic and adaptive configuration of interfaces with Components.
  4. Management and Control shall support
    1. Architectures that allow application-scenario dependent hierarchical execution of workflows, i.e. a combination of AIMs into computational graphs
    2. Supervised, unsupervised and reinforcement-based learning paradigms
    3. Computational graphs, such as Direct Acyclic Graph (DAG) as a minimum
    4. Initialisation of signalling patterns, communication and security policies between AIMs
  5. Storage shall support protocols to specify application-dependent requirements such as access time, retention, read/write throughput
  6. Access shall provide
    1. Static or slowly changing data with standard formats
    2. Data with proprietary formats

4.2 Systems requirements

The following requirements are not intended to apply to the MPAI-AIF standard, but should be used for assessing technologies

  1. Management and Control shall support asynchronous and time-based synchronous operation depending on application
  2. The Architecture shall support dynamic update of the ML models with seamless or minimal impact on its operation
  3. ML-based AIMs shall support time sharing operation enabling use of the same ML-based AIM in multiple concurrent applications
  4. AIMs may be aggregations of AIMs exposing new interfaces
  5. Complexity and performance shall be scalable to cope with different scenarios, e.g. from small MCUs to complex distributed systems
  6. The Architecture shall support workflows of a mixture of AI/ML-based and DP technology-based AIMs.

4.3       General requirements

The MPAI-AIF standard may include profiles for specific (sets of) requirements

5 Conclusions

When the definition of the MPAI-AIF Framework Licence will be completed, MPAI will issue a Call for Technologies that support the AI Framework with the requirem­ents given in this document.

Respondents will be requested to state in their submissions their intention to adhere to the Frame­work Licence developed for MPAI-AIF when licensing their technologies if they have been inc­luded in the MPAI-AIF standard.

The MPAI-AIF Framework Licence will be developed, as for all other MPAI Framework Licences, in compliance with the gener­ally accepted principles of competition law.

6        References

[1] MPAI Application Note#4 – MPAI-AIF Artificial Intelligence Framework

[2] MPAI Application Note#1 R1 – MPAI-CAE Context-based Audio Enhancement

[3] MPAI Application Note#2 R1 – MPAI-GSA Integrative Genomic/Sensor Analysis

[4] MPAI Application Note#3 R1 – MPAI-EVC AI-Enhanced Video Coding

[5] MPAI Application Note#5 R1 – MPAI-SPG Server-based Predictive Multiplayer Gaming

[6] MPAI Application Note#6 R1 – MPAI-MMC Multi-Modal Conversation

[7] MPAI-CAE Functional Requirements work programme

[8] MPAI-GSA Functional Requirements work programme

[9] MPAI-MMC Functional Requirements work programme

[10] MPAI-EVC Use Cases and Requirements

[11] Collaborative Evidence Conditions for MPAI-EVC Evidence Project R1

[12] Operational Guidelines for MPAI-EVC Evidence Project


Functional RequirementsApplication NoteFramework LicenceCall for TechnologiesAssessment of MPAI Submissions

MPAI Application Note #4

Artificial Intelligence Framework (MPAI-AIF)

Proponent: Andrea Basso.

 Description: The purpose of the MPAI framework is to enable the creation and automation of mixed ML-AI-DP processing and inference workflows at scale. The key components of the framework should address different modalities of operation (AI, ML and DP), data pipeline jungles and computing resource allocations including constrained hardware scenarios of edge AI devices.

The framework is depicted in Figure 1. It is composed by:

  1. Data Storage component
  2. Orchestrator
  3. Processing modules (PM), traditional or ML algorithms

Figure 1 – MPAI Framework

  1. MPAI Processing Modules (PM)

PMs are composed by 4 components as indicated in Figure 1:

  1. The processing element PM (ML or Traditional Data Processor)
  2. Interface to the common data storage format
  3. input interfaces
  4. output interfaces
  5. Orchestrator

The PMs in Figure 1 need to be executed and orchestrated, so that are being executed in the correct order and the timing where needed is respected e.g. that inputs to a PM are computed before the PM is executed. As reported in [6] the glue code between PMs of a complex ML system is often brittle and that custom scripts don’t scale beyond specific cases.

Therefore, it is important to have a specific orchestrator component that supports the implementation of several MPAI application scenarios and ease the efficient usage of PMs.

 As shown in figure 1 the orchestrator is characterized by:

  1. Interface to PMs
  2. Interface to the input
  3. Interface to the common data storage format

Note that the Orchestrator should be able to handle from simple orchestration tasks (i.e. represented by the execution of a script) to much more complex tasks with a topology of networked PMs that need to be synchronized according to a given time base.

  1. Data Storage

The Data Storage component encompass traditional storage and is referred to a variety of data types. Some examples: it stores the inputs and outputs of the individual PMs, data from the orchestrator PM’s state and intermediary results, shared data among PMs, information used by the orches­trator, orchestrator procedures as well domain knowledge data, data models, etc.

Comments:

Examples

1. MPAI-CAE

Examples of PMs in the MPAI-CAE application scenario are

 

  • process the signals from the environment captured by the microphones
  • performing dynamic signal equalization
  • selectively recognize and allow relevant environment sounds (i.e. the horn of a car).

2. MPAI-MMC

Examples of PMs in the MPAI-MMC scenario are

  • Speech recognition
  • Speech synthesis
  • Emotion recognition
  • Intention understanding
  • Gesture recognition
  • Knowledge fusion from different sources such as speech, facial expression, gestures, etc

An illustrative application of the MPAI-AIF Framework is given below

Figure 2 – MPAI-MMC in the MPAI-AIF Framework

Requirements:

MPAI has identified the following initial requirements:

  • Shall allow to orchestrate and automate the execution of AI mixed workflows for a variety of application scenarios
  • Agnostic to AI, ML or DP technology: The architecture should be general enough avoiding the imposition of limitations in terms of algorithmic structure, storage and communication and allow full interoperability of its components.
  • Allow uninterrupted functionality while algorithms or ML models are updated or retained. ML models once deployed may need to update often as more data is made available through their usage. Deployment of the updated models should happen seamlessly with no or minimal impact.
  • Support for distributed computing including combination of cloud and edge AI components.
  • Scalable: can be used in scenarios of different complexity
  • Efficient use of the computing and communication resources e.g. by supporting task sharing also for ML processing modules.
  • Support a common interface for processing modules
  • Support common data representation for storage
  • Support parallel and sequential combination of PMs
  • Support real time processing

 Object of standard:

MPAI has identified the following areas of possible standardization:

  • Architecture that specifies the roles of the components of the architecture
  • The modalities of the creation of the pipelines of PMs
  • Common PM interfaces specified in the application standards s8ch as MPAI-CAE, etc ..
  • Data storage interfaces

The current framework has been made as general as possible taking into consideration also current MPAI application standards. We are expecting the architecture to be enriched and extended according to the proposals of the MPAI contributors.

Benefits:

MPAI-AIF will bring benefits positively affecting

  1. Technology providers need not develop full applications to put to good use their technol­ogies. They can concentrate on improving the AI technologies of the PMs. Further, their technologies can find a much broader use in application domains beyond those they are accustomed to deal with.
  2. Equipment manufacturers and application vendors can tap from the set of technologies made available according to the MPAI-AIF standard from different competing sources, integrate them and satisfy their specific needs
  3. Service providers can deliver complex optimizations and thus superior user experience with minimal time to market as the MPAI-AIF framework enables easy combination of 3rd party components from both a technical and licensing perspective. Their services can deliver a high quality, consistent user audio experience with minimal dependency on the source by selecting the optimal delivery method
  4. End users enjoy a competitive market that provides constantly improved user exper­iences and controlled cost of AI-based products.

 Bottlenecks: the full potential of AI in MPAI-AIF would be limited by a market of AI-friendly processing units and by the introduction of the vast amount of AI technologies into products and services.

 Social aspects:

MPAI-AIF will enable the availability of a variety of AI-based products at lower price and faster.

Success criteria:

MPAI-AIF should create a competitive market of AI-based components expos­ing standard interfaces, processing units available to manufacturers, a variety of end user devices and trigger the implicit need felt by a user to have the best experience whatever the context.

References

[1] W. Wang, J. Gao, M. Zhang, S. Wang, G. Chen, T. K. Ng, B. C. Ooi, J. Shao, and M. Reyad, “Rafiki: machine learning as an analytics service system,” Proceedings of the VLDB Endowment, vol. 12, no. 2, pp. 128–140, 2018.

[2] Y. Lee, A. Scolari, B.-G. Chun, M. D. Santambrogio, M. Weimer, and M. Interlandi, “PRETZEL: Opening the black box of machine learning prediction serving systems,” in 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI18), pp. 611–626, 2018.

[3] “ML.NET [ONLINE].” https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet.

[69] D. Crankshaw, X. Wang, G. Zhou, M. J. Franklin, J. E. Gonzalez, and I. Stoica, “Clipper: A low-latency online prediction serving system.,” in NSDI, pp. 613–627, 2017.

[4] S. Zhao, M. Talasila, G. Jacobson, C. Borcea, S. A. Aftab, and J. F. Murray, “Packaging and sharing machine learning models via the acumos ai open platform,” in 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 841–846, IEEE, 2018.

[5] “Apache Prediction I/O.” https://predictionio.apache.org/.

[6] D.Sculley, G.Holt,D.Golovin,E. Davydov,T.Phillips,D.Ebner,V. Chaudhary,M. Young, J. Crespo, D. Dennison “Hidden technical debt in Machine learning systems Share” on NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems – Volume 2, December 2015, Pages 2503–2511