Search results for MPAI-AIF Framework Licence

Framework Licence

Artificial Intelligence Framework (MPAI-AIF)

1 Coverage

The MPAI AI Framework (MPAI-AIF) standard as will be defined in document Nxyz of Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI).

MPAI-AIF specifies a generic execution environment possibly integrating Machine Learning, Artificial Intelligence and legacy Data Processing components implementing application areas such as

  1. Context-based Audio Enhancement (MPAI-CAE)
  2. Integrative Genomic/Sensor Analysis (MPAI-GSA)
  3. AI-Enhanced Video Coding (MPAI-EVC)
  4. Server-based Predictive Multiplayer Gaming (MPAI-SPG)
  5. Multi-Modal Conversation (MPAI-MMC)
  6. Compression and Understanding of Industrial data (MPAI-CUI)

The six application areas are expected to become MPAI standards.

2 Definitions

Term Definition
Data Any digital representation of a real or computer-generated entity, such as moving pictures, audio, point cloud, computer graphics, sensor and actu­ator. Data includes, but is not restricted to, media, manufacturing, auto­mot­ive, health and generic data.
Development Rights Licence to use MPAI-AIF Essential IPRs to develop Implementations
Enterprise Any commercial entity that develops or implements the MPAI-AIF standard
Essential IPR Any Proprietary Rights, (such as patents) without which it is not possible on technical (but not commercial) grounds, to make, sell, lease, otherwise dispose of, repair, use or operate Implementations without infringing those Proprietary Rights
Framework Licence A document, developed in compliance with the gener­ally accepted principles of competition law, which contains the conditions of use of the Licence without the values, e.g., currency, percent, dates etc.
Implementation A hardware and/or software reification of the MPAI-AIF standard serving the needs of a professional or consumer user directly or through a service
Implementation Rights Licence to reify the MPAI-AIF standard
Licence This Framework Licence to which values, e.g., currency, percent, dates etc., related to a specific Intellectual Property will be added. In this Framework Licence, the word Licence will be used as singular. However, multiple Licences from different IPR holders may be issued
Profile A particular subset of the technologies that are used in MPAI-AIF standard and, where applicable, the classes, subsets, options and parameters relevant to the subset

3 Conditions of use of the Licence

  1. The Licence will be in compliance with generally accepted principles of competition law and the MPAI Statutes
  2. The Licence will cover all of Licensor’s claims to Essential IPR practiced by a Licencee of the MPAI-AIF standard.
  3. The Licence will cover Development Rights and Implementation Rights
  4. The Licence will apply to a baseline MPAI-AIF profile and to other profiles containing additional technologies
  5. Access to Essential IPRs of the MPAI-AIF standard will be granted in a non-discriminatory fashion.
  6. The scope of the Licence will be subject to legal, bias, ethical and moral limitations
  7. Royalties will apply to Implementations that are based on the MPAI-AIF standard
  8. Royalties will not be based on the computational time nor on the number of API calls
  9. Royalties will apply on a worldwide basis
  10. Royalties will apply to any Implementation
  11. An MPAI-AIF Implementation may use other IPR to extend the MPAI-AIF Implementation or to provide additional functionalities
  12. The Licence may be granted free of charge for particular uses if so decided by the licensors
  13. The Licences will specify
    1. a threshold below which a Licence will be granted free of charge and/or
    2. a grace period during which a Licence will be granted free of charge and/or
    3. an annual in-compliance royalty cap applying to total royalties due on worldwide rev­enues for a single Enterprise
  14. A preference will be expressed on the entity that should administer the patent pool of holders of Patents Essential to the MPAI-AIF standard
  15. The total cost of the Licences issued by IPR holders will be in line with the total cost of the licences for similar technologies standardised in the context of Standard Development Organisations

The total cost of the Licences will take into account the value on the market of the AI Framework


Use Cases and Functional RequirementsApplication NoteFramework LicenceCall for Technologies – Announcement

Use Cases and Functional Requirements

1 Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international association with the mission to develop AI-enabled data coding standards. Artificial Intelligence (AI) technologies have shown they can offer more efficient data coding than existing technol­ogies.

MPAI has analysed six use cases covering applic­ation areas benefiting from AI technol­ogies. Even though use cases are disparate, each of them can be implemented with a combination of processing modules performing functions that concur to achieving the inten­ded result.

MPAI has assessed that, leaving it to the market to develop individual implementations, would multiply costs and delay adoption of AI technologies, while modules with standard interfaces, combined and executed within the MPAI-specified AI Framework, will favour the emergence of horizontal markets where proprietary and competing module implemen­tations exposing standard interfaces will reduce cost, promote adoption and incite progress of AI technologies. MPAI calls these modules AI Modules (AIM).

MPAI calls the planned AI Framework standard as MPAI-AIF. As AI is a fast-moving field, MPAI expects that MPAI-AIF will be extended as new use cases will bring new requirements and new technologies will reach maturity.

To avoid the deadlock experienced in other high-technology fields, before engaging in the development of the the MPAI-AIF standard, MPAI will develop a Frame­work Licence (FWL) associated with the MPAI-AIF Architecture and Functional Requir­ements defined in this document. The FWL, essentially the business model that standard essential patent (SEP) holders will apply to monetise their Intellectual Properties (IP), but without values such as the amount or percentage of royalties or dates due, will act as Commercial Requirements for the standard and provide a clear IPR licensing framework.

This document contains a summary description of the six use cases (Section 2) followed by a section describing the architecture expected to become normative (Section 3). Section 4 lists the normative requirements identified so far.

2        Use Cases

The six use cases considered cover a broad area of application. Therefore, it is expected that the MPAI-AIF architecture can support a wide range of use cases of practical interest.

Each case is identified by its name and the acronym identifying the future MPAI standard. More information about MPAI-AIF can be found in [1].

2.1 Context-based Audio Enhancement (MPAI-CAE)

2.2 Integrative Genomic/Sensor Analysis (MPAI-GSA)

2.3 AI-Enhanced Video Coding (MPAI-EVC)

2.4 Server-based Predictive Multiplayer Gaming (MPAI-SPG)

2.5 Multi-Modal Conversation (MPAI-MMC)

2.6 Compression and Understanding of Industrial data (MPAI-CUI)

3        Architecture

The normative MPAI-AIF architecture enables the creation and automation of mixed ML-AI-DP processing and inference workflows at scale for the use cases considered above. It includes six basic normative elements of the Architecture called Components addressing different modalities of operation – AI, Machine Learning (ML) and Data Processing (DP), data pipelines jungles and computing resource allocations including constrained hardware scenarios of edge AI devices.

The normative reference diagram of MPAI-AIF is given by the following figure where APIs be­tween different Components at different level are shown.

Figure 3 – Proposed normative MPAI-AIF Architecture

  1. Management and Control

Management concerns the activation/disactivation/suspensions of AIMs, while Control supports complex application scenarios.

Management and Control handles simple orchestration tasks (i.e. represented by the execution of a script) and much more complex tasks with a topology of networked AIMs that can be syn­chronised according to a given time base and full ML life cycles.

  1. Execution

The environment where AIMs operate. It is interfaced with Management and Control and with Communication and Storage. It receives external inputs and produces the requested outputs both of which are application specific.

  1. AI Modules (AIM)

AIMs are units comprised of at least the following 3 functions:

  1. The processing element (ML or traditional DP)
  2. Interface to Communication and Storage
  3. Input and output interfaces (function specific)

AIMs can implement auto-configuration or reconfiguration of their ML-based computational models.

  1. Communication

Communication is required in several cases and can be implemented accordingly, e.g. by means of a service bus. Components can communicate among themselves and with outputs and Storage.

The Management and Control API implements one- and two-way signalling for computational workflow initialisation and control.

  1. Storage

Storage encompasses traditional storage and is referred to a variety of data types, e.g.:

  1. Inputs and outputs of the individual AIMs
  2. Data from the AIM’s state, e.g. with respect to traditional and continuous learning
  3. Data from the AIM’s intermediary results
  4. Shared data among AIMs
  5. Information used by Management and Control.
  6. Access

Access represents the access to static or slowly changing data that are required by the application such as domain knowledge data, data models, etc.

4 Requirements

4.1 Component requirements

  1. The MPAI-AIF standard shall include specifications of the interfaces of 6 Components
    1. Management and Control
    2. Execution
    3. AI Modules (AIM)
    4. Communication
    5. Storage
    6. Access
  2. MPAI-AIF shall support configurations where Components are distributed in the cloud and at the edge
  3. Management and Control shall enable operations on the general ML life cycle: the sequence of steps that and/or traditional data processing life cycle of
    1. Single AIMs, e.g. instantiation-configuration-removal, internal state dumping/retrieval, start-suspend-stop, train-retrain-update, enforcement of resource limits
    2. Combinations of AIMs, e.g. initialisation of the overall computational model, instan­tiation-removal-configuration of AIMs, manual, automatic, dynamic and adaptive configuration of interfaces with Components.
  4. Management and Control shall support
    1. Architectures that allow application-scenario dependent hierarchical execution of workflows, i.e. a combination of AIMs into computational graphs
    2. Supervised, unsupervised and reinforcement-based learning paradigms
    3. Computational graphs, such as Direct Acyclic Graph (DAG) as a minimum
    4. Initialisation of signalling patterns, communication and security policies between AIMs
  5. Storage shall support protocols to specify application-dependent requirements such as access time, retention, read/write throughput
  6. Access shall provide
    1. Static or slowly changing data with standard formats
    2. Data with proprietary formats

4.2 Systems requirements

The following requirements are not intended to apply to the MPAI-AIF standard, but should be used for assessing technologies

  1. Management and Control shall support asynchronous and time-based synchronous operation depending on application
  2. The Architecture shall support dynamic update of the ML models with seamless or minimal impact on its operation
  3. ML-based AIMs shall support time sharing operation enabling use of the same ML-based AIM in multiple concurrent applications
  4. AIMs may be aggregations of AIMs exposing new interfaces
  5. Complexity and performance shall be scalable to cope with different scenarios, e.g. from small MCUs to complex distributed systems
  6. The Architecture shall support workflows of a mixture of AI/ML-based and DP technology-based AIMs.

4.3       General requirements

The MPAI-AIF standard may include profiles for specific (sets of) requirements

5 Conclusions

When the definition of the MPAI-AIF Framework Licence will be completed, MPAI will issue a Call for Technologies that support the AI Framework with the requirem­ents given in this document.

Respondents will be requested to state in their submissions their intention to adhere to the Frame­work Licence developed for MPAI-AIF when licensing their technologies if they have been inc­luded in the MPAI-AIF standard.

The MPAI-AIF Framework Licence will be developed, as for all other MPAI Framework Licences, in compliance with the gener­ally accepted principles of competition law.

6        References

[1] MPAI Application Note#4 – MPAI-AIF Artificial Intelligence Framework

[2] MPAI Application Note#1 R1 – MPAI-CAE Context-based Audio Enhancement

[3] MPAI Application Note#2 R1 – MPAI-GSA Integrative Genomic/Sensor Analysis

[4] MPAI Application Note#3 R1 – MPAI-EVC AI-Enhanced Video Coding

[5] MPAI Application Note#5 R1 – MPAI-SPG Server-based Predictive Multiplayer Gaming

[6] MPAI Application Note#6 R1 – MPAI-MMC Multi-Modal Conversation

[7] MPAI-CAE Functional Requirements work programme

[8] MPAI-GSA Functional Requirements work programme

[9] MPAI-MMC Functional Requirements work programme

[10] MPAI-EVC Use Cases and Requirements

[11] Collaborative Evidence Conditions for MPAI-EVC Evidence Project R1

[12] Operational Guidelines for MPAI-EVC Evidence Project



MPAI receives technologies for its AI framework standard and calls for technologies supporting audio and human-machine conversation

Geneva, Switzerland – 17 february 2021. At its 5th General Assembly, Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI), an international, unaffiliated standards association

  1. Has kicked off work on its AI Framework (MPAI-AIF) standard after receiving substantial proposed technologies.
  2. Is calling for technologies to develop two standards related to audio (MPAI-CAE) and multimodal conversation (MPAI-MMC).
  3. Will soon be developing the Framework Licence for the next maturing project “Compression and Understanding of Industrial Data” (MPAI-CUI).

MPAI has reviewed responses to the call issued 2 months ago for tech­nologies supporting its AI Framework (MPAI-AIF) standard. The goal is to enable creation and automation of mixed Mach­ine Learning (ML) – Artificial Intelligence (AI) – Data Proces­sing (DP) and inference workflows, implemented as software, hardware or mixed software and hardware. MPAI-AIF will offer extended explainability to applications conforming to MPAI standards. The submissions received are enabling MPAI to develop the inten­ded standard whose publication is planned for July 2021.

MPAI has issued two Calls for Technologies supporting two new standards:

  1. The Context-based Audio Enhancement (MPAI-CAE) standard will improve the user exper­ien­ce for several audio-related applications in a variety of contexts such as in the home, in the car, on-the-go, in the studio etc. Examples of use are adding a desired emotion to a speech without emotion, preserving old audio tapes, improving the audioconference experience and removing unwanted sounds while keeping the relevant ones to a user walking in the street.
  2. The Multimodal Conversation (MPAI-MMC) standard will enable a human-mach­ine conver­sation that emulates human-human conversation in completeness and intensity by using AI. Examples of use are an audio-visual conversation with a machine where the machine is imper­sonated by a synthesised voice and an animated face, a request for information about an object while displaying it, a human question to a machine translated using a voice preserving the speech features of the human.

The content of the two Calls for Technologies will be introduced at two online conferences. Attendance is open to interested parties.

MPAI has developed the functional requirements for the Compression and Understanding of In­dustrial Data (MPAI-CUI) and, by decision of the General Assembly, MPAI Active Members may now develop the Framework Licence to facilitate the actual licences – to be developed outside of MPAI. The standard will be used to assess the risks faced by a company by using information from the flow of data produced.

The MPAI web site provides information about other AI-based standards being developed: AI-Enhanced Video Coding (MPAI-EVC) will improve the performance of existing video codecs, Integrative Genomic/Sensor Analysis (MPAI-GSA) will compress and understand the res­ults of combining genomic experiments with those produced by related devices/sensors, and Server-based Predictive Multiplayer Gaming (MPAI-SPG) will improve the user experience of online multiplayer game players.

MPAI develops data coding standards for applications that have AI as the core enabling technology. Any legal entity who supports the MPAI mission may join MPAI if it is able to contribute to the development of standards for the efficient use of data.

Visit the MPAI web site and contact the MPAI secretariat for specific information.

 


An introduction to the MPAI-AIF Call for Technologies

On 202/12/21 MPAI has held a teleconference to illustrate the MPAI-AIF Call for Technologies (CfT) and associated Framework Licence (FWL). This article summarises the main points illustrated at the teleconference: Whay and who is MPAI, the MPAI-AIF Functional Requirements, the MPAI_AIF Call for Technologies and the MPAI-AIF Framework Licence.

Miran Choi, an MPAI Director and Chair of the Communication Advisory Committee, recalled the reasons that led to the establishment of MPAI.

Over the past 3 decades, media compression standards have allowed manufacturing and services to boom. However, the technology momentum is progressively slowing while AI technologies are taking stage by offering more capabilities than traditional technologies, by being applicable to data other than audio/video and by being  supported by a global research effort. In addition to that, industry has recently suffered from the inadequacy of the FAIR, Reasonable and Non-Discriminatory (FRAND) model to deal with the tectonic changes of technology-intensive standards.

Miran then summarised the main characteristics of MPAI. A non-profit, unaffiliated and international association that develops

  • Standards for
    • AI-enabled data coding
    • Technologies that facilitate integration of data coding components in ICT systems and
  • Associated clear IPR licensing frameworks.

MPAI is the only standards organisation that has set AI as the key enabling technology for data coding standards. MPAI members come from industry, research and academia of 15 countries, representing a broad spectrum of technologies and applications.

The development of standards must obey rules of openness and due process, and MPAI has a rigorous process to develop standards in 6 steps

Step

Description

Use cases Collect/aggregate use cases in cohesive projects applicable across industries
Functional Requirements Identify the functional requirements the standard should satisfy
Commercial Requirements Develop and approve the framework licence of the stan­dard
Call for Technologies Publish a call for technologies supporting the functional and commercial requirements
Standard development Develop standard in an especially established Devel­opment Committee (DC)
MPAI standard Complete standard and obtain declarations from all Members

The transitions from one stage to the next are approved by the General Assembly.

The MPAI-AIF standard project is at the Call for Technologies stage.

Andrea Basso, Chair of the MPAI-AIF Development Committee (AIF-DC) in charge of the development of the MPAI-AIF standard introduced the motivations and functional requirements of the MPAI-AIF standard.

MPAI has developed several Use Cases for disparate applications coming to the conclusion that they can all be implemented with a combination of AI-based modules concurring to the achievement of the intended result. the history of media standards has shown the benefits of standardisation. Therefore, to avoid the danger of incompatible implementations of modules put to the market, where costs multiply at all levels and mass adoption of AI tech is delayed, MPAI seeks to standardise AI Modules (AIM) with standard interfaces, combined and executed within an MPAI-specified AI Framework. AIMs with standard interfaces will reduce overall design costs and increase component reusability,create favourable conditions leading to horizontal markets of competing implementations, and promote adoption and incite progress of AI technologies.

AIMs need an environment where they can be combined and executed. This is what MPAI- AIF – where AIF stands for AI Framework, is about. The AI Framework is depicted in the figure.

The AI Framework has 6 components: Management and Control, Execution, AI Modules, Communication, Storage and Access.

The MPAI functional requirements are

  1. Possibility to establish general Machine Learning and/or Data Processing life cycles
    1. for single AIMs to
      1. instantiate-configure-remove
      2. dump/retrieve internal state
      3. start-suspend-stop
      4. train-retrain-update
      5. enforce resource limits
      6. implement auto-configuration/reconfiguration of ML-based computational models of
    2. for multiple AIMs to
      1. initialise the overall computational model
      2. instantiate-remove-configure AIMs
      3. manually, automatically, dynamically and adaptively configure interfaces with Com­ponents
      4. one- and two-way signal for computational workflow initialisation and control of
      5. combinations of AIMs
  2. Application-scenario dependent hierarchical execution of workflows
  3. Topology of networked AIMs that can be synchronised according to a given time base and full ML life cycles
  4. Supervised, unsupervised and reinforcement-based learning paradigms
  5. Computational graphs, such as Direct Acyclic Graph (DAG) as a minimum
  6. Initialisation of signalling patterns, communication and security policies between AIMs
  7. Protocols to specify storage access time, retention, read/write throughput etc.
  8. Storage of Components’ data
  9. Access to
    1. Static or slowly changing data with standard formats
    2. Data with proprietary formats
  10. Possibility to implement AI Frameworks featuring
    1. Asynchronous and time-based synchronous operation depending on application
    2. Dynamic update of the ML models with seamless or minimal impact on its operation
    3. Time-sharing operation of ML-based AIMs shall enable use of the same ML-based AIM in multiple concurrent applications
    4. AIMs which are aggregations of AIMs exposing new interfaces
    5. Workflows that are a mixture of AI/ML-based and DP technology-based AIMs.
    6. Scalability of complexity and performance to cope with different scenarios, e.g. from small MCUs to complex distributed systems
  11. Possibility to create MPAI-AIF profiles

Panos Kudumakis, MPAI member, explained the MPAI-AIF Call For Technologies

  1. Who can submit
    1. All parties, including non members, who believe they have relevant technologies
    2. Responses submitted to secretariat who acknowledges via email
    3. Technologies submitted must
      1. Support the requirements of N74
      2. Be released according to the MPAI-AIF Framework Licence (N101) – if selected by MPAI for inclusion in MPAI-AIF
    4. MPAI will select the most suitable technologies on the basis of their technical merits for inclusion in MPAI-AIF.
    5. MPAI in not obligated to select a particular technology or to select any technology if those submitted are found inadequate.
  2. A submission shall contain
    1. Detailed documentation describing the proposed technologies.
    2. Annex A: Information Form (contact info, proposal summary).
    3. Annex B: Evaluation Sheet to be taken into consideration for self-evaluation (quantitative & qualitative) of the submission but will be filled out during the peer-to-peer evaluation phase.
    4. Annex C: Requirements Check List (N74) to be duly filled out indicating (using a table) which requirements identified are satisfied. If a requirement is not satisfied, the submission shall indicate the reason.
    5. Annex D: Mandatory text in responses
  3. A submission may contain
    1. Comments on the completeness and appropriateness of the MPAI-AIF requirements and any motivated suggestion to extend those requirements.
    2. A preliminary demonstration, with a detailed document describing it.
    3. Any other additional relevant information that may help evaluate the submission, such as additional use cases.
  4. Assessment
    1. Respondents must present their submission or proposal is discarded.
    2. If submission is accepted in whole or in part, submitter shall make available a working implementation, including source code (for use in MPAI-AIF Reference Software) before the technology is accepted for the MPAI-AIF standard.
    3. Software can be written in compiled or interpreted programming languages and in hardware description languages.
    4. A submitter who is not an MPAI member shall immediately join MPAI, else submission is discarded.
    5. Assessment guidelines form to aid peer-to-peer evaluation phase being finalised.
  5. Calendar
    1. Call for Technologies 16 Dec (MPAI-3)
    2. Presentation Conference Calls 21 Dec/07 Jan
    3. Notification of intention to submit 15 Jan
    4. Assessment form 20 Jan (MPAI-4)
    5. Submission deadline 15 Feb
    6. Calendar of evaluation of responses 17 Feb (MPAI-5)
    7. Approval of MPAI-AIF standard 19 July (estimate)

Davide Ferri, MPAI Director and Chair of AIF-FWL, the committee that developed the MPAI-AIF Framework Licence (FWL) explained that FWL covers the MPAI-AIF technology that specifies a generic execution environment, possibly integrating Machine Learning, Artificial Intelligence and legacy Data Processing components, implementing application areas such as

  1. Context-based Audio Enhancement (MPAI-CAE)
  2. Integrative Genomic/Sensor Analysis (MPAI-GSA)
  3. AI-Enhanced Video Coding (MPAI-EVC)
  4. Server-based Predictive Multiplayer Gaming (MPAI-SPG)
  5. Multi-Modal Conversation (MPAI-MMC)
  6. Compression and Understanding of Industrial data (MPAI-CUI)

These six application areas are expected to become MPAI standards.

The FWL includes a set of definitions that are omitted here. In particular the definition of Licence, namely, the Framework Licence to which values, e.g., currency, percent, dates etc., related to a specific Intellectual Property will be added.

The FWL is expressed in concise form as below

  1. The Licence will:
    1. be in compliance with generally accepted principles of competition law and the MPAI Statutes
    2. cover all of Licensor’s claims to Essential IPR practiced by a Licensee of the MPAI-AIF standard
    3. cover Development Rights and Implementation Rights
    4. apply to a baseline MPAI-AIF profile and to other profiles containing additional technologies
  2. Grant access to Essential IPRs of the MPAI-AIF standard in a non-discriminatory fashion.
  3. Have a scope to legal, bias, ethical and moral limitations
  4. Royalties will:
    1. apply to Implementations that are based on the MPAI-AIF standard
    2. not be based on the computational time nor on the number of API calls
    3. apply on a worldwide basis
    4. apply to any Implementation
  5. An MPAI-AIF Implementation may use other IPR to extend the MPAI-AIF Implementation or to provide additional functionalities
  6. The Licence may be granted free of charge for particular uses if so decided by the licensors
  7. The Licences will specify
    1. a threshold below which a Licence will be granted free of charge and/or
    2. a grace period during which a Licence will be granted free of charge and/or
    3. an annual in-compliance royalty cap applying to total royalties due on worldwide rev­enues for a single Enterprise
  8. A preference will be expressed on the entity that should administer the patent pool of holders of Patents Essential to the MPAI-AIF standard
  9. The total cost of the Licences issued by IPR holders will be in line with the total cost of the licences for similar technologies standardised in the context of Standard Development Organisations
  10. The total cost of the Licences will take into account the value on the market of the AI Framework technology Standardised by MPAI.

Miran reminded how easily legal entities or individuals representing a technical departments of a university supporting the MPAI mission and able to contribute to the development of MPAI standards can join MPAI. They should

  1. Choose one of the two classes of membership (until 2021/12/31):
    1. Principal Members, with the right to vote (2400 €)
    2. Associate Members, without the right to vote (480 €)
  2. Send secretariat@mpai.community
    1. a signed copy of Template for MPAI Membership applications
    2. a signed copy of the MPAI Statutes. Each page should be signed and initialled
    3. a copy of the bank transfer

MPAI issues a Call for Technologies supporting its AI Framework standard

Geneva, Switzerland – 16 December 2020. Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI), an international unaffiliated standards association, has approved a Call for Technologies (CfT) for publication at its 3rd General Assembly MPAI-3. The CfT concerns tech­nologies for MPAI-AIF, acronym of the MPAI AI Frame­work standard.

The goal of MPAI-AIF is to enable set up and execution of mixed processing and infer­ence work­flows made of Machine Learning, Artificial Intelligence and legacy Data Processing com­ponents called AI Modules (AIM).

The MPAI AI Framework standard will facilitate integration of AI and legacy data processing components through standard interfaces and methods. MPAI experts have already validated MPAI’s innovative approach in a sample micro controller-based implementation that is synergistic with MPAI-AIF standard development.

In line with its statutes, MPAI has developed the Framework Licence associated with the MPAI-AIF standard. Responses to the CfT shall be in line with the requirements laid down in the CfT and shall be supported by a statement that the respondent will licence their technologies, if adopted in the standard, according to the framework licence.

MPAI is also working on a range of standards for AIM input/output interfaces used in several application areas. Two candidate standards have completed the definition of Functional Requirements and have been promoted to the Commercial Requirements stage.

The two candidates are

  1. MPAI-CAE – Context-based Audio Enhancement uses AI to improve the user experience for a variety of uses such as entertainment, communication, teleconferencing, gaming, post-prod­uction, restoration etc. in the contexts of the home, the car, on-the-go, the studio etc. allowing a dynamically optimised user experience.
  2. MPAI-MMC – Multi-Modal Conversation uses AI to enable human-machine conversation that emulates human-human conversation in completeness and intensity.

MPAI adopts a light approach in the definition AIMs standardisation. Different implementors can produce AIMs of different performance still exposing the same standard interfaces. MPAI AIMs with different features from a variety of sources will promote hor­izontal markets of AI solutions that tap from and further promote AI innovation.

The MPAI web site provides more information about other MPAI standards: MPAI-CUI uses AI to compress and understand industrial data, MPAI-EVC to improve the performance of existing video codecs, MPAI GSA to to understand and compress the res­ult of combining genomic experiments with those produced by related devices, e.g. video, motion, location, weather, medical sensors, and MPAI-SPG to improve the user experience of online multiplayer games.

MPAI develops data coding standards for applications that have AI as core enabling technology. Any legal entity that supports the MPAI mission may join MPAI if it is able to contribute to the development of standards for the efficient use of Data.

Visit the MPAI home page and contact the MPAI secretariat for specific information.

 

 


MPAI commences development of the Framework Licence for the MPAI AI Framework

Geneva, Switzerland – 18 November 2020. The Geneva-based international Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) has concluded its second General As­sembly making a major step toward the development of its first standard called MPAI AI Frame­work, acronym MPAI-AIF.

MPAI-AIF has been designed to enable creation and automation of mixed processing and infer­ence workflows made of Machine Learning, Artificial Intelligence and traditional Data Processing components.

MPAI wishes to give as much information as possible to users of its standards. After approving the Functional Requirements, MPAI is now developing the Commercial Requirements, to be embodied in the MPAI-AIF Framework Licence. This will collect the set of conditions of use of the eventual licence(s), without values, e.g. currency, percentage, dates etc.

An optimal implementation of the MPAI use cases requires a coordinated combination of processing modules. MPAI has assessed that, by standardising the interfaces of Processing Mod­ules, to be executed in the MPAI AI-Framework, horizontal markets of competing standard implementations of processing modules will emerge.

The MPAI-AIF standard, that MPAI plans on delivering in July 2021, will reduce cost, promote adoption and incite progress of AI technologies while, if the market develops incompatible implementations, costs will multiply, and adoption of AI technologies will be delayed.

MPAI-AIF is the first of a series of standards MPAI has in its development pipeline. The following three work areas, promoted to Functional Requirements stage, will build on top of MPAI-AIF:

  1. MPAI-CAE – Context-based Audio Enhancement uses AI to improve the user experience for a variety of uses such as entertainment, communication, teleconferencing, gaming, post-prod­uction, restoration etc. in the contexts of the home, the car, on-the-go, the studio etc. allowing a dynamically optimised user experience.
  2. MPAI-GSA – Integrative Genomic/Sensor Analysis uses AI to understand and compress the results of high-throughput experiments combining genomic/proteomic and other data – for in­stance from video, motion, location, weather, medical sensors. The target use cases range from personalised medicine to smart farming.
  3. MPAI-MMC – Multi-Modal Conversation uses AI to enable human-machine conversation that emulates human-human conversation in completeness and intensity.

The MPAI web site provides more information about other MPAI standards: MPAI-EVC uses AI to improve the performance of existing video codecs, MPAI-SPG to improve the user experience of online multiplayer games and MPAI-CUI to compress and understand industrial data.

MPAI seeks the involvement of companies who can benefit from international data coding stan­dards and calls for proposals of standards. In a unique arrangement for a standards organisations, MPAI gives the opportunity, even to non-members, to accompany a proposal through the defin­ition of its goals and the development of functional requirements. More details here.

MPAI develops data coding standards for a range of applications with Artificial Intelligence (AI) as its core enabling technology. Any legal entity that supports the MPAI mission may join MPAI if it is able to contribute to the development of Technical Specifications for the efficient use of Data.

Visit the MPAI home page and  contact the MPAI secretariat for specific information.

 


MPAI launches 6 standard projects on audio, genomics, video, AI framework, multiuser online gaming and multimodal conversation

Geneva, Switzerland – 21 October 2020. The Geneva-based international Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) has concluded its first operational General Assembly adopting 6 areas of work, due to become standardisation projects.

MPAI-CAE – Context-based Audio Enhancement is an area that uses AI to improve the user experience for a variety of uses such as entertainment, communication, teleconferencing, gaming, post-production, restoration etc. in such contexts as in the home, in the car, on-the-go, in the studio etc. allowing a dynamically optimized user experience.

MPAI-GSA – Integrative Genomic/Sensor Analysis is an area that uses AI to understand and compress the results of high-throughput experiments combining genomic/proteomic and other data – for instance from video, motion, location, weather, medical sensors. The target use cases range. from personalised medicine to smart farming.

MPAI-SPG – Server-based Predictive Multiplayer Gaming uses AI to minimise the audio-visual and gameplay disruptions during an online real-time game caused by missing information at the server or at the client because of high latency and packet losses.

MPAI-EVC – AI-Enhanced Video Coding plans on using AI to further reduce the bitrate required to store and transmit video information for a variety of consumer and professional applications. One user of the MPAI-EVC standard is likely to be MPAI-SPG for improved compression and higher quality of cloud-gaming content.

MPAI-MMC – Multi-Modal Conversation aims to use AI to enable human-machine conversation that emulates human-human conversation in completeness and intensity

MPAI-AIF – Artificial Intelligence Framework is an area based on the notion of a framework populated by AI-based or traditional Processing Modules. As this is a foundational standard on which other planned MPAI standards such as MPAI-CAE, MPAI-GSA and MPAI-MMC, will be built, MPAI intends to move at an accelerated pace: Functional Requirements ready in November 2020, Commercial Requirements ready in December 2020 and Call for Technologies issued in January, 2021. The MPAI-AIF standard is planned to be ready before the summer holidays in 2021.

You can find more information about MPAI standards.

MPAI covers its Commercial Requirements needs with Framework Licences (FWL). These are the set of conditions of use of a license of a specific MPAI standard without the values, e.g. curren­cy, percentages, dates, etc. MPAI expects that FWLs will accelerate the practical use of its stan­dards.

MPAI develops data coding standards for a range of applications with Artificial Intelligence (AI) as its core enabling technology. Any legal entity that supports the MPAI mission may join MPAI if it is able to contribute to the development of Technical Specifications for the efficient use of Data.

Visit the MPAI home page and  contact the MPAI secretariat for spec­ific information.


MPAI-AIF

Artificial Intelligence Framework (MPAI-AIF)

AI Framework (MPAI-AIF) is an MPAI standard under development enabling environments (AIF) where AI Workflows (AIW) composed of basic components called AI Modules (AIM) can be created and executed in support of the application areas currently included in the MPAI work plan. It is a foundational MPAI standard on which other MPAI application standards will be built.

Publication is expected for 2021/10/27 (MPAI13).

See the public MPAI-AIF documents:

  1. Draft Standard
  2.  Call for Technologies
  3. Framework Licence
  4. Use Cases and Functional Requirements
  5. Application Note

 


Call for Technologies

Context-based Audio Enhancement (MPAI-CAE)

This document is also available in MS Word format as MPAI-CAE Call for Technologies

1       Introduction. 1

2       How to submit a response. 3

3       Evaluation Criteria and Procedure. 4

4       Expected development timeline. 4

5       References. 4

Annex A: Information Form.. 6

Annex B: Evaluation Sheet 7

Annex C: Requirements check list 10

Annex D: Technologies that may require specific testing. 11

Annex E: Mandatory text in responses. 12

1        Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international non-profit organisation with the mission to develop standards for Artificial Intelligence (AI) enabled digital data coding and for technologies that facilitate integration of data coding components into ICT systems. With the mechanism of Framework Licences, MPAI seeks to attach clear IPR licensing frameworks to its standards.

MPAI has found that the application area called “Context-based Audio Enhancement” is particul­arly relevant for MPAI standardisation because using context information to act on the input audio content can substantially improve the user experience of a variety of audio-related applications that include entertainment, communication, teleconferencing, gaming, post-produc­tion, restor­ation etc. for a variety of contexts such as in the home, in the car, on-the-go, in the studio etc.

Therefore, MPAI intends to develop a standard – to be called MPAI-CAE – that will provide standard tech­nologies to implement four Use Cases identified so far:

  1. Emotion-Enhanced Speech (EES)
  2. Audio Recording Preservation (ARP)
  3. Enhanced Audioconference Experience (EAC)
  4. Audio-on-the-go (AOG)

This document is a Call for Technologies (CfT) that

  1. Satisfy the MPAI-CAE Functional Requirements (N151) [4] and
  2. Are released according to the MPAI-CAE Framework Licence (N171) [6], if selected by MPAI for inclusion in the MPAI-CAE standard.

The standard will be developed with the following guidelines:

  1. To satisfy the Functional Requirements (N151) [1], available online. In the future, MPAI may decide to extend MPAI-CAE to support other Use Cases.
  2. To use, where feasible and desirable, the same basic tech­nol­ogies required by the companion document MPAI-MMC Use Cases and Functional Requir­ements [2].
  3. To be suitable for implementation as AI Modules (AIM) conforming to the emerging MPAI AI Framework (MPAI-AIF) standard based on the responses to the Call for Technologies (N100) [5] satisfying the MPAI-AIF Functional Requirements (N74) [4].

MPAI has decided to base its application standards on the AIM and AIF notions whose functional requirements have been identified in [1] rather than follow the approach of defining end-to-end systems. It has done so because:

  1. AIMs allow the reduction of a large problem to a set of smaller problems.
  2. AIMs can be independently developed and made available to an open competitive market.
  3. An implementor can build a sophisticated and complex system with potentially limited know­ledge of all the tech­nologies required by the system.
  4. An MPAI system has an inherent explainability.
  5. MPAI systems allow for competitive comparisons of functionally equivalent AIMs.

Respondents should be aware that:

  1. Use Cases that make up MPAI-CAE, the Use Cases themselves and the AIM internals will be non-normative.
  2. The input and output interfaces of the AIMs, whose requirements have been derived to support the Use Cases, will be normative.

Therefore, the scope of this Call for Technologies is restricted to technologies required to implement the input and output interfaces of the AIMs identified in N151 [1].

However, MPAI invites comments on any technology or architectural component identified in N151, specifically,

  1. Additions or removals of input/output signals to the identified AIMs with justification of the changes and identification of data formats required by the new input/output signals.
  2. Possible alternative partitioning of the AIMs implementing the example cases providing:
    1. Arguments in support of the proposed partitioning
    2. Detailed specifications of the input and output data of the proposed new AIMs
  3. New Use Cases fully described as in N151.

All parties who believe they have relevant technologies satisfying all or most of the requirements of one or more than one Use Case described in N151 are invited to submit proposals for consid­eration by MPAI. MPAI membership is not a prerequisite for responding to this CfT. However, proponents should be aware that, if their proposal or part thereof is accepted for inclusion in the MPAI-CAE standard, they shall immediately join MPAI, or their accepted technologies will be discarded.

MPAI will select the most suitable technologies based on their technical merits for inclusion in MPAI-CAE. However, MPAI in not obligated, by virtue of this CfT, to select a particular tech­nology or to select any technology if those submitted are found inadequate.

Submissions are due on 2021/04/12T23:59 UTC and should be sent to the MPAI secretariat (secretariat@mpai.community). The secretariat will acknowledge receipt of the submission via email. Submissions will be reviewed according to the schedule that the 7th MPAI General Assembly (MPAI-7) will define at its online meeting on 2021/04/14. For details on how submitters who are not MPAI members can attend the said review please contact the MPAI secretariat (secretariat@mpai.community).

2        How to submit a response

Those planning to respond to this CfT:

  1. Are advised that online events will be held on 2021/02/24 and 2021/03/10 to present the MPAI-CAE CfT and respond to questions. Logistic information on these events will be posted on the MPAI web site.
  2. Are requested to communicate their intention to respond to this CfT with an initial version of the form of Annex A to the MPAI secretariat (secretariat@mpai.community) by 2021/03/16. A potential submitter making a communication using the said form is not required to actually make a submission. A submission will be accepted even if the submitter did not communicate their intention to submit a response by the said date.
  3. Are advised to visit regularly the https://mpai.community/how-to-join/calls-for-technologies/ web site where relevant information will be posted.

Responses to this MPAI-CAE CfT shall/may include:

Table 1 – Mandatory and optional elements of a response

Item Status
Detailed documentation describing the proposed technologies mandatory
The final version of Annex A mandatory
The text of Annex B duly filled out with the table indicating which requirements identified in MPAI N151 [1] are satisfied. If all the requirements of a Use Case are not satisfied, this should be explained. mandatory
Comments on the completeness and appropriateness of the MPAI-CAE requirem­ents and any motivated suggestion to amend or extend those requirements. optional
A preliminary demonstration, with a detailed document describing it. optional
Any other additional relevant information that may help evaluate the submission, such as additional use cases. optional
The text of Annex E. mandatory

Respondents are invited to take advantage of the check list of Annex C before submitting their response and filling out Annex A.

Respondents are requested to present their submission (mandatory) at a meeting by teleconference that will be properly announced to submitters by the MPAI Secretariat. If no presenter will attend the meeting, the proposal will be discarded.

Respondents are advised that, upon acceptance by MPAI of their submission in whole or in part for further evaluation, MPAI will require that:

  • A working implementation, including source code, – for use in the development of the MPAI-CAE Reference Software and later publication as a standard by MPAI – be made available before the technology is accepted for inclusion in the MPAI-CAE standard. Software may be written in programming languages that can be compiled or interpreted and in hardware description languages.
  • The working implementation be suitable for operation in the MPAI AIF Framework (MPAI-AIF).
  • A non-MPAI member immediately join MPAI. If the non-MPAI member elects not to do so, their submission will be discarded. Direction on how to join MPAI can be found online.

Further information on MPAI can be obtained from the MPAI website.

3        Evaluation Criteria and Procedure

Proposals will be assessed using the following process:

  1. Evaluation panel is created from:
    1. All CAE-DC members attending.
    2. Non-MPAI members who are respondents.
    3. Non respondents/non MPAI member experts invited in a consulting capacity.
  2. No one from 1.1.-1.2. will be denied membership in the Evaluation panel.
  3. Respondents present their proposals.
  4. Evaluation Panel members ask questions.
  5. If required subjective and/or objective tests are carried out:
    1. Define required tests.
    2. Carry out the tests.
    3. Produce report.
  6. At least 2 reviewers will be appointed to review & report about specific points in a proposal if required.
  7. Evaluation panel members fill out Annex B for each proposal.
  8. Respondents respond to evaluations.
  9. Proposal evaluation report is produced.

4        Expected development timeline

Timeline of the CfT, deadlines and response evaluation:

Table 2 – Dates and deadlines

Step Date
Call for Technologies 2021/02/17
CfT introduction conference call 1 2021/02/24T14:00 UTC
CfT introduction conference call 2 2021/03/10T15:00 UTC
Notification of intention to submit proposal 2021/03/16T23.59 UTC
Submission deadline 2021/04/12T23.59 UTC
Evaluation of responses will start 2021/04/14 (MPAI-7)

Evaluation to be carried out during 2-hour sessions according to the calendar agreed at MPAI-7.

5        References

  1. MPAI-AIF Use Cases & Functional Requirements, N74; https://mpai.community/standards/mpai-aif/
  2. MPAI-AIF Call for Technologies, N100; https://mpai.community/standards/mpai-aif/#Technologies
  3. MPAI-AIF Framework Licence, MPAI N171; https://mpai.community/standards/mpai-aif/#Licence
  4. MPAI-CAE Use Cases & Functional Requirements; MPAI N151; https://mpai.community/standards/mpai-cae/#UCFR
  5. MPAI-CAE Call for Technologies, MPAI N152; https://mpai.community/standards/mpai-cae/#Technologies
  6. MPAI-CAE Framework Licence, MPAI N171; https://mpai.community/standards/mpai-cae/#Licence
  7. MPAI-MMC Use Cases & Functional Requirements; MPAI N153; https://mpai.community/standards/mpai-mmc/#UCFR
  8. MPAI-MMC Call for Technologies, MPAI N154; https://mpai.community/standards/mpai-mmc/#Technologies
  9. MPAI-MMC Framework Licence, N173; https://mpai.community/standards/mpai-mmc/#Licence

Annex A: Information Form

This information form is to be filled in by a Respondent to the MPAI-CAE CfT

  1. Title of the proposal
  2. Organisation: company name, position, e-mail of contact person
  3. What are the main functionalities of your proposal?
  4. Does your proposal provide or describe a formal specification and APIs?
  5. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

Annex B: Evaluation Sheet

NB: This evaluation sheet will be filled out by members of the Evaluation Team.

Proposal title:

Main Functionalities:

Response summary: (a few lines)

Comments on Relevance to the CfT (Requirements):

Comments on possible MPAI-CAE profiles[1]

Evaluation table:

Table 3Assessment of submission features

Note 1 The semantics of Submission features is provided by Table 4
Note 2 Evaluation elements indicate the elements used by the evaluator in assessing the submission
Note 3 Final Assessment indicates the ultimate assessment based on the Evaluation Elements

 

Submission features Evaluation elements Final Assessment
Completeness of description

Understandability

Extensibility

Use of Standard Technology

Efficiency

Test cases

Maturity of reference implementation

Relative complexity

Support of MPAI use cases

Support of non-MPAI use cases

Content of the criteria table cells:

Evaluation facts should mention:

  • Not supported / partially supported / fully supported.
  • What supported these facts: submission/presentation/demo.
  • The summary of the facts themselves, e.g., very good in one way, but weak in another.

Final assessment should mention:

  • Possibilities to improve or add to the proposal, e.g., any missing or weak features.
  • How sure the evaluators are, i.e., evidence shown, very likely, very hard to tell, etc.
  • Global evaluation (Not Applicable/ –/ – / + / ++)

New Use Cases/Requirements Identified:

(please describe)

  •  Evaluation summary:
  •  Main strong points, qualitatively:
  •  Main weak points, qualitatively:
  • Overall evaluation: (0/1/2/3/4/5)

0: could not be evaluated

1: proposal is not relevant

2: proposal is relevant, but requires significant more work

3: proposal is relevant, but with a few changes

4: proposal has some very good points, so it is a good candidate for standard

5: proposal is superior in its category, very strongly recommended for inclusion in standard

Additional remarks: (points of importance not covered above.)

The submission features in Table 3 are explained in the following Table 4.

Table 4 – Explanation of submission features

Submission features Criteria
Completeness of description Evaluators should

1.     Compare the list of requirements (Annex C of the CfT) with the submission.

2.     Check if respondents have described in sufficient detail to what part of the requirements their proposal refers to.

NB1: Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated.

NB2: Submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology.

Understandability Evaluators should identify items that are demonstrably unclear (incon­sistencies, sentences with dubious meaning etc.)
Extensibility Evaluators should check if respondent has proposed extensions to the Use Cases.

NB: Extensibility is the capability of the proposed solution to support use cases that are not supported by current requirements.

Use of standard Technology Evaluators should check if new technologies are proposed where widely adopted technologies exist. If this is the case, the merit of the new tech­nology shall be proved.
Efficiency Evaluators should assess power consumption, computational speed, computational complexity.
Test cases Evaluators should report whether a proposal contains suggestions for testing the technologies proposed
Maturity of reference implementation Evaluators should assess the maturity of the proposal.

Note 1: Maturity is measured by the completeness, i.e., having all the necessary information and appropriate parts of the HW/SW implementation of the submission disclosed.

Note 2: If there are parts of the implementation that are not disclosed but demonstrated, they will be considered if and only if such components are replicable.

Relative complexity Evaluators should identify issues that would make it difficult to implement the proposal compared to the state of the art.
Support of MPAI-CAE use cases Evaluators should check how many use cases are supported in the submission
Support of non MPAI-CAE use cases Evaluators should check whether the technologies proposed can demonstrably be used in other significantly different use cases.

Annex C: Requirements check list

Please note the following acronyms

KB Knowledge Base
QF Query Format

Table 5 – List of technologies identified in MPAI-CAE N151 [1]

Note: The numbers in the first column refer to the section numbers of N151 [1].

Technologies by Use Cases Response
Emotion-Enhanced Speech
4.2.4.1 Digital Speech Y/N
4.2.4.2 Emotion Y/N
4.2.4.3 Emotion KB query format Y/N
4.2.4.4 Emotion descriptors Y/N
Audio Recording Preservation
4.3.4.1 Digital Audio Y/N
4.3.4.2 Digital Video Y/N
4.3.4.3 Digital Image Y/N
4.3.4.4 Tape irregularity KB query format Y/N
4.3.4.5 Text Y/N
4.3.4.6 Packager Y/N
Enhanced Audioconference Experience
4.4.4.1 Digital Speech Y/N
4.4.4.2 Microphone geometry information Y/N
4.4.4.3 Output device acoustic model metadata KB query format Y/N
4.4.4.4 Delivery Y/N
Audio-on-the-go
4.5.4.1 Digital Audio Y/N
4.5.4.2 Microphone geometry information Y/N
4.5.4.3 Sound array Y/N
4.5.4.4 Sound categorisation KB query format Y/N
4.5.4.5 Sounds categorisation Y/N
4.5.4.6 User Hearing Profiles KB query format Y/N
4.5.4.7 Delivery Y/N

Respondent should consult the equivalent list in N154 [8] as some technologies are common or have a degree of similarity.

Annex D: Technologies that may require specific testing

Emotion-Enhanced Speech Speech features
Emotion-Enhanced Speech Emotion descriptors
Audio Recording Preservation Image features

Additional technologies may be identified during the evaluation phase.

Annex E: Mandatory text in responses

A response to this MPAI-CAE CfT shall mandatorily include the following text

<Company/Member> submits this technical document in response to MPAI Call for Technologies for MPAI project MPAI-CAE (N151).

 <Company/Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular <Company/Member> declares that  <Com­pany/Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-CAE (N171), alone or jointly with other IPR holders after the approval of the MPAI-CAE Technical Specif­ication by the General Assembly and in no event after commercial implementations of the MPAI-CAE Technical Specification become available on the market.

In case the respondent is a non-MPAI member, the submission shall mandatorily include the following text

If (a part of) this submission is identified for inclusion in a specification, <Company>  understands that  <Company> will be requested to immediately join MPAI and that, if  <Company> elects not to join MPAI, this submission will be discarded.

Subsequent technical contribution shall mandatorily include this text

<Member> submits this document to MPAI-CAE Development Committee (CAE-DC) as a con­tribution to the development of the MPAI-CAE Technical Specification.

 <Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular  <Company> declares that <Company> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-CAE (N171), alone or jointly with other IPR holders after the approval of the MPAI-CAE Technical Specification by the General Assembly and in no event after commercial implementations of the MPAI-CAE Technical Specification become available on the market.

[1] Profile of a standard is a particular subset of the technologies that are used in a standard and, where applicable, the classes, subsets, options and parameters relevan for the subset

 


Clarifications of Call for Technologies

MPAI-5 has approved the MPAI-CAE Use Cases and Functional Requirements (N151) as attachment to the Calls for Technologies N171. However, the source CAE-DC has identified some issues that are worth a clarificaton. This is posted on the MPAI web site and will be com­mun­icated directly to those who have informed the Secretariat of their intention to respond.

General issue

MPAI understands that the scope of N151 is very broad. Therefore it reiterates the point made in N152 that:

Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated. However, submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology.

Emotion-Enhanced Speech (Use case #1 in N151)

The Functional Requirements of the Use Case does not explicitly indicate the form in which speech without emotion and Emotion enter the EES system. Possible modalities are

  1. A speech file and a separate Emotion file where the sequence of Emotions carries time stamps
  2. An interleaved stream of speech and Emotions

MPAI welcomes proposals addressing these issues.

The assessment of submissions by Respondents who elect not to not answer this point will not influence the assessment of the rest of their submission

Audio Recording Preservation (Use case #2 in N151)

MPAI welcomes proposed semantics of the information conveyed in the Text output of the Musicological classifier. It is not clear what information the Musicological classifier is providing.

The assessment of submissions by Respondents who elect not to not answer this point will not influence the assessment of the rest of their submission

Enhanced Audioconference Experience (Use case #3 in N151)

The function of the Speech detection and separation AIM is described as “Separation of relevant speech vs non-speech signals”.

As the description is possibly misleading, we inform submitters that the correct description of the AIM should be “Separation of relevant speech vs other signals (including unwanted speech)”.

The assessment of submissions by Respondents who base their submission on the text in N151 will not be affected by this clarification.

References

  1. MPAI-CAE Use Cases & Functional Requirements; MPAI N151; https://mpai.community/standards/mpai-cae/#UCFR
  2. MPAI-CAE Call for Technologies, MPAI N152; https://mpai.community/standards/mpai-cae/#Technologies
  3. MPAI-CAE Framework Licence, MPAI N171; https://mpai.community/standards/mpai-cae/#Licence


Use Cases and Functional Requirements

Artificial Intelligence Framework (MPAI-AIF)

1       Introduction

2       Use Cases. 1

2.1       Context-based Audio Enhancement (MPAI-CAE) 2

2.2       Integrative Genomic/Sensor Analysis (MPAI-GSA) 3

2.3       AI-Enhanced Video Coding (MPAI-EVC) 6

2.4       Server-based Predictive Multiplayer Gaming (MPAI-SPG) 7

2.5       Multi-Modal Conversation (MPAI-MMC) 8

2.6       Compression and Understanding of Industrial data (MPAI-CUI) 9

3       Architecture. 10

4       Requirements. 11

4.1       Component requirements. 11

4.2       Systems requirements. 11

4.3       General requirements. 12

5       Conclusions. 12

6       References. 12

1        Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international association with the mission to develop AI-enabled data coding standards. Artificial Intelligence (AI) technologies have shown they can offer more efficient data coding than existing technol­ogies.

MPAI has analysed six use cases covering applic­ation areas benefiting from AI technol­ogies. Even though use cases are disparate, each of them can be implemented with a combination of processing modules performing functions that concur to achieving the inten­ded result.

MPAI has assessed that, leaving it to the market to develop individual implementations, would multiply costs and delay adoption of AI technologies, while modules with standard interfaces, combined and executed within the MPAI-specified AI Framework, will favour the emergence of horizontal markets where proprietary and competing module implemen­tations exposing standard interfaces will reduce cost, promote adoption and incite progress of AI technologies. MPAI calls these modules AI Modules (AIM).

MPAI calls the planned AI Framework standard as MPAI-AIF. As AI is a fast-moving field, MPAI expects that MPAI-AIF will be extended as new use cases will bring new requirements and new technologies will reach maturity.

To avoid the deadlock experienced in other high-technology fields, before engaging in the development of the the MPAI-AIF standard, MPAI will develop a Frame­work Licence (FWL) associated with the MPAI-AIF Architecture and Functional Requir­ements defined in this document. The FWL, essentially the business model that standard essential patent (SEP) holders will apply to monetise their Intellectual Properties (IP), but without values such as the amount or percentage of royalties or dates due, will act as Commercial Requirements for the standard and provide a clear IPR licensing framework.

This document contains a summary description of the six use cases (Section 2) followed by a section describing the architecture expected to become normative (Section 3). Section 4 lists the normative requirements identified so far.

2        Use Cases

The six use cases considered cover a broad area of application. Therefore, it is expected that the MPAI-AIF architecture can support a wide range of use cases of practical interest.

Each case is identified by its name and the acronym identifying the future MPAI standard. More information about MPAI-AIF can be found in [1].

2.1       Context-based Audio Enhancement (MPAI-CAE)

The overall user experience quality is highly dependent on the context in which audio is used, e.g.

  1. Entertainment audio can be consumed in the home, in the car, on public transport, on-the-go (e.g. while doing sports, running, biking) etc.
  2. Voice communications: can take place in the office, in the car, at home, on-the-go etc.
  3. Audio and video conferencing can be done in the office, in the car, at home, on-the-go etc.
  4. (Serious) gaming can be done in the office, at home, on-the-go etc.
  5. Audio (post-)production is typically done in the studio
  6. Audio restoration is typically done in the studio

By using context information to act on the content using AI, it is possible substantially to improve the user experience.

The following examples describe how MPAI-CAE can make the difference.

  1. Enhanced audio experience in a conference call

Often, the user experience of a video/audio conference can be marginal. Too much background noise or undesired sounds can lead to participants not understanding what participants are saying. By using AI-based adaptive noise-cancellation and sound enhancement, MPAI-CAE can virtually eliminate those kinds of noise without using complex microphone systems to capture environment characteristics.

  1. Pleasant and safe music listening while biking

While biking in the middle of city traffic, AI can process the signals captured from the environment by the microphones available in many earphones and earbuds (for active noise cancellation), adapt the sound rendition to the acoustic environment, provide an enhanced audio experience (e.g. performing dynamic signal equalization), improve battery life and selectively recognize and allow relevant environment sounds (i.e. the horn of a car). The user enjoys a satisfactory listening experience without losing contact with the acoustic surroundings.

  1. Emotion enhanced synthesized voice

Speech synthesis is constantly improving and finding several applications that are part of our daily life (e.g. intelligent assistants). In addition to improving the ‘natural sounding’ of the voice, MPAI-CAE can implement expressive models of primary emotions such as fear, happiness, sad­ness, and anger.

  1. Efficient 3D sound

MPAI-CAE can reduce the number of channels (i.e. MPEG-H 3D Audio can support up to 64 loudspeaker channels and 128 codec core channels) in an automatic (unsupervised) way, e.g. by mapping a 9.1 to a 5.1 or stereo (radio broadcasting or DVD), maintaining the musical touch of the composer.

  1. Speech/audio restoration

Audio restoration is often a time-consuming process that requires skilled audio engineers with specific experience in music and recording techniques to go over manually old audio tapes. MPAI-CAE can automatically remove anomalies from recordings through broadband denoising, declicking and decrackling, as well as removing buzzes and hums and performing spectrographic ‘retouching’ for removal of discrete unwanted sounds.

  1. Normalization of volume across channels/streams

Eighty-five years after TV has been first introduced as a public service, TV viewers are still strug­gling to adapt to their needs the different average audio levels from different broadcasters and, within a program, to the different audio levels of the different scenes.

MPAI-CAE can learn from user’s reactions via remote control, e.g. to a loud spot, and control the sound level accordingly.

  1. Automotive

Audio systems in cars have steadily improved in quality over the years and continue to be integrated into more critical applications. Today, a buyer takes it for granted that a car has a good automotive sound system. In addition, in a car there is usually at least one and sometimes two microphones to handle the voice-response system and the hands-free cell-phone capability. If the vehicle uses any noise cancellation, several other microphones are involved. MPAI-CAE can be used to improve the user experience and enable the full quality of current audio systems by reduc­ing the effects of the noisy automotive environment on the signals.

  1. Audio mastering

Audio mastering is still considered as an ‘art’ and the prerogative of pro audio engineers. Normal users can upload an example track of their liking (possibly obtained from similar musical content) and MPAI-CAE analyzes it, extracts key features and generate a master track that ‘sounds like’  the example track starting from the non-mastered track.  It is also possible to specify the desired style without an example and the original track will be adjusted accordingly.

More details on MPAI-CAE are found in [2,7]

2.2       Integrative Genomic/Sensor Analysis (MPAI-GSA)

Most experiment in quantitative genomics consist of a setup whereby a small amount of metadata – observable clinical score or outcome, desirable traits, observed behaviour – is correlated with, or modelled from, a set of data-rich sources. Such sources can be:

  1. Biological experiments – typically sequencing or proteomics/metabolomics data
  2. Sensor data – coming from images, movement trackers, etc.

All these data-rich sources share the following properties:

  1. They produce very large amounts of “primary” data as output
  2. They need “primary”, experiment-dependent, analysis, in order to project the primary data (1) onto a single point in a “secondary”, processed space with a high dimensionality – typically a vector of thousands of values
  3. The resulting vectors, one for each experiment, are then fed to some machine or statistical learning framework, which correlates such high-dimensional data with the low-dimensional metadata available for the experiment. The typical purpose is to either model the high-dimensional data in order to produce a mechanistic explanation for the metadata, or to produce a predictor for the metadata out of the high-dimensional data.
  4. Although that is not typically necessary, in some circumstances it might be useful for the statistical or machine learning algorithm to be able to go back to the primary data (1), in order to extract more detailed information than what is available as a summary in the processed high-dimensional vectors produced in (2).

It would be extremely beneficial to provide a uniform framework to:

  1. Represent the results of such complex, data-rich, experiments, and
  2. Specify the way the input data is processed by the statistical or machine learning stage

Although the structure above is common to a number of experimental setups, it is conceptual and never made explicit. Each “primary” data source can consist of heterogeneous information represented in a variety of formats, especially when genomics experiments are considered, and the same source of information is usually represented in different ways depending on the analysis stage – primary or secondary. That results in data processing workflows that are ad-hoc – two experiments combining different sets of sources will require two different workflows able to process each one a specific combination of input/output formats. Typically, such workflows will also be layered out as a sequence of completely separated stages of analysis, which makes it very difficult for the machine or statistical learning stage to go back to primary data when that would be necessary.

MPAI-GSA aims to create an explicit, general and reusable framework to express as many different types of complex integrative experiments as possible. That would provide (I) a compressed, optimized and space-efficient way of storing large integrative experiments, but also (II) the possibility of specifying the AI-based analysis of such data (and, possibly, primary analysis too) in terms of a sequence of pre-defined standardized algorithms. Such computational blocks might be partly general and prior-art (such as standard statistical algorithms to perform dimensional reduction) and partly novel and problem-oriented, possibly provided by commercial partners. That would create a healthy arena whereby free and commercial methods could be combined in a number of application-specific “processing apps”, thus generating a market and fostering innovation. A large number of actors would ultimately benefit from the MPAI-GSA standard – researchers performing complex experiments, companies providing medical and commercial services based on data-rich quantitative technologies, and the final users who would use instances of the computational framework as deployed “apps”.

The following examples describe typical uses of the MPAI-GSA framework.

  1. Medical genomics – sequencing and variant-calling workflows

In this use case, one would like to correlate a list of genomic variants present in humans and having a known effect on health (metadata) with the variants present in a specific individual (secondary data). Such variants are derived from sequencing data for the individual (primary data) on which some variant calling workflow has been applied. Notably, there is an increasing number of companies doing just that as their core business. Their products differ by: the choice of the primary processing workflow (how to call variants from the sequencing data for the individual); the choice of the machine learning analysis (how to establish the clinical importance of the variants found); and the choice of metadata (which databases of variants with known clinical effect to use). It would be easy to re-deploy their workflows as MPAI-GSA applications.

  1. Integrative analysis of ‘omics datasets

In this use case, one would like to correlate some macroscopic variable observed during a biological process (e.g. the reaction to a drug or a vaccine – metadata) with changes in tens of thousands of cell markers (gene expression estimated from RNA; amount of proteins present in the cell – secondary data) measured through a combination of different high-throughput quantitative biological experiments (primary data – for instance, RNA-sequencing, ChIP-sequencing, mass spectrometry). This is a typical application in research environments (medical, veterinary and agricultural). Both primary and secondary analysis are performed with a variety of methods depending on the institution and the provider of bioinformatics services. Reformulating such methods in terms of MPAI-GSA would help reproducibility and standardisation immensely. It would also provide researchers with a compact way to store their heterogeneous data.

  1. Single-cell RNA-sequencing

Similar to the previous one, but in this case at least one of the primary data sources is RNA-sequencing performed at the same time on a number (typically hundred of thousands) of different cells – while bulk RNA sequencing mixes together RNAs coming from several thousands of different cells, in single-cell RNA sequencing the RNAs coming from each different cell are separately barcoded, and hence distinguishable. The DNA barcodes for each cell would be metadata here. Cells can then be clustered together according to the expression patterns present in the secondary data (vectors of expression values for all the species of RNA present in the cell) and, if sufficient metadata is present, clusters of expression patterns can be associated with different types/lineages of cells – the technique is typically used to study tissue differentiation. A number of complex algorithms exist to perform primary analysis (statistical uncertainty in single-cell RNA-sequencing is much bigger than in bulk RNA-sequencing) and, in particular, secondary AI-based clustering/analysis. Again, expressing those algorithms in terms of MPAI-GSA would make them much easier to describe and much more comparable. External commercial providers might provide researchers with clever modules to do all or part of the machine learning analysis.

  1. Experiments correlating genomics with animal behaviour

In this use case, one wants to correlate animal behaviour (typically of lab mice) with their genetic profile (case of knock-down mice) or the previous administration of drugs (typically encountered in neurobiology). Hence primary data would be video data from cameras tracking the animal; secondary data would be processed video data in the form of primitives describing the animal’s movement, well-being, activity, weight, etc.; and metadata would be a description of the genetic background of the animal (for instance, the name of the gene which has been deactivated) or a timeline with the list and amount of drugs which have been administered to the animal. Again, there are several companies providing software tools to perform some or all of such analysis tasks – they might be easily reformulated in terms of MPAI-GSA applications.

  1. Spatial metabolomics

One of the most data-intensive biological protocols nowadays is spatial proteomics, whereby in-situ mass-spec/metabolomics techniques are applied to “pixels”/”voxels” of a 2D/3D biological sample in order to obtain proteomics data at different locations in the sample, typically with sub-cellular resolution. This information can also be correlated with pictures/tomograms of the sample, to obtain phenotypical information about the nature of the pixel/voxel. The combined results are typically analysed with AI-based technique. So primary data would be unprocessed metabolomics data and images, secondary data would be processed metabolomics data and cellular features extracted from the images, and metadata would be information about the sample (source, original placement within the body, etc.). Currently the processing of spatial metabolomics data is done through complex pipelines, typically in the cloud – having these as MPAI-GSA applications would be beneficial to both the researchers and potential providers of computing services.

  1. Smart farming

During the past few years, there has been an increasing interest in data-rich techniques to optimise livestock and crop production (so called “smart farming”). The range of techniques is constantly expanding, but the main ideas are to combine molecular techniques (mainly high-throughput sequencing and derived protocols, such as RNA-sequencing, ChIP-sequencing, HiC, etc.; and mass-spectrometry – as per the ‘omics case at point 2) and monitoring by images (growth rate under different conditions, sensor data, satellite-based imaging) for both livestock species and crops. So this use case can be seen as a combination of cases 2 and 4. Primary sources would be genomic data and images; secondary data would be vectors of values for a number of genomic tags and features (growth rate, weight, height) extracted from images; metadata would be information about environmental conditions, spatial position, etc. A growing number of companies are offering services in this area – again, having the possibility of deploying them as MPAI-GSA applications would open up a large arena where academic or commercial providers would be able to meet the needs of a number of customers in a well-defined way.

More details on MPAI-GSA are found in [3,8].

2.3       AI-Enhanced Video Coding (MPAI-EVC)

MPAI has carried out an investigation on the performance improvement of AI-enhanced HEVC, AI-enhanced VVC and End-to-end AI-based video coding. Preliminary evidence offered by the investigation suggests that by replacing and/or enhancing existing sel­ected HEVC and VVC coding tools with AI-based tools, the objectively measured compres­sion performance may be im­proved by up to around 30%. These results were obtained by combining somewhat heterog­en­eous data from experiments reported in the liter­ature.

The reported initial results, however, do indicate that AI can bring significant im­prov­ements to existing video coding technologies. Therefore, MPAI is investigating the feasibility of improving the coding efficiency by about 25% to 50% over an existing standard with an acceptable increase in complexity using technologies reported in the literature. If the investigation is successful, MPAI will develop a standard called MPAI AI-Enhanced Video Coding (MPAI-EVC).

The investigation showed that encouraging results can be obtained from new types of AI-based coding schemes – called end-to-end. These schemes, while promising, still need substantial more research.

MPAI is also aware of ongoing research targeted at hybrid schemes where AI-based technologies are added to the existing codecs as an enhancement layer without making any change to the base-layer codec itself, thus providing backward-compatible solutions.

At this stage MPAI conducts two parallel activities

  1. Collaborative activity targeting a scientifically sound assessment of the improvements achieved by state-of-the-art research. To the extent possible this should be done with the participation of the authors of major improvements
  2. Thorough development of requirements that the MPAI-EVC should satisfy.

The choice of the starting point (the existing codec), starting from which an AI-enhanced video codec should be developed, is an issue because high-performance video codecs have typically many essential patents (SEP) holders. They should all be convinced to allow MPAI to extend the selected starting point with AI-based tools that satisfy the – still to be defined – MPAI-EVC framework licence. As the result of such an endeavour is not guaranteed, MPAI is planning to pick Essential Video Coding (MPEG-5 EVC) as the starting point. EVC baseline is reported not to be encumbered by IPR.  Additionally, EVC Main Profile is reported to have a limited number of SEP holders. As an EVC patent holder has announced the release of a full implem­entation of EVC as Open Source Software (OSS), the choice of EVC as the starting point would also make available a working code base. The choice between the EVC baseline and main profile is TBD.

The following figures represent the block diagrams of 3 potential configurations to be adopted by the MPAI-EVC standard.

The green circles of Figure 1 indicate traditional video coding tools that could be enhanced or replaced by AI-enabled tools. This will be taken as the basis of the collaborative activity men­tioned above.

Figure 1 A reference diagram for the Horizontal Hybrid approach

In Figure 2 a traditional video codec is enhanced by an AI Enhancement codec.

Figure 2 A reference diagram for the Vertical Hybrid approach

More details on MPAI-EVC are found in [4,10.11.12]

2.4       Server-based Predictive Multiplayer Gaming (MPAI-SPG)

There are two basic approaches to online gaming:

  1. Traditional online gaming: the server receives a sequence of data from the client(s) and sends an input-dependent sequence of data to the client(s) which use the data to create appropriate video frames.
  2. Cloud gaming: the server receives a sequence of data from the client(s) and sends an input-dependent sequence of video frames to the client(s). In a cloud gaming scenario, all clients run in the cloud on virtual machines.

In case the connection has temporary high latency or packet loss (in the following called network disruption) two strategies may be used to make up for missing information

  1. Client-side prediction when information from the client does not reach the server or from the server does not reach the client
  2. Server-side prediction when information from the client does not reach the server

In a game a finite state Game machine calculates the logic of the game from the inputs received from the game controller. The client reacts to such user input before the server has acknowledged the input and updated its Game state. If an updated Game state from the server is missing, the client predicts the Game state locally and produces a video frame that is potentially wrong. When the right information from the server reaches the client, the client Game state is reconciliated with the server Game state.

For example, in a first-person shooter game where a fast pace shooting takes place, player A aims their weapon at the position that player B held some milliseconds in the past. Therefore, when Player A fires, player B is long gone. In order to remediate this, when the server gets the information regarding the shooting, which is precise because it carries timestamps, it knows exactly where player A’s the weapon was aiming at and the past position of player B. The server thus processes the shot at that past time, reconciles all the states and updates the clients. In spite of the belated reconciliation, however. the situation is not satisfactory because player B was shot in the past and, in the few milliseconds of differ­ence, player B may have taken cover.

AI and ML can provide more efficient solutions than available today to compensate network disruption and improve user experience to both traditional online gaming and cloud-based gaming.

An AI machine can collect data from multiple clients, perform a much better prediction of each move of each participant and perform sophisticated reconcil­iations. Information from the game engine – inputs from clients and reconciliation info – can be used in the video encoding process to improve the encoding (i.e. motion estimation), thus making possible encoding at higher frame rates for a high-quality gaming exper­ience even on low performing hardware.

Here are two examples of known games that illustrate how MPAI standards can feasibility and user experience.

Example 1: Racing games

During an online racing game, players can see lagging reactions to their moves and an overall low-quality presentation because of network disruption. Usually the information on the screen predic­ted by a client is wrong if the online client information cannot reach the server and the clients involved in the online game on time. In a car racing game, the player may see at time t0 a vehicle going straight to the wall when it is reaching a curve. At time t1, after some seconds, the same vehicle is “teleported” to the actual position yielding an awful player experience.

AI can mitigate this issue, offering a better game experience to players. Data from the different online games are collected and used to predict a meaningful path or the correct behaviour in the time information does not reach the clients.

Example 2: Zombie games

In some traditional online video games, specific information is displayed differently on different clients because it is too onerous to compute all the outcomes of players’ actions in a physically consistent way. An example is provided by zombie games: the result of killing hordes of zombies in each client is visually different from client to client.

A server-based predictive input can support the online architecture and enable it to provide an equal outcome on all clients. In a massive multiplayer hack&slash game, the result of the different combats among players yields the same live visual online experience to each player.

More details on MPAI-SPG are found in [5]

2.5       Multi-Modal Conversation (MPAI-MMC)

A useful application of AI is in the conversational partner which provides the user with information, entertains, chats and answers questions through the speech interface. However, an application should include more than just a speech interface to provide a better service to the user. For example, emotion recognizer and gesture interpreter are needed for better multi-modal interfaces.

Multi-modal conversation (MPAI-MMC) aims to enable human-machine conversation that emulates human-human conversation in completeness and intensity by using AI.

The example of MMC is the conversation between a human user and a computer/robot as in the following list. The input from the user can be voice, text or image or combination of different inputs. Considering emotion of the human user, MMC will output responses in a text, speech, music depending on the user’s needs.

  • Chats: “I am bored. What should I do now?” – “You look tired. Why don’t you take a walk?”
  • Question Answering: “Who is the famous artist in Barcelona?” – “Do you mean Gaudi?”
  • Information Request: “What’s the weather today?” – “It is a little cloudy and cold.”
  • Action Request: “Play some classical music, please” – “OK. Do you like Brahms?”

More details on MPAI-MMC are found in [6]

2.6       Compression and Understanding of Industrial data (MPAI-CUI)

Most economical organizations, e.g., companies, etc., produce large quantities of data, often because these are required by regulation. Users of these data maybe the company itself or Fintech and Insurtech services who need to access the flow of company data to assess and mon­itor financial and organizational performance, as well as the impact of vertical risks (e.g., cyber, seismic, etc.).

The sheer amount of data that need to be exchanged is an issue. Analysing those data by humans is typically on­erous and may miss vitally important information. Artificial intelligence (AI) may help reduce the amount of data with a controlled loss of information and extract the most relevant information from the data. AI is considered the most promising means to achieve the goal.

Unfortunately, the syntax and semantics of the flow of data is high dependent on who has produced the data. The format of the date is typically a text file with a structure not designed for indexing, search and ex­traction. Therefore, in order to be able to apply AI technologies to meaningfully reduce the data flow, it is necessary to standardize the formats of the components of the data flow and make the data “AI friendly”.

Recent regulations are imposing a constant monitoring (ideally monthly). Thus, there is the pos­sibility to have similar blocks of data in temporally consecutive sequences of data.

The company generating the flow data may need to perform compression for its own need (e.g., identifying core and non-core data). Subsequent entities may perform further data compres­sion.

In general, compressed data should allow for easy data search and extraction.

MPAI-CUI may be used in a variety of contexts

  1. To support the company’s board in deploying efficient strategies. A company can analyse its financial performance, identifying possible clues to the crisis or risk of bankruptcy years in advance. It may help the board of directors and decision-makers to make the proper decisions to avoid these situations, conduct what-if analysis, and devise efficient strategies.
  2. To assess the financial health of companies that apply for funds/financial help. A financial institution that receives a request for financial help from a troubled company, can access its financial and organizational data and make an AI-based assessment of that company, as well as a prediction of future performance. This aids the financial institution to take the right decision in funding or not that company, having a broad vision of its situation.
  3. To assess the risk in different fields considering non-core data (e.g., non-financial data). Accurate and targeted sharing of core and non-core data that ranges from the financial and organizational information to other types of risks that affect the business continuity (e.g., environmental, seismic, infrastructure, and cyber).
  4. To analyse the effects of disruptions on the national economy, e.g., performance evaluation by pre/post- pandemic analysis.

3        Architecture

The normative MPAI-AIF architecture enables the creation and automation of mixed ML-AI-DP processing and inference workflows at scale for the use cases considered above. It includes six basic normative elements of the Architecture called Components addressing different modalities of operation – AI, Machine Learning (ML) and Data Processing (DP), data pipelines jungles and computing resource allocations including constrained hardware scenarios of edge AI devices.

The normative reference diagram of MPAI-AIF is given by the following figure where APIs be­tween different Components at different level are shown.

Figure 3 – Proposed normative MPAI-AIF Architecture

  1. Management and Control

Management concerns the activation/disactivation/suspensions of AIMs, while Control supports complex application scenarios.

Management and Control handles simple orchestration tasks (i.e. represented by the execution of a script) and much more complex tasks with a topology of networked AIMs that can be syn­chronised according to a given time base and full ML life cycles.

  1. Execution

The environment where AIMs operate. It is interfaced with Management and Control and with Communication and Storage. It receives external inputs and produces the requested outputs both of which are application specific.

  1. AI Modules (AIM)

AIMs are units comprised of at least the following 3 functions:

  1. The processing element (ML or traditional DP)
  2. Interface to Communication and Storage
  3. Input and output interfaces (function specific)

AIMs can implement auto-configuration or reconfiguration of their ML-based computational models.

  1. Communication

Communication is required in several cases and can be implemented accordingly, e.g. by means of a service bus. Components can communicate among themselves and with outputs and Storage.

The Management and Control API implements one- and two-way signalling for computational workflow initialisation and control.

  1. Storage

Storage encompasses traditional storage and is referred to a variety of data types, e.g.:

  1. Inputs and outputs of the individual AIMs
  2. Data from the AIM’s state, e.g. with respect to traditional and continuous learning
  3. Data from the AIM’s intermediary results
  4. Shared data among AIMs
  5. Information used by Management and Control.
  6. Access

Access represents the access to static or slowly changing data that are required by the application such as domain knowledge data, data models, etc.

4        Requirements

4.1       Component requirements

  1. The MPAI-AIF standard shall include specifications of the interfaces of 6 Components
    1. Management and Control
    2. Execution
    3. AI Modules (AIM)
    4. Communication
    5. Storage
    6. Access
  2. MPAI-AIF shall support configurations where Components are distributed in the cloud and at the edge
  3. Management and Control shall enable operations on the general Machine Learning and/or traditional Data Processing life cycle of
    1. Single AIMs, e.g. instantiation-configuration-removal, internal state dumping/retrieval, start-suspend-stop, train-retrain-update, enforcement of resource limits
    2. Combinations of AIMs, e.g. initialisation of the overall computational model, instan­tiation-removal-configuration of AIMs, manual, automatic, dynamic and adaptive configuration of interfaces with Components.
  4. Management and Control shall support
    1. Architectures that allow application-scenario dependent hierarchical execution of workflows, i.e. a combination of AIMs into computational graphs
    2. Supervised, unsupervised and reinforcement-based learning paradigms
    3. Computational graphs, such as Direct Acyclic Graph (DAG) as a minimum
    4. Initialisation of signalling patterns, communication and security policies between AIMs
  5. Storage shall support protocols to specify application-dependent requirements such as access time, retention, read/write throughput
  6. Access shall provide
    1. Static or slowly changing data with standard formats
    2. Data with proprietary formats

4.2       Systems requirements

The following requirements are not intended to apply to the MPAI-AIF standard, but should be used for assessing technologies

  1. Management and Control shall support asynchronous and time-based synchronous operation depending on application
  2. The Architecture shall support dynamic update of the ML models with seamless or minimal impact on its operation
  3. ML-based AIMs shall support time sharing operation enabling use of the same ML-based AIM in multiple concurrent applications
  4. AIMs may be aggregations of AIMs exposing new interfaces
  5. Complexity and performance shall be scalable to cope with different scenarios, e.g. from small MCUs to complex distributed systems
  6. The Architecture shall support workflows of a mixture of AI/ML-based and DP technology-based AIMs.

4.3       General requirements

The MPAI-AIF standard may include profiles for specific (sets of) requirements

5        Conclusions

When the definition of the MPAI-AIF Framework Licence will be completed, MPAI will issue a Call for Technologies that support the AI Framework with the requirem­ents given in this document.

Respondents will be requested to state in their submissions their intention to adhere to the Frame­work Licence developed for MPAI-AIF when licensing their technologies if they have been inc­luded in the MPAI-AIF standard.

The MPAI-AIF Framework Licence will be developed, as for all other MPAI Framework Licences, in compliance with the gener­ally accepted principles of competition law.

6        References

[1] MPAI Application Note#4 – MPAI-AIF Artificial Intelligence Framework

[2] MPAI Application Note#1 R1 – MPAI-CAE Context-based Audio Enhancement

[3] MPAI Application Note#2 R1 – MPAI-GSA Integrative Genomic/Sensor Analysis

[4] MPAI Application Note#3 R1 – MPAI-EVC AI-Enhanced Video Coding

[5] MPAI Application Note#5 R1 – MPAI-SPG Server-based Predictive Multiplayer Gaming

[6] MPAI Application Note#6 R1 – MPAI-MMC Multi-Modal Conversation

[7] MPAI-CAE Functional Requirements work programme

[8] MPAI-GSA Functional Requirements work programme

[9] MPAI-MMC Functional Requirements work programme

[10] MPAI-EVC Use Cases and Requirements

[11] Collaborative Evidence Conditions for MPAI-EVC Evidence Project R1

[12] Operational Guidelines for MPAI-EVC Evidence Project


Call for technologies

Multimodal Conversation

This document is also available in MS Word format MPAI-MMC Call for Technologies

1       Introduction.

2       How to submit a response.

3       Evaluation Criteria and Procedure.

4       Expected development timeline.

5       References.

Annex A: Information Form..

Annex B: Evaluation Sheet

Annex C: Requirements check list

Annex D: Technologies that may require specific testing.

Annex E: Mandatory text in responses.

1        Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international non-profit organisation with the mission to develop standards for Artificial Intelligence (AI) enabled digital data coding and for technologies that facilitate integration of data coding components into ICT systems. With the mechanism of Framework Licences, MPAI seeks to attach clear IPR licensing frameworks to its standards.

MPAI has found that the application area called “Context-based Audio Enhancement” is particul­arly relevant for MPAI standardisation because using context information to act on the input audio content can substantially improve the user experience of a variety of audio-related applications that include entertainment, communication, teleconferencing, gaming, post-produc­tion, restor­ation etc. for a variety of contexts such as in the home, in the car, on-the-go, in the studio etc.

Therefore, MPAI intends to develop a standard – to be called MPAI-MMC – that will provide standard tech­nologies to implement four Use Cases identified so far:

  1. Conversation with emotion
  2. Multimodal Question Answering
  3. Personalized Automatic Speech Translation

This document is a Call for Technologies (CfT) that

  1. Satisfy the MPAI-MMC Functional Requirements of N153 [6] and
  2. Are released according to the MPAI-MMC Framework Licence (N173) [9], if selected by MPAI for in­clusion in the MPAI-MMC standard.

The standard will be developed with the following guidelines:

  1. To satisfy the MPAI-MMC Functional Requirements (N153) [6]. In the future, MPAI may decide to extend MPAI-MMC to support other Use Cases as a part of this MPAI-CAE standard or as a future extension of it.
  2. To use, where feasible and desirable, the same basic tech­nol­ogies required by the companion document MPAI-CAE Use Cases and Functional Requir­ements [3].
  3. To be suitable for implementation as AI Modules (AIM) conforming to the emerging MPAI AI Framework (MPAI-AIF) standard based on the responses to the MPAI-AIF Call for Technologies (N100) [1] MPAI-AIF Functional Requirements (N74) [1].

MPAI has decided to base its application standards on the AIM and AIF notions whose functional requirements have been identified in [1] rather than follow the approach of defining end-to-end systems. It has done so because:

  1. AIMs allow the reduction of large problems to sets of smaller problems.
  2. AIMs can be independently developed and made available to an open competitive market.
  3. An application developer can build a sophisticated and complex system MPAI system with potentially limited know­ledge of all the tech­nologies required by the system.
  4. An MPAI system has a high-level of inherent explainability.
  5. MPAI systems allow for competitive comparisons of functionally equivalent AIMs.

Respondents should be aware that:

  1. Use Cases that make up MPAI-MMC, the Use Cases themselves and the AIM internals will be non-normative.
  2. The input and output interfaces of the AIMs, whose requirements have been derived to support the Use Cases, will be normative.

Therefore, the scope of this Call for Technologies is restricted to technologies required to implement the input and output interfaces of the AIMs identified in N153 [6].

However, MPAI invites comments on any technology or architectural component identified in N153, specifically,

  1. Additions or removals of input/output signals to the identified AIMs with justification of the changes and identification of data formats required by the new input/output signals.
  2. Possible alternative partitioning of the AIMs implementing the example cases providing:
    1. Arguments in support of the proposed partitioning
    2. Detailed specifications of the input and output data of the proposed new AIMs
  3. New Use Cases fully described as in N153.

All parties who believe they have relevant technologies satisfying all or most of the requirements of one or more than one Use Case described in N153 are invited to submit proposals for consid­eration by MPAI. MPAI membership is not a prerequisite for responding to this CfT. However, proponents should be aware that, if their proposal or part thereof is accepted for inclusion in the MPAI-MMC standard, they shall immediately join MPAI, or their accepted technologies will be discarded.

MPAI will select the most suitable technologies based on their technical merits for inclusion in MPAI-MMC. However, MPAI in not obligated, by virtue of this CfT, to select a particular tech­nology or to select any technology if those submitted are found inadequate.

Submissions are due on 2021/04/12T23:59 UTC and should be sent to the MPAI secretariat (secretariat@mpai.community). The secretariat will acknowledge receipt of the submission via email. Submissions will be reviewed according to the schedule that the 7th MPAI General Assembly (MPAI-7) will define at its online meeting on 2021/04/14. For details on how submitters who are not MPAI members can attend the said review please contact the MPAI secretariat (secretariat@mpai.community).

2        How to submit a response

Those planning to respond to this CfT:

  1. Are advised that online events will be held on 2021/02/24 and 2021/03/10 to present the MPAI-MMC CfT and respond to questions. Logistic information on these events will be posted on the MPAI web site.
  2. Are requested to communicate their intention to respond to this CfT with an initial version of the form of Annex A to the MPAI secretariat (secretariat@mpai.community) by 2021/03/16. A potential submitter making a communication using the said form is not required to actually make a submission. A submission will be accepted even if the submitter did not communicate their intention to submit a response by the said date.
  3. Are advised to visit regularly the https://mpai.community/how-to-join/calls-for-technologies/ web site where relevant information will be posted.

Responses to this MPAI-MMC CfT shall/may include:

Table 1 – Mandatory and optional elements of a response

Item Status
Detailed documentation describing the proposed technologies mandatory
The final version of Annex A mandatory
The text of Annex B duly filled out with the table indicating which requirements identified in MPAI N151 [3] are satisfied. If all the requirements of a Use Case are not satisfied, this should be explained. mandatory
Comments on the completeness and appropriateness of the MPAI-MMC functional requirem­ents and any motivated suggestion to amend and/or extend those requir­ements. optional
A preliminary demonstration, with a detailed document describing it. optional
Any other additional relevant information that may help evaluate the submission, such as additional use cases. optional
The text of Annex E. mandatory

Respondents are invited to take advantage of the check list of Annex C before submitting their response and filling out Annex A.

Respondents are mandatorily requested to present their submission at a teleconference meeting to be properly announced to submitters by the MPAI Secretariat. If no presenter od a submission will attend the meeting, the submission will be discarded.

Respondents are advised that, upon acceptance by MPAI of their submission in whole or in part for further evaluation, MPAI will require that:

  • A working implementation, including source code – for use in the development of the MPAI-MMC Reference Software and later publication as an MPAI standard– be made available before the technology is accepted for inclusion in the MPAI-MMC standard. Software may be written in programming languages that can be compiled or interpreted and in hardware description languages.
  • The working implementation be suitable for operation in the MPAI AIF Framework (MPAI-AIF).
  • A non-MPAI member immediately join MPAI. If the non-MPAI member elects not to do so, their submission will be discarded. Direction on how to join MPAI can be found online.

Further information on MPAI can be obtained from the MPAI website.

3        Evaluation Criteria and Procedure

Proposals will be assessed using the following process:

  1. Evaluation panel is created from:
    1. All MMC-DC members attending.
    2. Non-MPAI members who are respondents.
    3. Non respondents/non MPAI member experts invited in a consulting capacity.
  2. No one from 1.1.-1.2. will be denied membership in the Evaluation panel.
  3. Respondents present their proposals.
  4. Evaluation Panel members ask questions.
  5. If required subjective and/or objective tests are carried out:
    1. Define required tests.
    2. Carry out the tests.
    3. Produce report.
  6. At least 2 reviewers will be appointed to review & report about specific points in a proposal if required.
  7. Evaluation panel members fill out Annex B for each proposal.
  8. Respondents respond to evaluations.
  9. Proposal evaluation report is produced.

4        Expected development timeline

Timeline of the CfT, deadlines and response evaluation:

Table 2 – Dates and deadlines

Step Date
Call for Technologies 2021/02/17
CfT introduction conference call 1 2021/02/24T14:00 UTC
CfT introduction conference call 2 2021/03/10T15:00 UTC
Notification of intention to submit proposal 2021/03/16T23.59 UTC
Submission deadline 2021/04/12T23.59 UTC
Evaluation of responses will start 2021/04/14 (MPAI-7)

Evaluation to be carried out during 2-hour sessions according to the calendar agreed at MPAI-7.

5        References

  1. MPAI-AIF Use Cases and Functional Requirements, N74; https://mpai.community/standards/mpai-aif/
  2. MPAI-AIF Call for Technologies, MPAI N100; https://mpai.community/standards/mpai-aif/#Technologies
  3. MPAI-AIF Framework Licence, MPAI N171; https://mpai.community/standards/mpai-aif/#Licence
  4. MPAI-CAE Use Cases & Functional Requirements; MPAI N151; https://mpai.community/standards/mpai-cae/#UCFR
  5. MPAI-CAE Call for Technologies, MPAI N152; https://mpai.community/standards/mpai-cae/#Technologies
  6. MPAI-CAE Framework Licence, MPAI N171; https://mpai.community/standards/mpai-cae/#Licence
  7. Draft MPAI-MMC Use Cases & Functional Requirements; MPAI N153; https://mpai.community/standards/mpai-mmc/#UCFR
  8. Draft MPAI-MMC Call for Technologies, MPAI N154; https://mpai.community/standards/mpai-mmc/#Technologies
  9. MPAI-MMC Framework Licence, MPAI N173; https://mpai.community/standards/mpai-mmc/#Licence

Annex A: Information Form

This information form is to be filled in by a Respondent to the MPAI-MMC CfT

  1. Title of the proposal
  2. Organisation: company name, position, e-mail of contact person
  3. What are the main functionalities of your proposal?
  4. Does your proposal provide or describe a formal specification and APIs?
  5. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

Annex B: Evaluation Sheet

NB: This evaluation sheet will be filled out by members of the Evaluation Team.

Proposal title:

Main Functionalities:

Response summary: (a few lines)

Comments on Relevance to the CfT (Requirements):

Comments on possible MPAI-MMC profiles[1]

Evaluation table:

Table 3 – Assessment of submission features

Note 1 The semantics of Submission features is provided by Table 4
Note 2 Evaluation elements indicate the elements used by the evaluator in assessing the submission
Note 3 Final Assessment indicates the ultimate assessment based on the Evaluation Elements

 

Submission features Evaluation elements Final Assessment
Completeness of description

Understandability

Extensibility

Use of Standard Technology

Efficiency

Test cases

Maturity of reference implementation

Relative complexity

Support of MPAI use cases

Support of non-MPAI use cases

Content of the criteria table cells:

Evaluation facts should mention:

  • Not supported / partially supported / fully supported.
  • What supported these facts: submission/presentation/demo.
  • The summary of the facts themselves, e.g., very good in one way, but weak in another.

Final assessment should mention:

  • Possibilities to improve or add to the proposal, e.g., any missing or weak features.
  • How sure the evaluators are, i.e., evidence shown, very likely, very hard to tell, etc.
  • Global evaluation (Not Applicable/ –/ – / + / ++)

 New Use Cases/Requirements Identified:

(please describe)

Evaluation summary:

  • Main strong points, qualitatively:
  •  Main weak points, qualitatively:
  • Overall evaluation: (0/1/2/3/4/5)

0: could not be evaluated

1: proposal is not relevant

2: proposal is relevant, but requires significant more work

3: proposal is relevant, but with a few changes

4: proposal has some very good points, so it is a good candidate for standard

5: proposal is superior in its category, very strongly recommended for inclusion in standard

Additional remarks: (points of importance not covered above.)

The submission features in Table 3 are explained in the following Table 4.

Table 4 – Explanation of submission features

Submission features Criteria
Completeness of description Evaluators should

1.     Compare the list of requirements (Annex C of the CfT) with the submission.

2.     Check if respondents have described in sufficient detail to what part of the requirements their proposal refers to.

NB1: Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated.

NB2: Submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology.

Understandability Evaluators should identify items that are demonstrably unclear (incon­sistencies, sentences with dubious meaning etc.)
Extensibility Evaluators should check if respondent has proposed extensions to the Use Cases.

NB: Extensibility is the capability of the proposed solution to support use cases that are not supported by current requirements.

Use of standard Technology Evaluators should check if new technologies are proposed where widely adopted technologies exist. If this is the case, the merit of the new tech­nology shall be proved.
Efficiency Evaluators should assess power consumption, computational speed, computational complexity.
Test cases Evaluators should report whether a proposal contains suggestions for testing the technologies proposed
Maturity of reference implementation Evaluators should assess the maturity of the proposal.

Note 1: Maturity is measured by the completeness, i.e., having all the necessary information and appropriate parts of the HW/SW implementation of the submission disclosed.

Note 2: If there are parts of the implementation that are not disclosed but demonstrated, they will be considered if and only if such components are replicable.

Relative complexity Evaluators should identify issues that would make it difficult to implement the proposal compared to the state of the art.
Support of MPAI-MMC use cases Evaluators should check how many use cases are supported in the submission
Support of non MPAI-MMC use cases Evaluators should check whether the technologies proposed can demonstrably be used in other significantly different use cases.

Annex C: Requirements check list

Please note the following acronyms

KB Knowledge Base
QF Query Format

Table 5 – List of technologies identified in MPAI-MMC N153 [6]

Note: The numbers in the first column refer to the section numbers of N153 [6].

Technologies by Use Cases Response
Conversation with Emotion
4.2.4.1 Text Y/N
4.2.4.2 Digital Speech Y/N
4.2.4.3 Digital Video Y/N
4.2.4.4 Emotion Y/N
4.2.4.5 Emotion KB (speech) query format Y/N
4.2.4.6 Emotion KB (text) query format Y/N
4.2.4.7 Emotion KB (video) query format Y/N
4.2.4.8 Meaning Y/N
4.2.4.9 Dialog KB query format – by tomorrow Y/N
4.2.4.10 Input to speech synthesis (Reply) Y/N
4.2.4.11 Input to face animation Y/N
Multimodal Question Answering
4.3.4.1 Text Y/N
4.3.4.2 Digital Speech Y/N
4.3.4.3 Digital Image Y/N
4.3.4.4 Image KB query format Y/N
4.3.4.5 Object identifier Y/N
4.3.4.6 Meaning Y/N
4.3.4.7 Intention KB query format Y/N
4.3.4.8 Online dictionary query format Y/N
Personalized Automatic Speech Translation
4.4.4.1 Text Y/N
4.4.4.2 Digital Speech Y/N
4.4.4.3 Speech features Y/N
4.4.4.4 Language identification Y/N
4.4.4.5 Translation results Y/N

Respondent should consult the equivalent list in N152 [5]

Annex D: Technologies that may require specific testing

Conversation with Emotion Speech features
Conversation with Emotion Text features
Conversation with Emotion Video features
Multimodal Question Answering Image features
Personalised Automatic Speech Translation Speech features

 Additional technologies may be identified during the evaluation phase.

Annex E: Mandatory text in responses

A response to this MPAI-MMC CfT shall mandatorily include the following text

<Company/Member> submits this technical document in response to MPAI Call for Technologies for MPAI project MPAI-MMC (N153).

 <Company/Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular <Company/Member> declares that  <Com­pany/Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-MMC (N171), alone or jointly with other IPR holders after the approval of the MPAI-MMC Technical Specif­ication by the General Assembly and in no event after commercial implementations of the MPAI-MMC Technical Specification become available on the market.

In case the respondent is a non-MPAI member, the submission shall mandatorily include the following text

If (a part of) this submission is identified for inclusion in a specification, <Company>  understands that  <Company> will be requested to immediately join MPAI and that, if  <Company> elects not to join MPAI, this submission will be discarded.

Subsequent technical contribution shall mandatorily include this text

<Member> submits this document to MPAI-MMC Development Committee (MMC-DC) as a con­tribution to the development of the MPAI-MMC Technical Specification.

 <Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular  <Company> declares that <Company> or its successors will make available the terms of the Licence related to its Essential Patents according to the MPAI-MMC Framework Licence (N173), alone or jointly with other IPR holders after the approval of the MPAI-MMC Technical Specification by the General Assembly and in no event after commercial implementations of the MPAI-MMC Technical Specification become available on the market.

[1] Profile of a standard is a particular subset of the technologies that are used in a standard and, where applicable, the classes, subsets, options and parameters relevan for the subset.


Template for responses to the Call for Technologies

This document is also available in MS Word format Template for responses to the MPAI-MMC Call for Technologies

Abstract

This document is provided as a help to those who intend to submit responses to the MPAI-CAE Call for Technologies. Text in red(as in this sentence) provides guidance to submitters and should not be included in a submission. Text in green shall be mandatorily included in a submission. If a submission does not include the green text, the submission will be rejected.

If the submission is in multiple files, each file shall include the green statement.

Text in white is the text suggested to respondents for use in a submission.

1        Introduction

This document is submitted by <organisation name> (if an MPAI Member) and/or by <organ­is­ation name>, a <company, university etc.> registered in … (if a non-MPAI member) in response to the MPAI-MMC Call for Technol­ogies issued by Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) on 2021/02/17 as MPAI document N154.

In the opinion of the submitter, this document proposes technologies that satisfy the requirements of MPAI-MMC Use Cases & Functional Requirements issued by MPAI on 2020/02/17 as MPAI document N153.

Possible additions

This document also contains comments on the requirements as requested by N153.

This document also contains proposed technologies that satisfy additional requirements as allowed by N153.

<Company and/or Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular <Company and or Member> declares that  <Company and or Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the MPAI-MMC Framework Licence (N171), alone or jointly with other IPR holders after the approval of the MPAI-MMC Technical Specif­ication by the MPAI General Assembly and in no event after commercial implem­entations of the MPAI-MMC Technical Specification become available on the market.

< Company and/or Member> acknowledges the following points:

  1. MPAI in not obligated, by virtue of this CfT, to select a particular technology or to select any technology, if those submitted are found inadequate.
  2. MPAI may decide to use the same technology for functionalities also requested in the MPAI-CAE Call for Technologies (N172) and associated Functional Requirements (N171).
  3. A representative of <Company and/or Member> shall present this submission at a CAE-DC meeting communicated by MPAI Secretariat (mailto:secretariat@mpai.community). If no <Company and/or Member> will attend the meeting and present the submission, this submission will be discarded.
  4. <Company and/or Member> shall make available a working implementation, including source code – for use in the development of the MPAI-CAE Reference Software and eventual public­ation by MPAI as a normative standard – before the technology submitted is accepted for the MPAI-CAE standard
  5. The software submitted may be written in programming languages that can be compiled or interpreted and in hardware description languages, upon acceptance by MPAI for further eval­uation of their submission in whole or in part.
  6. <Company> shall immediately join MPAI upon acceptance by MPAI for further evaluation of this submission in whole or in part.
  7. If <Company> does not join MPAI, this submission shall be discarded.

2        Information about the submission

This information corresponds to Annex A on N154. It is included here for submitter’s convenience.

  1. Title of the proposal
  2. Organisation: company name, position, e-mail of contact person
  3. What are the main functionalities of your proposal?
  4. Does your proposal provide or describe a formal specification and APIs?
  5. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

3        Comments on/extensions to requirements (if any)

 

4        Overview of Requirements supported by submission

Please answer Y or N. Detail on the specific answers can be provided in the submission.

Technologies by Use Cases Response
Conversation with Emotion
4.2.4.1 Text Y/N
4.2.4.2 Digital Speech Y/N
4.2.4.3 Digital Video Y/N
4.2.4.4 Emotion Y/N
4.2.4.5 Emotion KB (speech) query format Y/N
4.2.4.6 Emotion KB (text) query format Y/N
4.2.4.7 Emotion KB (video) query format Y/N
4.2.4.8 Meaning Y/N
4.2.4.9 Dialog KB query format – by tomorrow Y/N
4.2.4.10 Input to speech synthesis (Reply) Y/N
4.2.4.10 Input to face animation Y/N
Multimodal Question Answering
4.3.4.1 Text Y/N
4.3.4.2 Digital Speech Y/N
4.3.4.3 Digital Image Y/N
4.3.4.4 Image KB query format Y/N
4.3.4.5 Object identifier Y/N
4.3.4.6 Meaning Y/N
4.3.4.7 Intention KB query format Y/N
4.3.4.8 Online dictionary query format Y/N
Personalized Automatic Speech Translation
4.4.4.1 Text Y/N
4.4.4.2 Digital Speech Y/N
4.4.4.3 Speech features Y/N
4.4.4.4 Language identification Y/N
4.4.4.5 Translation results Y/N

5        New Proposed requirements (if any)

1. Y/N
2. Y/N
3. Y/N

6. Detailed description of submission

6.1       Proposal chapter #1

6.2       Proposal chapter #2

….

7        Conclusions

 

Clarifications of the Call for Technologies

MPAI-5 has approved the MPAI-MMC Use Cases and Functional Requirements as attachment to the Call for Technologies N173. However, MMC-DC has identified some issues that are worth a clarification. This is posted on the MPAI web site and will be com­mun­icated to those who have informed the Secretariata of their intention to respond.

General issue

MPAI understands that the scope of both N151 and N153 is very broad. Therefore it reiterates the point made in N152 and N154 that:

Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated. However, submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology.

Multimodal Question Answering (Use case #2 in N153)

MPAI welcomes submission that propose a standard set of “type of question intentions” and the means to indicate the language used in the Query Format.

MPAI welcomes proposals that propose a concept format for Reply in addition to or instead of a text format.

The assessment of submissions by Respondents who elect to not consider this point in their submission will not influence the assessment of the rest of their submission

References

  1. MPAI-MMC Use Cases & Functional Requirements; MPAI N153; https://mpai.community/standards/mpai-mmc/#UCFR
  2. MPAI-MMC Call for Technologies, MPAI N154; https://mpai.community/standards/mpai-mmc/#Technologies
  3. MPAI-MMC Framework Licence, N173; https://mpai.community/standards/mpai-mmc/#Licence