Compression and Understanding of Industrial Data (MPAI-CUI)

1       Introduction. 1

2       How to submit a response. 1

3       Evaluation Criteria and Procedure. 1

4       Expected development timeline. 1

5       References. 1

Annex A: Information Form.. 1

Annex B: Evaluation Sheet 1

Annex C: Requirements check list 1

Annex D: Technologies that may require specific testing. 1

Annex E: Mandatory text in responses. 1

1        Introduction

Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is an international non-profit organisation with the mission to develop standards for Artificial Intelligence (AI) enabled digital data coding and for technologies that facilitate integration of data coding components into ICT systems. With the mechanism of Framework Licences, MPAI seeks to attach clear IPR licen­sing frameworks to its standards.

MPAI has found that the application area called “Compression and Understanding of  Industrial Data” is particul­arly relevant for MPAI standardisation because AI allows for substantial reduction of the amount of information produced by companies and for more in-depth analysis of the data to be carried out.

Therefore, MPAI intends to develop a standard – to be called MPAI-CUI – that will provide standard tech­nologies to implement several Use Cases, the first of which is:

  1. AI-based Performance Prediction (APP)

This document is a Call for Technologies (CfT) that

  1. Satisfy the MPAI-CUI Functional Requirements (N200) [4] and
  2. Are released according to the MPAI-CUI Framework Licence (N202) [5], if selected by MPAI for inclusion in the MPAI-CUI standard.

The standard will be developed with the following guidelines:

  1. To satisfy the Functional Requirements (N200) [4]. In the future, MPAI may decide to extend MPAI-CUI to support other Use Cases.
  2. To be suitable for implementation as AI Modules (AIM) conforming to the MPAI AI Framework (MPAI-AIF) standard which is being based on the responses to the Call for Technologies (N100) [1] satisfying the MPAI-AIF Functional Requirements (N74) [1].

Rather than follow the approach of defining end-to-end systems, MPAI has decided to base its application standards on the AIM and AIF notions. The AIF functional requirements have been identified in [1], while the AIM requirements are Use Case-specific. It has done so because:

  1. AIMs allow the reduction of a large problem to a set of smaller problems.
  2. AIMs can be independently developed and made available to an open competitive market.
  3. An implementor can build a sophisticated and complex system with potentially limited know­ledge of all the tech­nologies required by the system.
  4. MPAI systems are inherently explainable.
  5. MPAI systems allow for competitive comparisons of functionally equivalent AIMs.

Respondents should be aware that:

  1. The currently addressed MPAI-CUI Use Case and the AIM internals will be non-normative.
  2. The input and output interfaces of the AIMs, whose requirements have been derived to support the Use Case, will be normative.

Therefore, the scope of this Call for Technologies is restricted to technologies required to implement the input and output interfaces of the AIMs identified in N200 [4].

However, MPAI invites comments on any technology or architectural component identified in N200, specifically,

  1. Additions or removals of input/output data to the identified AIMs with justification of the changes and identification of data formats required by the new input/output signals.
  2. Possible alternative partitioning of the AIMs implementing the Use Case providing:
    1. Arguments in support of the proposed partitioning.
    2. Detailed specifications of the input and output data of the proposed new AIMs.
  3. New fully described Use Cases.

All parties who believe they have relevant technologies satisfying all or most of the requirements of the Use Case described in N200 are invited to submit proposals for consid­eration by MPAI. MPAI membership is not a prerequisite for responding to this CfT. However, proponents should be aware that, if their proposal or part thereof is accepted for inclusion in the MPAI-CUI standard, they shall immediately join MPAI, or their accepted technologies will be discarded.

MPAI will select the most suitable technologies based on their technical merits for inclusion in MPAI-CUI. However, MPAI in not obligated, by virtue of this CfT, to select a particular tech­nology or to select any technology if those submitted are found inadequate.

Submissions are due on 2021/05/10T23:59 UTC and should be sent to the MPAI secretariat (secretariat@mpai.community). The secretariat will acknowledge receipt of the submissions via email. Submissions will be reviewed according to the schedule that the 8th MPAI General Assembly (MPAI-8) will define at its online meeting on 2021/05/12. Please contact the MPAI secretariat (secretariat@mpai.community),for details on how submitters who are not MPAI members can attend the said review.

2        How to submit a response

Those planning to respond to this Call for Technologies are:

  1. Advised that online events will be held on 2021/03/31 and 2021/04/07 to present the MPAI-CUI Call for Technologies and respond to questions. Logistic information on these events will be posted on the MPAI web site.
  2. Requested to communicate their intention to respond to this CfT with an initial version of the form of Annex A to the MPAI secretariat (secretariat@mpai.community) by 2021/04/13. A potential submitter making a communication using the said form is expected but not required to actually make a submission. A submission will be accepted even if the submitter did not communicate their intention to submit a response by the said date.
  3. Advised to visit regularly the MPAI web site where relevant information will be posted.

Responses to this MPAI-CUI CfT shall/may include the elements described in Table 1:

Table 1 – Mandatory and optional elements of a response

Item Status
Detailed documentation describing the proposed technologies mandatory
The final version of Annex A mandatory
The text of Annex B duly filled out with the table indicating which requirements identified in MPAI N200 [4] are satisfied. If all Functional Requirements are not satisfied, this should be explained. mandatory
Comments on the completeness and appropriateness of the Functional Requir­ements and any motivated suggestion to amend or extend them. optional
A preliminary demonstration, with a detailed document describing it. optional
Any other additional relevant information that may help evaluate the submission, such as additional use cases. optional
The text of Annex E. mandatory

Respondents are invited to take advantage of the check list of Annex C before submitting their response and filling out Annex A.

Respondents are requested to present their submission (mandatory) at a meeting by teleconference that the MPAI Secretariat will properly announce to submitters. If no presenter will attend the meeting, the proposal will be discarded.

Respondents are advised that, upon acceptance by MPAI of their submission in whole or in part for further evaluation, MPAI will require that:

  • A working implementation, including source code – to be used in the development of the MPAI-CUI Reference Software and later publication as a standard by MPAI – be made available before the technology is accepted for inclusion in the MPAI-CUI standard. Software may be written in a programming language that can be compiled or interpreted or in a hardware description language.
  • The working implementation be suitable for operation in the MPAI AI Framework (MPAI-AIF).
  • A non-MPAI member immediately join MPAI. If the non-MPAI member elects not to do so, their submission will be discarded. Direction on how to join MPAI can be found online.

Further information on MPAI can be obtained from the MPAI website.

3        Evaluation Criteria and Procedure

Proposals will be assessed using the following process:

  1. Evaluation panel is created from:
    1. All CUI-DC members attending.
    2. Non-MPAI members who are respondents.
    3. Non respondents/non MPAI member experts invited in a consulting capacity.
  2. No one from 1.1.-1.2. will be denied membership in the Evaluation panel.
  3. Respondents present their proposals.
  4. Evaluation Panel members ask questions.
  5. If required subjective and/or objective tests are carried out:
    1. Define required tests.
    2. Carry out the tests.
    3. Produce report.
  6. At least 2 reviewers will be appointed to review & report about specific points in a proposal if required.
  7. Evaluation panel members fill out Annex B for each proposal.
  8. Respondents respond to evaluations.
  9. Proposal evaluation report is produced.

4        Expected development timeline

Timeline of the CfT, deadlines and response evaluation:

Table 2 – Dates and deadlines

Step Date Meeting
Call for Technologies 2021/03/17 MPAI-6
CfT introduction conference call 1 2021/03/31T15:00 UTC
CfT introduction conference call 2 2021/04/07T15:00 UTC
Notification of intention to submit proposal 2021/04/13T23.59 UTC
Submission deadline 2021/05/10T23.59 UTC
Evaluation of responses will start 2021/05/12 MPAI-8

Evaluation to be carried out during 2-hour sessions according to the calendar agreed at MPAI-8.

5        References

  1. MPAI-AIF Use Cases & Functional Requirements, N74; https://mpai.community/standards/mpai-aif/
  2. MPAI-AIF Framework Licence, MPAI N101; https://mpai.community/standards/mpai-aif/#Licence
  3. MPAI-AIF Call for Technologies, N100; https://mpai.community/standards/mpai-aif/#Technologies
  4. MPAI-CUI Use Cases & Functional Requirements; MPAI N200; https://mpai.community/standards/mpai-cui/#UCFR
  5. MPAI-CUI Framework Licence, MPAI N201; https://mpai.community/standards/mpai-cui/#Licence
  6. MPAI-CUI Call for Technologies, MPAI N202; https://mpai.community/standards/mpai-cui/#Technologies

Annex A: Information Form

This information form is to be filled in by a Respondent to the MPAI-CUI CfT.

The purpose of this Annex is to collect data that facilitate the organisation of submission evalu­ation. Therefore, submitters are requested to only provide such data as Use Case(s) considered, types of technologies proposed, special requirements for (optional) demonstration and any other information that is functional to the evaluation of the submission.

  1. Title of the proposal
  2. Organisation: company name, position, e-mail of contact person
  3. What are the main functionalities of your proposal?
  4. Does your proposal provide or describe a formal specification and APIs?
  5. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

Parties sending this Annex A are

  1. This Annex A should be only sent to the Secretariat
  2. Points 1., 3., 4., and 5. above will be made known to MPAI members. Point 2. will not be disclosed.
  3. The full submissions will be made available to MPAI members after the submission deadline of 2021/04/10.
  4. The Secretariat will not accept any confidential inforamtion at the time expression of interest is communicated to the Secretariat.

Annex B: Evaluation Sheet

NB: This evaluation sheet will be filled out by members of the Evaluation Team.

Proposal title:

Main Functionalities:

Response summary: (a few lines)

Comments on Relevance to the CfT (Requirements):

Comments on possible MPAI-CUI profiles[1]

Evaluation table:

Table 3 – Assessment of submission features

Note 1 The semantics of Submission features is provided by Table 4
Note 2 Evaluation elements indicate the elements used by the evaluator in assessing the submission
Note 3 Final Assessment indicates the ultimate assessment based on the Evaluation Elements

 

Submission features Evaluation elements Final Assessment
Completeness of description

Understandability

Extensibility

Use of Standard Technology

Efficiency

Test cases

Maturity of reference implementation

Relative complexity

Support of non-MPAI use cases

Content of the criteria table cells:

Evaluation facts should mention:

  • Not supported / partially supported / fully supported.
  • What supported these facts: submission/presentation/demo.
  • The summary of the facts themselves, e.g., very good in one way, but weak in another.

Final assessment should mention:

  • Possibilities to improve or add to the proposal, e.g., any missing or weak features.
  • How sure the evaluators are, i.e., evidence shown, very likely, very hard to tell, etc.
  • Global evaluation (Not Applicable/ –/ – / + / ++)

New Use Cases/Requirements Identified:

(please describe)

Evaluation summary:

  • Main strong points, qualitatively:
  •  Main weak points, qualitatively:
  • Overall evaluation: (0/1/2/3/4/5)

0: could not be evaluated

1: proposal is not relevant

2: proposal is relevant, but requires significant more work

3: proposal is relevant, but with a few changes

4: proposal has some very good points, so it is a good candidate for standard

5: proposal is superior in its category, very strongly recommended for inclusion in standard

Additional remarks: (points of importance not covered above.)

The submission features in Table 3 are explained in the following Table 4.

Table 4 – Explanation of submission features

Submission features Criteria
Completeness of description Evaluators should

1.     Compare the list of requirements (Annex C of the CfT) with the submission.

2.     Check if respondents have described in sufficient detail to what part of the requirements their proposal refers to.

NB1: Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated.

NB2: Submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology.
Understandability Evaluators should identify items that are demonstrably unclear (incon­sistencies, sentences with dubious meaning etc.)
Extensibility Evaluators should check if respondent has proposed extensions to the Use Cases.

NB: Extensibility is the capability of the proposed solution to support use cases that are not supported by current requirements.
Use of standard technology Evaluators should check if new technologies are proposed while widely adopted technologies exist. If this is the case, the merit of the new tech­nology shall be proved.
Efficiency Evaluators should assess power consumption, computational speed, computational complexity.
Test cases Evaluators should report whether a proposal contains suggestions for testing the technologies proposed
Maturity of reference implementation Evaluators should assess the maturity of the proposal.

Note 1: Maturity is measured by its completeness, i.e., by disclosing all the necessary information and appropriate parts of the HW/SW implem­entation of the submission.

Note 2: If there are parts of the implementation that are not disclosed but demonstrated, they will be considered if and only if such com­ponents are replicable.
Relative complexity Evaluators should identify issues that would make it difficult to implement the proposal compared to the state of the art.
Support of non MPAI-CUI use cases Evaluators should check whether the technologies proposed can demonstrably be used in other significantly different use cases.

Annex C: Requirements check list

Please note the following acronyms

KB Knowledge Base
QF Query Format

Table 5 – List of technologies identified in MPAI-CUI N200 [4]

Note: The numbers in the first column refer to the section numbers of N200 [4].

Technologies Response
4.1.4.1 Governance data (raw) Y/N
4.1.4.2 Financial statement data (raw) Y/N
4.1.4.3 Risk assessment technical data (raw) Y/N
4.1.4.4 Governance Y/N
4.1.4.5 Financial statement Y/N
4.1.4.6 Risk assessment technical data Y/N
4.1.4.7 Financial features Y/N
4.1.4.8 Governance features Y/N
4.1.4.9 Severity Y/N
4.1.4.10 Decision Tree Y/N
4.1.4.11 Default probability Y/N
4.1.4.12 Adequacy of organisational model Y/N
4.1.4.13 Business continuity index Y/N

Annex D: Technologies that may require specific testing

Financial features
Governance features
Decision Tree

Additional technologies may be identified during the evaluation phase.

Annex E: Mandatory text in responses

A response to this MPAI-CUI CfT shall mandatorily include the following text

<Company/Member> submits this technical document in response to MPAI Call for Technologies for MPAI project MPAI-CUI (N202).

 <Company/Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular <Company/Member> declares that  <Com­pany/Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-CUI (N201), alone or jointly with other IPR holders after the approval of the MPAI-CUI Technical Specif­ication by the General Assembly and in no event after commercial implementations of the MPAI-CUI Technical Specification become available on the market.

In case the respondent is a non-MPAI member, the submission shall mandatorily include the following text

If (a part of) this submission is identified for inclusion in a specification, <Company>  understands that  <Company> will be requested to immediately join MPAI and that, if  <Company> elects not to join MPAI, this submission will be discarded.

Subsequent technical contribution shall mandatorily include this text

<Member> submits this document to MPAI-CUI Development Committee (CUI-DC) as a con­tribution to the development of the MPAI-CUI Technical Specification.

 <Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular  <Company> declares that <Company> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-CUI (N201), alone or jointly with other IPR holders after the approval of the MPAI-CUI Technical Specification by the General Assembly and in no event after commercial implementations of the MPAI-CUI Technical Specification become available on the market.

[1] Profile of a standard is a particular subset of the technologies that are used in a standard and, where applicable, the classes, subsets, options and parameters relevan for the subset

Use Cases and Functional Requirements – Framework Licence – Call for Technologies – Template for responses to the Call for Technologies – Application Note

Template for responses to the MPAI-CUI Call for Technologies

Abstract

This document is provided as a help to those who intend to submit responses to the MPAI-CUI Call for Technologies. Text in res(as in this sentence) provides guidance to submitters and should not be included in a submission. Text in green shall be mandatorily included in a submission. If a submission does not include the green text, the submission will be rejected.

If the submission is in multiple files, each file shall include the green statement.

Text in white is the text suggested to respondents for use in a submission.

1        Introduction

This document is submitted by <organisation name> (if an MPAI Member) and/or by <organ­is­ation name>, a <company, university etc.> registered in … (if a non-MPAI member) in response to the MPAI-CUI Call for Technol­ogies issued by Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) on 2021/03/17 as MPAI document N191.

In the opinion of the submitter, this document proposes technologies that satisfy the requirements of MPAI document MPAI-CUI Use Cases & Functional Requirements issued by MPAI on 2020/03/17 as MPAI document N151.

Possible additions

This document also contains comments on the requirements as requested by N200.

This document also contains proposed technologies that satisfy additional requirements as allowed by N189.

<Company and/or Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N80), in particular <Company and or Member> declares that  <Company and or Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the MPAI-CUI Framework Licence (N201), alone or jointly with other IPR holders after the approval of the MPAI-CUI Technical Specif­ication by the MPAI General Assembly and in no event after commercial implem­entations of the MPAI-CAE Technical Specification become available on the market.

< Company and/or Member> acknowledges the following points:

  1. MPAI in not obligated, by virtue of this CfT, to select a particular technology or to select any technology if those submitted are found inadequate.
  2. <Company and/or Member> plans on having a representative to present this submission at a CAE-DC meeting communicated by MPAI Secretariat (mailto:secretariat@mpai.community). <Company and/or Member> acknowledges that, in no representative will attend the meeting and present the submission, this sub­mission will be discarded.
  3. <Company and/or Member> plans on making available a working implementation, including source code – for use in the development of the MPAI-CUI Reference Software and eventual public­ation by MPAI as a normative standard – before the technology submitted is accepted for the MPAI-CUI standard.
  4. The software submitted may be written in a programming language that can be compiled or interpreted or in a hardware description language, upon acceptance by MPAI for further eval­uation of their submission in whole or in part.
  5. <Company> shall immediately join MPAI upon acceptance by MPAI for further evaluation of this submission in whole or in part.
  6. If <Company> does not join MPAI, this submission shall be discarded.

2        Information about the submission

This information corresponds to Annex A on N191. It is included here for submitter’s convenience.

  1. Title of the proposal
  2. Organisation: company name, position, e-mail of contact person
  3. What are the main functionalities of your proposal?
  4. Does your proposal provide or describe a formal specification and APIs?
  5. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

3        Comments on/extensions to requirements (if any)

 

4        Overview of Requirements supported by the submission

Please answer Y or N. Detail on the specific answers can be provided in the submission.

Technologies Response
4.1.4.1 Governance data (raw) Y/N
4.1.4.2 Financial statement data (raw) Y/N
4.1.4.3 Risk assessment technical data (raw) Y/N
4.1.4.4 Governance Y/N
4.1.4.5 Financial statement Y/N
4.1.4.6 Risk assessment technical data Y/N
4.1.4.7 Financial features Y/N
4.1.4.8 Governance features Y/N
4.1.4.9 Severity Y/N
4.1.4.10 Decision Tree Y/N
4.1.4.11 Default probability Y/N
4.1.4.12 Adequacy of organisational model Y/N
4.1.4.13 Business continuity index Y/N

5        New Proposed requirements (if any)

 

1. Y/N
2. Y/N
3. Y/N

 

 

6. Detailed description of submission

6.1       Proposal chapter #1

6.2       Proposal chapter #2

….

7        Conclusions