3 Evaluation Criteria and Procedure. 3
4 Expected development timeline. 4
Annex 3: Requirements check list 8
Annex 4: Mandatory text in responses. 9
1 Introduction
Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI[1]) is an international non-for-profit organisation with the mission to develop standards for Artificial Intelligence (AI) enabled digital data coding and for technologies that facilitate integration of data coding components into ICT systems [1]. With the mechanism of Framework Licences, MPAI intends to facilitate the establishment of patent pool inheriting clear IPR licensing frameworks from the standard development process [2].
MPAI standardisation is based on the following assumptions:
- Development, implementation and secure distribution of MPAI Standards requires a governed ecosystem [3].
- MPAI Application Standards are executed in an MPAI-specified secure environment called AI Framework [4].
Typically, AI Modules (AIM) are implemented with Neural Networks. During the last decade, Neural Networks have been deployed in an increasing variety of domains, but solutions and especially deep neural networks are costly. Therefore, MPAI has found the application area called “Neural Network Watermarking” to be relevant for MPAI standardization as there is a need for assessing the performances of traceability technologies.
In response to this need, MPAI has established NNW-DC. The DC has developed the Technical Specification: Neural Network Traceability (MPAI-NNT) V1.0 that specifies methods to evaluate the following aspects of Active (Watermarking) and Passive (Fingerprinting) Neural Network Traceability Methods:
- The ability of a Neural Network Traceability Detector/Decoder to detect/decode/match Traceability Data when the traced Neural Network has been modified.
- The computational cost of injecting, extracting, detecting, decoding, or matching Traceability Data.
- Specifically for active tracing methods, the impact of inserted Traceability Data on the performance of a neural network and on its inference.
The Call for Technologies: Neural Network Watermarking (MPAI-NNW) – Technologies (NNW-TEC) requests Neural Network Traceability Technologies that make it possible:
- To verify that the data provided by an Actor, and received by another Actor is not compromised, i.e. it can be used for the intended scope.
- To identify the Actors providing and receiving the data.
- To evaluate the quality of solutions supporting points 1.a and 1.b above implemented with the proposed Neural Network Traceability Technologies.
An Actor is a process producing, providing, processing, or consuming information.
This document is a Call for Technologies (CfT) that
- Satisfy the NNW-TEC V1.0 Use Cases and Functional Requirements (N2417) [5] and
- Are released according to the NNW-TEC V1.0 Framework Licence (N2418) [6], if selected by MPAI for inclusion in the NNW-TEC
All parties who believe they have relevant technologies satisfying all or most of the requirements of one or more than one Use Case described in N2417 [5] are invited to submit proposals for consideration by MPAI. MPAI membership is not a prerequisite for responding to this CfT. However, proponents should be aware that, if their proposal or part thereof is accepted for inclusion in the NNW-TEC standard, they will be requested to immediately join MPAI, or lose the opportunity to have their accepted technologies included in the standard.
MPAI will select the most suitable technologies based on their technical merits for inclusion in the NNW-TEC standard. However, MPAI in not obligated, by virtue of this CfT, to select a particular technology or to select any of the proposed technology if those submitted are found inadequate.
Submissions shall be received to the MPAI secretariat (secretariat@mpai.community) by 2025/09/27 T23:59 UTC. The secretariat will acknowledge receipt of the submission via email. Submissions will be reviewed according to the schedule that the 60th MPAI General Assembly (MPAI-60) will define at its online meeting on 2025/09/30. For details on how non MPAI members who have made a submission can attend the said review sessions, details are available by contacting the MPAI secretariat (secretariat@mpai.community).
2 How to submit a response
Those planning to respond to this CfT are:
- Advised that an online event will be held on 2025/07/01 to present the MPAI-NNT CfT. The recording of that event will be made available at: https://mpai.community/community/presentations/
- Requested to communicate their intention to respond to this CfT with an initial version of the form of Annex 1 to the MPAI secretariat (secretariat@mpai.community) by 2025/08/11. Submission of an Annex 1 helps MPAI to properly plan for the revision of submissions. However, it is not a commitment to make a submission and a response to this Call not preceded by the submission of Annex 1 will still be accepted.
- Advised to visit regularly the https://mpai.community/standards/mpai-nnw/ web site where relevant information will be posted.
Responses to this NNW-TEC CfT may/shall include:
Table 1 – Optional and mandatory elements of a response
| Item | Status |
| Detailed documentation describing the proposed technologies | mandatory |
| The final version of Annex 1 | mandatory |
| The text of Annex 2 duly filled out with the table indicating which Functional Requirements identified in MPAI N2417 [5] are satisfied. If some of the Functional Requirements of a Use Case are not satisfied, this should be explained. | mandatory |
| Comments on the completeness and appropriateness of the NNW-TEC Functional Requirements and any motivated suggestion to amend and/or extend those Requirements. | optional |
| A preliminary demonstration, with a detailed document describing it. | optional |
| Any other additional relevant information that may help evaluate the submission, such as additional use cases. | optional |
| The text of Annex 4. | mandatory |
Respondents are invited
- To take advantage of the check list of Annex 3 before submitting their response and filling out Annex 1.
- To use the Template for Respomses [7] in their resposes.
Respondents are mandatorily requested to present their submission at a teleconference meeting to be properly announced to submitters by the MPAI Secretariat. If no presenter of a submission will in attendance to that meeting, the submission will be discarded.
Respondents are advised that, upon acceptance by MPAI of their submission in whole or in part for further evaluation, MPAI will require that:
- A working implementation, including source code – for use in the development of the NNW-TEC Reference Software and later publication as an MPAI standard – be made available before the technology is accepted for inclusion in the NNW-TEC standard. Software may be written in programming languages that can be compiled or interpreted. Hardware Description Language implementations are also accepted.
- A non-MPAI member immediately join MPAI. If the non-MPAI member elects not to do so, their submission will be discarded. Direction on how to join MPAI can be found online.
Further information on MPAI can be obtained from the MPAI website.
3 Evaluation Criteria and Procedure
Proposals will be assessed using the following process:
- Evaluation panel is created from:
- NNW-DC members in attendance.
- Non-MPAI members who are respondents.
- Non respondents/non MPAI member experts invited in a consulting capacity.
- No one from 1.1.-1.2. is denied membership in the Evaluation panel.
- Respondents present their proposals.
- Evaluation Panel members ask questions.
- If required objective tests are carried out after:
- The required tests are defined.
- The required texts are carried out.
- A report is produced.
- If required, at least 2 reviewers are appointed to review and report about specific points in a proposal.
- Evaluation panel members fill out Annex 2 for each proposal.
- Respondents respond to evaluations.
- Proposal evaluation report is produced.
4 Expected development timeline
Timeline of the CfT, deadlines and response evaluation:
Table 2 – Dates and deadlines
| Step | Date | Time |
| Call for Technologies issued | 2025/06/11 | |
| Notification of intention to submit proposal sent | 2025/08/11 | 23.59 UTC |
| Submission deadline | 2025/09/27 | 23.59 UTC |
| Evaluation of responses starts | 2025/09/30 (MPAI-60) |
Evaluation to be carried out during 2-hour sessions according to the calendar agreed at MPAI-60.
5 References
- MPAI Standards Resources; https://mpai.community/standards/resources/
- MPAI Patent Policy; https://mpai.community/about/the-mpai-patent-policy/
- Technical Specification: Governance of the MPAI Ecosystem (MPAI-GME) V1.1
- Technical Specification: AI Framework (MPAI-AIF) V2.1
- Use Cases and Functional Requirements: Neural Network Watermarking (MPAI-NNW) – Technologies (NNW-TEC) V1.0; MPAI N2417
- Framework Licence: Neural Network Watermarking (MPAI-NNW) – Technologies (NNW-TEC) V1.0; MPAI N2418
- Template for Responses: Neural Network Watermarking (MPAI-NNW) – Technologies (NNW-TEC) V1.0; V1.0; MPAI N2419
Annex 1: Information Form
This information form is to be filled in by a Respondent to the NNW-TEC CfT
- Title of the proposal
- Organisation: company name, position, e-mail of contact person
- What are the main functionalities of your proposal?
- Will you provide a demonstration to show how your proposal meets the evaluation criteria?
Annex 2: Evaluation Sheet
NB: This evaluation sheet will be filled out by members of the Evaluation Team.
Proposal title:
Main Functionalities:
Response summary: (a few lines)
Comments on Relevance to the CfT (Requirements):
Evaluation table:
Table 3 – Assessment of submission features
| Note 1 | The semantics of Submission Features is provided by Table 4 |
| Note 2 | Evaluation elements indicate the elements used by the evaluator in assessing the submission |
| Note 3 | Final Assessment indicates the ultimate assessment based on the Evaluation Elements |
| Submission features | Criteria |
| Completeness of description | Evaluators should
1. Compare the list of requirements (Annex 3 of the CfT) with the submission. 2. Check if respondents have described in sufficient detail to what part of the requirements their proposal refers to. NB1: Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated. NB2: Submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology. |
| Understandability | Evaluators should identify items that are demonstrably unclear (inconsistencies, ambiguous sentences, etc.) |
| Extensibility | Evaluators should check if respondent has proposed extensions to the Use Cases.
NB: Extensibility is the capability of the proposed solution to support use cases that are not supported by current requirements. |
| Efficiency | Evaluators should assess power consumption, computational speed, computational complexity. |
| Test cases | Evaluators should report whether a proposal contains suggestions for testing the technologies proposed |
| Maturity of reference implementation | Evaluators should assess the maturity of the proposal.
Note 1: Maturity is measured by the completeness, i.e., having all the necessary information and appropriate parts of the HW/SW implementation of the submission disclosed. Note 2: If there are parts of the implementation that are not disclosed but demonstrated, they will be considered if and only if such components are replicable. |
Content of the criteria table cells:
Evaluation facts should mention:
- Not supported / partially supported / fully supported.
- What supported these facts: submission/presentation/demo.
- The summary of the facts themselves, e.g., very good in one way, but weak in another.
Final assessment should mention:
- Possibilities to improve or add to the proposal, e.g., any missing or weak features.
- How sure the evaluators are, i.e., evidence shown, very likely, very hard to tell, etc.
- Global evaluation (Not Applicable/ –/ – / + / ++)
New Use Cases/Requirements Identified:
(please describe)
Evaluation summary:
- Main strong points (qualitative description):
- Main weak points (qualitative description):
- Overall evaluation: (0/1/2/3/4/5)
0: could not be evaluated
1: proposal is not relevant
2: proposal is relevant, but requires significant more work
3: proposal is relevant, but with a few changes
4: proposal has some very good points, so it is a good candidate for standard
5: proposal is superior in its category, very strongly recommended for inclusion in standard
Additional remarks: (points of importance not covered above.)
Table 4 explains the submission features of Table 3.
Table 4 – Explanation of submission features
| Submission features | Criteria |
| Completeness of description | Evaluators should
1. Compare the list of requirements (Annex 3 of the CfT) with the submission. 2. Check if respondents have described in sufficient detail to what part of the requirements their proposal refers to. NB1: Completeness of a proposal for a Use Case is a merit because reviewers can assess that the components are integrated. NB2: Submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered instead of a submission that is complete but has a less performing technology. |
| Understandability | Evaluators should identify items that are demonstrably unclear (inconsistencies, ambiguous sentences, etc.) |
| Extensibility | Evaluators should check if respondent has proposed extensions to the Use Cases.
NB: Extensibility is the capability of the proposed solution to support use cases that are not supported by current requirements. |
| Efficiency | Evaluators should assess power consumption, computational speed, computational complexity. |
| Test cases | Evaluators should report whether a proposal contains suggestions for testing the technologies proposed |
| Maturity of reference implementation | Evaluators should assess the maturity of the proposal.
Note 1: Maturity is measured by the completeness, i.e., having all the necessary information and appropriate parts of the HW/SW implementation of the submission disclosed. Note 2: If there are parts of the implementation that are not disclosed but demonstrated, they will be considered if and only if such components are replicable. |
Annex 3: Requirements check list
Table 5 – List of technologies identified in NNW-TEC N2417 [5]
| Requirement0 | Response |
| 5. Service and application scenarios | |
| Additional/Comments on illustrative use cases | Y/N |
| 6. Functional requirements | |
| An Actor applying a tracking technology should be able to detect whether a content (model or inference) includes the tracking technology. | Y/N |
| An Actor applying a tracking technology should be able to detect whether the Source data has been processed during the transmission from a Source to a Sink. | Y/N |
| An Actor transmitting data including a tracking technology should be able to decide whether the data preserves:
a. The syntax of the Source data or data resulting from any processing. b. The semantics of the Source data or data resulting from any processing. |
Y/N |
| An Actor transmitting data including a tracking technology should be able to decide whether the data preserves:
a. The syntax of the Source data or data resulting from any processing. b. The semantics of the Source data or data resulting from any processing. |
Y/N |
| An Actor applying a watermarking technology to Source data should be able to retrieve the payload from the NN or the NN inference, to:
a. Allow the identification of the sender, b. Allow the identification of the receiver, c. Verify whether the data was sent by the intended sender, d. Verify whether the data was meant for the intended receiver, e. Verify whether the data was received by the intended receiver. |
Y/N |
| An Actor applying a fingerprinting technology to Source data should be able to identify the NN or the NN inference. | Y/N |
Annex 4: Mandatory text in responses
A response to this MPAI-NNW CfT shall mandatorily include the following text
<Company/Member> submits this technical document in response to MPAI Call for Technologies for Neural Network Watermarking (MPAINNW-TEC) (N2416).
<Company/Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N2115), in particular <Company/Member> declares that <Company/Member> or its successors will make available the terms of the Licence related to its Essential Patents according to the Framework Licence of MPAI-NNW (N2418), alone or jointly with other IPR holders after the approval of the MPAI-NNW Technical Specification by the General Assembly and in no event after commercial implementations of the MPAI-NNW Technical Specification become available on the market.
In case the respondent is a non-MPAI member, the submission shall mandatorily include the following text
If (a part of) this submission is identified for inclusion in a specification, <Company> understands that <Company> will be requested to immediately join MPAI and that, if <Company> elects not to join MPAI, this submission will be discarded.
Subsequent technical contribution shall mandatorily include this text
<Member> submits this document to MPAI-NNW Development Committee (NNW-DC) as a contribution to the development of the MPAINNW-TEC Technical Specification.
<Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N2115), in particular <Company> declares that <Company> or its successors will make available the terms of the Licence related to its Essential Patents according to the MPAI-NNW Framework Licence (N2418), alone or jointly with other IPR holders after the approval of the MPAINNW-TEC Technical Specification by the General Assembly and in no event after commercial implementations of the MPAINNW-TEC Technical Specification become available on the market.
[1] All terms and acronyms are defined at https://dbmpai.rf.gd
