Abstract

This document requests parties having rights to technologies satisfying the Use Cases and Functional Requirements [3] and the Framework Licence [4] of the planned Technical Specification: Up-sampling Filter for Video Applications (MPAI-UFV) V1.0 to respond to this Call for Technologies preferably using the Template for Responses [5]. The goal of MPAI-UFV V1.0 is to develop a standard up-sampling filter providing optimal performance when applied to a video to generate a video with a higher number of lines and pixels. The submissions received will be assessed, collaboratively improved if found suitable, and used in the development of the planned MPAI-UFV Technical Specification.

1               Introduction

Established in September 2020, MPAI is an international non-profit organisation with the mission to develop standards for Artificial Intelligence (AI)-enabled data coding and technologies facilitating integration of data coding components into Information and Communication Technology (ICT) systems [1]. The MPAI Patent Policy [2] guides the accomplishment of this mission.

So far, MPAI has developed thirteen Technical Specifications relevant to its mission such as execution environment of multi-component AI applications, portable avatar format, object and scene description, neural network watermarking, context-based audio enhancements, multimodal human-machineinteraction, company performance prediction, metaverse, profiles, qualifiers, and ecosystem governance. Seven Technical Specifications have been adopted by IEEE without modification (3300-2024, 3301-2024, 3302-2022, 3303-2023, 3304-2023, 3305-2024, 3306-2024, 3307-2024) and more are in the pipeline. Several other standard projects – such as AI for health, online gaming and XR venues – are under way and are expected to deliver specifications in the next few months.

MPAI specifications are the result of a process whose main steps are:

  1. Development of functional requirements in an open environment.
  2. Adoption of “commercial requirements” (Framework Licence) by MPAI Principal Members setting guidelines for the future commercial licence to be issued by standard essential patent (SEP) holders.
  3. Publication of a Call for Technologies – such as this document – referring to the Functional and Commercial Requirements and inviting parties who accept to license their technologies according to the Framework Licence – if their technologies are accepted to be part of the target Technical Specification – to submit a Response.

2               Scope and purpose

This Call for Technologies: Up-sampling Filter for Video applications (MPAI-UFV) V1.0 invites any party able and wishing to contribute to the development of the planned MPAI-UFV Technical Specification to submit a response. If their technologies are selected by MPAI for possible modification and inclusion in the planned MPAI-UFV Technical Specification, they are required to eventually license their technologies in a future commercial licence respecting the guidelines of the Framework Licence: Up-sampling Filter for Video applications (MPAI-UFV) V1.0 [4].

Any respondent who is not an MPAI member and wishes to participate in the development of MPAI-UFV shall join MPAI. If they own accepted technologies and do not join MPAI, they lose the opportunity to have their technologies included in the planned MPAI-UFV Technical Specification.

The planned MPAI-UFV Technical Specification will be developed using technologies that satisfy the following mandatory requirements:

  1. Be part of responses to this Call submitted by parties accepting the MPAI-UFV Framework Licence [4] and satisfy the MPAI-UFV Use Cases and Functional Requirements [3].
  2. Be integrated by subsequent Members’ contributions during the development of the MPAI-UFV Technical Specification. Such contributions shall be submitted stating acceptance of the MPAI-UFV Framework Licence [4].

Therefore, the scope of this Call is restricted to responses whose specification in the planned MPAI-UFV Technical Specification conforms to [3]. However, respondents are welcome to additionally:

  1. Comment on any technical element of Use Cases and Functional Requirements: Up-sampling Filter for Video applications [3].
  2. Propose Technologies based on motivated proposals to extend or add new functional requirements in Use Cases and Functional Requirements [3] provided they:
  3. Are in line with the scope of Use Cases and Functional Requirements [3].
  4. Satisfy the Framework Licence [4].

MPAI membership is not a prerequisite for responding to this Call for Technologies. However, proponents should be aware that, if their Submission or part thereof is accepted for inclusion in the planned MPAI-UFV Technical Specification, they will be required to immediately join MPAI, or lose the opportunity to have their accepted technologies included in the standard.

MPAI will select the most suitable technologies based on their technical merits. However, MPAI in not obligated, by virtue of this Call, to select a particular technology or to select any of the proposed technologies if those submitted are found to be inadequate.

Note that in the future, MPAI may decide to further extend the planned MPAI-UFV or a new part of a future MPAI-UFV series of Technical Specifications.

Test Sequences provided by MPAI to proponents

MPAI will provide the following material:

  1. Original Test Sequences with the following features:
    1. 8 and 10 bits/pixels per component.
    2. YCbCr colour spaces with 4:2:0 sampling.
    3. Progressive scan.
  2. Encoded Test Sequences with the following features:
    1. Random Access Test conditions: Intra, P and B pictures.
    2. Low Delay Test conditions: Intra and P pictures.
    3. QP Values: 22, 27, 32, 37, 42.
    4. AVC, HEVC and VVC codecs.

MPAI will provide the link to the folder to those expressing an interest to respond to this Call per Annex 1.

3               Components of a submission

Responses to this Call for Technologies shall/may include:

Table 1 – Optional and mandatory elements of a response

Note: the materials listed in the table shall be uploaded to the repository (registration required)

Item Status
The completed version of Annex 1 mandatory
A complete description of the proposed up-sampling filter with a level of detail allowing an expert in the field to develop an implement the proposed filter. mandatory
The following software elements to be uploaded the MPAI storage at https://experts.mpai.community/live/nextcloud/ (a filled one copy of Annex 1 should be sent to the secretariat to have registration approved):

1.     Docker image that contains the encoding and decoding environments, encoder, decoder, and bitstreams.

2.     Up-sampling filter in source code (preferred) or executable form. Note that proponents of accepted proposals will be requested to provide the source code of the up-sampling filter.

3.     Python scripts to enable testers to carry out the Performance Test.

 mandatory
The following results:

1.     Tables of objective quality results obtained by the submitter with their proposed solution.

2.     The decoded Test Sequences.

3.     The up-sampling results for SD to HD and HD to 4K obtained  with the proposed solution.

4.     The VMAF-BD Rate assessment provided with a graph for each QP and a table with minimum, maximum and average value for each sequence.

5.     A Complexity assessment using MAC/pixel and number of parameters of the submitted up-sampling fiter. Use of SADL (6) is recommended.

 mandatory
The text of Annex 2. mandatory
Comments on the completeness and appropriateness of the MPAI-UFV Functional Requirements and any motivated suggestion to amend and/or extend those Requirements. optional
A preliminary real-time demonstration, with a detailed document describing it. optional
Any other additional relevant information that may help evaluate the submission. optional

4               How to submit a response

Those planning to respond to this Call are:

  1. Advised that the MPAI-UFV Call for Technologies will be presented online on 2024/11/27.
  2. Requested to communicate their intention to respond to this Call with an initial version of the form of Annex 1to the MPAI secretariat by 2024/12/17. Submission of a duly filled out Annex 1 helps MPAI to properly plan for assessing the submissions. This is not a requirement, however, and the submission of a respondent to this Call who did not submit Annex 1 will still be accepted.
  3. Encouraged to regularly visit the MPAI-EVC webpage where relevant additional information may be posted.
  4. Required to deliver mandatorily their submissions to the MPAI secretariat by 2025/02/11 T23:59 UTC. The secretariat will acknowledge receipt of the submission via email.
  5. Required to attend the review of submissions according to the schedule that the 53rd MPAI General Assembly (MPAI-53) will define at its online meeting on 2025/02/19. The MPAI secretariat will inform submitters about how non MPAI members can attend the said review sessions. Respondents shall present their submission at such online review sessions. If no presenter of a submission will be in attendance, the submission will be discarded.

Further information on MPAI can be obtained from the MPAI website.

5               Evaluation

5.1       Process

Submissions will be evaluated by an Evaluation Team tasked to perform a Technical Evaluation and a General Evaluation. The Evaluation Team is created from:

  1. MPAI Member representatives in attendance.
  2. Non-MPAI Member representatives who are respondents to any of the received submissions.
  3. Non respondent experts/non MPAI Member representatives invited in a consulting capacity.
  4. No one from 1. and 2. Will be denied membership in the Evaluation Panel if they request it.

Proposals will be assessed using the following process:

  1. Each Respondent presents their proposal.
  2. Evaluation Team members ask questions.
  3. Evaluation Team organises the Tests.
  4. A volunteer member of the Evaluation Team executes the docker image of a Respondent and computes the values obtained using the test set provided by the respected academic.
  5. The Objective Quality Evaluation will use the VMAF-BD Rate metrics to compare the sequences obtained by up-sampling with the bicubic filter the AVC, HEVC and VVC co-decoded with QP Values: 22, 27, 32, 37, 42 with the same co-decoded sequences up-sampled with the proposed up-sampling algorithm.
  6. Latency is the number of frames used by the upsampling process. Note that the actual number of frames used to produce the response should be specified.
  7. The Complexity evaluation will use MAC/pixel and number of parameters.

5.2       General Evaluation

Evaluation Team members will fill out an Evaluation Form containing the following elements.

Proposal title:

Response summary: (a few lines)

Comments on possible MPAI-UFV profiles1

Assessment of submission features:

Note 1 The semantics of submission features is given by Table 3.
Note 2 Evaluation Elements indicate the elements used by Evaluators in assessing the submission.

Table 2 – Features to be considered in a submission

Submission features Evaluation Elements
Completeness of description A detailed description is recommended 
Understandability of description A detailed description is recommended 
Performance of proposal Values of the requested metrics (VMAF BD-rate)
Latency Number of frames actually used by proponent
Complexity of proposal MAC/Pixel mandatorily, complexity computed using SADL reccomended
Software implementability A detailed description is recommended 
Model scalability Performances values for different versions of algorithm

Evaluation summary:

  1. Main strong points, qualitatively:
  2. Main weak points, qualitatively:

Additional remarks: (points of importance not covered above.)

The submission features in Table 2 are explained in the following Table 3.

Table 3 – Explanation of submission features

Submission features Criteria
Completeness of description Evaluators should: check if respondents have described in sufficient detail how the requirements are supported by the proposal.

Note: Submissions will be judged for the merit of what is proposed. A submission on a single technology that is excellent may be considered despite complete but less performing submission.
Understandability Evaluators should identify items that are demonstrably unclear (inconsistencies, sentences with dubious meaning etc.)
Performance of proposal The performance in terms of VMAF-BD Rate provided with graph for each QPs and a table with minimum, maximum and average value for each sequence.
Latency For the purpose of the test, latency is the number of delayed frames that the algorithm uses to calculate the response.
Proposal should specify the actual number of frames used in the response.
Complexity Evaluators should assess computational speed, computational complexity and identify issues that would make it difficult to implement the proposal compared to the state of the art.
Software implementability  Possibility to run real time the upsampling algorithm on an affordable laptop.
Model scalability Possibility to have different versions of the same algorithm suitable for different needs of the user (e.g., saving energy, hardware constraint)

Software implementability and Model scalability are desiderable features.

6               Expected timeline

Timeline of the Call for Technologies, deadlines and response evaluation:

Table 4 – Dates and deadlines

Step Date Time
MPAI-UFV Call for Technologies issued. 2024/11/20 17:00 UTC
MPAI-UFV Call for Technologies presented online. 2024/11/27 14:00 UTC
Notification of intention to submit a proposal (Annex 1). 2024/12/17 23:59 UTC
Response submission deadline. 2025/02/11 23:59 UTC
Start of response evaluation. 2025/02/19 (MPAI-53)

The Evaluation Panel and its subgroups – if any – will conduct its work during 1-hour online sessions according to the calendar agreed at MPAI-53.

The MPAI-UFV Call for Technologies requests submitters who decide to participate in the development of the standard after the Test, to enable MPAI to develop the standard smoothly and effectively by committing to the following:

  1. To collaborate with all other submitters.
  2. To make eventually available their encoding and decoding programs in source code if (parts of) their submission is accepted.
  3. To make available their own training and testing code in case this is required to facilitate the development of the standard.
  4. To execute code if requested to progress the development of the standard, based on a fair sharing of the work.

7               References

  1. MPAI Statutes
  2. MPAI Patent Policy
  3. MPAI; Use Cases and Functional Requirements: Up-sampling Filter for Video Applications (MPAI-UFV); N2078
  4. MPAI; Framework Licence: Use Cases and Functional Requirements: Up-sampling Filter for Video Applications (MPAI-UFV); N2079
  5. MPAI; Template for Responses: Up-sampling Filter for Video Applications (MPAI-UFV); N2080.
  6. SADL https://github.com/InterDigitalInc/SADL

Annex 1 Registration Form

This Registration Form is to be filled in by a prospective Respondent to this MPAI-UFV Call for Technologies.

  1. Title of the proposal.
  2. Organisation: company name, position, e-mail of contact person.
  3. What are the main functionalities of your proposal?
  4. Does your proposal provide or describe a formal specification and APIs?
  5. Will you provide a demonstration to show how your proposal meets the evaluation criteria?

Annex 2 Mandatory text in a submission

A submission in response to Call for Technologies: Up-sampling Filter for Video Applications (MPAI-UFV) shall mandatorily include the following text:

<Company/Member> submits this technical document in response to Call for Technologies: Up-sampling Filter for Video Applications (MPAI-UFV) V1.0 (N2077).

 <Company/Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N421), in particular <Company/Member> declares that  <Company/Member> or its successors will make available the terms of the Licence related to its Essential Patents according to Framework Licence: Up-sampling Filter for Video Applications (MPAI-UFV) V1.0 (N2079), alone or jointly with other IPR holders after the approval of the planned MPAI-UFV by the General Assembly and in no event after commercial implementations of MPAI-UFV become available on the market.

In case the respondent is a non-MPAI member, the submission shall mandatorily include the following text:

If (a part of) this submission is identified for inclusion in a specification, <Company> understands that <Company> will be requested to immediately join MPAI and that, if <Company> elects not to join MPAI, this submission will be discarded.

Subsequent technical contribution shall mandatorily include this text

<Member> submits this document to MPAI as a contribution to the development of the planned MPAI-UFV Technical Specification.

 <Member> explicitly agrees to the steps of the MPAI standards development process defined in Annex 1 to the MPAI Statutes (N421), in particular  <Company> declares that <Company> or its successors will make available the terms of the Licence related to its Essential Patents according to the MPAI-UFV Framework Licence (N2079), alone or jointly with other IPR holders after the approval of the MPAI-UFV V1.0 Technical Specification by the General Assembly and in no event after MPAI-UFV V1.0 commercial implementations become available on the market.