<-Complexity Reduction    Go to ToC 

(Informative)

1. Introduction

This chapter provides the Neural Network weights obtained by applying the process specified in:

  1.  Design Procedure applied to
    1. The standard-definition to high-definition up-sampling, and
    2. High-definition to Ultra High-Definition up-sampling.
  2.  Complexity Reduction applied to the Neural Network of point 1, namely:
    1. The standard-definition to high-definition up-sampling, and
    2. High-definition to Ultra High-Definition up-sampling.

2. Test conditions

Table 1 provides the test conditions employed for the performance verification of the un-pruned and pruned up-sampling filters.

Table 1 – Test conditions for performance verification

Standard sequences  CatRobot, FoodMarket4, ParkRunning3.
Bits/sample  8 and 10 bit-depth per component.
Colour space  YCbCr with 4:2:0 sub sampling.
Encoding technologies  AVC, HEVC, and VVC.
Encoding settings  Random Access and Low Delay at QPs 22, 27, 32, 37, 42, 47.
Up-sampling SD to HD and HD to UHD.
Metrics BD-Rate, BD-PSNR and BD-VMAF
Deep-learning structure Same for all QPs

Table 2 includes the performance results luminance only for Video sequences

  1. unpruned and pruned up-sampling filters,
  2. for SD to HD, HD to UHD and for SD to HD using the HD to UHD parameters
  3. for videos that have been encoded with HEVC and VVC
  4. in Low Delay (LD) and Random Access (RA) coding settings.

2. Performance results

Results show an impressive improvement for all coding technologies, and encoding options for all three objective metrics when compared with the currently used traditional bicubic interpolation.

Table 2 – Performance of the EVC-UFV Up-sampling Filter

HEVC (LD) VVC (LD) HEVC (RA) VVC (RA)
Unpruned SD to HD (using own trained filter)
Unpruned HD to UHD (using own trained filter)
Unpruned SD to HD (using HD to UHD filter)
Pruned SD to HD (using own trained filter) 12.2% 13.8% 17.3% 22.5%
Pruned HD to UHD (using own trained filter) 6% 6.5% 6.0% 7.9%
Pruned SD to HD (using HD to UHD filter) 11.6% 11.4% 15.3% 19.9%

Table 3 provides the same information for YUV sequences.

Table 3 – Performance of the EVC-UFV Up-sampling Filter

HEVC (LD) VVC (LD) HEVC (RA) VVC (RA)
Unpruned SD to HD (using own trained filter)
Unpruned HD to UHD (using own trained filter)
Unpruned SD to HD (using HD to UHD filter)
Pruned SD to HD (using own trained filter)
Pruned HD to UHD (using own trained filter)
Pruned SD to HD (using HD to UHD filter)

3. Weights of the complexity-reduced network

The un-pruned and pruned weights of the SD (540×960) to HD (1080×1920) and HD (1080×1920) to UHD (2160×3840) up-sampling filters can be downloaded from the MPAI Git.

Table 4 provides all the up-sampling filters’ weights As they are watermarked, the performance obtained by using these weights is slightly inferior to the ones reported in Section 2.

Table 4 – Watermarked weights of up-sampling filters

Un-pruned Pruned
SD to HD Weights Weights
HD to UHD Weights Weights

The number of parameters of the pruned filteris about  40% of the un-pruned filter.

The loss in performance of the pruned filter is less than 1% in BD-rate compared to the performance of the un-pruned filter.

<-Complexity Reduction    Go to ToC