<-Complexity Reduction    Go to ToC 

(Informative)

1. Introduction

This chapter provides the Neural Network weights obtained by applying the process specified in:

  1.  Design Procedure applied to
    1. The standard-definition to high-definition up-sampling, and
    2. High-definition to Ultra High-Definition up-sampling.
  2.  Complexity Reduction applied to the Neural Network of point 1, namely:
    1. The standard-definition to high-definition up-sampling, and
    2. High-definition to Ultra High-Definition up-sampling.
  3. Procedure to generate an up-sampled image.

2. Test conditions

Table 1 provides the test conditions employed for the performance verification of the un-pruned and pruned up-sampling filters.

Table 1 – Test conditions for performance verification

Standard sequences  CatRobot, FoodMarket4, ParkRunning3.
Bits/sample  8 and 10 bit-depth per component.
Colour space  YCbCr with 4:2:0 sub sampling.
Encoding technologies  AVC, HEVC, and VVC.
Encoding settings  Random Access and Low Delay at QPs 22, 27, 32, 37, 42, 47.
Up-sampling SD to HD and HD to UHD.
Metrics BD-Rate, BD-PSNR and BD-VMAF
Deep-learning structure Same for all QPs

Table 2 includes the performance results luminance only for Video sequences

  1. unpruned and pruned up-sampling filters,
  2. for SD to HD, HD to UHD and for SD to HD using the HD to UHD parameters
  3. for videos that have been encoded with HEVC and VVC
  4. in Low Delay (LD) and Random Access (RA) coding settings.

2. Performance results

Results show an impressive improvement for all coding technologies, and encoding options for all three objective metrics when compared with the currently used traditional bicubic interpolation.

Table 2 – Performance of the EVC-UFV Up-sampling Filter

HEVC (LD) VVC (LD) HEVC (RA) VVC (RA)
Unpruned SD to HD (using own trained filter) 12.08% 13.74% 17.14% 22.5%
Unpruned HD to UHD (using own trained filter) 4.05% 4.39% 6.29% 8.49%
Unpruned SD to HD (using HD to UHD filter) 11.79% 13.45% 15.67% 20.38%
Pruned SD to HD (using own trained filter) 12.2% 13.8% 17.3% 22.5%
Pruned HD to UHD (using own trained filter) 6.0% 6.5% 6.0% 7.9%
Pruned SD to HD (using HD to UHD filter) 11.6% 11.4% 15.3% 19.9%

Table 3 provides the same information for YUV sequences.

Table 3 – Performance of the EVC-UFV Up-sampling Filter

HEVC (LD) VVC (LD) HEVC (RA) VVC (RA)
Unpruned SD to HD (using own trained filter) U = 7.75 %
V = 9.58%		 U = 6.60%
V = 7.93%		 U = 10.90%
V = 12.90%		 U = 16.90%
V = 1803%
Unpruned HD to UHD (using own trained filter) U = 7.83%
V = 8.15%		 U = 7.20%
V = 7.41%		 U = 9.39%
V = 9.46%		 U = 10.64%
V = 10.92%
Unpruned SD to HD (using HD to UHD filter) U = 8.60%
V = 10.51%		 U = 6.18%
V = 7.48%		 U = 10.33%
V = 12.17%		 U = 14.52%
V = 15.68%
Pruned SD to HD (using own trained filter) U = 8.38%
V = 10.20		 U = 6.47%
V = 7.75%		 U = 11.29%
V = 13.20%		 U = 16.75%
V = 17.81%
Pruned HD to UHD (using own trained filter) U = 6.92%
V = 7.24%		 U = 6.81%
V = 7.07%		 U = 8.44%
V = 8.63%		 U = 9.99 %
V = 10.39%
Pruned SD to HD (using HD to UHD filter) U = 7.84%
V = 9.64%		 U = 5.79%
V = 7.07%		 U = 9.52%
V = 11.32%		 U = 14.06%
V = 15.16%

3. Testing the up-sampling procedure

The software at MPAI Git enables users of this Technical Specification to:

  1. Upload an SD or HD image
  2. Select the type of weights – pruned or unpruned to be used to up-sample the provided image.
  3. Download the up-sampled image.

Note that the image includes a small DeepCamera logo at the centre of the image

The number of parameters of the pruned filter is about  40% of the un-pruned filter.

The loss in performance of the pruned filter is less than 1% in BD-rate compared to the performance of the un-pruned filter.

<-Complexity Reduction    Go to ToC