1. Introduction

This chapter specifies

  1. The steps required to implement the pruning method that enables a User to produce a Up-sampling network with reduced complexity starting from a fine-tuned model produced based on the procedure specified in Design Procedure.
  2. The weights reduced-complexity neural networks referenced in UFV Filter Specification.

2. Pruning Algorithm

The procedure requires a fine-tuned or pre-trained model (in the following “Model”) and a dataset to which the Pruning Algorithm specified in this chapter is applied.

The Pruning Algorithm is based on DepGraph (1, 2). This requires a dataset containing Standard Definition (SD), High Definition (HD), and 4k resolution (4K). The dataset may be the same as used in the process specified in Design Procedure. If dataset is not available, the dataset shall be developed using the procedure of Design Procedure. 

  1. Set the Performance Criteria that the Pruned Model should satisfy.
  2. Compute the Dependency Graph and Pruning Groups.  
  3. Starting from the non-pruned Model, execute the following Steps with values of N – depending on the desired convergence speed – until the Pruned Model satisfies the complexity (number of parameters) and performance criteria:
    1. Set the target Threshold to number*N% for the Channels having the Step equal to number. E.g., if N is 10 and the current Step is 4, the Threshold is set to 40%.
    2. Apply Sparsity Learning to the Model.
    3. Evaluate the Importance of each Channel in each Layer of the Model. The Importance is defined as the average of all Parameters of the Channel divided by the number of Parameters in the Channel.
    4. Perform the following steps until the percentage of surviving Parameters is greater than Threshold:
      1. For each Layer, remove the Channel that has the lowest Importance.
      2. For each removed Channel, remove all the Channels that are in the same Pruning Groups.
    5. Retrain the Pruned Model for the Model Recovery Phase for a number Epochs that…. At the end retain the Model with the highest Validation score.
    6. Save the Pruned Model.
  4. Select the Pruned Model having satisfying the established complexity (number of parameters) and Performance Criteria.

The Reference Software of the Pruning Algorithm is available at Git.