This chapter will deal with two watermarking-related cases:

  • NNs for which the watermark is added after the NNs model was created.
  • NNs for which the watermark is added during the training of the NNs model.

1       Post-training watermark embedding

The Imperceptibility evaluation specifies the Means that enable a Tester to evaluate the differences in performance of a NN before and after the watermark embedding process. There are two cases:

  1. The NN has the input and output data format with specified semantics.
  2. The input and output data format of the NN do not have specified semantics.

1.1      NN with I/O data format has specified semantics

In this section, two actors are involved: the NN Watermarking provider requesting a Tester to evaluate the Imperceptibility performance of their NN Watermarking Method.

The Tester shall adopt the following procedure:

  1. Define a pair of training and testing datasets with a size with at least an order of magnitude more entries than trainable Weights.
  2. Select:
    1. A set of M unwatermarked NNs trained on the training dataset.
    2. D data Payloads corresponding to the pre-established Payload size.
  3. Apply the NN Watermarking Method to the M
  4. Process the training dataset and the D data Payload (if needed).
  5. Feed the M unwatermarked NN with the test dataset
  6. Measure the task-dependent quality of the produced inference.
  7. Feed the M x D watermarked NN with the same test dataset
  8. Measure the task-dependent quality of the produced inference, informative examples of quality evaluation are provided in Annex 5.
  9. Provide the task-dependent quality of the produced inference measured in 6 and 7.

1.2      NN with I/O data format has no specified semantics

In this section, two actors are involved: the NN Watermarking provider requesting a Tester to evaluate the Imperceptibility performance of their Watermarking Method.

The workflow of the process shall be the following:

  1. Tester connects the NN to other NN until the input and output of the resulting configuration have input / output formats with specified semantics.
  2. Tester applies all the steps in 6.1.1.

2       In-training watermark embedding

The Imperceptibility evaluation specifies the Means for evaluating the performance of a watermarked NN. The workflow of the process shall evaluate the watermarked NN as an NN.