The Computational cost evaluation specifies the Means that enable a Tester to evaluate the computational cost of:

  • Injecting, in terms of memory footprint, time to process an epoch, and number of epochs necessary to insert the watermark.
  • Detecting or decoding, in terms of memory footprint and time for the detector or the decoder to produce the expected result.

1       Computational cost of injecting a watermark

The Computational cost evaluation specifies the Means that enable a Tester to evaluate the computational cost of the injection using neural network watermarking method under testing.

The following four elements shall be used to characterize the injection process:

  1. The memory footprint.
  2. The time to execute the operation required by one epoch normalized according to the number of batches processed in one epoch.
  3. In case of the injection is done concurrently with the training of the network, the number of epochs required to insert the watermark.
  4. The time for the watermarked neural network to compute an inference.

The Tester shall evaluate the Computational cost of the injection according to the following workflow:

  1. Define a pair of training and testing datasets with a size with at least an order of magnitude more entries than trainable parameters.
  2. Select:
    1. The training dataset (if needed).
    2. A set of M unwatermarked NNs trained on the training dataset.
    3. D data payloads corresponding to the pre-established payload size.
  3. Apply the watermarking technology to the M NNs using the D data payloads.
  4. Record the corresponding M x D set of values characterizing the processing.
  5. Provide the statistical average of the values over the total number of tests (e. M x D) for one of the informative Testing Environments of Table 3.

2       Computational cost of detecting/decoding

The MPAI Computational cost evaluation specifies the Means that enable a Tester to evaluate the computational cost of the detecting/decoding of a neural network watermarking methods.

We use the total duration and the memory footprint to characterize the detecting/decoding process.

The Tester shall evaluate the Computational cost of detecting/decoding according to the following workflow:

  1. Select a set of M unwatermarked NNs, D data payloads corresponding to the pre-established payload size and, if needed, the train dataset.
  2. Apply the watermarking technology to the M NNs with the D data payloads
  3. Evaluate the Robustness of the detector:
  4. Apply the Watermark detector to any the M x D NNs.
  5. Record the corresponding M x D set of values characterizing the processing.
  6. Evaluate the Robustness of the decoder:
  7. Apply the Watermark decoder to any the M x D NNs.
  8. Record the corresponding M x D set of values characterizing the processing.
  9. Provide the statistical average of the values over the total number of tests (over M x D) for one of the informative Testing Environments of Table 3.

Table 3. Testing Environments (informative)

  Testing environment
Medium – Single GPU (16GB/6144 CUDA cores)

– 8 cores CPU (2.6GHz)
Large – Double GPU (32GB/12288 CUDA cores)

– 16 cores CPU (3.4GHz)