Introduction
During the last decade, Neural Networks have been deployed in an increasing variety of domains and the production of Neural Networks became costly, in terms of both resources (GPUs, CPUs, memory) and time. Moreover, users of Neural Network based services more and more express their needs for a certified service quality.
NN Traceability offers solutions to satisfy both needs, ensuring that a deployed Neural Network is traceable and any tampering detected.
Inherited from the multimedia realm, watermarking assembles a family of methodological and application tools allowing to imperceptibly and persistently insert some metadata (payload) into an original NN model. Subsequently, detecting/decoding this metadata from the model itself or from any of its inferences provides the means to trace the source and to verify the authenticity.
An additional traceability technology is fingerprinting that relates to a family of methodological and applicative tools allowing to extract some salient information from the original NN model (a fingerprint) and to subsequently identify that model based on the extracted information.
Therefore, MPAI has found the application area called “Neural Network Watermarking” to be relevant for MPAI standardization as there is a need for both Nural Network Traceability technologies and for assessing the performances such technologies.
MPAI available standards
In response to these needs, MPAI has established the Neural Network Watermarking Development Committee (NNW-DC). The DC has developed Technical Specification: Neural Network Watermarking (MPAI-NNW) – Traceability (NNW-NNT) V1.0. This specifies methods to evaluate the following aspects of Active (Watermarking) and Passive (Fingerprinting) Neural Network Traceability Methods:
- The ability of a Neural Network Traceability Detector/Decoder to detect/decode/match Traceability Data when the traced Neural Network has been modified,
- The computational cost of injecting, extracting, detecting, decoding, or matching Traceability Data,
- Specifically for active tracing methods, the impact of inserted Traceability Data on the performance of a neural network and on its inference.
MPAI-NNW Future Standards
During its 57th GA held on June 11th, MPAI released a Call for Neural Network Watermarking (MPAI-NNW) – Technologies (NNW-TEC). This call requests Neural Network Traceability Technologies that make it possible to:
- verify that the data provided by an Actor, and received by another Actor is not compromised, i.e. it can be used for the intended scope,
- identify the Actors providing and receiving the data, and
- evaluate the quality of solutions supporting the previous two items.
An Actor is a process producing, providing, processing, or consuming information.
MPAI NNT Actors
Four types of Actors are identified as playing a traceability-related role in the use cases.
- NN owner – the developer of the NN, who needs to ensure that ownership of NN can be claimed.
- NN traceability provider – the developer of the traceability technology able to carry a payload in a neural network or in an inference.
- NN customer – the user who needs the NN owner’s NN to make a product or offer a service.
- NN end-user – the user who buys an NN-based product or subscribes to an NN-based service.
Examples of Actors are:
- Edge-devices and software
- Application devices and software
- Network devices and software
- Network services
MPAI NNT Use cases
MPAI use cases relate to both the NN per se (i.e., to the data representation of the model) and to the inference of that NN (i.e., to the result produced by the network when fed with some input data), as illustrated in Figure 1.
Figure 1: Synopsis of NNT generic use cases: Identify the ownership of an NN, Identify an NN (e.g. DOI) and Verify integrity of an NN
The NNW-TEC use cases document is available; it includes sequence diagrams describing the positions and actions of the four main Actors in the workflow.
MPAI NNT Service and application scenarios
MPAI NNT is relevant for services and applications benefitting from one or several conventional NN tasks such as:
- Video/image/audio/speech/text classification
- Video/image/audio/speech/text segmentation
- Video/image/audio/speech/text generation
- Video/image/audio/speech decoding
Figures 2, 3 and 4 present three typologies of services and applications aggregating the generic use cases presented above.
The first example (Traceable newsletter service – Figure 2) covers the case where an end-user subscribes to a newsletter that is produced by a Generative AI service (provided by an NN customer), according to the end-user profile. In such a use case, a malicious user might try to temper the very production of the personalized content or to modify it during its transmission.
The second example (Autonomous vehicle services – Figure 3) deals with the traceability and authenticity of the multimodal content that is exchanged in various ways: (1) the car A (acting as an NN end-user) sends to a server (acting as an NN customer or owner) acquired signals for data-processing, (2) An embedded AI transmits instructions such as braking, turning, or accelerating to the car (NN owner and end-user), (3) Another vehicle B in the environment transmits environmental information to vehicle A. Various types of malicious attacks with critical consequences can be envisaged: AI interception and corruption (e.g. adversarial learning), les données can be corrupted in their transmission from and/or to the autonomous vehicle (forced connection interruption or data modification).
The third example (AI generated or processed information services – Figure 4) shows how NNT can be beneficial when real images are modified by a deepfake process. A user capturing a video sequence by a connected camera would like to appear as the archetype secret agent (say, James Bond) by interacting with a generative AI service remotely accessible in the network. This module synthesizes novel audiovisual content, which is then rendered on a large display for the user to enjoy. Such services are not immune from security threats: the attacker can intercept the encoded stream prior to its arrival at the trusted AI server and processed is through a malicious edge-deployed generative AI or it can affect the very trusted generative AI service (e.g. by employing some adversarial training techniques).
Figure 2: Autonomous vehicle services
Figure 3: AI-generated newsletter example
Figure 4: AI generated or processed information services
