<- Definitions     Go to ToC       Metadata ->

1       AI Framework Components

1.1       Components for Basic Functionalities

1.2       Components for Security Functionalities

2       AI Framework Implementations

3       AIMs

3.1       Implementation types

3.2       Combination

3.3       Hardware-software compatibility

3.4       Actual implementations

1       AI Framework Components

This MPAI-AIF Version adds a Secure Profile with Security functionalities on top of the Basic Profile of Version 1.1 with the following restrictions:

  • There is only one AIW containing only one AIM – which may be a Composite AIM.
  • The AIM implementer guarantees the security of the AIM by calling the security API.
  • The AIF application developer cannot access securely the Composite AIM internals.

1.1      Components for Basic Functionalities

Figure 2 specifies the MPAI-AIF Components supported by MPAI-AIF Version 1.1.

Figure 2 – The MPAI-AIF V1 Reference Model

The specific functions of the Components are:

  1. Controller:
    • Provides basic functionalities such as scheduling, communication between AIMs and with AIF Components such as AIM Storage and Global Storage.
    • Acts as a resource manager, according to instructions given by the User through the User Agent.
    • Can interact by default to all the AIMs in a given AIF.
    • Activates/suspends/resumes/deactivates AIWs based on User’s or other inputs.
    • May supports complex application scenarios by balancing load and resources.
    • Accesses the MPAI Store APIs to download AIWs and AIMs.
    • Exposes three APIs:
      • AIM APIs enable AIMs to communicate with it (register themselves, communicate and access the rest of the AIF environment). An AIW is an AIM with additional metadata. Therefore, an AIW uses the same AIM API.
      • User APIs enable User or other Controllers to perform high-level tasks (e.g., switch the Controller on and off, give inputs to the AIW through the Controller).
      • Controller-to-Controller API enables interactions among Controllers.
    • May run an AIW on different computing platforms and may run more than one AIW.
    • May communicate with other Controllers.
  2. Communication: connects the AIF Components via Events or Channels connecting an output Port of an AIM with an input Port of another AIM. Communication has the following characteristics:
    • The Communication Component is turned on jointly with the Controller.
    • The Communication Component needs not be persistent.
    • Channels are unicast and may be physical or logical.
    • Messages are transmitted via Channels. They are composed of sequences of Records and may be of two types:
      • High-Priority Messages expressed as up to 16-bit integers.
      • Normal-Priority Messages expressed as MPAI-AIF defined types (6.1.1).
        • Messages may be communicated through Channels or Events.
  1. AI Module (AIM): a data processing element with a specified Function receiving AIM-specific inputs and producing AIM-specific outputs having the following characteristics:
    • Communicates with other Components through Ports or Events.
    • Includes at least one input Port and one output Port.
    • May incorporate other AIMs.
    • May be hot-pluggable, and dynamically register and disconnect itself on the fly.
    • May be executed:
      • Locally, e.g., it encapsulates hardware physically accessible to the Controller.
      • On different computing platforms, e.g., in the cloud or on groups of drones, and encapsulates communication with a remote Controller.
  2. AI Workflow (AIW): an organised aggregation of AIMs receiving AIM-specific inputs and producing AIM-specific outputs according to its Function implementing a Use Case that is either proprietary or specified by an MPAI Application Standard.
  3. Global Storage: stores data shared by AIMs.
  4. AIM Storage: stores data of individual AIMs.
  5. User Agent: interfaces the User with an AIF through the Controller.
  6. Access: provides access to static or slowly changing data that is required by AIMs such as domain knowledge data, data models, etc.
  7. MPAI Store: stores Implementations for users to download by secure protocols.

Note: When different Controllers running on separate computing platforms (Group Elements) interact with one another, they cooperate by requesting one or more Controllers in range to open Remote Ports. The Controllers on which the Remote Ports are opened can then react to information sent by other Controllers in range through the Remote Ports and implement a collective behaviour of choice. For instance: there is a main Controller and the other Controllers in the Group react to the information it sends; or there is no main Controller and all Controllers in the Group behave according to a collective logic specified in the Controllers.

1.2      Components for Security Functionalities

The AIF Components have the following features:

  1. The AIW
    • The AIMs in the AIW trust each other and communicate without special security concerns.
    • Communication among AIMs in the Composite AIM is non-secure.
  2. The Controller
    • Communicates securely with the MPAI-Store and the User Agent (Authentication, Attestation, and Encryption).
    • Accesses Communication, Global Storage, Access and MPAI Store via Trusted Services API.
    • Is split in two parts:
      • Secure Controller accesses Secure Communication and Secure Storage.
      • Non-Secure Controller can access the non-secure parts of the AIF.
    • Interfaces with the User Agent in the area where non-secure code is executed.
    • Interface with the Composite AIM in the area where secure code is executed,
  3. AIM/AIW Storage
    • Secure Storage functionality is provided through key exchange.
    • Non-secure functionality is provided without reference to secure API calls.
  4. The AIW/AIMs call the Secure Abstraction Layer via API.
  5. The AIMs of a Composite AIM shall run on the same computing platform.

Figure 3 specifies the MPAI-AIF Components operating in the secure environment created by the Secure Abstraction Layer.

Figure 3 – The MPAI-AIF V2 Reference Model

2       AI Framework Implementations

MPAI-AIF enables a wide variety of Implementations:

  1. AIF Implementations can be tailored to different execution environments, e.g., High-Performance Computing systems or resource-constrained computing boards. For instance, the Controller might be a process on a HPC system or a library function on a computing board.
  2. There is always a Controller even if the AIF is a lightweight Implementation.
  3. The API may have different MPAI-defined Profiles to allow for Implementations:
    • To run on different computing platforms and different programming languages.
    • To be based on different hardware and resources available.
  4. AIMs may be Implemented in hardware, software and mixed-hardware and software.
  5. Interoperability between AIMs is ensured by the way communication between AIMs is defined, irrespective of whether they are implemented in hardware or software.
  6. Use of Ports and Channels ensures that compatible AIM Ports may be connected irrespective of the AIM implementation technology.
  7. Message generation and Event management is implementation independent.

3       AIMs

3.1      Implementation types

AIMs can be implemented in either hardware or software keeping the same interfaces independent of the implementation technology. However, the nature of the AIM might impose constraints on the specific values of certain API parameters and different Profiles may impose different constraints. For instance, Events (easy to accommodate in software but less so in hardware); and persistent Channels (easy to make in hardware, less so in software).

While software-software and hardware-hardware connections are homogeneous, a hybrid hardware-software scenario is inherently heterogeneous and requires the specification of additional communication protocols, which are used to wrap the hardware part and connect it to software. A list of such protocols is provided by the MPAI Ontology [11].

Examples of supported architectures are:

  • CPU-based devices running an operating system.
  • Memory-mapped devices (FPGAs, GPUs, TPUs) which are presented as accelerators.
  • Cloud-based frameworks.
  • Naked hardware devices (i.e., IP in FPGAs) that communicate through hardware Ports.
  • Encapsulated blocks of a hardware design (i.e., IP in FPGAs) that communicate through a memory-mapped bus. In this case, the Metadata associated with the AIM (see 6.3) shall also specify the low-level communication protocol used by the Ports.

3.2      Combination

MPAI-AIF supports the following ways of combining AIMs:

  • Software AIMs connected to other software AIMs resulting in a software AIM.
  • Non-encapsulated hardware blocks connected to other non-encapsulated hardware blocks, resulting in a larger, non-encapsulated hardware AIM.
  • Encapsulated hardware blocks connected to either other encapsulated hardware blocks or other software blocks, resulting in a larger software AIM.

Connection between a non-encapsulated hardware AIM and a software AIM is not supported as in such a case direct communication between the AIMs cannot be defined in any meaningful way.

3.3      Hardware-software compatibility

To achieve communication among AIMs irrespective of their implementation technology, the requirements considered in the following two cases should be satisfied:

  1. Hardware AIM to Hardware AIM: Each named type in a Structure is transmitted as a separate channel. Vector types are implemented as two channels, one transmitting the size and the second transmitting the data.
  2. All other combinations: Fill out a Structure by recursively traversing the definition (breadth-first). Sub-fields are laid down according to their type, in little-endian order.

3.4      Actual implementations

3.4.1     Hardware

Metadata ensures that hardware blocks can be directly connected to other hardware/software blocks, provided the specification platforms for the two blocks have compatible interfaces, i.e., they have compatible Ports and Channels.

3.4.2     Software

Software Implementations shall ensure that Communication among different constituent AIMs, and with other AIMs outside the block, is performed correctly.

In addition, AIM software Implementations shall contain a number of well-defined steps so as to ensure that the Controller is correctly initialised and remains in a consistent internal state, i.e.:

  1. Code registering the different AIMs used by the AIW. The registration operation specifies where the AIMs will be executed, either locally or remotely. The AIM Implementations are archives downloaded from the Store containing source code, binary code and hardware designs executed on a local machine/HPC cluster/MPC machine or a remote machine.
  2. Code starting/stopping the AIMs.
  3. Code registering the input/output Ports for the AIM.
  4. Code instantiating unicast channels between AIM Ports belonging to AIMs used by the AIW, and connections from/to the AIM being defined to/from remote AIMs.
  5. Registering Ports and connecting them may result in a number of steps performed by the Controller – some suitable data structure (including, for instance, data buffers) will be allocated for each Port or Channel, in order to support the functions specified by the Controller API called by the AIM (8.3).
  6. Explicitly write/read data to/from, any of the existing Ports.
  7. In general, arbitrary functionality can be added to a software AIM. For instance, depending on the AIM Function, one would typically link libraries that allow a GPU or FPGA to be managed through Direct Memory Access (DMA), or link and use high-level libraries (e.g., Tensor­Flow) that implement AI-related functionality.
  8. The API implementation depends on the architecture the Implementation is designed for.

 <- Definitions     Go to ToC       Metadata ->