1 Definition

Visual Qualifier is a set of Data providing additional information on Visual Data for potential use by a machine.

Visual Object includes Visual Qualifier in addition to Visual Data. It is specified by MPAI-OSD V1.1.

2 Functional Requirements

Visual Qualifier must allow the expression of the following Qualifier elements:

  1. Sub-Types
    1. Colour
    2. Transparency
    3. Brightmess
  2. Formats
    1. Content
      1. Sampling
        1. Time
        2. Space
      2. SD
        1. Static
        2. Dynamic
      3. 3D
        1. Static
        2. Dynamic
    2. Transport
  3. Attributes
    1. Source
      1. Real
      2. Synthetic
    2. Metadata
      1. Visual Metadata Formats
      2. Object Identity
      3. Content Description
    3. Device
      1. Device ID
      2. Device Location
    4. Sensor Characteristics

3 Syntax

https://schemas.mpai.community/TFA/V1.0/data/VisualQualifier.json

4 Semantics

  1. Sub-Types

    1. Colour
      1. Definition: the parameters that characterise the digital representation of the colour information of a Visual instance.
      2. Parameters
        1. Colour Formats
          1. Definition: the specific mapping of the colour space
          2. Mappings
            1. Rec. ITU-R BT. 709
            2. Rec. ITU-R BT. 2020
            3. SMPTE ST 2036-1
        2. Alpha-channel
          1. Definition: a number that represents the degree of transparency of a colour of a Visual instance.
          2. Number
            1. A number between 0 and 1.
        3. Brightness
          1. Definition: the expression of the maximum value of intensity than a Visual instance can take
          2. Units of measure
            1. Nit (candelas/sqm)
        4. YUV
          1. Definition; The specific types of YUV transformation of the three colour axes.
          2. Types
            1. Y’UV
            2. Y’PbPr
            3. Y’CbCr
            4. YDbDr
            5. Y’IQ
        5. Colour Subsampling
          1. Definition: The specific  method of subsampling the colour information
          2. Methods
            1. 4:4:4
            2. 4:2:2
            3. 4:1:1
        6. YMCK
          1. Definition: the specific colour axis rotation used by the printing industry with the addition of the Black channel.

  2. Formats

    1. Content
        1. Sampling
          1. Definitions: The characteristics of the transformation from a Visual instance in a real  space to a Visual entity in the Virtual Space.
            1. Characteristics
              1. Time
                1. Definition: method to define the period between digital representations of snapshots of a Visual instance.
                2. Methods
                  1. Time between snapshots.
              2. Space
                1. Definitions: method to represent the density of visual samples in the Virtual Space.
                2. Methods
                  1. Dots per inch (dpi)
              3. Sample Precision:
                1. Definition: Integer representing the number of bits/pixel
        2. 2D
          1. Definition: the digital representation of a time-independent or time-dependent Visual instance as 2D information
          2. Methods
            1. Static
              1. BMP
              2. Bounding Box
              3. JPEG (ISO/IEC 10918-1:1994)
              4. JPEG 2000 (ISO/IEC 15444-1:2019)
              5. JPEG XS (ISO/IEC 21122-1:2024)
              6. PNG
              7. RAW
              8. SVG
              9. TIFF
            2. Dynamic
              1. AVC (ISO/IEC 14496-10:2022)
              2. AV1
              3. EVC (ISO/IEC 23094-1:2020)
              4. HEVC (ISO/IEC 23008-2:2023)
              5. LCEVC (ISO/IEC 23094-2:2021)
              6. VVC (ISO/IEC 23090-2:2023)
          3. 3D
            1. Definition: digital representation of a time-independent or time-dependent Visual instance as 3D information
            2. Methods
              1. Static
                1. 3DS
                2. Bounding Box
                3. glTF
                4. G-PCC (ISO/IEC 23090-9:2023)
                5. OBJ
                6. USD
                7. V-PCC (ISO/IEC 23090-5:20230)
              2. Dynamic
                1. FBX
                2. glTF
                3. G-PCC (ISO/IEC 23090-9:2023)
                4. USD
                5. V-PCC (ISO/IEC 23090-5:20230)
    2. Transport
      1. Definition: the type of data arrangement used to transport a Visual instance
      2. Methods
        1. File
          1. AVI
          2. EXIF
          3. JPEG XS (ISO/IEC 21122-3:2022)
          4. MP4 (ISO/IEC 14496-12:2022)
        2. Stream
          1.  DASH (ISO/EC 23009-1:2022)
          2. HTTP Live Streaming
          3. WebRTC
          4. MPEG-2 TS (ISO/IEC 13818-1:2023)

  3. Attributes

    1. Source Type
      1. Definition: the types of a Visual instance.
      2. Types:
        1. Real
          1. Raster
        2. Synthetic
          1. Raster
          2. Vector
    2. Metadata
      1. Definition: the types of data formats attached to a Visual instance.
      2. Methods
        1. Visual Metadata Formats
          1. Dublin Core (ISO 15836-1:2017)
          2. IPTC Photo Metadata
          3. XMP
        2. Object IDs
          1. Instance Identifier
        3. Entity Internal Status
          1. Personal Status
    3. Device
      1. Definition: elements of the device that captured the Visual instance.
      2. Elements
        1. Device ID
          1. Definition: an identifier of the device that captured the Speech instance
          2. Types
            1. A string.
        2. Device Location
          1. Definition: the position and orientation of the device that captured a Visual instance in a real or virtual space.
            1. Position and orientation
              1. Point of View
        3. Sensor characteristics
          1. Definition: sensor features having an impact on the captured Visual instance
          2. Features