This image was generated from a set of captured digital images used to train a Generative Adversarial Network, so would be classified as “trainedAlgorithmicMedia” in the proposed new version of the Digital Source Type CV. Source: Public Domain via Wikimedia Commons

A hot topic in media circles these days is “synthetic media”. That is, media that was created either partly or fully by a computer. Usually the term is used to describe content created either partly or wholly by AI algorithms.

IPTC’s Video Metadata Working Group has been looking at the topic recently and we concluded that it would be useful to have a way to describe exactly what type of content a particular media item is. Is it a raw, unmodified photograph, video or audio recording? Is it a collage of existing photos, or a mix of synthetic and captured content? Was it created using software trained on a set of sample images or videos, or is it purely created by an algorithm?

We have an existing vocabulary that suits some of this need: Digital Source Type. This vocabulary was originally created to be able to describe the way in which an image was scanned into a computer, but it also represented software-created images at a high level. So we set about expanding and modifying that vocabulary to cover more detail and more specific use cases.

It is important to note that we are only describing the way a media object has been created: we are not making any statements about the intent of the user (or the machine) in creating the content. So we deliberately don’t have a term “deepfake”, but we do have “trainedAlgorithmicMedia” which would be the term used to describe a piece of content that was created by an AI algorithm such as a Generative Adversarial Network (GAN).

Here are the terms we propose to include in the new version of the Digital Source Type vocabulary. (New terms and definition changes are marked in bold text. Existing terms are included in the list for clarity.)

Term IDdigitalCapture
Term nameOriginal digital capture sampled from real life
Term descriptionThe digital media is captured from a real-life source using a digital camera or digital recording device
ExamplesDigital photo or video taken using a digital SLR or smartphone camera
Term IDnegativeFilm
Term nameDigitised from a negative on film
Term descriptionThe digital media was digitised from a negative on film on any other transparent medium
ExamplesFilm scanned from a moving image negative
Term IDpositiveFilm
Term nameDigitised from a positive on film
Term descriptionThe digital media was digitised from a positive on a transparency or any other transparent medium
ExamplesDigital photo scanned from a photographic positive
Term IDprint
Term nameDigitised from a print on non-transparent medium
Term descriptionThe digital image was digitised from an image printed on a non-transparent medium
ExamplesDigital photo scanned from a photographic print
Term IDhumanEdited
Term nameOriginal media with minor human edits
Term descriptionMinor augmentation or correction by a human, such as a digitally-retouched photo used in a magazine
ExamplesVideo camera recording, manipulated digitally by a human editor
Term IDcompositeCapture
Term nameComposite of captured elements
Term descriptionMix or composite of several elements that are all captures of real life
Examples* A composite image created by a digital artist in Photoshop based on several source images
* Edited sequence or composite of video shots
Term IDalgorithmicallyEnhanced
Term nameAlgorithmically-enhanced media
Term descriptionMinor augmentation or correction by algorithm
ExamplesA photo that has been digitally enhanced using a mechanism such as Google Photos’ “de-noise” feature
Term IDdataDrivenMedia
Term nameData-driven media
Term descriptionDigital media representation of data via human programming or creativity
ExamplesA representation of a distant galaxy created by analysing the outputs of a deep-space telescope (as opposed to a regular camera)
An infographic created using a computer drawing tool such as Adobe Illustrator or AutoCAD
Term IDdigitalArt
Term nameDigital art
Term descriptionMedia created by a human using digital tools
Examples* A cartoon drawn by an artist into a digital tool using a digital pencil, a tablet and a drawing package such as Procreate or Affinity Designer
* A scene from a film/movie created using Computer Graphic Imagery (CGI)
* Electronic music composition using purely synthesised sounds
Term IDvirtualRecording
Term nameVirtual recording
Term descriptionLive recording of virtual event based on synthetic and optionally captured elements
Examples* A recording of a computer-generated sequence, e.g. from a video game
* A recording of a Zoom meeting
Term IDcompositeSynthetic
Term nameComposite including synthetic elements
Term descriptionMix or composite of several elements, at least one of which is synthetic
Examples* Movie production using a combination of live-action and CGI content, e.g. using Unreal engine to generate backgrounds
* A capture of an augmented reality interaction with computer imagery superimposed on a camera video, e.g. someone playing Pokemon Go
Term IDtrainedAlgorithmicMedia
Term nameTrained algorithmic media
Term descriptionDigital media created algorithmically using a model derived from sampled content
Examples* Image based on deep learning from a series of reference examples
* A “speech-to-speech” generated audio or “deepfake” video using a combination of a real actor and an AI model
* “Text-to-image” using a text input to feed an algorithm that creates a synthetic image
Term IDalgorithmicMedia
Term nameAlgorithmic media
Term descriptionMedia created purely by an algorithm not based on any sampled training data, e.g. an image created by software using a mathematical formula
Examples* A purely computer-generated image such as a pattern of pixels generated mathematically e.g. a Mandelbrot set or fractal diagram
* A purely computer-generated moving image such as a pattern of pixels generated mathematically

We propose that the following term, which exists in the current DigitalSourceType CV, be retired:

Term IDRETIRE: softwareImage
Term nameCreated by software
Term descriptionThe digital image was created by computer software
NoteWe propose that trainedAlgorithmicMedia or algorithmnicMedia be used instead of this term.

We welcome all feedback from across the industry to these proposed terms.

Please contact Brendan Quinn, IPTC Managing Director at mdirector@iptc.org use the IPTC Contact Us form to send your feedback.