A hot topic in media circles these days is “synthetic media”. That is, media that was created either partly or fully by a computer. Usually the term is used to describe content created either partly or wholly by AI algorithms.

IPTC’s Video Metadata Working Group has been looking at the topic recently and we concluded that it would be useful to have a way to describe exactly what type of content a particular media item is. Is it a raw, unmodified photograph, video or audio recording? Is it a collage of existing photos, or a mix of synthetic and captured content? Was it created using software trained on a set of sample images or videos, or is it purely created by an algorithm?

We have an existing vocabulary that suits some of this need: Digital Source Type. This vocabulary was originally created to be able to describe the way in which an image was scanned into a computer, but it also represented software-created images at a high level. So we set about expanding and modifying that vocabulary to cover more detail and more specific use cases.

It is important to note that we are only describing the way a media object has been created: we are not making any statements about the intent of the user (or the machine) in creating the content. So we deliberately don’t have a term “deepfake”, but we do have “trainedAlgorithmicMedia” which would be the term used to describe a piece of content that was created by an AI algorithm such as a Generative Adversarial Network (GAN).

Here are the terms we propose to include in the new version of the Digital Source Type vocabulary. (New terms and definition changes are marked in bold text. Existing terms are included in the list for clarity.)

Term ID	digitalCapture
Term name	Original digital capture sampled from real life
Term description	The digital media is captured from a real-life source using a digital camera or digital recording device
Examples	Digital photo or video taken using a digital SLR or smartphone camera

Term ID	negativeFilm
Term name	Digitised from a negative on film
Term description	The digital media was digitised from a negative on film on any other transparent medium
Examples	Film scanned from a moving image negative

Term ID	positiveFilm
Term name	Digitised from a positive on film
Term description	The digital media was digitised from a positive on a transparency or any other transparent medium
Examples	Digital photo scanned from a photographic positive

Term ID	print
Term name	Digitised from a print on non-transparent medium
Term description	The digital image was digitised from an image printed on a non-transparent medium
Examples	Digital photo scanned from a photographic print

Term ID	humanEdited
Term name	Original media with minor human edits
Term description	Minor augmentation or correction by a human, such as a digitally-retouched photo used in a magazine
Examples	Video camera recording, manipulated digitally by a human editor

Term ID	compositeCapture
Term name	Composite of captured elements
Term description	Mix or composite of several elements that are all captures of real life
Examples	* A composite image created by a digital artist in Photoshop based on several source images * Edited sequence or composite of video shots

Term ID	algorithmicallyEnhanced
Term name	Algorithmically-enhanced media
Term description	Minor augmentation or correction by algorithm
Examples	A photo that has been digitally enhanced using a mechanism such as Google Photos’ “de-noise” feature

Term ID	dataDrivenMedia
Term name	Data-driven media
Term description	Digital media representation of data via human programming or creativity
Examples	A representation of a distant galaxy created by analysing the outputs of a deep-space telescope (as opposed to a regular camera) An infographic created using a computer drawing tool such as Adobe Illustrator or AutoCAD

Term ID	digitalArt
Term name	Digital art
Term description	Media created by a human using digital tools
Examples	* A cartoon drawn by an artist into a digital tool using a digital pencil, a tablet and a drawing package such as Procreate or Affinity Designer * A scene from a film/movie created using Computer Graphic Imagery (CGI) * Electronic music composition using purely synthesised sounds

Term ID	virtualRecording
Term name	Virtual recording
Term description	Live recording of virtual event based on synthetic and optionally captured elements
Examples	* A recording of a computer-generated sequence, e.g. from a video game * A recording of a Zoom meeting

Term ID	compositeSynthetic
Term name	Composite including synthetic elements
Term description	Mix or composite of several elements, at least one of which is synthetic
Examples	* Movie production using a combination of live-action and CGI content, e.g. using Unreal engine to generate backgrounds * A capture of an augmented reality interaction with computer imagery superimposed on a camera video, e.g. someone playing Pokemon Go

Term ID	trainedAlgorithmicMedia
Term name	Trained algorithmic media
Term description	Digital media created algorithmically using a model derived from sampled content
Examples	* Image based on deep learning from a series of reference examples * A “speech-to-speech” generated audio or “deepfake” video using a combination of a real actor and an AI model * “Text-to-image” using a text input to feed an algorithm that creates a synthetic image

Term ID	algorithmicMedia
Term name	Algorithmic media
Term description	Media created purely by an algorithm not based on any sampled training data, e.g. an image created by software using a mathematical formula
Examples	* A purely computer-generated image such as a pattern of pixels generated mathematically e.g. a Mandelbrot set or fractal diagram * A purely computer-generated moving image such as a pattern of pixels generated mathematically

We propose that the following term, which exists in the current DigitalSourceType CV, be retired:

Term ID	RETIRE: softwareImage
Term name	Created by software
Term description	The digital image was created by computer software
Note	We propose that trainedAlgorithmicMedia or algorithmnicMedia be used instead of this term.

 

We welcome all feedback from across the industry to these proposed terms.

Please contact Brendan Quinn, IPTC Managing Director at mdirector@iptc.org use the IPTC Contact Us form to send your feedback.