NewsML-G2 2.35

Preview. Built from the published NewsML-G2 documents by tools/prosedoc. This layout is a harness for reviewing structure — it is not the published site.

18. Designing a NewsML-G2 feed from scratch

18.1. Introduction

When designing a NewsML-G2 feed, some fundamental questions need to be asked, such as:

  • What is the provider’s business value proposition and how must this be supported by the NewsML-G2 implementation?

  • What are the needs and capabilities of customers? It may be the customer who is driving the move the NewsML-G2, but customers will often have issues that need to be resolved and which require sympathetic handling.

  • What limitations – if any – do the provider’s existing systems and processes impose on the NewsML-G2 project?

This chapter gives general guidance on the issues an implementer of a feed might encounter, and where necessary points to more detailed information on specific topics described elsewhere in the Guidelines.

As a general principle, the IPTC recommends that providers strive for inter-operability, so where possible, stick to using the recommended "mainstream" properties as identified by the Property Usage sheet below. The sheet lists the NewsML-G2 properties which are most widely used by currently existing NewsML-G2 feeds. As there may be more than one way to implement a feature; in such cases the IPTC recommends choosing the most widely-used property as indicated by the sheet.

Also consider using existing IPTC NewsCodes Schemes for property values where appropriate, in preference to creating proprietary schemes that accomplish a similar purpose. A table showing IPTC Controlled Vocabulary Usage is shown below.

18.2. Content-driven design

The design of a NewsML-G2 feed will depend on the content type being conveyed, since each type will have specific metadata and processing needs.

For all content types, note the basic NewsML-G2 rule: a single logical item of content per News Item, which can be represented by more than one technical representation. For multiple pieces of content in a named relationship, use the Package Item, and for sending one or more NewsML-G2 Items in a transmission workflow, the News Message may be used.

18.2.1. Content choices

Text

The most commonly-used NewsML-G2 container for text is <inlineXML> wrapping XHMTL content, although NITF is also used. Note that any well-formed XML may be conveyed in <inlineXML> so this can be aligned to a more specific business purpose such as XBRL (Extended Business Reporting Language).

It is also possible to convey a text-based object as binary content, for example as a PDF, using <remoteContent> or as plain text using <inlineData>.

When converting from legacy text formats such as IPTC7901, the initial advantages of NewsML-G2 include the ability to separate metadata from the content, and the use of globally-unique controlled values to classify content.

Going further, there are more specialised structures available for implementers to provide financial information, express quantitative data, and give precise geographic details in a machine-readable form.

Pictures

As binary content, pictures (and graphics) are conveyed using the <remoteContent> wrapper. The Pictures and Graphics chapter gives an overview of NewsML-G2 features and the relationship between NewsML-G2 metadata and embedded metadata such as the "IPTC Core" fields in JPEG/TIFF files. There is also a chapter on Mapping Embedded Photo Metadata to NewsML-G2 that provides more detailed information on this issue.

Video

The first consideration for video is the business requirement, since this drives the technical design in one of two directions:

  • Simple one-part video – probably delivered in multiple formats to suit different technical platforms

  • Multi-part video delivered as a reference to a single file (and potentially multiple technical renditions) that requires each part or shot to have its own administrative and descriptive metadata.

NewsML-G2 serves both of these content models, which are described in the Video chapter.

18.3. The Basics

The Chapter on NewsML-G2 Basics has a description of the mandatory and commonly-used properties.

18.4. Identification and version control

Choose a scheme for generating the @guid of NewsML-G2 Items. The IPTC has registered a namespace urn "newsml" and a specification for this purpose (see Item Identifier) but other schemes such as TAG URI are available.

Decide how Item version will be generated: it must start at "1" and ideally should increment by "1" each time a new version is published, although a new version may have any number that is greater than the previous version number. Providers may also use <link> to inform receivers of the previous version and provide a location or id for it (see Using Links for more details on this)

18.5. Conformance

Since version 2.24, NewsML-G2 supports only one schema (the "All-Power" schema), therefore the conformance attribute must be present with a value of "power".

18.6. Validation

When designing and testing the feed, validation against the IPTC scheme files will be essential. The IPTC recommends that validation should not be required in production: it can be resource intensive and introduces a potential point of failure. If a provider chooses to validate production documents, they must NOT validate against the schema files hosted by the IPTC.

18.7. Catalogs and Controlled Vocabularies

Even after following the advice about using NewsCodes (above) it is almost certain that some provider specific schemes will be needed. See the section on Controlled Vocabularies and QCodes for detailed information on how to create and maintain CVs and Catalogs.

18.8. Rights Information

See Chapter on Rights Metadata to go beyond the basics of expressing rights information.

18.9. Management Metadata

18.9.1. Timestamps

There are four timestamps available in NewsML-G2 documents, two are properties of Item Metadata and relate to management of the Item; two are properties of Content Metadata and are part of the Administrative properties of the Item’s content. The four properties are:

  • itemMeta/firstCreated

  • itemMeta/versionCreated

  • contentMeta/contentCreated

  • contentMeta/contentModified

NewsML-G2 timestamps in Item Metadata express the full date and time of the creation or update of the Item. Content Metadata timestamps may be truncated from the right because the precise date and time of the creation of content (for example an historic photograph) may be unknown. Note that any timestamp that contains the time MUST contain a time offset or Z for UTC, for example the format YYYY-MM-DDThh:mm:ss(+-)hh:mm:ss, or YYYY-MM-DDThh:mm:ssZ.

The single mandatory timestamp for a NewsML-G2 document is <versionCreated>, but the following rules MUST be followed when any other timestamp is present.

Data Value rules
  1. If <contentCreated> is used, it MUST NOT be later than <versionCreated>.

  2. If <contentModified> is present, <contentCreated> SHOULD also be present and MUST be earlier than <contentModified>.

  3. If <contentModified> is present, it MUST NOT be later than <versionCreated>.

  4. If <firstCreated> is present, it MUST NOT be later than <versionCreated>.

Data Processing rules
  1. The recipient processor MUST first check if a <contentModified> element is present.

  2. If not it MUST check if a <contentCreated> element is present.

  3. If not it SHOULD assume that the content was created at the time indicated by <versionCreated> element in itemMeta.

For a full description of the timestamps and processing rules, see the NewsML-G2 Specification, which may be downloaded. This can be downloaded by visiting newsml-g2.org/spec.

A number of existing implementations use only itemMeta/versionCreated for text, but for picture, graphics and video, it is generally accepted that contentMeta/contentCreated and contentMeta/contentModified should be used as, for example, the moment when a picture is taken is considered significant.

18.9.2. Publishing Status/Embargoed

Both properties of <itemMeta> have "implied" values, in that there are default values that are assumed if the actual properties are absent. Publishing Status defaults to "usable" and Embargoed to "none". There are separate topics for these properties in the chapter on Generic Processes and Conventions.

Links identify and can also locate NewsML-G2 Items and other non-G2 resources that may be used to supplement the understanding of an Item’s content. The default relationship to the target Item is "See Also" but other relationships such as "Translated From" and "Derived From" are defined in the IPTC Item Relationship NewsCodes.

18.10. Property Usage

The following property usage table was compiled from a survey of IPTC members.

Property name Text Photo / Graphic Video News Package

News Message

header

High

High

High (in package)

Medium

sent

Mandatory

Mandatory

Mandatory

Mandatory

sender

High

Medium+

High

Medium

transmitId

Medium

Medium+

High

High

priority

High

Medium

Medium

Medium

origin

Medium

Medium

No

Medium

timestamp

Medium

Low+

Medium

No

destination

High - with channel

Medium+ - with channel

Medium

Medium

channel

High - with destination

Medium+ - with destination

High

Medium

News Item

@guid

Mandatory

Mandatory

Mandatory

Mandatory

@version

Always

Always

Always

Always

Catalogs/Hop History

catalogRef

Mandatory

Mandatory

Mandatory

Mandatory

catalog

Mandatory

Mandatory

Mandatory

Mandatory

hopHistory

Low

Low

No

No

rightsInfo

accountable

Low

Low

No

Medium

copyrightHolder

High

High

High

High

copyrightNotice

High

High

High

Medium

usageTerms

Medium

Low+

Medium

Low+

itemMeta

itemClass

Mandatory

Mandatory

Mandatory

Mandatory

provider

Mandatory

Mandatory

Mandatory

Mandatory

versionCreated

Mandatory

Mandatory

Mandatory

Mandatory

firstCreated

Medium

High

High

Medium+

embargoed

Always

Always

Always

Always

pubStatus

High

High

High

High

role

Medium

Low+

High

Medium

fileName

Medium

Medium

Medium

Medium

generator

Medium

Medium

High

High

profile

Medium

Low+

High

Medium+

service

Medium

Low+

High

Medium

title

Medium

Low

No

Medium

edNote

High

Low+

Low+

No

memberOf

Medium

Low

No

No

instanceOf

Medium

Low

No

No

signal

High

Medium+

High

Medium

altRep

No

No

No

No

deliverableOf

No

No

No

No

hash

No

No

No

No

link

Medium

Low

Medium

Low

contentMeta

icon

No

Low

Low

No

urgency

High

Medium+

High

Medium

contentCreated

Medium

Medium+

Medium

Medium

contentModified

Medium

Medium

Low

Low+

located

High

Low

High

Low+

infoSource

High

High

High

Medium

creator

High

High

Medium

Medium

contributor

Medium

Medium+

Medium

Low+

audience

Medium

No

No

Low+

exclAudience

Low

Low

No

No

altId

High

High

High

Medium+

rating

No

Low

No

No

userInteraction

No

No

No

No

language

Medium

Medium

High

Medium+

genre

Medium+

Low

Medium

No

keyword

Medium+

Low

No

Medium

subject

High

High

High

High

bag

No

No

No

No

slugline

High

Medium+

High

High

headline

High

High

High

High

dateline

Medium+

Low

Low

Medium+

by

Medium

Medium+

Medium

No

creditline

Medium

High

No

Medium

description

Medium

High

High

Medium

partMeta

No

Low

Low

No

Helper structures

assert

~

~

~

~

inlineRef

~

~

~

~

derivedFrom

Low

No

No

No

contentSet

inlineXML

High

n/a

n/a

n/a

inlineData

Low

n/a

n/a

n/a

remoteContent

No

High

n/a

n/a

groupSet

main/root Group

n/a

n/a

High

High

subgroups

n/a

n/a

High

High

itemRef

n/a

n/a

High

High

Key

Mandatory

Mandatory per the NewsML-G2 specification

Always

Explicitly defined or implied by absence

High

Expected to be present

Medium

May be present

Low

Unlikely to be present (currently)

No

Not present (currently)

~

Technical helpers: These provide richer information about another explicit property

n/a

Not applicable

red

Mandatory mark-up

18.11. IPTC Controlled Vocabulary Usage

The table below compares the usage of IPTC CVs (where available) and proprietary CVs for commonly-used and mandatory properties, compiled from a survey of IPTC members.

Property name IPTC CV(s) Scheme Alias Provider CVs

News Message header

destination

n/a

~

Medium

News Item

catalog info

~

~

rightsInfo

~

No

itemMeta

itemClass

http://cv.iptc.org/newscodes/ninature

ninat

~

provider

http://cv.iptc.org/newscodes/newsprovider

nprov

No

pubStatus

http://cv.iptc.org/newscodes/pubstatusg2

stat

~

role

~

Medium

generator

~

Medium

service

~

High

edNote

~

Medium

memberOf

~

High

instanceOf

~

High

signal

http://cv.iptc.org/newscodes/signal

sig

High

link

http://cv.iptc.org/newscodes/itemrelation

irel

High

contentMeta

~

located

~

High

infoSource

http://cv.iptc.org/newscodes/contentprodpartyrole

cpprole

Medium

creator

~

High

contributor

~

High

audience

~

Medium

exclAudience

~

High

altId

~

Medium

genre

http://cv.iptc.org/newscodes/genre

genre

High

subject

http://cv.iptc.org/newscodes/mediatopic

medtop

~

http://cv.iptc.org/newscodes/subjectcode

subj

~

http://cv.iptc.org/newscodes/cpnature

cpnat

~

http://cv.iptc.org/newscodes/howextracted

howextr

~

http://cv.iptc.org/newscodes/whypresent

why

~

description

http://cv.iptc.org/newscodes/descriptionrole

drol

Medium

partMeta

~

Medium

tech helpers

assert

~

High

contentSet

remoteContent

http://cv.iptc.org/newscodes/rendition

rnd

High

groupSet

itemRef

~

High

Key

Red

Mandatory per specification

High

Expected to be present

Medium

May be present

Low

Rarely present

No

Available but not used

~

Unavailable or not applicable