Preview. Built from the published NewsML-G2 documents by tools/prosedoc. This layout is a harness for reviewing structure — it is not the published site.
18. Designing a NewsML-G2 feed from scratch
18.1. Introduction
When designing a NewsML-G2 feed, some fundamental questions need to be asked, such as:
-
What is the provider’s business value proposition and how must this be supported by the NewsML-G2 implementation?
-
What are the needs and capabilities of customers? It may be the customer who is driving the move the NewsML-G2, but customers will often have issues that need to be resolved and which require sympathetic handling.
-
What limitations – if any – do the provider’s existing systems and processes impose on the NewsML-G2 project?
This chapter gives general guidance on the issues an implementer of a feed might encounter, and where necessary points to more detailed information on specific topics described elsewhere in the Guidelines.
As a general principle, the IPTC recommends that providers strive for inter-operability, so where possible, stick to using the recommended "mainstream" properties as identified by the Property Usage sheet below. The sheet lists the NewsML-G2 properties which are most widely used by currently existing NewsML-G2 feeds. As there may be more than one way to implement a feature; in such cases the IPTC recommends choosing the most widely-used property as indicated by the sheet.
Also consider using existing IPTC NewsCodes Schemes for property values where appropriate, in preference to creating proprietary schemes that accomplish a similar purpose. A table showing IPTC Controlled Vocabulary Usage is shown below.
18.2. Content-driven design
The design of a NewsML-G2 feed will depend on the content type being conveyed, since each type will have specific metadata and processing needs.
For all content types, note the basic NewsML-G2 rule: a single logical item of content per News Item, which can be represented by more than one technical representation. For multiple pieces of content in a named relationship, use the Package Item, and for sending one or more NewsML-G2 Items in a transmission workflow, the News Message may be used.
18.2.1. Content choices
Text
The most commonly-used NewsML-G2 container for text is <inlineXML> wrapping XHMTL content, although NITF is also used. Note that any well-formed XML may be conveyed in <inlineXML> so this can be aligned to a more specific business purpose such as XBRL (Extended Business Reporting Language).
It is also possible to convey a text-based object as binary content, for example as a PDF, using <remoteContent> or as plain text using <inlineData>.
When converting from legacy text formats such as IPTC7901, the initial advantages of NewsML-G2 include the ability to separate metadata from the content, and the use of globally-unique controlled values to classify content.
Going further, there are more specialised structures available for implementers to provide financial information, express quantitative data, and give precise geographic details in a machine-readable form.
Pictures
As binary content, pictures (and graphics) are conveyed using the <remoteContent> wrapper. The Pictures and Graphics chapter gives an overview of NewsML-G2 features and the relationship between NewsML-G2 metadata and embedded metadata such as the "IPTC Core" fields in JPEG/TIFF files. There is also a chapter on Mapping Embedded Photo Metadata to NewsML-G2 that provides more detailed information on this issue.
Video
The first consideration for video is the business requirement, since this drives the technical design in one of two directions:
-
Simple one-part video – probably delivered in multiple formats to suit different technical platforms
-
Multi-part video delivered as a reference to a single file (and potentially multiple technical renditions) that requires each part or shot to have its own administrative and descriptive metadata.
NewsML-G2 serves both of these content models, which are described in the Video chapter.
18.3. The Basics
The Chapter on NewsML-G2 Basics has a description of the mandatory and commonly-used properties.
18.4. Identification and version control
Choose a scheme for generating the @guid of NewsML-G2 Items. The IPTC has registered a namespace urn "newsml" and a specification for this purpose (see Item Identifier) but other schemes such as TAG URI are available.
Decide how Item version will be generated: it must start at "1" and ideally should increment by "1" each time a new version is published, although a new version may have any number that is greater than the previous version number. Providers may also use <link> to inform receivers of the previous version and provide a location or id for it (see Using Links for more details on this)
18.5. Conformance
Since version 2.24, NewsML-G2 supports only one schema (the "All-Power" schema), therefore the conformance attribute must be present with a value of "power".
18.6. Validation
When designing and testing the feed, validation against the IPTC scheme files will be essential. The IPTC recommends that validation should not be required in production: it can be resource intensive and introduces a potential point of failure. If a provider chooses to validate production documents, they must NOT validate against the schema files hosted by the IPTC.
18.7. Catalogs and Controlled Vocabularies
Even after following the advice about using NewsCodes (above) it is almost certain that some provider specific schemes will be needed. See the section on Controlled Vocabularies and QCodes for detailed information on how to create and maintain CVs and Catalogs.
18.8. Rights Information
See Chapter on Rights Metadata to go beyond the basics of expressing rights information.
18.9. Management Metadata
18.9.1. Timestamps
There are four timestamps available in NewsML-G2 documents, two are properties of Item Metadata and relate to management of the Item; two are properties of Content Metadata and are part of the Administrative properties of the Item’s content. The four properties are:
-
itemMeta/firstCreated
-
itemMeta/versionCreated
-
contentMeta/contentCreated
-
contentMeta/contentModified
NewsML-G2 timestamps in Item Metadata express the full date and time of the creation or update of the Item. Content Metadata timestamps may be truncated from the right because the precise date and time of the creation of content (for example an historic photograph) may be unknown. Note that any timestamp that contains the time MUST contain a time offset or Z for UTC, for example the format YYYY-MM-DDThh:mm:ss(+-)hh:mm:ss, or YYYY-MM-DDThh:mm:ssZ.
The single mandatory timestamp for a NewsML-G2 document is <versionCreated>, but the following rules MUST be followed when any other timestamp is present.
Data Value rules
-
If <contentCreated> is used, it MUST NOT be later than <versionCreated>.
-
If <contentModified> is present, <contentCreated> SHOULD also be present and MUST be earlier than <contentModified>.
-
If <contentModified> is present, it MUST NOT be later than <versionCreated>.
-
If <firstCreated> is present, it MUST NOT be later than <versionCreated>.
Data Processing rules
-
The recipient processor MUST first check if a <contentModified> element is present.
-
If not it MUST check if a <contentCreated> element is present.
-
If not it SHOULD assume that the content was created at the time indicated by <versionCreated> element in itemMeta.
For a full description of the timestamps and processing rules, see the NewsML-G2 Specification, which may be downloaded. This can be downloaded by visiting newsml-g2.org/spec.
A number of existing implementations use only itemMeta/versionCreated for text, but for picture, graphics and video, it is generally accepted that contentMeta/contentCreated and contentMeta/contentModified should be used as, for example, the moment when a picture is taken is considered significant.
18.9.2. Publishing Status/Embargoed
Both properties of <itemMeta> have "implied" values, in that there are default values that are assumed if the actual properties are absent. Publishing Status defaults to "usable" and Embargoed to "none". There are separate topics for these properties in the chapter on Generic Processes and Conventions.
18.9.3. Links
Links identify and can also locate NewsML-G2 Items and other non-G2 resources that may be used to supplement the understanding of an Item’s content. The default relationship to the target Item is "See Also" but other relationships such as "Translated From" and "Derived From" are defined in the IPTC Item Relationship NewsCodes.
18.10. Property Usage
The following property usage table was compiled from a survey of IPTC members.
| Property name | Text | Photo / Graphic | Video | News Package |
|---|---|---|---|---|
News Message |
||||
header |
High |
High |
High (in package) |
Medium |
sent |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
sender |
High |
Medium+ |
High |
Medium |
transmitId |
Medium |
Medium+ |
High |
High |
priority |
High |
Medium |
Medium |
Medium |
origin |
Medium |
Medium |
No |
Medium |
timestamp |
Medium |
Low+ |
Medium |
No |
destination |
High - with channel |
Medium+ - with channel |
Medium |
Medium |
channel |
High - with destination |
Medium+ - with destination |
High |
Medium |
News Item |
||||
@guid |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
@version |
Always |
Always |
Always |
Always |
Catalogs/Hop History |
||||
catalogRef |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
catalog |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
hopHistory |
Low |
Low |
No |
No |
rightsInfo |
||||
accountable |
Low |
Low |
No |
Medium |
copyrightHolder |
High |
High |
High |
High |
copyrightNotice |
High |
High |
High |
Medium |
usageTerms |
Medium |
Low+ |
Medium |
Low+ |
itemMeta |
||||
itemClass |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
provider |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
versionCreated |
Mandatory |
Mandatory |
Mandatory |
Mandatory |
firstCreated |
Medium |
High |
High |
Medium+ |
embargoed |
Always |
Always |
Always |
Always |
pubStatus |
High |
High |
High |
High |
role |
Medium |
Low+ |
High |
Medium |
fileName |
Medium |
Medium |
Medium |
Medium |
generator |
Medium |
Medium |
High |
High |
profile |
Medium |
Low+ |
High |
Medium+ |
service |
Medium |
Low+ |
High |
Medium |
title |
Medium |
Low |
No |
Medium |
edNote |
High |
Low+ |
Low+ |
No |
memberOf |
Medium |
Low |
No |
No |
instanceOf |
Medium |
Low |
No |
No |
signal |
High |
Medium+ |
High |
Medium |
altRep |
No |
No |
No |
No |
deliverableOf |
No |
No |
No |
No |
hash |
No |
No |
No |
No |
link |
Medium |
Low |
Medium |
Low |
contentMeta |
||||
icon |
No |
Low |
Low |
No |
urgency |
High |
Medium+ |
High |
Medium |
contentCreated |
Medium |
Medium+ |
Medium |
Medium |
contentModified |
Medium |
Medium |
Low |
Low+ |
located |
High |
Low |
High |
Low+ |
infoSource |
High |
High |
High |
Medium |
creator |
High |
High |
Medium |
Medium |
contributor |
Medium |
Medium+ |
Medium |
Low+ |
audience |
Medium |
No |
No |
Low+ |
exclAudience |
Low |
Low |
No |
No |
altId |
High |
High |
High |
Medium+ |
rating |
No |
Low |
No |
No |
userInteraction |
No |
No |
No |
No |
language |
Medium |
Medium |
High |
Medium+ |
genre |
Medium+ |
Low |
Medium |
No |
keyword |
Medium+ |
Low |
No |
Medium |
subject |
High |
High |
High |
High |
bag |
No |
No |
No |
No |
slugline |
High |
Medium+ |
High |
High |
headline |
High |
High |
High |
High |
dateline |
Medium+ |
Low |
Low |
Medium+ |
by |
Medium |
Medium+ |
Medium |
No |
creditline |
Medium |
High |
No |
Medium |
description |
Medium |
High |
High |
Medium |
partMeta |
No |
Low |
Low |
No |
Helper structures |
||||
assert |
~ |
~ |
~ |
~ |
inlineRef |
~ |
~ |
~ |
~ |
derivedFrom |
Low |
No |
No |
No |
contentSet |
||||
inlineXML |
High |
n/a |
n/a |
n/a |
inlineData |
Low |
n/a |
n/a |
n/a |
remoteContent |
No |
High |
n/a |
n/a |
groupSet |
||||
main/root Group |
n/a |
n/a |
High |
High |
subgroups |
n/a |
n/a |
High |
High |
itemRef |
n/a |
n/a |
High |
High |
Key |
||||
Mandatory |
Mandatory per the NewsML-G2 specification |
|||
Always |
Explicitly defined or implied by absence |
|||
High |
Expected to be present |
|||
Medium |
May be present |
|||
Low |
Unlikely to be present (currently) |
|||
No |
Not present (currently) |
|||
~ |
Technical helpers: These provide richer information about another explicit property |
|||
n/a |
Not applicable |
|||
red |
Mandatory mark-up |
|||
18.11. IPTC Controlled Vocabulary Usage
The table below compares the usage of IPTC CVs (where available) and proprietary CVs for commonly-used and mandatory properties, compiled from a survey of IPTC members.
| Property name | IPTC CV(s) | Scheme Alias | Provider CVs |
|---|---|---|---|
News Message header |
|||
destination |
n/a |
~ |
Medium |
News Item |
|||
catalog info |
~ |
~ |
|
rightsInfo |
~ |
No |
|
itemMeta |
|||
itemClass |
|
~ |
|
provider |
|
No |
|
pubStatus |
|
~ |
|
role |
~ |
Medium |
|
generator |
~ |
Medium |
|
service |
~ |
High |
|
edNote |
~ |
Medium |
|
memberOf |
~ |
High |
|
instanceOf |
~ |
High |
|
signal |
|
High |
|
link |
|
High |
|
contentMeta |
~ |
||
located |
~ |
High |
|
infoSource |
|
Medium |
|
creator |
~ |
High |
|
contributor |
~ |
High |
|
audience |
~ |
Medium |
|
exclAudience |
~ |
High |
|
altId |
~ |
Medium |
|
genre |
|
High |
|
subject |
|
~ |
|
|
~ |
||
|
~ |
||
|
~ |
||
|
~ |
||
description |
|
Medium |
|
partMeta |
~ |
Medium |
|
tech helpers |
|||
assert |
~ |
High |
|
contentSet |
|||
remoteContent |
|
High |
|
groupSet |
|||
itemRef |
~ |
High |
| Key | |
|---|---|
|
Mandatory per specification |
High |
Expected to be present |
Medium |
May be present |
Low |
Rarely present |
No |
Available but not used |
~ |
Unavailable or not applicable |