At the 2020 IPTC Spring Meeting in May, the IPTC Standards Committee endorsed version 2.29 of our flagship news markup and syndication standard, NewsML-G2.

Version 2.29, including the 2.29 XML Schema, example documents and schema documentation, can be downloaded from https://iptc.org/std/NewsML-G2/2.29/ or from the NewsML-G2 GitHub repository at https://github.com/iptc/newsml-g2.

The NewsML-G2 Guidelines have been updated to reflect the latest changes in the schema. The NewsML-G2 Specification will be updated soon.

Summary of changes in NewsML-G2 2.29

<assert qcode="example-org:123">
    <name>Acme Financials</name>
    <organisationDetails>
        <affiliation role="example-affrole:memberOf" type="cpnat:organisation" qcode="example-fingroup:ABC"/>

<assert qcode="example-pers:22">
    <name>John Smith</name>
    <personDetails>
        <affiliation role="example-affrole:employedBy" type="cpnat:organisation" qcode="example-club:A"/>
        <affiliation role="example-affrole:supporterOf" type="cpnat:organisation" qcode="example-club:B"/>

For any questions regarding NewsML-G2, please post to the NewsML-G2 discussion forum.

On July 1st 2020, IPTC was invited to participate in an online workshop held by the Arab States Broadcasting Union (ASBU).

In a joint presentation, Brendan Quinn (IPTC Managing Director) and Robert Schmidt-Nia (Chair of IPTC and consultant with DATAGROUP Consulting Services) spoke on behalf of IPTC and Jürgen Grupp (data architect with the German public broadcaster SWR) spoke on behalf of the European Broadcasting Union.

The invitation was extended to IPTC and EBU because ASBU is looking at creating a common framework for sharing content between ASBU member broadcasters.

Jürgen Grupp started with an overview of why metadata is important in broadcasting and media organisations, and introduced the EBU’s high-level architecture for media, the EBU Class Conceptual Data Model (CCDM). and the EBUCore metadata set. Jürgen then gave examples of how CCDM and EBUCore are implemented by some European broadcasters.

Next, Brendan Quinn introduced IPTC and the IPTC News Architecture, the underlying logical model behind all of IPTC’s standards. We then took a deep dive into some video-related NewsML-G2 constructs like partMeta (used to describe metadata for parts of a video such as the rights and descriptive metadata for multiple time-based shots within a single video file) and contentSet (used to link to multiple renditions of the same video in different formats, resolutions or quality levels).

Then Robert Schmidt-Nia described some real-world examples of implementation of NewsML-G2 and the IPTC News Architecture at broadcasters and news agencies in Europe, in particular touching on the real-world issues of whether to “push” content or to create a “content API” that customers can use to select and download the content that they would like.

A common theme throughout our presentations was that the representation of the data in XML, RDF, JSON or some other format is relatively easy to change, but the important decision is what logical model to use and how to agree on the meaning (semantics) of terms and vocabularies.

A robust question and answer period touched on wide-ranging issues from the choices between XML, RDF and JSON, extending standardised models and vocabularies, and what decisions should be made to decide how to proceed.

This was one of the first meetings of ASBU on this topic and we look forward to assisting them further on their journey to metadata-based content sharing between their members.

This post is part of a series about the IPTC Spring Meeting 2019 in Lisbon, Portugal. See Day 2 writeup and the day 3 writeup.

Last week brought IPTC members together for our twice-yearly Face-to-Face Meeting to discuss news credibility, taxonomies and controlled vocabularies, updates in sports standards and much more!

This year’s IPTC Spring Meeting was in Lisbon, Portugal, and over 40 IPTC member delegates, member experts and invited guests gathered for three days to discuss all the latest developments in news and media technology.

On Monday, IPTC Chair and Director of Information Management for Associated Press Stuart Myles gave a great introduction and overview of what was to come in the meeting. After everyone introduced themselves, Stuart discussed some changes that the IPTC Board has been thinking about, including looking at updating the Mission and Vision of the organisation to reflect how we operate in 2019.

Then Robert Schmidt-Nia from dpa Deutsche Presse-Agentur introduced their C-POP project (in collaboration with STT and the Sanoma group in Finland) which follows on from the Performing Content we saw at the previous meeting in Toronto. It was interesting hearing about the agency’s shift in focus from a strict business-to-business model to a “B2B2C” model thinking about what consumers needed and how agencies could help publishers to deliver on the needs of readers and subscribers, ideally using feedback from publishers to agencies on how well their content is performing according to real metrics like loyalty and subscription revenue. IPTC will be involved in the C-POP project so you can expect to hear more about this in the future.

On the same topic, Andy Read from BBC gave an overview of the “Telescope” internal measurement tool, showing how BBC staff can view in real time how their content is being consumed by region, topic or device.

James Logan from the BBC and Brendan Quinn of IPTC gave an overview of IPTC’s work with news trust and credibility projects The Trust Project and the Journalism Trust Initiative. We decided at the Autumn 2018 Meeting that IPTC wouldn’t create its own standard around news credibility, disinformation and “fake news”, but that we would work with existing groups and help them to incorporate their standards in IPTC’s work. With The Trust Project, that has been going well, and we are almost ready to publish some best practices on implementing the Trust Project’s Trust Indicators in NewsML-G2 content. Trust Project indicators are already used in schema.org markup by over 120 news providers so it’s great to see such strong uptake.

Separately we have been working with Reporters Sans Frontières’ Journalism Trust Initiative which is at an earlier stage and is looking at documenting general standards for trustworthy and ethical journalism. IPTC is part of the JTI’s Technical Task Force which is working with the drafting teams on making their statements specific enough to be answered with data and indexed by machines. Hopefully it will end up with similar indicators to the Trust Project indicators 

With both news credibility projects, some questions still need to be addressed, such as assessing the credibility of claims (when a news organisation says they are trustworthy, how can you trust them!), and how these trust indicators work in a multi-provider workflow: if a news agency sends some content to a publisher who then merges it with original reportage, who determines the trust indicators that are attached to the final story? There is definitely a lot more work to do!

On the same topic, Dave Compton of Refinitiv gave an update on how the News Architecture Working Group has been looking at the Trust Project’s Trust Indicators and working them into NewsML-G2. As far as we have seen so far, no updates to the NewsML-G2 standard are necessary to support the new work. Martin Vertel from dpa showed us the API he created to give dpa’s clients access to Trust Project indicators for dpa stories. Building it with a browser-based JavaScript module opens up some interesting possibilities.

Joaquim Carreira from local agency Lusa showed us the “Combate Às Fake News” project focussing on media literacy and helping readers to know what to look for, including the idea of a “nutrition label” for news content looking at criteria such as factuality, readability and use of emotional language.

The day was rounded off with Johan Lindgren of Swedish agency TT presenting the recent work of IPTC’s Sports Content Working Group. The group has recently been tidying up the spec and incorporating suggestions for changes, plus looking at eSports and Chess as two non-traditional sports that are both seeing an increase in interest – in the case of eSports, it is becoming a huge industry. Our tests showed that in simple cases eSports results can be addressed with existing SportsML 3 structures, but to handle more detailed play-by-play results we may need to at least introduce a new controlled vocabulary. Please let us know if you would like to implement SportsML for eSports!

Johan also presented the draft of SportsML 3.1 to be voted on by the IPTC Standards Committee.

Stay tuned for an update on Days 2 and 3!

We were proud to be involved at last week’s Metadata Exchange for News interoperability demo organised by DPP (formerly known as the Digital Production Partnership).

DPP’s “Metadata Exchange for News” is an industry initiative aimed at making the news production process easier.

The DPP team looked around for existing standards on which to base their work, and when they found IPTC’s NewsML-G2, they realised that it exactly matched their requirements. NewsML-G2’s generic PlanningItem and NewsItem structure meant that it could easily be used to manage news production workflows with no customisation required.

We were treated to a demo of a full news production workflow in the DPP’s offices at ITV in London on February 6th.

A full news production workflow

As you can see from the diagram, the workflow involves these steps:

• An editor creates a planning record for a news item using Wolftech’s planning system, describing metadata for the planned story
• The system sends the planning item as NewsML-G2 to Sony’s XDCAM Air system which converts it to Sony’s proprietary planning metadata and sends it directly to a camera
• XDCAM Air retrieves the footage from the camera, links it to the planning metadata using the NewsML-G2 IDs, back into XDCAM Air which is then retrieved by some simple custom web services
• The web services send NewsML-G2 NewsItem metadata along with the MP4 video file to Ooyala’s Flex Media Platform via an Amazon Web Services S3 bucket
• Ooyala Flex Media Platform sends the media and metadata to the platforms that require it, in this case the Reuters Connect video browsing and distribution platform.

The NewsML-G2 integrations were built for the demo but the idea is that they will soon become standard features of the products involved. All parties reported that implementing NewsML-G2 was fast and fairly painless!

Thanks to all involved and special thanks to Abdul Hakim of DPP for leading the project and organising the demo day.

Look out for an IPTC Webinar on this topic soon!

An updated version 2.25 of NewsML-G2 is available as Developer Release

•  XML Schemas and the corresponding documentation are updated
• the Structure Matrix Excel sheet is updated

Packages of version 2.25 files can be downloaded:

• All XML Schemas plus Structure Matrix from https://www.iptc.org/std/NewsML-G2/NewsML-G2_2.25.zip 
• The same without XML Schema documentation in HTML: https://www.iptc.org/std/NewsML-G2/NewsML-G2_2.25-noXMLdocu.zip    
• New: in the newsml-g2 repository on GitHub: https://github.com/iptc/newsml-g2

All changes of version 2.25 can be found on that page: http://dev.iptc.org/G2-Approved-Changes

An important decision was taken: the Core Conformance Level will not be developed any further as all recent Change Requests were in fact aiming at features of the Power Conformance Level,  changes of the Core Level were only a side effect.

The Core Conformance Level specifications of version 2.24 will stay available and valid, find them at http://dev.iptc.org/G2-Standards#CCLspecs  

For the new version 2.23 of NewsML-G2, the specification part of the Annual Release is now available and can be downloaded from the NewsML-G2 Release Section of the IPTC Developer Site.

The NewsML-G2 standard provides state-of-the-art XML format metadata to combine rich functionality, ease of use, compactness and compatibility with the Semantic Web. It is a single format for exchanging text, images, video, audio news and event or sports data – and packages thereof.

This specification part of the Annual Release of NewsML-G2 v2.23 includes the XML Schemas and the Structure Matrix document. The updated Quick Start Guides, Implementation Guidelines and full specifications will be released in October. This is part of an ongoing incremental development of NewsML-G2, as providers expand their content use-cases.

Three changes/improvements in version 2.23 are:

1. It allows the addition of further Rights Expression properties <rightsInfo> and now covers these Rights cases:
   –       Allows embedding or referencing a rights expression
   –       Allows use of XML or JSON as format for embedding
2.  It allows address properties (locality, area, country, etc.) to include a World Region.
3.  It allows the addition of facets to the Item Class property, this provides for more flexibility.

Please visit the IPTC Standards Page for a full list of the available IPTC standards.

When IPTC member Sourcefabric presents their flagship product Superdesk – an extensible end-to-end news production, curation and distribution platform – they always recognize the importance of IPTC standard NewsML-G2 as its backbone.

As Sourcefabric CTO Holman Romero explained at the IPTC Summer Meeting in Stockholm (13 – 15 June 2016), Superdesk was built on the principles of the News Architecture part of the NewsML-G2 specification by the IPTC. This is because Superdesk is not a traditional Web CMS, but rather a platform developed from the ground up for journalists to manage the numerous processes of a newsroom.

“Superdesk is more than a news management tool built for journalists by journalists, for creation, archiving, distribution, workflow structure, and editorial communications,” Romero said. “We at Sourcefabric also see it as the cornerstone of the new common open-source code base for quality, professional journalism.”

NewsML-G2 is a blueprint that provides all the concepts and business logic for a news architecture framework. It also standardises the handling of metadata that ultimately enables all types of content to be linked, searched, and understood by end users. NewsML-G2 metadata properties are designed to comply with RDF, the data model of the Semantic Web, enabling the development of new applications and opportunities for news organisations in evolving digital markets.

There were several important factors that led Sourcefabric, Europe’s largest developer of open source tools for news media, to the decision to use NewsML-G2:

• IPTC has established credibility as a consortium of the world’s leading news agencies and publishers. Additionally, NewsML-G2 has been adopted by some of the world’s major news agencies as the standard de facto for news distribution. “Why reinvent the wheel?” Romero said. “IPTC’s standards are based on years of experience of top news industry professionals.”

• NewsML-G2 met the requirements of Sourcefabric’s content model: granularity, structured data, flexibility, and reusability.

• NewsML-G2 met the requirements for Sourcefabric’s design principals:
  • Every piece of content is a News Item.
  • Content types are text, image, video, audio.
  • Content profiles support the creation of story profiles and templates.
  • It can format items for content packages and highlights.

• Content can be created once and used in many places. Sourcefabric refers to this as the COPE model: Create Once, Publish Everywhere. This structure enables and frees the content to be used seamlessly and automatically across multiple channels and devices, and in a variety of previously impossible contexts.

Sourcefabric also stresses the importance of metadata, the building blocks of “structured journalism.” As explained by Romero in a recent blog post: “The foundation of structured journalism is built on the ability to access and locate enormous amounts of data from all over the web and from within the system itself (i.e. content from previous articles). Without providing valuable metadata for each of your stories and subsequent pieces of visual collateral, finding key information located inside of them becomes infinitely more difficult.”

News organizations that use Superdesk include the Australian news agency AAP and Norweigian News Agency NTB. Other IPTC standards supported by the platform are NewsML 1, ninjs, NITF, Subject Codes, IPTC 7901.

Source code repositories are publicly available in Github: https://github.com/superdesk

About Sourcefabric: Sourcefabric’s mission is to make professional-grade technology available to all who believe that quality independent journalism has a fundamental role to play in any healthy society. They generate revenue by IT services – managed hosting, SaaS, custom development, integration into existing workflows – as well as project-by-project funding, grants, donations.