3 days ago
Standards For Data Provenance And Digital Preservation
securing data provenance
Long term digital preservation of data and other digital content faces several challenges. The first is degradation of the recorded information due to physical damage over time or to thermally driven erasure of the data. The second hurdle is obsolescence of the technology (hardware and software) needed to read back the recorded information. The third hurdle is the widespread destruction or alteration of information (intentional or not), which has occurred several times in human history, and which is likely to occur again.
The first and second challenges are generally met by making multiple copies of data, ideally in multiple places, or migrating the data to new media when the old media is close to becoming obsolete or close to losing data. The last obstacle requires new methodologies around determining data provenance and whether that data has been altered. Organizations such as the IEEE and the Society of Motion Pictures and Television Engineers, or SMPTE are starting efforts to develop methodologies and standards on digital data provenance.
SMPTE just announced that it is creating a Content Provenance and Authenticity, CPA, in Media Study Group. This effort will assess how current content provenance and authenticity technologies affect media production and distribution. An important element of this will be about the carriage of content provenance information in MXF files, due to an urgent industry need.
MXF stands for Material Exchange Format. It is a container format primarily used in professional broadcasting and video production for exchanging audio-video materials. MXF files can contain video, audio and metadata and they support various compression schemes and data types. MXF is an open standard designed to simplify file-based media workflows and interoperability between different systems.
The scope of the project will be to identify content provenance and authenticity technologies, areas of work and activities in other professional media organizations, and make recommendations where SMPTE can update existing or create new standards to support the flow of content provenance and authenticity information. The group will also gather use cases and requirements, and summarize those findings and recommendations in one or more study group reports.
The group includes representatives of the SMPTE Standards Community from Ross Video, SONY, Adobe, The European Broadcasting Union, and Metaglue.
The IEEE standards association is also engaged in initial efforts in creating a study group around the creation of potential standards on Global Data Veracity as well. I am hoping that all the groups working on methods for data provenance can work together to make compatible standards and best practices.
There are both immediate and long-term needs for modern methods for digital data provenance. History has shown that knowledge has been and can be intentionally suppressed or otherwise made hard to find and that many older books and other types of historical documents simply have been lost over time. In addition, today digital data can be subject to tampering and modification for criminal, religious or political reasons. We need approaches to preserve digital data in its original form over long periods of time and to make that data available to future generations as well as to preserve the integrity of current media workflows.
SMPTE announced that it has created a Content Provenance and Authenticity in Media Study Group. The provenance of digital content needs standards and best practices to preserve workflow integrity and enable long term digital data preservation.
