In the file-based workflows of digital cinematography today the secure and reliable transfer of media files is one of the essential activities to be performed in a production. The Media Hash List specification ensures that the complete source media is transferred without any alterations. This article explains the details of a Media Hash List, why you should care, and how you can implement it as part of your workflow
TABLE OF CONTENTS
- What is a Media Hash List?
- Why should you care?
- How Does it Work?
- Does Limecraft Support Media Hash Lists?
- Recommended reading
What is a Media Hash List?
A Media Hash List (MHL) in video production is a structured log file that stores checksums (hash values) for media files. It ensures data integrity during file transfers, backups, and archiving processes.
Applications of MHL:
- File Verification: Each media file’s hash value is generated using cryptographic algorithms like MD5, SHA-1, or SHA-256.
- Transfer Integrity: When files are copied or moved, their hashes are recalculated and compared to the original values to detect corruption or incomplete transfers.
- Standardised Format: MHL files typically follow a structured format that includes metadata like file paths, hash algorithms used, and timestamps.
- System integration: MHL files can be generated and verified automatically by various media management tools and software used in post-production workflows.
Example Use Case:
During the production of a film, media files are copied from camera storage cards to external drives. An MHL file is created and saved alongside the media files. Before editing, the post-production team verifies the files using the MHL to ensure no data was corrupted during transfer.
Why should you care?
To make sure that the copy is an exact replica of the original, the best solution are checksums, which are originating for example in cryptography. Media Hash List utilises the widely used SHA-1, MD5, or xxhash hashing algorithms. While being similar to cross sums, these algorithms are mathematically much more advanced and guarantee that no two distinct files possess the same checksum.
Therefore, you can detect any change in a file by comparing it to its checksum. If the file wasn’t altered, the checksums match. If the file was altered, it is by definition another file and therefore, the checksums don’t match.
How Does it Work?
Inventory List for Folders
The Media Hash List (MHL) standard specifies how a common inventory list of a folder’s content looks like. MHL is an XML format, that basically lists all the files in a folder and all of its subfolders as well as the corresponding checksums. This list is accompanied by additional information about the creation of the MHL file.
Technically speaking, the MHL file comprehends all the information necessary to tell if the folder structure or any of the contained files was modified since the MHL file was created.
Detection of changes
Within a production process, files get copied and moved multiple times. During each file operation a multitude of unintended changes can occur, mostly due to human error, but because of technical failures:
- Empty files
- Changed files
- Change of folder structure
- Incomplete copy of a folder structure
- Removal or missing of files
- Addition of files
To detect these errors, MHL stores a map of the original folder structure in each MHL file. Therefore, every change to the folder structure can be detected by comparing it to the original map.
Ready for Long-Term Archiving
The Media Hash List is an XML format, that has also been created to be human-readable to a certain extent. Thus the format is self-describing enough, so that you can easily build support for MHL in you own software and system environment. At the same time, this also ensures archivability, since you can be sure this format will still be readable by future generations.
Does Limecraft Support Media Hash Lists?
When using Limecraft for ingesting creating and ingesting dailies, log files are created by Limecraft Edge and they are accessible by different means.
Accessing the Log Files on Disc
When using Limecraft Edge for backing up or offloading original files to intermediary storage or transferring dailies to shared storage, the log files are stored alongside the media in each folder in json format and in human-readable PDF format.
Accessing the Log Files in your Limecraft Workspace
Upon successful ingest, assuming you have sufficient access permissions, log files can also be access via your Limecraft Workspace via Card Overview > Show Manifest.
