web page software

Integrating Media Fingerprinting in Production and Delivery Workflows

When it comes to preserving, manipulating, and distributing digital media assets, the content owner faces a series of complex, overlapping challenges and intertwining business initiatives that require the business planning and production organizations to creatively design closed-loop processes that ensure the longevity and relevancy of the final production.

These challenges are universal in the industry, regardless whether the entity is a studio, broadcaster, video streaming portal, corporation, university, or government archive – they all have the capacity to create, and the need to preserve and identify multimedia footage, specifically video.

The Industry’s Current Realities

In the Media and Entertainment industry, there remains tens of thousands of film titles and historical broadcasts still in analogue form – film, tape, cells, etc. Many of these studios own over 100 years of content!

Studios, broadcasters and agencies universally agree that physical tapes and reels are fragile, actual storage space requirements are burgeoning, maintenance costs are sharply rising, and physical degradation of these tapes is unavoidable. All are huge issues that threaten their business’s ability to be financially responsible to their shareholders.  

This has spawned a rush to “digitize” everything. Digital files not only provide more flexibility in terms of handling and delivery, but also make it easier to catalog through tools like digital asset management systems (DAM) while simultaneously reducing many of the operational expense costs associated with physical media. Unfortunately, merely digitizing assets is not enough. Companies then find themselves grappling with new challenges and new threats.

Some of these new challenges are shared across industries and are discussed below.

Reality #1 – Asset management efficiency and metadata conformance

Any digital asset needs to be organized and stored in adherence within an overall database schema that will make the DAM system run effectively. Serving as the librarian function, the DAM’s value is maximized only if assets can be located quickly and accurately. All DAM systems permit some sort of “search” functionality – most commonly by “keyword.” The keyword is simply an alphanumeric string that describes the asset. Keywords, frame rate, shot location, duration, keyframes etc. are all attributes that can be used to describe the media, and the bundle of information that is attached to any individual asset is collectively known as “metadata.” The key to asset identification is ensuring that the metadata is uniformly consistent and applied equally across all assets in the library, regardless of project, age, media format, producer, etc. However, if any attribute is inconsistent, it adversely affects the searchability of the asset and hence the value of the library. Providing metadata that is accurate, relevant and consistently formatted across all of the different asset logging projects is the primary need of any successful metadata conformance initiative.

Solution - ivitec MediaSeeker Core Platform’s unique fingerprinting technology can be used to create a unique, permanent, representation for the media asset. This string is always consistent and is stored in the video fingerprints database so that it may always be found, regardless of the overall quality of the rest of the metadata representation. Fingerprinting is an extremely cost-effective and timely tool to supplement metadata conformance initiatives.

Reality #2 – Proliferation of downstream production versions

Unlike analogue formats, digital files are easily duplicated, manipulated, and stored. As a result, each stage of media production may create multiple versions of the same basic sequence. Each version may have a slightly different special effect, rendering, or perspective, but the essence data is largely the same.

A sampling of multi-format distribution challenges is described above in Figure 1.
For example, television shows are distributed all around the world where they are further edited or converted to regional standards. Subtitled, these region-specific versions are eventually sent back to the originating studio for contractual purposes where every version is received and checked-in, manually reanalyzed, cataloged and then saved in the studio archives. Although the changes are relatively minor and may be small in scope, each review cycle incurs substantial labor and duplicative (digital) storage costs.

In the commercial news space, stock footage (or past footage) from a previous news story is often reused or rebroadcast for different news programs in the future. Again, the entire new broadcast will be shot-listed and stored in the archive. This manual cataloging process often fails to describe the asset properly, causing whatever minimal or incomplete metadata to be the sole organizational index for this new asset.

Moreover, university, government and industry film archives often face the monumental task of reassembling complete versions of old films based on multiple versions, some of which are missing or contain additional frames. Completing an accurate restoration has enormous significance for the archivist – their job is to preserve a cultural or historical legacy.

Solution – By creating fingerprints for each news story, broadcast, or studio master, ivitec MediaSeeker Core Platform’s analysis engine can be used to instantly locate all varieties of the source footage - regardless of the age, edits, language, small differences in graphic overlays, subtitles, or overall completeness of the broader metadata representation. This enables much faster production and more accurate decision-making for the news director, archive manager, and editors. Improved accuracy and more timely productions lead to the availability of a higher value piece.

Reality #3 – Need to optimize sales opportunities / revenues - proliferation of multiple consumption (distribution) file formats

Companies everywhere are under increased pressure to grow sales and the “top line.” As traditional broadcast and cable viewership stagnates, companies rush to embrace “new media:” internet, IP streaming, mobile streaming, etc. The proliferation of new consumption venues introduces a stark new reality – that of “unauthorized content” from “unauthorized distributors.”  

Several high profile legal cases pitting Hollywood studios against video sharing portals (e.g. Viacom vs. YouTube, 2007) are clear examples of how critical this topic has become. The studios blame these sites for copyright infringement leading to lost revenue. The video sites deny wrongdoing, claiming they are not responsible for the copyrighted material that users post. And so the debate ensues about the definition of compliance to the Digital Millennium Copyright Act (DMCA).

However, the real issue is: “How does a studio implement a comprehensive distribution strategy that includes the commercial requirements for accelerating the availability of down-converted broadcasts to authorized distribution partners?” 

The market demands stories that are relevant, new, or compelling. If there is demand, there is value. The “freshness” of the story directly impacts the willingness of a user to pay or an advertiser to tie in to the asset – leading to sales opportunities. Unhappy with the lack of availability, creative consumers have taken matters into their own hands and use consumer-grade video capture tools and their arbitrary production values to rip-down traditional on air or cable broadcasts and make them available on the video sharing sites to “provide a service” for others with similar interests.

But beyond lost revenue opportunities, these copies are of substandard quality and negatively impact the studio’s brand. This is potentially a much larger issue.

Solution – By integrating the ivitec MediaSeeker into the production workflow, finished stories and programs can be fingerprinted automatically. The fingerprints are stored in a central database/metadata archive. This fingerprint remains constant even as the media is sent downstream for transcoding and metadata reconformed for the requirements of the specific video sharing or platform provider (fingerprints for identical videos, regardless of format, remain identical and identifiable). Regardless of whether this “quality” media arrives at a sanctioned partner or if a consumer rips an on-air broadcast and uploads to an unauthorized video-sharing site, the ivitec MediaSeeker analysis engine can be employed to scan a library of properties and determine where a copy of the footage is present, be it the entire copy or merely a few frames.  

Rather than manually scanning a list of exception sites, MediaSeeker’s APIs will allow integration into a business intelligence system to permit automated comparisons to the company’s official partner list. Once exceptions to this list are flagged, the business development or legal department can then decide whether to proceed with legal action or engage in a broader business discussion to sanction a new partner – possibly opening more sales potential and more advertising opportunities.  

Reality #4 – Unexpected/under-anticipated operating expenses with maintaining digital storage archives

As mentioned earlier, DAM systems will catalog and maintain insight to the whereabouts of all titles and projects being worked on. Additionally, archive systems keep copies of all titles for historical and compliance purposes.

While they are easier to find in digital form, duplicate digital files have a much higher proliferation potential than if the media remained in analogue form. Across the industry, it has been estimated that up to 97% of files in a digital library may be duplicate material**. Obviously, this adds to the cost of provisioning and maintaining the SAN/NAS hardware -- rack space is increasingly at a premium, while power and HVAC costs are also rising. Additionally, bloated disk libraries can make the performance of the overall system slower, impacting productivity. Having too many copies of the same thing makes it harder for users to distinguish which copy they actually want to use.
Conversely, there may be corporate requirements to find or “rebuild” asset libraries with missing or lost clips that can be reintegrated into the overall library.  

Solution – By integrating the ivitec MediaSeeker into the production and archive workflow, finished stories and programs can be fingerprinted automatically. The fingerprints are stored in a central database/metadata archive. This fingerprint remains constant regardless of the media’s archive format, thus allowing the archivist the opportunity to more confidently purge intermediary / working copies of media that are no longer required, saving disk space and allowing the DAM to run more efficiently; or identifying unique clips that have not already been ingested and stored in the archive.

Reality #5 – Expectations of production workflow integration and scaling difficulties

Asset Management systems are trying to extend their basic “librarian” capability and are competing with broadcast automation systems to become a “workflow” management tool.  

For both, we find that in the increasingly distributed (WAN-based) production model, DAMs are ill-suited to properly handle the networking, protocol and hardware interface requirements, and most lack robust integration points, making it very difficult to integrate third-party technologies in order to provide a holistic solution. Additionally, the computational requirements necessary to support the various functions require multiple server nodes to be tied together in order to provide a reasonable QoS (Quality of Service) to the personnel who are connected to the system. Many legacy implementations have been very expensive, highly customized, time consuming, and unstable. As a result of a less than optimal experience, management is reluctant to try new technologies regardless of their potential value since they fear “disrupting the apple cart.”

Solution – The ivitec MediaSeeker Core Platform makes an extremely rich and robust library of API function calls available which permit custom integration into any DAM, automation, archive, playout, or business intelligence system. Custom statistics, report views, conditional execution of fingerprinting tasks, fingerprinting analysis, and searches are all possible manually through a UI, and programmatically through these APIs. Additionally, custom queries and features may be developed by the customer’s development staff with knowledge of basic scripting technologies supported in the SOAP standard, such as C#.Net, or Java,.

The ivitec fingerprinting and comparison engines are robust and inherently multi-core enabled. Per ivitec’s benchmark implementation, a single CPUbased PC running MediaSeeker can ingest, fingerprint, and catalog between 2000 and 10,000 hours of content within a single catalog.

If rack space is at a premium, the MediaSeeker APIs can be passed across the internet and to an ivitec SaaS service, where computational throughput is highly elastic, yet customer security is ensured. The secure fingerprinting database remains property of the customer, and may be kept in the cloud, in house, or a combination thereof. This ensures the maximum deployment flexibility for the organization.

Reality #6 – Dissatisfaction with current security technologies like DRM and watermarking

Many companies have already experimented with DRM technologies and watermarking and have had mixed results. More often, they are disappointed that watermarking technology merely provides a purely forensic capability to possibly recover where a production leak has occurred, but neither piracy prevention nor supplemental searchability in a library. Additionally, creating watermarks is computationally intensive and may be removed or obfuscated through any number of file editing techniques. This makes watermarking of limited value when implemented in a vacuum.  

Digital Rights Management was heralded as a panacea against piracy and copyright infringement when first introduced in the last 1990s. Individual or group access rights basically “tethered” the asset to be manipulated or viewed by specific individuals or groups. Useful in a closed production environment, this concept was rapidly circumvented as the industry’s production schedules were compressed requiring ‘round-the-clock access to the required files, and pressure on production cost controls forced the outsourcing of many functions like rendering, colorization, compositing, etc. Such requirements made using DRM across the wide-area network, very hard to set up and manage, and hence increasingly easier to bypass.

There is a misconception in the industry that fingerprinting may become yet another technological fad.  

Solution – It is important to remember that fingerprinting is a completely orthogonal technology to DRM and watermarking. In fact, when used in conjunction with the previous, a comprehensive media security strategy may be formed.  

However, by itself, fingerprinting is not a security tool. It should be considered a production and distribution tool. When used specifically from these perspectives, it is important to understand that the ivitec MediaSeeker fingerprint is unique to a complete media asset, or set of frames. The fingerprint is permanent and stored in a database, does not travel with the asset itself, and hence cannot be altered or removed.  

From a distribution perspective, consumer access rights may be served from the content aggregator, or embedded in a programmable chip or software applet that runs at the various points of consumption. Since the fingerprint is static, it can be used as a baseline to perform web-based comparisons against media assets at any number of video sharing or streaming sites.