Sign of the Times: Enhancing IMF with Auxilary Image Tracks

As a UK Public Service Broadcaster, ITV is regulated by Ofcom, which oversees the provision of accessible content on our broadcast platforms. One of the ways that we make our content more accessible is by providing content that has been translated into British Sign Language, or BSL. BSL is the primary language of around 150,000 people in the UK, and given that the grammar, lexicon and structure of BSL is very different to spoken English, providing content that has been well translated can be much more engaging than just adding subtitles.

As it stands, IMF for Broadcast & Online doesn’t support our signing workflow, so, in the DPP working group, we’ve been working towards creating a plug-in that could bring the benefits of IMF to the creation, storage and distribution of signed content.

IMF is all about components

If you’ve made it this far, you probably already have a pretty good idea of what IMF is. If not, then start here and work your way through all the DPP publications on the subject.

It’s fair to say though that the primary purpose of IMF is to simplify and automate the way we handle multiple versions, enabling content businesses to store only the master content and not all of the derivatives.

But that only works if you have one video track as the master. Historically, what IMF hasn’t done is allow you to store additional video components that are intended to be overlaid onto the main content. So in order to add a signed video track into IMF, you would need to render it over the programme content, and therefore create another version of the content, thus defeating the purpose of using IMF in the first place!

The need for a plug-in

The DPP has been working with its members to publish a plug-in that optionally extends SMPTE ST 2067 (the core IMF Standard), SMPTE TSP 2121-1 (the DPP ProRes Application), and SMPTE TSP 2121-4 (the DPP JPEG2000 Application Constraints) to allow for multiple concurrent video tracks. ‘Optionally’ is the key here, by their nature, plug-ins in IMF guarantee backwards compatibility. If the receiver of an IMF package doesn’t understand the plug-in, it can safely ignore it and process the programme anyway.

The primary element of the plug-in is the AuxImageSequence, which extends IMF by allowing multiple video tracks. The AuxImageSequence sits alongside the MainImageSequence, and allows the CPL to reference more than one video track, allowing the user to stack up different components of the image. Embedding an alpha channel into the AuxImageSequence is a key part of this, allowing each track to control how much of the image is shown and how much is transparent.

The AuxImageSequence is definitely part of the solution, but we still need to define how to take the multiple layers of video and render them into a single image. To help facilitate immediate implementation, this will need to be based on a common, agreed approach, so any device will be able to directly layer the AuxImageSequence on top of the MainImageSequence with no other metadata or external control. The DPP working group has proposed a set of initial constraints. These constraints will simplify the job of the downstream processor that is rendering the image. For example, by allowing only one AuxImageSequence we don’t need to worry about track hierarchy, and ensuring that the tracks are the same resolution removes the need to independently scale them. Once OPLs are a bit more mature (read more about those here), and MetaRes is adopted, then you’ll be able to have more active control of the presentation rather than relying on these constraints.

The use cases

BSL translation of content is just one use case to which we believe this plug-in is applicable. Localising graphics, masking parts of a complied version and flexible branding are also areas that could benefit. In theory, any use case where the content owner wants to overlay any part of the image could benefit.

Get involved

If you’re a DPP member and would like to participate in our IMF work, or if you'd like to discuss membership for your company, contact Rowan.

Rowan de Pomerai

CTO


×

MEMBERS

Enter your email to download

If you work for a DPP member company, you'll be able to log in or create your account. Otherwise, we'll help you learn more about how to access this download.

Not sure if your company is a member? Check our Members Directory

Download this file

Download Now Actionable Insight Summary

Great news. Your company is a member!

Register in a few quick and easy steps to get access to all DPP documents.

Create your login

Join the DPP for full access

This download is exclusively available to DPP members. If you think that your company is a member, you might want to try again.

If you're not yet a member, you can find out more about the benefits of membership.

Learn more

Think there's a mistake? Contact us.

Thanks!

We have emailed you a link to download this file.

Continue

Oops!

Something went wrong.

Start again

If the problem persists, please get in touch.

Account suspended

Unfortunately it looks like your account has been suspended.

Use different email

Please verify your account

We can see you have created an account with us, but you need to verify your email address.

Thanks

Please check your email to verify your account.

Just a few more details!

We would like to send you occasional updates on our work, publications, and events. You can unsubscribe at any time. Can we keep you informed by email?

Please choose an option

It's time to reverify your account

From time to time we need to reverify your email address for security reasons. Click the button below and we'll send you a link straightaway.

System unavailable

We're sorry, it looks like we can't register your account right now. Please check again later.

If the problem persists, please contact membership@thedpp.com.

Account Problem

We're very sorry, it looks like we can't retrieve your account right now. The membership team have been notified and should be in touch shortly.

If the problem persists, please contact membership@thedpp.com.

Coming Soon

This download is not yet available - check back soon!

Learn more

Payment Checkout

Set at runtime

Continue

Payment Complete

Thank you, we have emailed you a link to download this file. The link will be active for 24 hours. If you are having problems please contact us