Connections

Cloud-based artificial intelligence (AI) and machine learning (ML) solutions are transforming and accelerating virtually every aspect of content creation and distribution, according to Digital Nirvana.

As these technologies continue to grow in power and sophistication, they’re bringing new levels of accuracy, efficiency, compliance and cost savings to broadcast operations and content creation.

One critical way in which AI and ML helps is by saving time while creating content and then when distributing it, Russell Wise, SVP of sales and marketing at Digital Nirvana, said Oct. 20 during the online Media & Entertainment Day event.

During the Distribution & APIs breakout session “How AI/ML Empower Revolutionizing Content Creation & Distribution,” he and Russell Vijayan, head of AI products and services at Digital Nirvana, illustrated real-world applications of AI and ML that the company has implemented for content producers and distributors. They examined use cases across the media delivery chain, including production, post-production and distribution.

They first looked at a use case that the company deployed with a leading online entertainment company they didn’t identify. “They’re well-known and, for them, the most important thing, is delivering news and entertainment very quickly,” Wise said.

The business challenges the company experienced with transcription included the facts that: Daily show content had to be consistently created in a two-hour window; footage and field recordings were not searchable based on spoken content; and it took 15-20 hours to create a standard montage clip, according to Digital Nirvana.

The solutions to those challenges that Digital Nirvana was able to provide were: Portal/application programming interface (API)-based speech-to-text conversion and returning to Production Asset Management (PAM) and Media Asset Management (MAM); and searchable video reducing standard montage clip creation to two hours.

“Both in the production and post-production phases… the big challenge was just time,” according to Wise. The company “needed to create new content with captions in a two-hour window,” but there was just a “tremendous amount of footage coming in from the field that was not searchable, had no context and no scripting and, from that, they needed to complete content and deliver that to an OTT provider in the format expected in a very short time frame,” he said.

Digital Nirvana provided the company with the ability to go to specific places in a piece of video and, by clicking on search terms, find all the clips they need much faster, allowing it to produce content “within the timeline that they were given,” explained Vijayan.

“The second challenge, of course, is that the content had to go out that also had to be captioned accurately,” Wise pointed out. The company, however, “did not have time to use a third-party resource to send out the content, have it captioned and turned around,” he said, adding: “They wanted an in-house workflow where they could automatically use speech-to-text for captioning and assess quality, and also publish this to the streaming platform” the content was being distributed on.

When it comes to captioning for Web and over-the-top services, the challenges the company had were automating the in-house captioning process and compliance with streaming platform standards, according to Digital Nirvana.

Solutions that Digital Nirvana provided to solve those challenges included: Automatic transcripts created in real-time; caption conversion using presets, audio sync and natural language processing (NLP); and parameters and NLP-based auto-quality control (QC) for compliance with various streaming platforms.

“Here we are using a combination of technologies” to overcome the challenges,  Vijayan explained. “We created automatic transcripts which could” go in as metadata and, at the same time, be used as raw material for creating captions, he noted, adding: “Once the transcripts are created, now there is a challenge of converting that into subtitles or captions…. We applied a layer of NLP on top of the standard parameters. That would help split the captions, arrange the captions in relative order so that the user doesn’t need to spend much time going and rearranging each of those captions and changing titles…. Now, once that is done, we also gave the capability for them to compare their current output with the streaming platform’s guidelines.”

The user is “alerted if there is any non-compliance item … in relation to streaming platforms,” Vijayan said, adding: “That helped them to turnaround much faster and also ensure there’s no rejection from the streaming platform side.”

The captions created using this system have 98-99% accuracy, Vijayan said, noting that, in the process, what previously took one hour to accomplish now takes 10-15 minutes.

The third challenge was “something we see sort of across the board: These days, everybody needs to localize their content – specifically the captions – into other languages,” in this case, English to Spanish, Wise said.

The specific challenges were the repurposing of captions for different countries and using existing closed captions in another language and generating translations, according to Digital Nirvana. The solutions were automatic translations that can be created under three minutes and importing existing captions and generating new translations.

“Content has gone global for a while now,” Vijayan noted, adding: “You take any platform, you see a lot more languages. Whether it is dubs or subs, you see a lot more additional languages for the same content.”

In the specific case of the company Digital Nirvana was helping, requirements included the ability to create Spanish captions for English captions much faster because the company only had two hours to “complete this whole process,” Vijayan said.

“What we did differently for them is we used our linguistic research and created a set of algorithms that will dictate as to how much content needs to be translated … to get a much more accurate result,” Vijayan said.

Moving on to distribution and video metadata, Wise pointed to use cases with two multichannel video programming distributors. In both cases, they had original content but most of the distribution is other peoples’ content — 300-400 channels worth of it, he said, noting both companies used compliance logging systems.

The goals are “mining” data from the distributed content and “making better decisions” about what place you “get the most bang for the buck with [an] ad placement,” while figuring out how to track or decide what product placement is going to the video, Wise said.

Digital Nirvana solutions provide: The ability to accurately identify spots based on content (viewed on mobile phones, in cars, etc.); identify and replace brands within content based on consumer data; and a reduction of effort in identifying explicit content, according to the company.

As an example, Vijayan noted there may be a Microsoft Surface tablet on the screen and a company might want to replace it with an Apple iPad. Or when an actor opens a fridge, a company might want to replace what is inside it for a beer ad, he said.

A “common theme among all customers” is that “many have existing workflows in place and they need to add some element of AI or ML to maximize or better use that workflow,” according to Wise. “But we also see clients who are willing to just adapt a workflow off the shelf and deploy it in their organization and use it from day one,” he said.

“The key, I think, is… having a full set of capability of microservices to accomplish that full range of tasks,” he added.

Click here to access the full presentation.

M&E Day was sponsored by IBM Security, Microsoft Azure, SHIFT, Akamai, Cartesian, Chesapeake Systems, ContentArmor, Convergent Risks, Deluxe, Digital Nirvana, edgescan, EIDR, PK, Richey May Technology Solutions, STEGA, Synamedia and Signiant and was produced by MESA, in cooperation with NAB Show New York, and in association with the Content Delivery & Security Association (CDSA) and the Hollywood IT Society (HITS).