Creating an AI-driven video analysis platform with audience personalization data for Zentiment

Partner Overview

Zentiment, founded by experienced film and TV trailer editors, aims to revolutionize the post-production process for short-form content using AI technologies.

The team has edited trailers for various film and TV productions, including X Factor, the Olympics, FIFA World Cups, political campaigns, major TV series, films, and documentaries.

Zentiment aims to make the trailer creation process more efficient and insightful to support its clients. With enough data combined with sentiment analysis, the team could predict how an audience might respond to any specific moment in a trailer.

This would increase the accuracy and relevance of Zentiment’s feedback on trailers for clients. The team could also suggest alternate moments from the film that would more effectively communicate an emotion to the audience.

Challenges

The traditional trailer creation process for films and TV shows is often slow and subjective while lacking data-driven insights – leading to several key issues:

#1 Inefficient creation process:

• Time-consuming and resource-intensive
• Requires multiple iterations and approvals
• Difficult to target different segments and platforms efficiently

#2 Lack of data-driven insights:

• No pre-release audience feedback
• Decisions based on intuition and personal biases of a small group
• Disconnect between creative intent and actual audience reception

#3 Limited optimization:

• Difficult to rapidly iterate based on feedback
• A one-size-fits-all approach to creation
• Inability to predict response before releasing marketing material

Project Overview

Zentiment commissioned Neurons Lab to conduct a generative AI feasibility assessment and proof of concept (PoC) for their innovative video editing and audience evaluation project.

Our solution can analyze an entire movie and identify moments to consider putting in a trailer based on various criteria. It tags sections of the film based on their suitability for the trailer.

Zentiment’s initial expectations were for an AI solution providing sentiment analysis. However, we exceeded these expectations, providing a PoC capable of analyzing and tagging:

• Sentiment
• Character actions
• Dialogue
• Mood
• Scene descriptions
• And more

In the long term, Zentiment aims for a product that facilitates the creation of the right trailer for the right audience. This will support filmmakers in their efforts to increase ticket sales and audience viewing figures.

Solution

The user journey is as follows:

• A user visits the platform’s front end and uploads a new video.

• This automatically triggers a Lambda function through an S3 event set in the input folder.

• Lambda launches an ECS task and passes it to the S3 bucket and file to process.

• The video file is processed, and the results are sent to the output folder in the S3 bucket.

• The front end fetches the results from the API deployed on an ECS service.

• The user makes a request to generate a trailer script for a processed video.

• The front end calls the API, which runs an ECS task to create the trailer script.

Example output

Here is an illustrative example of AI’s video analysis and feature tagging capabilities. Based on the Deadpool & Wolverine trailer above, we used multimodal AI to produce the following 5-act analysis:

• 0:00-0:18 – Introduction and setup of Deadpool’s emotional state.

• 0:19-0:45 – Introduction of Wolverine and his past.

• 0:46-1:19 – The mission and Wolverine’s reluctance.

• 1:20-1:51 – Wolverine’s acceptance and the movie title reveal.

• 1:52-2:00 – Humorous ending scene.

Here is a snapshot of the AI-driven analysis for Act 3:

Part 3: 0:46-1:19

1. Main Characters

• Deadpool: Still in his red and black suit.

• Wolverine: Still in his yellow and blue X-Men suit.

• A young girl: Appears briefly in a flashback, seemingly close to Wolverine.

2. Emotions

• Deadpool: Determined and passionate about saving his world.

• Wolverine: Grieving, haunted by his past, and reluctant to get involved.

3. Actions

• Deadpool: Shows Wolverine a photograph and pleads for his help.

• Wolverine: Initially rejects Deadpool’s plea but later seems to reconsider.

4. Objects

• Photograph: Held by Deadpool, depicts a group of people, presumably his loved ones.

• Wolverine’s suit: Now damaged and bloodstained, emphasizing the hardships he has faced.

• Tank: Wolverine is seen battling enemies alongside a tank in a flashback.

5. Setting

• The desolate landscape continues.

• A diner where Deadpool and Wolverine have a tense conversation.

• Various action-packed flashbacks from Wolverine’s past.

6. Plot

• Deadpool reveals that his world is in danger and he needs Wolverine’s help to save it.

• The flashbacks emphasize Wolverine’s tragic past and his desire to leave his superhero life behind.

7. Visual and Audio Effects

• Dramatic music continues, building tension.

• More action-packed flashbacks with intense sound effects.

• Slow-motion shots emphasize the emotional weight of certain scenes.

8. Pace

• The pace fluctuates between fast-paced action sequences and slower, dialogue-heavy moments.

9. Cultural or Social Values

• The importance of friendship and teamwork is highlighted, as Deadpool seeks Wolverine’s help despite their differences.

• The video also touches upon the themes of loss, grief, and the burden of heroism.

10. Colors and Visuals

• The contrast between Deadpool’s bright red suit and Wolverine’s more muted yellow and blue continues to symbolize their differing personalities and approaches.

• The desolate landscape remains a constant visual reminder of the stakes involved.

11. Humor

• Despite the serious tone, Deadpool’s dialogue includes moments of humor, providing comic relief.

Results

In only two sprints, each lasting just two weeks, Neurons Lab delivered the PoC on time and received very positive feedback from Zentiment.

To complete the PoC, we:

• Prepared pre-processed PoC data sets
• Produced documentation for the codebase, model architecture, and technical specifications
• Developed the CI/CD pipeline for Infrastructure as code deployment
• Developed the Infrastructure as Code for quick build and deployment
• Created an AI solution development roadmap

We also outlined how to proceed with a potential full-scale development of the full AI Editing Engine at a later date.

Feedback

“The talent in the project team is exceptional. Thanks to the team, we pivoted from our initial idea to a more ambitious one. The approach, presentations, and level of innovation were real game-changers, so you have exceeded our expectations.”

Tommy Duncan, CEO and Co-founder, Zentiment

About Neurons Lab

Neurons Lab is an AI consultancy that provides end-to-end services – from identifying high-impact AI applications to integrating and scaling the technology. We empower companies to capitalize on AI’s capabilities.

As an AWS Advanced Partner, our global team comprises data scientists, subject matter experts, and cloud specialists supported by an extensive talent pool of 500 experts. We solve the most complex AI challenges, mobilizing and delivering with outstanding speed to support urgent priorities and strategic long-term needs.

Ready to leverage AI for your business? Get in touch with the team here.