Storia at the Oscars

• Back

stylized video generation

Films are defined among other things by the tonality and visual aesthetic of their scenes. These are aspects that the directors of photography and cinematographers on a set spend countless hours deciding on and refining. This is all to ensure that the actual shooting of a scene is as efficient as possible, since it is such an expensive part of the film lifecycle.What if you could experiment with the tonality of a scene in minutes not days without requiring expensive reshooting?Consider, for example, this action sequence from the Academy-award winning All Quiet on the Western Front

Using generative AI techniques, we are able to take the original sequence (top) and completely change the visual feel of the scene, introducing more bright colors, high contrast shadows, and an almost video game appearance.Or consider this sequence from the same film, where we have also introduced a darker, more demonic appearance to the scene. It almost has the look of a Resident Evil game.

We are also able to completely change the appearance of characters with the desired tone in mind.Continuing with the demonic aesthetic, notice the transformation of Michelle Yeoh from the Oscar-winner Everything Everywhere All at Once:

Or if we want a more light-hearted transformation, notice how we can change the appearance of Felix Kammerer from All Quiet on the Western Front to a soft, ethereal character like a Medieval painting.

If we want to change the style of a sequence from live-action to something more animated and cartoonish, notice the transformation of this scene from The Banshees of Inisherin:

We can adapt a similar transformation to Jamie Lee Curtis from Everything Everywhere All at Once.

In all situations, generative AI has enabled us to completely change the aesthetic of a video shot without having to reshoot anything. Within minutes, we are able to experiment with the look and feel of a film.

• Back

Talent swap

Casting the right actor/actress in a film can make or break a production.How can we de-risk the possibility of making a wrong casting decision and explore what a film would look like with a certain character played by other actors?Generative AI can help answer the "what if" question by swapping out which actor plays a certain part. We call this cinematic character adaptation.In one famous example of the casting predicament, famous actor and artist Harry Styles was almost cast for the part of Elvis in the Oscar-nominated Elvis.What would the film have looked like if Harry Style had replaced Austin Butler?

• Back

Stylized Marketing Creative

Marketing creative is crucial for ensuring the right message is conveyed to moviegoers about a film.How do we quickly experiment with the appearance of certain marketing creative? This can be done cheaply and quickly using generative AI. Think minutes, not hours or days. All we have to do is provide a textual description of the style we want, and AI handles the rest.Consider this adaptation of the promotional poster for Guillermo del Toro's lovely Academy award-winning film Pinocchio. We can see what the poster would look like as a pencil drawing:

Or as a cartoon adaptation of Orson Welles' Citizen Kane:

Or as an illustrated Pixar film:

Or if we are feeling especially inspired, here's Pinocchio as a cartoon Spiderman:

We may even want to see what the live-action remake of an animated film could look like, as in this promotional poster from Best Animated Short winner The Boy, the Mole, the Fox, and the Horse:

• Back

ai soundtrack adaptation

It's common in music to sample and make covers from existing tracks, providing them distinct musical interpretations. However mixing music typically requires an expert level knowledge of music theory and associated audio editing software.What if we could just take an existing track and adapt it to new styles by describing in words what interpretation we want?Consider for example the song Voodoo Mama from the Oscar-nominated score from Babylon.AI can allow us to adapt it to a techno style by just providing the textual description "techno beats deadmau5, house music":

Or if we wanted to riff on the Oscar-nominated song This is a Life from Mitski here's an AI-generated rendition of "90s hip-hop Wu Tang Clan":

Or "electric guitar blues, stevie ray vaughan":

• Back

branded creative augmentation

When it comes to promoting film characters, we may also want to associate certain brands with them. As an example, imagine having a lead character in a film wearing a shirt from Adidas or some other apparel brand.Adapting these brands may become time-consuming but we would like to be able to just touch up existing marketing creative we already have.In the case of Marcel from Marcel the Shell with Shoes On we may want to selectively promote Balenciaga shoes.Here's a before shot:

And a shot with Marcel wearing the branded shoes:

In this case, generative AI has enabled us to augment an image by simply taking the original photo and providing a description of the branded apparel we want ("balenciaga shoes"). No photo editing software needed.