AI Knitted Short Story - 'Edith & Fred' - Runway ML Gen 3 Vid to Vid.

AIAnimation
16 Sept 202403:11

TLDRIn this creative video, the artist shares a unique animated short created using a blend of live portrait and Runway ML's Gen 3 video-to-video technology. The artist played both characters, using live portrait to animate character images generated by mid-journey. Despite inconsistencies due to technical limitations, the woolen yarn knitted style adds charm to the narrative about Edith and Fred, who ponder the nature of their reality while sitting on a bench. The video showcases the potential of Runway ML's Gen 3 for stylizing videos and invites viewers to explore its capabilities further.

Takeaways

  • 🎥 The creator shares an animated short story using a combination of live portrait and Runway ML's Gen 3 video approach.
  • 🎭 They acted out both roles themselves and used live portrait to animate character images generated with mid-journey.
  • 🧶 The animation style applied is a woolen yarn knitted style, which adds a unique texture to the characters.
  • 🔄 Despite technical limitations causing changes between each 10-second clip, these variations fit the narrative.
  • 📹 The video includes a dialogue between two characters, Edith and Fred, discussing the familiarity of their situation.
  • 🤔 The characters ponder if their reality is a dream and what would happen if one of them woke up.
  • 🎨 There are occasional inconsistencies in the character design, such as the female character Edith having a mustache or beard.
  • 📹 The creator has published a course on how to use live portrait, covering installation, character design, and editing.
  • 🔗 The creator also explored Runway ML's Gen 3 video-to-video feature in a separate YouTube video.
  • 🚀 The video serves as an example of how to create stylized animations using Runway ML's Gen 3 for various artistic effects.

Q & A

  • What is the main theme of the animated short story 'Edith & Fred'?

    -The main theme of the animated short story 'Edith & Fred' revolves around the concept of reality and the possibility of living in a dream, where the characters question the nature of their existence and whether they might be part of a dream that could end at any moment.

  • How was the animation style of the short story achieved?

    -The animation style was achieved by filming the creator acting out both roles, applying that animation to character images generated using mid-journey, and then running the content through Runway ML's Gen 3 to apply a woolen yarn knitted style.

  • What is the significance of the bench and the time of day in the story?

    -The bench and the time of day are significant as they represent a recurring setting where the characters Edith and Fred always end up, hinting at a sense of déjà vu and the repetitive nature of their encounters, which could suggest a dream-like or predetermined scenario.

  • What is the Matrix reference in the dialogue between Edith and Fred?

    -The Matrix reference is to the scene where the black cat walks past the characters multiple times, causing one of them to question the nature of reality. This is used by Fred to compare their situation, suggesting that their experiences might not be real.

  • Why does the female character, Edith, sometimes have a mustache or beard in the video?

    -Edith sometimes has a mustache or beard due to limitations in the animation process. Despite having the correct seed number and consistent text prompt, there are inconsistencies between each 10-second clip, which the creator mentions as fitting the narrative.

  • What is the significance of the characters knitting in the story?

    -The act of knitting by Edith is used as a metaphor for the creation of their reality. It suggests that their world might be as malleable and intricate as a knitted piece, and that it could unravel if one of them wakes up from the supposed dream.

  • What does the character Fred mean when he says 'maybe they stay, maybe they unravel'?

    -Fred's statement 'maybe they stay, maybe they unravel' refers to the potential outcomes if one of them wakes up from the dream they believe they are in. 'Stay' implies that the other person remains in the dream, while 'unravel' suggests that the dream and their existence within it might end or fall apart.

  • How long did it take to create the entire animated short?

    -The exact time is not specified in the transcript, but the creator mentions that despite the inconsistencies, the process did not take too long, indicating a relatively efficient creation process.

  • What is the course mentioned by the creator, and what does it cover?

    -The course is about using Live Portrait and covers how to install it locally or run it in the cloud, as well as the process from script to character design and using Live Portrait to editing the final animation.

  • What is the new video feature in Runway ML Gen 3 that the creator explored?

    -The new video feature in Runway ML Gen 3 is the video-to-video approach, which allows for the transformation of live-action footage into various stylized forms, such as cartoon, CG, porcelain, or paper textures.

  • What is the creator's hope for future improvements in Runway ML's technology?

    -The creator hopes that future improvements in Runway ML's technology will allow for more consistency between generations of video content, reducing the inconsistencies observed in the current Gen 3 output.

Outlines

00:00

🎨 'Creating a Short Animated Story with Live Portrait and Runway ML'

The creator shares a unique animated short film made using a blend of live portrait animation and Runway ML's Gen 3 video approach. They filmed themselves acting out two roles, applied animation to character images generated with Midjourney, and then processed the content through Runway ML to achieve a knitted woolen style. Despite inconsistencies due to technical limitations, the creator found the variations fitting for the narrative. The story revolves around two characters discussing the familiarity of their situation, questioning the reality of their existence, and pondering the implications of one waking up from a dream. The creator also mentions a course on using Live Portrait and a new video exploring Runway ML's video-to-video feature.

Mindmap

Keywords

💡Animated Short

An 'animated short' refers to a brief film that uses animation techniques to tell a story. In the context of the video, the creator has used a combination of live-action and animation to craft a narrative. The animated short serves as a creative medium to explore themes and ideas in a visually engaging way. The video showcases the process of creating such a short, where the creator filmed themselves acting out roles and then applied animation to character images.

💡Runway ML Gen 3

Runway ML Gen 3 is a video processing tool that applies various styles to video content. In the video, the creator uses this tool to give their animated short a 'woolen yarn knitted style,' which is a unique visual effect that adds texture and a handmade feel to the characters and scenes. This tool is integral to the video's aesthetic and contributes to the storytelling by enhancing the visual narrative.

💡Live Portrait

Live Portrait is a technology that allows for the creation of animated characters from live-action footage. The creator uses this technology to animate their own performance, capturing the essence of their expressions and movements. This is applied to character images generated through another tool, creating a seamless blend of live-action and animation. The use of Live Portrait in the video demonstrates how technology can be used to bring characters to life in an animated short.

💡Mid Journey

Mid Journey is likely a reference to a tool or platform used to generate character images. In the video script, it is mentioned as a source for creating character designs that are then animated using Live Portrait. This tool plays a role in the pre-production stage of the animated short, helping the creator to visualize and develop the characters before they are brought to life through animation.

💡Woolen Yarn Knitted Style

The 'woolen yarn knitted style' is a visual effect applied to the animated short through Runway ML Gen 3. It gives the video a distinctive appearance, as if the characters and scenes were made of knitted wool. This style is not only aesthetically pleasing but also serves a thematic purpose, adding a cozy and tactile quality to the narrative that complements the story's themes of comfort and familiarity.

💡Deja Vu

Deja Vu is a French term that translates to 'already seen' and describes the sensation of experiencing something as if it has happened before. In the video, one of the characters mentions Deja Vu to describe their feeling of familiarity with the recurring scenario of meeting at the same bench and time. This concept is central to the narrative, as it leads to a philosophical discussion about the nature of reality and dreams.

💡Dream

The concept of a 'dream' is a recurring theme in the video's dialogue. The characters speculate about the possibility of their experiences being part of a dream, which leads to a deeper conversation about the nature of reality. Dreams are used as a metaphor to explore the idea that life might be illusory or that our perceptions of reality are not always reliable. This theme is integral to the story, as it encourages viewers to question their own experiences and the world around them.

💡Inconsistencies

In the context of the video, 'inconsistencies' refer to the variations that occur between different segments of the animated short. Despite the creator's efforts to maintain a consistent style, there are changes in the appearance of the characters, such as the occasional addition of a mustache or beard. These inconsistencies, while unintended, are embraced by the creator as part of the narrative's charm, adding a quirky element to the storytelling.

💡Narrative

The 'narrative' is the story being told in the animated short. It revolves around two characters, Edith and Fred, who find themselves in a recurring situation that feels like a dream. The narrative is used to explore themes of reality, memory, and the comfort of familiar routines. The video's narrative is engaging and thought-provoking, inviting viewers to consider the nature of their own experiences and the world they perceive.

💡Video to Video Feature

The 'video to video feature' is a capability within Runway ML Gen 3 that allows for the transformation of one video style to another. In the video, the creator mentions exploring this feature, which suggests the potential for further creative experimentation with video content. This feature is part of the broader discussion about the tools and techniques available for video production and animation, showcasing the possibilities for creative expression.

💡Stylized

To be 'stylized' in the context of the video means to give the characters and scenes a distinct visual appearance that deviates from a realistic representation. The creator uses Runway ML Gen 3 to apply various styles, such as woolen yarn knitted, cartoon, CG, porcelain, or paper textures, to create a stylized look. This approach enhances the artistic quality of the animated short and contributes to its unique aesthetic, setting it apart from more traditional animation styles.

Highlights

Creator shares an animated short made with a mix of live portrait and Runway ML's Gen 3 video approach.

The creator acted out both roles and used live portrait to animate character images generated by mid-journey.

Runway ML's Gen 3 was used to apply a woolen yarn knitted style to the animated content.

Despite limitations, changes between 10-second clips fit the narrative of the story.

The animated short tells a story of two characters, Edith and Fred, who always end up at the same bench at the same time of day.

Characters discuss the feeling of déjà vu and the possibility of their reality being a dream.

Edith wonders if she might have been knitting in her chair before appearing in the dream.

Fred and Edith contemplate what would happen to the other if one of them wakes up from the dream.

The characters express a desire to stay in the dream longer due to its comfort.

The creator acknowledges inconsistencies in the animation but notes they suit the narrative.

The female character, Edith, occasionally appears with a mustache or beard due to animation limitations.

The creator offers a course on using live portrait for character design and animation.

The course covers installation, character design, and editing using live portrait.

The creator suggests using Runway ML's Gen 3 to stylize videos with various textures like cartoon, CG, or porcelain.

A new video exploring Runway ML's video to video feature in Gen 3 was uploaded to YouTube.

The creator expresses hope for improved consistency in future generations of Runway ML's video approach.

The video ends with a cheerful sign-off and music.

Casual Browsing

Awesome Video ReStyling - Runway ML Gen 3

2024-09-20 13:34:00

How To Use Runway Gen-3 Alpha - (Runway Gen-3 Tutorial) - Gen 3 Alpha Guide

2024-09-20 13:52:00

Runway Gen 3 is Crazy! Runway AI Tutorial

2024-09-20 15:37:00

Runway Gen-3 Alpha Turbo Image to Video

2024-09-20 11:58:00

Runway Gen-3 Alpha (All Videos)

2024-09-20 12:51:00

Create Cinematic AI Videos with Runway Gen-3

2024-09-08 13:17:00