That's what I think for now. Letting writers show their work and previs. We can't even get a walking talking animation correct let alone a fight scene. But other things like backgrounds and non organic movement (spaceships) it's there or it can do half the job.
I just happened to come across this Starwars animation on the stableDiffusion subreddit, which is among the most impressive ones I have seen up to date. [https://www.reddit.com/r/StableDiffusion/comments/1906h3o/star\_wars\_death\_star\_explosion\_anime\_style/](https://www.reddit.com/r/stablediffusion/comments/1906h3o/star_wars_death_star_explosion_anime_style/)
While this particular example may be impressive, I still agree with what you said.
It would be cool to see a model trained on cheap 80's and 90's OVA, which, for my taste, totally had a charm of their own, by the way they were exploiting very basic animation techniques for very effective storytelling.
I know of a video game franchise where this kind of AI animation would work extraordinarily well for -- Disgaea.
The entire franchise is mostly still images with slight movements. Hell, what I saw in this video is better than what I typically see in their games. If they can get an AI model to do most of the work, the cost to animate their games would probably decrease enormously.
And they could probably use it. The company that makes the games is very small and almost went bankrupt once already.
The current problem is copyright, you need to prove that you own all material that the model was trained on (at least that's how it worked on steam and similar last time I checked) otherwise you're too big of liability. So if you don't already have enough art/assets to train the model on, it doesn't make sense to rely on it, and if you do, well then you already have tons of art to use. Good way to ease the workload for the franchise though.
Making consistent content made with poop is commercially viable these days.
AI has been assisting animation for like a year already. You just don’t notice it because people are too busy making things with it.
Yup. The real line in the sand is what people "count". I can anecdotally confirm we've been using AI in some form for years now. Full screen hero animated characters in AAA films, maybe not. BG elements and grunt work? Absolutely at *least* since 2018. We're using AI for roto and upscaler/denoisers these days.
On the art side it's much more complicated and different companies are testing different levels of legal flexibility: Liability if you train across IP's even if you own both, exposure to litigation if you use a 3rd party, some other things I cant remember. (This is all off the top of my head from a company meeting a year ago, no idea where it stands now). *Personally* I predict art teams will be training focused proprietary models as a compliment to standard workflows for some time. For pros there is definitely a point where text-to-prompt is more effort than getting what you want 'the old way'.
Sure, but that's missing the point. Which is that what op posted is nice, but its not even close to actual animation just because there's slight motion in the pictures.
Omg you people are crazy, how can't you see a bigger picture, few year ago there was some pixels blob, now this. Can you imagine what would happen in another few years I would say, but clearly you can't...
The problem with people is their own egos are too large. They can't believe that after thousands of years of being the dominant intelligent species on the planet their creativity is about to be upstaged by "machines".
Logic and reasoning is ineffective against these people. They're pure creatures of emotion.
Oh man, your words are like a light at the end of a long tunnel. Reasonable and forward thinkers are rare in this world. It's not only that people are going to be upstaged, they are going to be crushed. In 10 to 20 years, people will become useless in most fields compared to AI. At the same time, productivity will soar through the roof. I can't even imagine how the world is going to look then, and thankfully, I am not responsible for finding jobs for people without a job.
Yeah, I'm sure if current progress continues, sometime this century AI is going to be better than humans in every single field. There's no need to continue with the traditional approach to the economy and jobs at that point, we've been long overdue for a paradigm shift.
>Reasonable and forward thinkers are rare in this world.
It's a good thing forums like reddit exist where you can occasionally discuss topics with like-minded people. I recommend checking out r/singularity
GANs are never going to fully substitute for a professional art team.
I swear people just naturally underestimate just how much goes into making professional media.
>GANs are never going to fully substitute for a professional art team.
First off, diffusion models aren't GANs, and are provably much better than them at generation (there was even a research paper done on this). There's a way higher degree of control, which means professionals can control their process waay better than they could with GANs.
Secondly, OP may be a bit too aggressive with their 6-month timeline. But you said "never". People don't underestimate media, you underestimate AI and its potential. Do you seriously believe AI will **never** fully substitute a professional team? Not in a decade? Multiple decades? Multiple centuries?
What an incredibly naive and shortsighted opinion.
Yeah, claiming never is a pretty ignorant statement. In fact, AI has also been shown to have superior originality in initial recent studies as AI has begun to develop for things like art and eventually we will likely reach a point where you can propose a scenario or genre and the AI will be able to create it immediately. Thus it is only natural we can use AI to create stuff, ourselves, in the coming future.
I think some of these people are simply in denial atm.
I know every single step of the production process that goes into making movies. I also have experience with 3D modeling and some special effects creation. I understand perfectly how much work is involved, which is why I'm telling you that it will be replaced. It's not because it's easy work, but because it's hard, expensive, and tedious work! Yes, not every pixel will be perfect, and there may be a few glitches in the animation, but it will be 10,000 times cheaper to produce. You can't compete with that, and I can clearly see that we are getting closer to that reality with each passing day.
Not only will movies be replaced, but the entire concept of how we interact with movies will be new. Each of us, even after the movie ends, will have the opportunity to chat with our favorite characters and even carry them into other movies. We can place them in various, crazy environments and see how our favorite characters try to adapt. It is possible to do this with current-day technologies; it will be a bit janky, but it's possible. Just wait a few years [(It's already possible to talk with your characters)](https://www.youtube.com/watch?v=enWO16x6tRM) , add visual novel style and you are good to go... I am telling that you could be a God in your movie in your characters life, not a person just looking from a far. You can't tell me that today's media have anything who can compete with that. Few times it was tried with some TV shows which allowed viewers to make decisions but it was just too expensive and limited in scope.
In the not too distant future
You will be able to take any movie
The best script, best director, highest budget, etc...
And with the power of AI
Replace all characters
With big boob waifus
Because OP is claiming we'll get commercially viable animation in 6 months, but it took longer than that to get commercially viable photos, actually good animation is WAY harder.
How are there still legitimate skeptics of AI’s potential among StableDiffusion subscribers after the insane progress we’ve seen in just the last 18 months? OK it might not be 6 months, but I could legitimately see commercially viable AI animation in 1-2 years, which is insanely soon. That’s literally just 1-2 major production cycles for major media companies.
Nothing wrong with lowering expectations and chilling. 1 year ago people were saying the same thing about how we'll have the ability to make full AI shows by the end of 2023. And while there has been about 4 major breakthroughs during that time, it ain't as fast as what those people were hyping it out be.
Limited animation is a form of animation too. If an animator added narrative and acting to this type of "slideshow" of moving pictures, they could produce something akin to the limited animation of cheap Hanna Barbera productions.
It might not be the best animation technically, but it could be commercially viable. People are already watching YouTube videos composed of AI animations that could be cynically be called "just slideshows." That's commercial viability right there. Flash animations used to have a lot of commercial viability too, even when their quality was obviously far below traditional commercial animation. Just because a cheap form of animation looks mostly like cheap crap doesn't mean that it's not viable commercially.
It took quite a while to go from still images to this. To assume that the entire field of animation will be solved in 6 months is dumb as heck. It shows a massive lack of understanding in the complexity of expressing motion, never mind doing it with a cohesive style across many shots.
We are far more sensitive to oddities in motion than in images. Our brain is more open to extra fingers or eyes than it is to broken unnatural movement. It’s going to have to get much closer to solving motion to be commercially viable. Assuming we are talking about actually producing work comparable to what is crafted by humans professionally.
Humans create oddities in animation/graphic work already. Modern CGI is full of uncanny valley and poor physics implementations, see the train carriage in the Godzilla movie.
You’re not really saying anything other than “more development is required”, which is a different way of saying the same thing you’re arguing against. The development is happening and it is improving at a rapid rate.
> It shows a massive lack of understanding in the complexity of expressing motion, never mind doing it with a cohesive style across many shots.
Slightly rephrasing this, you get the arguments that were made ~2 years ago for why image generation is so difficult (how can one part of the image have proper context of the other, it won't be consistent!) There is immense complexity in current image generation that already has to handle the hard parts of expressing motion (like how outpainting can be used to show the same cartoon character in a different pose), and physics (one cool example was an early misunderstanding DALLE2 had when generating rainbows and tornados, they would tend to spiral around the tornado like it was getting sucked in.) It's not a trivial leap from current models, but it's a very expected leap. The right data is very important here, but vision models which can now label every frame in a video with detailed text may unlock new training methods (there are so many ideas here, they are being tried, some of them will likely succeed.)
That's not how this works, video methods are different than image methods sometimes. 6 months of image gen to image gen saw massive improvements. Video gen has been around for a while, so 6 months of video gen improving on video gen is huge.
so far:
did some animations of paintings for a castle museum,
did a 8 minute history of fashion for fashionweek,
did preproduction work on a sci-fi movie about ai,
did two workshops for a production company about SD,
did a flower themed music video that is also title track for a new crime thriller movie coming out soon,
and right now i'm working on a series of images of robots for a cover for a new album for a well sized duo making electronic music.
I've been doing design and "creative technology" stuff for almost 25 years, so it's mainly just that the clients I already had now are asking for ai stuff, because I show them ai experiments I'm doing and I always had curious clients. But honestly I think some guys are much much better at finding clients than me, I'm by no means the best out there at anything.
> and right now i'm working on a series of images of robots for a cover for a new album for a well sized duo making electronic music.
Wow, that's **DAFT** you crazy **PUNK**...
Interesting. I've done a few suprisingly big projects as well, including a commercial for a huge company (for social media — small scale) where it was 100% AI.
We should connect. I get inquiry I can't serve at the moment, and my network leans too heavily Midjourney/Runway/Pika over SD.
> 6 months
not gonna happen. Early milestones are easy. For comparison, look at automated driving, where everyone is having a really hard time on the final hurdles, which are REALLY difficult to overcome.
I assume similar problem will crop up with AI animation when it comes to trying to incorporating real **action** and **interaction** instead of just static moving images.
(show me a convincing AI animation of someone eating a meal with fork and knife and I might change my mind)
To generate a complex scene, an AI has to understand it. The context, the whys and hows. That's part of the reason diffusion models find text and interactions like eating and fighting tricky. An even harder task would be to generate a coherent, consistent multipanel comic book. Extended animation would be as hard or harder than that.
The thing is, it's possible that these things will be solved in the not-too-distant future. One could imagine multimodal GPT-6 being able to plan such a thing. But if an AI is able to understand how to manipulate and eat spaghetti or generate a comic book then it can also do a lot of other things that the world is absolutely not ready for.
Basically, custom AI-generated movies will only exist if the world is just about to get very strange and terrifying.
> (show me a convincing AI animation of someone eating a meal with fork and knife and I might change my mind)
When Will Smith finally eats that spaghetti animators can start worrying.
Long way to go. No offense, but as far as animation goes, this is really a glorified slideshow with a few ambient effects. In time that will change of course.
**Defaulted to one day.**
I will be messaging you on [**2024-01-05 23:15:03 UTC**](http://www.wolframalpha.com/input/?i=2024-01-05%2023:15:03%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/StableDiffusion/comments/18ylel0/im_calling_it_6_months_out_from_commercially/kgco6cg/?context=3)
[**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FStableDiffusion%2Fcomments%2F18ylel0%2Fim_calling_it_6_months_out_from_commercially%2Fkgco6cg%2F%5D%0A%0ARemindMe%21%202024-01-05%2023%3A15%3A03%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2018ylel0)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
As a tool to speed up normal animation work, maybe. As a full replacement to do the whole thing.. bruh.. There is barely any motion in these, just like the same shit that all the video stuff was 6 months ago. Some consistency progress has been made but nowhere remotly close to compete with regular animation for atleast a few years.
Comics of all sorts would benefit greatly from current tech, but i imagine the general "oh god, ai" sentiment that stuff like the opening for Secret invasion got will keep the tech 'taboo' for a while even there. Especially given how many braindead "artists" there are out there that dont get that AI is there for them to use to make their work, not to replace them.
Yes, and using this that someone recently shared [https://www.reddit.com/r/StableDiffusion/comments/18x96lo/videodrafter\_contentconsistent\_multiscene\_video/](https://www.reddit.com/r/StableDiffusion/comments/18x96lo/videodrafter_contentconsistent_multiscene_video/)
Means we will have consistent characters, environments, and objects (like cars, etc.) between scenes and they're moving much further beyond mere camera movement to actual understanding the actions of a description (like a person washing clothes, or an animal doing something specific, etc.).
Just for easier access and those that might overlook it it links to a hugging page but there is another link there to this more useful page of info [https://videodrafter.github.io/](https://videodrafter.github.io/)
But that video literally shows that it's not consistent at all, there's a shit ton of warping and changing. And despite what you're claiming, all those examples are super static.
You misunderstood. You're confusing quality of the generations with prompt and detail consistency between scenes as well as actions.
When you look at their examples they're clearly the same people, items, and environments between different renders. The prompt will understand actor A, Bob, or however you use him from one scene to the next as the same person for rendering. The same applies to, say a certain car model/paint job/details like broken mirror, etc. or a specific type of cake. That living room layout? The same each time they revisit the living room. Yes, the finer details are a bit warped as it still can improve overall generation just like other video generators and even image generators but that is less important than the coherency and prompt achievements here. It also recognizes actual actions like reading, washing something, or other specific actions rather than just the basic panning many tools currently only offer (though Pika 1.0 has dramatically improved on this point as well).
They're short frame generations so of course they're relatively static. The entire point is this technique is able to make much longer sequences of animation with this tech as it matures which is the current big bottleneck in AI video generation due to inability to understand subjects in a scene, context, and consistency. It is no surprise it didn't come out day 1 perfect and the end of AI video development.
EDIT: The amount of upvotes the above post is getting indicates a surprising number of people aren't reading properly and doing exactly what is mentioned in my first paragraph confusing what the technology is intended for.
Not necessarily. As long as it can understand the prompts, context, and concepts like physics, kinematics, etc. it can actually do so without such extreme training. This is the benefit of an approach like Dall-E 3 vs SD's approach, but it is also more complex to develop though we've been seeing real strides such as this videodrafter or Pika 1.0.
As for open source... oh boy, I don't expect such quality and tech available anytime soon, especially since SD is simply so far behind at this point its absurd while emad's ego runs the company into the ground.
>Not necessarily. As long as it can understand the prompts, context, and concepts like physics, kinematics, etc. it can actually do so without such extreme training.
So you're expecting an extremely powerful (better than GPT-4) open source LLM to be combined with an extremely powerful open source video generator that would need to be light years ahead of what we're seeing today?
C'mon... Sometimes you guys sound downright delusional.
Are you just ill in the head? The resource I posted and Pika 1.0 already do what you claim is impossible. Already does it. Now. Not in the future, but the present. Does not require anything light yeras ahead and GPT-4 is irrelevant from this so why you brought that up beyond you just being, ironically, delusional is beyond me. It is, and has been, clear you don't understand the fundamentals behind the involved technologies. Why you are posting here is anyone's guess.
Granted, I definitely remember you, the guy who said this was impossible like a month ago and uh...
Wow, your predictions freaking suck. You got demolished then and here we are a month later showing just how absurdly delusional you have been.
EDIT: He was so embarrassed he posted a response and then immediately blocked me to have the "last word" and make it look like he is correct to anyone viewing as if I couldn't refute him. Seriously, repeatedly showing he has serious issues. Sadly, he clearly has never seen Pika Labs 1.0 current new quality offerings. I can only pity the guy at this point.
>Are you just ill in the head? The resource I posted and Pika 1.0 already do what you claim is impossible.
No, it doesn't. It's not anywhere even close to where it needs to be. There's a good chance it won't be good enough for decades.
If you want (just in case) to try a real quick&dirty solution, check this: https://mangatv.shop/story/photo-realistic-young-sam-inspired-by-detective-tales-investigates-the-disappearance-of-cookies-his-quest-leads-him-to-a
And yep, it is a slideshow, I'm thinking about Video Diffusion but this is $0.2 for 4 seconds, too much for my unit economy
Awesome work! I’m thinking there will pushback from commentators regarding linear narratives, but the way we can consume content can shift in ways culture didn’t expect and emerging demonstrations like this can be at the forefront of narratives through technologies like VR, AR, Mixed Reality, holograms.
It seems like we are getting a major advancement every 4 months right now. Stability (emad) made it clear a year ago in a press event that their planned direction is in animation and 3d models. We are seeing that with the models being released both directly and adjacent to stable diffusion.
I think in the short term we are going to see SVD training create a jump in video. Right now it seems to struggle with complex and animated images.
I actually hate the Hollywood trend of 2-3 second shots but it does allow for AI to slip in somewhat in the vein of our busy cameras. Still a lot of challenges here, like persistent details and settings and keeping things more grounded and less psychedelic, but that might be doable if one is clever enough I suppose.
The real mastery is going to be when we can create something like a Casablanca where we're not just constantly sucking the DP's dick and treating the audience like infants that don't know where to look. When we're able to hold a busy shot for a minute or two and let the world exist inside the frame without things going nuts. Or have Jackie Chan style action instead of cutting every single "punch".
The SD parts of this were using [Leonardo.ai](https://Leonardo.ai) and EverArt.
**Workflow:** Midjourney/Leonardo/Dalle-3 --> Photoshop/Canva (sometimes) --> Magnific (sometimes) --> Runway Gen 2 | Trained a model on those images using EverArt | ElevenLabs (speech-to-speech) | Lalamu Studio for lip-sync
This is so dope!!!
Im super curious how was that image made the one right sfter the title screen, the one that looks like a super market, been trying to achieve an anime style street scenes, can't seem to achieve it..
Thanks! The SD parts of this were using Leonardo.ai and EverArt.
Workflow: Midjourney/Leonardo/Dalle-3 --> Photoshop/Canva (sometimes) --> Magnific (sometimes) --> Runway Gen 2 | Trained a model on those images using EverArt | ElevenLabs (speech-to-speech) | Lalamu Studio for lip-sync
Im just glad Im not into anime. The genre is already plagued by excessive corner-cutting, and they will definitely try to ride this thing into their grave.
Commercially viable in 6 months? No, there are already too many issues with image generation, we are lacking major tools in order to make good looking animation or video
it depends on your definition of **commercially viable** . Course people are making money from ai video already for months. Ai images - for years already.
That’s amazing, do you have any tips? I’m trying to use AI to animate our D&D sessions.
Of course not at a professional level, yet, it’s just a hobby now. But any tips would be helpful
Edit: oh shit, you have a whole YouTube channel. That’ll be super helpful
I don’t want to just push you to my course, but I do have an AI animation course with a couple of free lessons https://aianimationacademy.thinkific.com/courses/AIAnimation
It’s hard to just name a tip that’d be meaningful without knowing what your challenges are, but if you have something specific you want to work on, I’m happy to reply here
I’ll be the pessimist and say it’s at least 2 years off. If not more. I work in the animation industry and there are currently too many issues for it to be “Commercially” viable.
Id say in 6 months we’ll likely see indie stuff starting.
Grain of salt: I’m always open to being proven wrong.
It's still too slow, even with high end GPUs like 4090 it takes around one minute to generate one clip from one image.
But I agree, in 6-12 it will be completely solved.
TBH the progress seems to be more exponential now, so you might be right. We need people who can build smarter systems and do animation like how we do, and then probably a few other problems to solve with consistency.
AI animation could be utilized right now in commercial animated productions. It already has in some limited cases.
It's just a matter of time until we see the first narrative feature to be made entirely of AI generated animations. Even if no more new tools came out, it would happen with just what is already here. We don't need more advanced AIs or anything, we just need more time.
Is there a market for generated movies that people didn’t craft? Personally I find it a fun tool to play around with and visualize stuff, but I’d never pay money for it.
You can do it now, if you seperately generate the characters from the background.
You just need pose animations and use pose detection. Mask out the characters and place them in the generated background. That way, it's only the characters that have poor temporal consistancy. Instead of the whole frame.
I think you're right. It's very time-consuming though, as you're going to run into a lot of lighting and color grade issues trying to make the layers blend well.
But it's still way more efficient than traditional animation.
I'd say could be less than 6 months before using commercially. The first thing is, to be used in commercially, the entire frame need not be the film. I mean, different elements in the cg/VFX shot can be animated using this. I have used video ai already in my film project and no one knows.. 😁 that's because I used it as a element in my 3d scene.
But just that saved me lot of time and energy and also helped me make something new.
So 6 months, it's gonna be something else, if controlnets and vram requirments are taken care of.
And we can't tell if any other company comes up with mindfick update to any of the current ai videos, or maybe we upgrade image process workflow to video, new controlnets for consistency and motion. Anything is possible with ai, but one thing for sure... It's gonna be faster than u expect... Exponential growth.. exciting times. Can't imagine how the work output will be in an year
I don't think so for animation... But I would really like a Comic or Manga with slightly animated cells. Something between real animation and static comics
maybe a tad longer still got a bit of artifacts and stuff same with "realistic" videos but we are getting close! in 10-15 years maybe the next blockbuster movie can be made entirely by YOU!!!!!!
That has some potential (joined sub).
You probably know more about comics and certainly understand what you're making better than I do, but just as some comic-naive feedback: I hope you're planning on adding some more sophisticated page layouts. Interesting panel aspect ratios, positions, overlaps, framing variations, and flow across the page really juice up a comic. Also, the 'camera' angles for each panel feel like they are lacking something as compared to commercial comics. I don't know enough about composition to articulate it precisely (it feels a bit monotonous and somewhat disorienting in places), but if you have enough control over the generation it feels like that might be an area that could be juiced-up a bit.
Thanks for the feedback. It has evolved a lot since it’s initial inception and chapter 3 which is coming soon I’m really trying to push some new ideas. Appreciate you checking it out!
Yeah, it'll be great just to have it read a book and process it into a cartoon.
Translate the TTS, identify characters and maintain a voice for them.
Determine the type of voice for each character.
Manage each character with a seed. Keep the consistency of locations, and backgrounds with their own seeds.
Change the locations based on time, and age the characters based on dates in the story.
On one hand, it has a chance to elevate what would be nonexistent or decent works to better ones. On the other hand, it has a significantly higher chance of flooding various markets with subpar garbage, making finding good, new, media even more difficult.
True convincing movement is orders of magnitude more difficult than this. Google’s Videopoet is the best I’ve seen but they can only do around 2 seconds (probably more behind closed doors).
Is there any good background music generation as well? Then I might even take a shot to put an idea of a game I had for a long time into fruition. But I do need music for various situations including epic and mystery themes.
We're definitely close.
But are we "self-driving car" close? The kind of close that never actually gets there because of some last problem, or the realization that there are a billion edge cases to handle?
It's definitely interesting watching the development, but the future has a way of not turning out the way you think it will.
EverArt (uses SD) is the secret to a lot of the consistency in getting, with the old man in particular.
You can sometimes get two in the same shot, rarely 3, but very high chance of wonkiness.
There’s some shots with the old man’s back facing the camera and a tree monster that’s close enough to the other shots that is an example of it getting two in one shot.
Really boring animation, perhaps.
Quality is getting there, but control and dynamism are very, very lacking.
Dynamism is easier to solve, I believe, but control is difficult. It'll take a while before AI can precisely understand how you want different characters to behave, How to achiece precise camera controls, convey clear and coherent emotions, do speech motions, etc.
A hybrid between an animation and a visual novel, on the other hand, is entirely possible. Not nearly anywhere close to as good as full anime or a pixar film, but a different way of storytelling nonetheless.
Not a chance. This is like 5 frames of motion in any given scene and even that looks shit. AI still can't get hands right and you think it's going to learn full motion in the next 6 months?
There has been literal garbage in commercial animation for decades now, AI have been commercially viable since almost day one, but being commercially viable for ***good*** animation is another thing entirely which I don't think you fully understand the difference of.
Well you're calling it badly. This is nowhere near commercially viable animation (there's barely any animation at all) and it won't be in 6 months either. I'm calling it.
Visual novels going to be epic in the next year.
That's what I think for now. Letting writers show their work and previs. We can't even get a walking talking animation correct let alone a fight scene. But other things like backgrounds and non organic movement (spaceships) it's there or it can do half the job.
>We can't even get a walking talking animation correct let alone a fight scene 4:50, that guy has an extra finger, for gods sake! xD
I just happened to come across this Starwars animation on the stableDiffusion subreddit, which is among the most impressive ones I have seen up to date. [https://www.reddit.com/r/StableDiffusion/comments/1906h3o/star\_wars\_death\_star\_explosion\_anime\_style/](https://www.reddit.com/r/stablediffusion/comments/1906h3o/star_wars_death_star_explosion_anime_style/) While this particular example may be impressive, I still agree with what you said. It would be cool to see a model trained on cheap 80's and 90's OVA, which, for my taste, totally had a charm of their own, by the way they were exploiting very basic animation techniques for very effective storytelling.
I know of a video game franchise where this kind of AI animation would work extraordinarily well for -- Disgaea. The entire franchise is mostly still images with slight movements. Hell, what I saw in this video is better than what I typically see in their games. If they can get an AI model to do most of the work, the cost to animate their games would probably decrease enormously. And they could probably use it. The company that makes the games is very small and almost went bankrupt once already.
The current problem is copyright, you need to prove that you own all material that the model was trained on (at least that's how it worked on steam and similar last time I checked) otherwise you're too big of liability. So if you don't already have enough art/assets to train the model on, it doesn't make sense to rely on it, and if you do, well then you already have tons of art to use. Good way to ease the workload for the franchise though.
Bruh imagine visual novel ai book readers with an epic voice and ai generated backgrounds that sorta match the scene.
Blockbuster Ao3! lets goooo
Or illustrated podcast drama series.
Making consistent content made with poop is commercially viable these days. AI has been assisting animation for like a year already. You just don’t notice it because people are too busy making things with it.
We noticed it, it was the opening sequence of Secret Invasion, and it was fucking garbage. I think their model had four entire photos in it.
When it is used well you do not notice it* Good artists can hide it well.
Yup. The real line in the sand is what people "count". I can anecdotally confirm we've been using AI in some form for years now. Full screen hero animated characters in AAA films, maybe not. BG elements and grunt work? Absolutely at *least* since 2018. We're using AI for roto and upscaler/denoisers these days. On the art side it's much more complicated and different companies are testing different levels of legal flexibility: Liability if you train across IP's even if you own both, exposure to litigation if you use a 3rd party, some other things I cant remember. (This is all off the top of my head from a company meeting a year ago, no idea where it stands now). *Personally* I predict art teams will be training focused proprietary models as a compliment to standard workflows for some time. For pros there is definitely a point where text-to-prompt is more effort than getting what you want 'the old way'.
These are still just slide shows, relax
Better than the big boob waifus that get upvoted here every day
Sure, but that's missing the point. Which is that what op posted is nice, but its not even close to actual animation just because there's slight motion in the pictures.
Omg you people are crazy, how can't you see a bigger picture, few year ago there was some pixels blob, now this. Can you imagine what would happen in another few years I would say, but clearly you can't...
RemindMe! One Year
Six months
The problem with people is their own egos are too large. They can't believe that after thousands of years of being the dominant intelligent species on the planet their creativity is about to be upstaged by "machines". Logic and reasoning is ineffective against these people. They're pure creatures of emotion.
Oh man, your words are like a light at the end of a long tunnel. Reasonable and forward thinkers are rare in this world. It's not only that people are going to be upstaged, they are going to be crushed. In 10 to 20 years, people will become useless in most fields compared to AI. At the same time, productivity will soar through the roof. I can't even imagine how the world is going to look then, and thankfully, I am not responsible for finding jobs for people without a job.
Yeah, I'm sure if current progress continues, sometime this century AI is going to be better than humans in every single field. There's no need to continue with the traditional approach to the economy and jobs at that point, we've been long overdue for a paradigm shift. >Reasonable and forward thinkers are rare in this world. It's a good thing forums like reddit exist where you can occasionally discuss topics with like-minded people. I recommend checking out r/singularity
GANs are never going to fully substitute for a professional art team. I swear people just naturally underestimate just how much goes into making professional media.
>GANs are never going to fully substitute for a professional art team. First off, diffusion models aren't GANs, and are provably much better than them at generation (there was even a research paper done on this). There's a way higher degree of control, which means professionals can control their process waay better than they could with GANs. Secondly, OP may be a bit too aggressive with their 6-month timeline. But you said "never". People don't underestimate media, you underestimate AI and its potential. Do you seriously believe AI will **never** fully substitute a professional team? Not in a decade? Multiple decades? Multiple centuries? What an incredibly naive and shortsighted opinion.
Yeah, claiming never is a pretty ignorant statement. In fact, AI has also been shown to have superior originality in initial recent studies as AI has begun to develop for things like art and eventually we will likely reach a point where you can propose a scenario or genre and the AI will be able to create it immediately. Thus it is only natural we can use AI to create stuff, ourselves, in the coming future. I think some of these people are simply in denial atm.
I know every single step of the production process that goes into making movies. I also have experience with 3D modeling and some special effects creation. I understand perfectly how much work is involved, which is why I'm telling you that it will be replaced. It's not because it's easy work, but because it's hard, expensive, and tedious work! Yes, not every pixel will be perfect, and there may be a few glitches in the animation, but it will be 10,000 times cheaper to produce. You can't compete with that, and I can clearly see that we are getting closer to that reality with each passing day. Not only will movies be replaced, but the entire concept of how we interact with movies will be new. Each of us, even after the movie ends, will have the opportunity to chat with our favorite characters and even carry them into other movies. We can place them in various, crazy environments and see how our favorite characters try to adapt. It is possible to do this with current-day technologies; it will be a bit janky, but it's possible. Just wait a few years [(It's already possible to talk with your characters)](https://www.youtube.com/watch?v=enWO16x6tRM) , add visual novel style and you are good to go... I am telling that you could be a God in your movie in your characters life, not a person just looking from a far. You can't tell me that today's media have anything who can compete with that. Few times it was tried with some TV shows which allowed viewers to make decisions but it was just too expensive and limited in scope.
debatable
In the not too distant future You will be able to take any movie The best script, best director, highest budget, etc... And with the power of AI Replace all characters With big boob waifus
What a time to be alive
Have you just invented mobile gacha games?
Yeah, agreed. I find mine difficult to fap to.
Big boob waifus are the driving force behind developing AI.
Slideshows are content. People would totally watch a great story based on slideshows on YT.
Exists. Motion comics.
I mean, people watch people react to people eating fastfood on YT. The bar is pretty much lying on the ground.
I don't know why you're being down voted, this is literally what comics are.
Because OP is claiming we'll get commercially viable animation in 6 months, but it took longer than that to get commercially viable photos, actually good animation is WAY harder.
How are there still legitimate skeptics of AI’s potential among StableDiffusion subscribers after the insane progress we’ve seen in just the last 18 months? OK it might not be 6 months, but I could legitimately see commercially viable AI animation in 1-2 years, which is insanely soon. That’s literally just 1-2 major production cycles for major media companies.
I think it will be closer to 2 years but your not wrong
When I saw this post, I thought 2 years as well.
Nothing wrong with lowering expectations and chilling. 1 year ago people were saying the same thing about how we'll have the ability to make full AI shows by the end of 2023. And while there has been about 4 major breakthroughs during that time, it ain't as fast as what those people were hyping it out be.
Limited animation is a form of animation too. If an animator added narrative and acting to this type of "slideshow" of moving pictures, they could produce something akin to the limited animation of cheap Hanna Barbera productions. It might not be the best animation technically, but it could be commercially viable. People are already watching YouTube videos composed of AI animations that could be cynically be called "just slideshows." That's commercial viability right there. Flash animations used to have a lot of commercial viability too, even when their quality was obviously far below traditional commercial animation. Just because a cheap form of animation looks mostly like cheap crap doesn't mean that it's not viable commercially.
Anime is 3 images per second
This is good, but it's only like motion comics level.
[удалено]
It took quite a while to go from still images to this. To assume that the entire field of animation will be solved in 6 months is dumb as heck. It shows a massive lack of understanding in the complexity of expressing motion, never mind doing it with a cohesive style across many shots.
There's a HUGE difference between 'commercially viable' and 'solving animation'. Nice try tho lol
We are far more sensitive to oddities in motion than in images. Our brain is more open to extra fingers or eyes than it is to broken unnatural movement. It’s going to have to get much closer to solving motion to be commercially viable. Assuming we are talking about actually producing work comparable to what is crafted by humans professionally.
Humans create oddities in animation/graphic work already. Modern CGI is full of uncanny valley and poor physics implementations, see the train carriage in the Godzilla movie. You’re not really saying anything other than “more development is required”, which is a different way of saying the same thing you’re arguing against. The development is happening and it is improving at a rapid rate.
> It shows a massive lack of understanding in the complexity of expressing motion, never mind doing it with a cohesive style across many shots. Slightly rephrasing this, you get the arguments that were made ~2 years ago for why image generation is so difficult (how can one part of the image have proper context of the other, it won't be consistent!) There is immense complexity in current image generation that already has to handle the hard parts of expressing motion (like how outpainting can be used to show the same cartoon character in a different pose), and physics (one cool example was an early misunderstanding DALLE2 had when generating rainbows and tornados, they would tend to spiral around the tornado like it was getting sucked in.) It's not a trivial leap from current models, but it's a very expected leap. The right data is very important here, but vision models which can now label every frame in a video with detailed text may unlock new training methods (there are so many ideas here, they are being tried, some of them will likely succeed.)
That's not how this works, video methods are different than image methods sometimes. 6 months of image gen to image gen saw massive improvements. Video gen has been around for a while, so 6 months of video gen improving on video gen is huge.
Bro, it paid my rent the last 6 months.
What sort of work/clients have you been getting. What sort of work do they want you to do for them?
so far: did some animations of paintings for a castle museum, did a 8 minute history of fashion for fashionweek, did preproduction work on a sci-fi movie about ai, did two workshops for a production company about SD, did a flower themed music video that is also title track for a new crime thriller movie coming out soon, and right now i'm working on a series of images of robots for a cover for a new album for a well sized duo making electronic music.
Need to get like you! How do you find clients? I have all the technicals nailed but I’m not sure how to find clients.
I've been doing design and "creative technology" stuff for almost 25 years, so it's mainly just that the clients I already had now are asking for ai stuff, because I show them ai experiments I'm doing and I always had curious clients. But honestly I think some guys are much much better at finding clients than me, I'm by no means the best out there at anything.
Ah I see. Good job!
> and right now i'm working on a series of images of robots for a cover for a new album for a well sized duo making electronic music. Wow, that's **DAFT** you crazy **PUNK**...
Interesting. I've done a few suprisingly big projects as well, including a commercial for a huge company (for social media — small scale) where it was 100% AI. We should connect. I get inquiry I can't serve at the moment, and my network leans too heavily Midjourney/Runway/Pika over SD.
[https://www.instagram.com/the.emperor.of.antarctica/](https://www.instagram.com/the.emperor.of.antarctica/) always open to talk about work :)
Awesome work.
Amazing showcase! Did you create this all using SD ?
> 6 months not gonna happen. Early milestones are easy. For comparison, look at automated driving, where everyone is having a really hard time on the final hurdles, which are REALLY difficult to overcome. I assume similar problem will crop up with AI animation when it comes to trying to incorporating real **action** and **interaction** instead of just static moving images. (show me a convincing AI animation of someone eating a meal with fork and knife and I might change my mind)
To generate a complex scene, an AI has to understand it. The context, the whys and hows. That's part of the reason diffusion models find text and interactions like eating and fighting tricky. An even harder task would be to generate a coherent, consistent multipanel comic book. Extended animation would be as hard or harder than that. The thing is, it's possible that these things will be solved in the not-too-distant future. One could imagine multimodal GPT-6 being able to plan such a thing. But if an AI is able to understand how to manipulate and eat spaghetti or generate a comic book then it can also do a lot of other things that the world is absolutely not ready for. Basically, custom AI-generated movies will only exist if the world is just about to get very strange and terrifying.
> (show me a convincing AI animation of someone eating a meal with fork and knife and I might change my mind) When Will Smith finally eats that spaghetti animators can start worrying.
Long way to go. No offense, but as far as animation goes, this is really a glorified slideshow with a few ambient effects. In time that will change of course.
> In time that will change of course. Maybe. Our exciting initial progress is likely to stall and / or plateau for years at some point.
RemindMe! 5 years.
**Defaulted to one day.** I will be messaging you on [**2024-01-05 23:15:03 UTC**](http://www.wolframalpha.com/input/?i=2024-01-05%2023:15:03%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/StableDiffusion/comments/18ylel0/im_calling_it_6_months_out_from_commercially/kgco6cg/?context=3) [**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FStableDiffusion%2Fcomments%2F18ylel0%2Fim_calling_it_6_months_out_from_commercially%2Fkgco6cg%2F%5D%0A%0ARemindMe%21%202024-01-05%2023%3A15%3A03%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2018ylel0) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
Workflow??
As a tool to speed up normal animation work, maybe. As a full replacement to do the whole thing.. bruh.. There is barely any motion in these, just like the same shit that all the video stuff was 6 months ago. Some consistency progress has been made but nowhere remotly close to compete with regular animation for atleast a few years. Comics of all sorts would benefit greatly from current tech, but i imagine the general "oh god, ai" sentiment that stuff like the opening for Secret invasion got will keep the tech 'taboo' for a while even there. Especially given how many braindead "artists" there are out there that dont get that AI is there for them to use to make their work, not to replace them.
6 years more like it
Yes, and using this that someone recently shared [https://www.reddit.com/r/StableDiffusion/comments/18x96lo/videodrafter\_contentconsistent\_multiscene\_video/](https://www.reddit.com/r/StableDiffusion/comments/18x96lo/videodrafter_contentconsistent_multiscene_video/) Means we will have consistent characters, environments, and objects (like cars, etc.) between scenes and they're moving much further beyond mere camera movement to actual understanding the actions of a description (like a person washing clothes, or an animal doing something specific, etc.). Just for easier access and those that might overlook it it links to a hugging page but there is another link there to this more useful page of info [https://videodrafter.github.io/](https://videodrafter.github.io/)
But that video literally shows that it's not consistent at all, there's a shit ton of warping and changing. And despite what you're claiming, all those examples are super static.
You misunderstood. You're confusing quality of the generations with prompt and detail consistency between scenes as well as actions. When you look at their examples they're clearly the same people, items, and environments between different renders. The prompt will understand actor A, Bob, or however you use him from one scene to the next as the same person for rendering. The same applies to, say a certain car model/paint job/details like broken mirror, etc. or a specific type of cake. That living room layout? The same each time they revisit the living room. Yes, the finer details are a bit warped as it still can improve overall generation just like other video generators and even image generators but that is less important than the coherency and prompt achievements here. It also recognizes actual actions like reading, washing something, or other specific actions rather than just the basic panning many tools currently only offer (though Pika 1.0 has dramatically improved on this point as well). They're short frame generations so of course they're relatively static. The entire point is this technique is able to make much longer sequences of animation with this tech as it matures which is the current big bottleneck in AI video generation due to inability to understand subjects in a scene, context, and consistency. It is no surprise it didn't come out day 1 perfect and the end of AI video development. EDIT: The amount of upvotes the above post is getting indicates a surprising number of people aren't reading properly and doing exactly what is mentioned in my first paragraph confusing what the technology is intended for.
In 5 years we are able to type any prompt we want and get a movie.
I don't want a single prompt, I want to put a whole book in and get either the whole thing as a movie or each chapter as a tv show.
I didn't say how long the prompt was :)
Who will own that technology? They will censor what you can make.
If we are lucky, no one, it will be open source.
There’s no way. It would have to be trained on actual movies for that to happen. The film studios will go scorched Earth and that’ll be the end of it.
Not necessarily. As long as it can understand the prompts, context, and concepts like physics, kinematics, etc. it can actually do so without such extreme training. This is the benefit of an approach like Dall-E 3 vs SD's approach, but it is also more complex to develop though we've been seeing real strides such as this videodrafter or Pika 1.0. As for open source... oh boy, I don't expect such quality and tech available anytime soon, especially since SD is simply so far behind at this point its absurd while emad's ego runs the company into the ground.
>Not necessarily. As long as it can understand the prompts, context, and concepts like physics, kinematics, etc. it can actually do so without such extreme training. So you're expecting an extremely powerful (better than GPT-4) open source LLM to be combined with an extremely powerful open source video generator that would need to be light years ahead of what we're seeing today? C'mon... Sometimes you guys sound downright delusional.
Are you just ill in the head? The resource I posted and Pika 1.0 already do what you claim is impossible. Already does it. Now. Not in the future, but the present. Does not require anything light yeras ahead and GPT-4 is irrelevant from this so why you brought that up beyond you just being, ironically, delusional is beyond me. It is, and has been, clear you don't understand the fundamentals behind the involved technologies. Why you are posting here is anyone's guess. Granted, I definitely remember you, the guy who said this was impossible like a month ago and uh... Wow, your predictions freaking suck. You got demolished then and here we are a month later showing just how absurdly delusional you have been. EDIT: He was so embarrassed he posted a response and then immediately blocked me to have the "last word" and make it look like he is correct to anyone viewing as if I couldn't refute him. Seriously, repeatedly showing he has serious issues. Sadly, he clearly has never seen Pika Labs 1.0 current new quality offerings. I can only pity the guy at this point.
>Are you just ill in the head? The resource I posted and Pika 1.0 already do what you claim is impossible. No, it doesn't. It's not anywhere even close to where it needs to be. There's a good chance it won't be good enough for decades.
You need A LOT more than selective prallax and semianimated elements on a still frame to have a commercially viable AI animation....
This looks incredible
Thank you
I knew ai anime will be a thing at some point but I didn't think it was THIS close.
If you want (just in case) to try a real quick&dirty solution, check this: https://mangatv.shop/story/photo-realistic-young-sam-inspired-by-detective-tales-investigates-the-disappearance-of-cookies-his-quest-leads-him-to-a And yep, it is a slideshow, I'm thinking about Video Diffusion but this is $0.2 for 4 seconds, too much for my unit economy
If you mean compelling images with subtle motion then yes. Still probably at least a year or two away from complex motion such as moving humans.
Awesome work! I’m thinking there will pushback from commentators regarding linear narratives, but the way we can consume content can shift in ways culture didn’t expect and emerging demonstrations like this can be at the forefront of narratives through technologies like VR, AR, Mixed Reality, holograms.
It seems like we are getting a major advancement every 4 months right now. Stability (emad) made it clear a year ago in a press event that their planned direction is in animation and 3d models. We are seeing that with the models being released both directly and adjacent to stable diffusion. I think in the short term we are going to see SVD training create a jump in video. Right now it seems to struggle with complex and animated images.
I actually hate the Hollywood trend of 2-3 second shots but it does allow for AI to slip in somewhat in the vein of our busy cameras. Still a lot of challenges here, like persistent details and settings and keeping things more grounded and less psychedelic, but that might be doable if one is clever enough I suppose. The real mastery is going to be when we can create something like a Casablanca where we're not just constantly sucking the DP's dick and treating the audience like infants that don't know where to look. When we're able to hold a busy shot for a minute or two and let the world exist inside the frame without things going nuts. Or have Jackie Chan style action instead of cutting every single "punch".
new to animation with SD how would i make something like this?
The SD parts of this were using [Leonardo.ai](https://Leonardo.ai) and EverArt. **Workflow:** Midjourney/Leonardo/Dalle-3 --> Photoshop/Canva (sometimes) --> Magnific (sometimes) --> Runway Gen 2 | Trained a model on those images using EverArt | ElevenLabs (speech-to-speech) | Lalamu Studio for lip-sync
This is so dope!!! Im super curious how was that image made the one right sfter the title screen, the one that looks like a super market, been trying to achieve an anime style street scenes, can't seem to achieve it..
I'll take the over on that
What an amazing short, unbelievable that all made by AI
Man, this is crazy good. What was your workflow/process like?
Thanks! The SD parts of this were using Leonardo.ai and EverArt. Workflow: Midjourney/Leonardo/Dalle-3 --> Photoshop/Canva (sometimes) --> Magnific (sometimes) --> Runway Gen 2 | Trained a model on those images using EverArt | ElevenLabs (speech-to-speech) | Lalamu Studio for lip-sync
Are we there yet? u/TheReelRobot
Very much so. It’s my fulltime, well paying job now. Lots of launches next month that’ll explain more
Can't wait to follow your progress. Thank you!
u/thereelrobot it's been 6 months. What is a good animation currently?
the futures gonna suck complete ass
Im just glad Im not into anime. The genre is already plagued by excessive corner-cutting, and they will definitely try to ride this thing into their grave.
This looks great. It reminds me of the witcher video game where it had comics in it. How'd you make this? Do you have a tutorial you could link me to?
Dude. This is amazing. Great job
i really dislike art based AI applications
Commercially viable in 6 months? No, there are already too many issues with image generation, we are lacking major tools in order to make good looking animation or video
Ugh. This is depressing.
it depends on your definition of **commercially viable** . Course people are making money from ai video already for months. Ai images - for years already.
The tech is just a tool, what I care for is the story, but the story here, unfortunately, was lacking
2 weeks left before the revolution, lol
You was right
It’s my fulltime job now
That’s amazing, do you have any tips? I’m trying to use AI to animate our D&D sessions. Of course not at a professional level, yet, it’s just a hobby now. But any tips would be helpful Edit: oh shit, you have a whole YouTube channel. That’ll be super helpful
I don’t want to just push you to my course, but I do have an AI animation course with a couple of free lessons https://aianimationacademy.thinkific.com/courses/AIAnimation It’s hard to just name a tip that’d be meaningful without knowing what your challenges are, but if you have something specific you want to work on, I’m happy to reply here
That’s exactly what I was looking for, actually. Thank you so much!
How to get character consistency?
Maybe. I'd say 5 years to creating or editing a film, TV show to an extraordinary degree...maybe.
This looks like shit
I’ll be the pessimist and say it’s at least 2 years off. If not more. I work in the animation industry and there are currently too many issues for it to be “Commercially” viable. Id say in 6 months we’ll likely see indie stuff starting. Grain of salt: I’m always open to being proven wrong.
honestly just use what you got here with a smidge of aftereffects and it's already possible
IIRC, there has been an anime that was made with AI already. Forgot the name, sad enough, maybe that is because I don't watch Anime
That was only 1 video, not an actual series.
It's still too slow, even with high end GPUs like 4090 it takes around one minute to generate one clip from one image. But I agree, in 6-12 it will be completely solved.
I'm all in for less crappy CGI in anime!
Prompt ?
u/savevideo
I said 2 years one year and a half ago. You are not wrong.
So... Too late for a manwha?
TBH the progress seems to be more exponential now, so you might be right. We need people who can build smarter systems and do animation like how we do, and then probably a few other problems to solve with consistency.
AI animation could be utilized right now in commercial animated productions. It already has in some limited cases. It's just a matter of time until we see the first narrative feature to be made entirely of AI generated animations. Even if no more new tools came out, it would happen with just what is already here. We don't need more advanced AIs or anything, we just need more time.
Is there a market for generated movies that people didn’t craft? Personally I find it a fun tool to play around with and visualize stuff, but I’d never pay money for it.
How can I build like this video? Let me know tool or github address!
You can do it now, if you seperately generate the characters from the background. You just need pose animations and use pose detection. Mask out the characters and place them in the generated background. That way, it's only the characters that have poor temporal consistancy. Instead of the whole frame.
I think you're right. It's very time-consuming though, as you're going to run into a lot of lighting and color grade issues trying to make the layers blend well. But it's still way more efficient than traditional animation.
RemindMe! 6 month
I'd say could be less than 6 months before using commercially. The first thing is, to be used in commercially, the entire frame need not be the film. I mean, different elements in the cg/VFX shot can be animated using this. I have used video ai already in my film project and no one knows.. 😁 that's because I used it as a element in my 3d scene. But just that saved me lot of time and energy and also helped me make something new. So 6 months, it's gonna be something else, if controlnets and vram requirments are taken care of. And we can't tell if any other company comes up with mindfick update to any of the current ai videos, or maybe we upgrade image process workflow to video, new controlnets for consistency and motion. Anything is possible with ai, but one thing for sure... It's gonna be faster than u expect... Exponential growth.. exciting times. Can't imagine how the work output will be in an year
More than that
that sucks
It is coming, in 6 months ai will be able to generate full videos without morph effect
What do you base this on?
Really incredible!
Maybe netflix will stop canceling all the good stuff then!
why dont people just take the last frame from those short generated animations and feed it again to the animation ai to make the next part?
Which part of this was SD? EverArt? Some info on the workflow would be good
x-post this to /r/MarkMyWords
I don't think so for animation... But I would really like a Comic or Manga with slightly animated cells. Something between real animation and static comics
How do you animate the mouth to mimic words like that?
maybe a tad longer still got a bit of artifacts and stuff same with "realistic" videos but we are getting close! in 10-15 years maybe the next blockbuster movie can be made entirely by YOU!!!!!!
What model did you use?
[I am making an AI comic](https://imgur.com/gallery/u71j64o) and you better bet I’ll be trying to animate panels and certain scenes.
That has some potential (joined sub). You probably know more about comics and certainly understand what you're making better than I do, but just as some comic-naive feedback: I hope you're planning on adding some more sophisticated page layouts. Interesting panel aspect ratios, positions, overlaps, framing variations, and flow across the page really juice up a comic. Also, the 'camera' angles for each panel feel like they are lacking something as compared to commercial comics. I don't know enough about composition to articulate it precisely (it feels a bit monotonous and somewhat disorienting in places), but if you have enough control over the generation it feels like that might be an area that could be juiced-up a bit.
Thanks for the feedback. It has evolved a lot since it’s initial inception and chapter 3 which is coming soon I’m really trying to push some new ideas. Appreciate you checking it out!
Also, for the record you are the first person to follow my Subreddit. Remember this day if it ever gets popular lol
Mind sharing your workflow?
I'll take that bet. 6 months from now, we won't even have dalle3 level still image generations at home yet.
It just takes the right team of (of one maybe) of writer, director and editor, then rendering like mad.
Filling in inbetweens of keyframes seemlessly would be the greatest revolution to animation
r/terrifyingasfuck
Good job!
What ai tool was used to generate this
Honestly. I cant wait for ai to remember your character. Have this story for years but never could draw it right.
Yeah, it'll be great just to have it read a book and process it into a cartoon. Translate the TTS, identify characters and maintain a voice for them. Determine the type of voice for each character. Manage each character with a seed. Keep the consistency of locations, and backgrounds with their own seeds. Change the locations based on time, and age the characters based on dates in the story.
On one hand, it has a chance to elevate what would be nonexistent or decent works to better ones. On the other hand, it has a significantly higher chance of flooding various markets with subpar garbage, making finding good, new, media even more difficult.
What tools were used for this
True convincing movement is orders of magnitude more difficult than this. Google’s Videopoet is the best I’ve seen but they can only do around 2 seconds (probably more behind closed doors).
So Stable Diffusion wasn't used at all in the workflow?
Brace yourselves for movie fan fictions
I believe in 1-2 years, individual creators will have access to an entire suite of AI tools that can do the work of major production studios.
What program was used for this?
6 months seems aggressive
Only on Youtube
And for Zack Snyder style slow mo shots
Damned villagers burning people alive.
Is there any good background music generation as well? Then I might even take a shot to put an idea of a game I had for a long time into fruition. But I do need music for various situations including epic and mystery themes.
They should be able to handle transition shots.
We're definitely close. But are we "self-driving car" close? The kind of close that never actually gets there because of some last problem, or the realization that there are a billion edge cases to handle? It's definitely interesting watching the development, but the future has a way of not turning out the way you think it will.
What are you using here? And do you think you could have the same character in multiple shots?
EverArt (uses SD) is the secret to a lot of the consistency in getting, with the old man in particular. You can sometimes get two in the same shot, rarely 3, but very high chance of wonkiness. There’s some shots with the old man’s back facing the camera and a tree monster that’s close enough to the other shots that is an example of it getting two in one shot.
Ah cool, Thanks.
Maybe for Hallmark movies that no one actually watches. But if you can get just an ai to do it. Imagine what a person utilizing ai as a tool could do.
SCP animations are gonna get wild
18 Months is more realistic
Really boring animation, perhaps. Quality is getting there, but control and dynamism are very, very lacking. Dynamism is easier to solve, I believe, but control is difficult. It'll take a while before AI can precisely understand how you want different characters to behave, How to achiece precise camera controls, convey clear and coherent emotions, do speech motions, etc. A hybrid between an animation and a visual novel, on the other hand, is entirely possible. Not nearly anywhere close to as good as full anime or a pixar film, but a different way of storytelling nonetheless.
Not a chance. This is like 5 frames of motion in any given scene and even that looks shit. AI still can't get hands right and you think it's going to learn full motion in the next 6 months?
Is this made with stable video?
Great video. Sound is the problem. Music way to loud over voices. I didn understand a word the tree thing said.
There has been literal garbage in commercial animation for decades now, AI have been commercially viable since almost day one, but being commercially viable for ***good*** animation is another thing entirely which I don't think you fully understand the difference of.
Not in the next 10 years. Yikes you guys have no o Idea how tech improves
nice
Well you're calling it badly. This is nowhere near commercially viable animation (there's barely any animation at all) and it won't be in 6 months either. I'm calling it.