Impressive Cases of Using Artificial Imagination
Artificial imagination-related techniques have been burning hot this year. More and more Text-to-image AI programs/tools emerged to allow people to synthesize new images pretty quickly. The Dall-E 2 by OpenAI, MidJourney, and Stable Diffusion by Stability.ai are the most recent well-known text-to-image model that creates images from textual descriptions. It is conceivable that as these technologies become more mature, they may profoundly impact fields related to visual images, such as film, video, games, advertising, graphic arts, etc. Now they are just starting, and AI-driven image generation brings us a fresh feeling.
Here are two compelling cases showing the great prospect of artificial imagination.
This House Does Not Exist
This House Does Not Exist is an eye-opening case. It’s an AI-powered program developed by entrepreneur @levelsi that allows users to generate realistic images of modern architecture homes. I did house rendering through 3Ds Max and V-ray by myself, so I am surprised to see the potential AI can help generate such beautiful and realistic home images within a short time.
This project uses a latent text-to-image diffusion model to generate realistic images of modern houses. Diffusion Models are the key AI technology behind the program. Over the past year, Diffusion models have seen a surge in popularity, rapidly displacing Generative Adversarial Networks (GANs) as the preferred method for AI-based image generation.
The website of This House Does Not Exist is straightforward and fun to use. No complex information architecture or too many buttons, only focus on displaying the fantastic house images made by AI. You can view AI-created houses through building materials, like stone and glass, or through the scene. For example, if you select Beach as the house scene, house images with the Beach will filter out. Further, you can vote for the best images by clicking Upvote at the bottom of the image. You will be surprised how awesome AI can render such realistic and impressive images.
The Crow
Text-to-image is the first step; the next will be text-to-video AI generation tools. This second case shows the potential of AI generates animated scenes. Three minutes animation “The Crow,” brought by Artist Glenn Marshall, won prizes in Cannes and Linz. Glenn Marshall brings artificial imagination techniques to the video. In which he transformed a dancer from an original video ‘Painted’ short on YouTube into a crow using AI technology.
The basic technique is feeding each video frame into an AI process, which changes the image according to a CLIP-guided text-to-image prompt. After the final synthesis, as the animation describes, it is “a haunting and riveting work that follows the raven through an ephemeral dance in a barren post-apocalyptic landscape until its inevitable demise.”
The Crow is a great example to show that the images/animation area cocreates with AI can extend the human sensory experience. With a vast image database and much faster than manual image generation, the AI model has shown its fantastic potential to help artists create diverse scenes beyond human imagination. It heralds a fast change in the film materials and tools we use to make movies and videos. Perhaps sooner, artists and video makers will become accustomed to adopting Pytti Colab notebook and use the AI model, like the Stable Diffusion model, to transform video/image material and produce newer expressions.
These applications are currently in the trial stage, and if subsequent text-to-image AI tools do not help people solve the real problem, they may only be short-lived. But for artists, filmmakers, and designers, as Glenn Marshall said in an interview, the best way is to immerse in the latest technology (AI) and be part of the coding and development community that is creating art and animation that no human eye has seen before.
