There’s a reason that artificial intelligence has taken center stage in humanity’s collective mind in recent months, and it’s not simply because of the proliferation of new tools that are now ubiquitous parts of our lives. AI is not a new feature of our society; machine learning has been influencing civilization as we know it for decades.
But over the last year, some of our most brilliant and inventive minds have helped realize AI models that force us to interrogate deeply-held assumptions about the human condition. The dynamic at the core of what makes us uncomfortable about these technologies is that artificial intelligence has always been the reflective study of what makes us who we are. No other scientific field of inquiry so directly leans on the unsettling issues of our own existential identities. And that is, without hyperbole, terrifying.
This is exactly what underpins much of the reactionary, anti-AI activism that has become so common in the last year, despite claims that concerns are solely rooted in ethical and moral worries. Yes, there are legitimate concerns about ethically building and using AI-based tools — these questions only differ in degree, not kind, to ones we have always faced. But it’s hard not to notice the distinct undertone of worry that human exceptionalism has increasingly fewer places in which to hide.
If a machine can touch upon the realms of human capability that have for thousands of years been stalwart representatives and conduits of what we might think of as “the soul,” what, then, does that mean for such a concept?
What the surge of AI-based technologies in the last year has meant for our species is that we no longer have the luxury of considering these questions in the abstract. And whatever time remains for us to try to contain these conversations to the philosophical is exponentially dwindling; development in AI, by its very nature, is lightning fast.
We should welcome the challenge. Artificial intelligence is here, and it’s not going away. Beyond representing a healthy approach to challenging circumstances, embracing, engaging, and being galvanized by what is unquestionably a turning point in human history is our only chance of utilizing this technology for good. For every doomsday scenario we can think of regarding AI, there exists another potential outcome, one that is almost frightening in its ability to inspire.
At nft now, we’ve had the pleasure of watching the AI art community grow into the diverse and amazing movement that it is today. Like everyone else on the planet, we have watched in awe as we covered the most important advancements in artificial intelligence in the last year.
Today, we’re proud to launch Now AI, our new channel dedicated to the fast-paced field of AI. This weekly column will highlight the most relevant, curious, and captivating stories and developments in the space to serve the community that continues to inspire us. We hope you’ll join us for the journey, from the alarming to the enthralling.
“The rule for all terrors is head straight into them,” the great philosopher Alan Watts once observed. “When you are sailing in a storm, you don’t let a wave hit your boat on the side. You go bow into the wave — and ride it.”
In the Know: The week’s top AI stories
🔎 Using AI to supercharge scientific discovery: Study
- A new paper published in the journal Nature entitled “Accelerating science with human-aware artificial intelligence” posits that artificial intelligence can contribute to the development of scientific inquiry and discovery by generating hypotheses that humans simply can’t consider.
🛑 SEC Chair says securities laws “may be implemented” depending on AI’s use
- During remarks given at the National Press Club in Washington D.C., United States Securities and Exchange Commission (SEC) Chair Gary Gensler underscored AI’s potential to help and harm society, warning that it could lead to financial collapse. Gensler also noted that securities laws “may be implicated depending upon how AI technology is used.”
🌀 For better or worse, advertising companies are starting to embrace AI
- While AI makes creating and tracking ads easier, the industry is on edge for fear of an AI automation takeover. The associated legal and logistical challenges to making that a reality, however, are mounting.
🎨 AI art to be allowed at the 2023 Colorado State Fair
- AI art movement diehards will recognize the art fair name. Last year, Colorado resident Jason Allen entered an AI-collaborative artwork, Théâtre D’opéra Spatial, to the Colorado State Fair, winning the fair’s digital art competition and generating controversy online about AI art’s ethical implications. This year, the fair has announced that it will allow AI art in its competitions, granted it is labeled as an AI creation.
/Promptly: Roope Rainisto on How to Master AI Video
The AI-generated video revolution is here. Much like how text-to-image AI generators took the world by storm in the summer of 2022, 2023 is shaping up to be the year of text-to-video experimentation and discovery. If you’ve had the pleasure of watching the myriad uncanny AI videos that have been popping up online in recent months — occasionally parodying food commercials, like this unofficial AI-generated Pepsi ad — then you’ve likely seen the work of tools like Runway’s Gen-2 model or the more recent Zeroscope.
Runway’s official tagline for its text-to-video product offerings — “No Lights. No camera. All action.” — is at once trite and exhilarating, underscoring the potential for tools like Gen-2 to completely upend the film industry, from indie productions to high-budget Hollywood affairs. If that sounds hyperbolic, just take a look at some of the entries to Claire Silver’s fifth AI art contest, which revolves around short-form AI video and animation.
More sophisticated AI video tools and productions are going to arrive on the scene quicker than you might realize. Before they do, they’re likely to play a small but growing role in studio productions, just as text-to-image AI models did in the opening credits to Marvel’s Secret Invasion last month.
Whether you’re interested in playing around with AI video for fun or want to employ these tools for something more serious, knowing how to use them will make all the difference. To get a better understanding of AI video and how to make it, we sat down with Roope Rainisto, the man behind the ground-breaking Life In West America and Reworld NFT collections and one of the leading figures of the AI post-photography and post-video movement.
Just five of Rainisto’s AI video creations have garnered more than 3 million views on TikTok in recent weeks, and few know their way around these programs as well as he does. In this week’s edition of /Promptly, our series that breaks down AI tools from the experts who use them, Rainisto tells us what to do (and what to avoid doing) to bring your video ideas to life with Runway’s Gen-2 text-to-video model and the newly released Zeroscope.
Tip 1: Use Runways’ Gen-2 model for consistent results
Runway is an applied AI research company that provides a host of tools to its users from text-to-image, image-to-image, inpainting functionality, and more. But it’s most well-known for its Gen-1 and Gen-2 text-to-video AI models, which Rainisto has used to create some wonderfully absurd short story creations in recent months, including Duck Story.
One of the benefits of using Gen-2, Rainisto says, is its reliability and consistency in its outputs. If you’re looking for aesthetically pleasing that are going to give you pretty much what you’ve described in your prompts, Runway is the way to go.
“Runway is the more mature option in a way,” Rainisto said while speaking to nft now. “It has an extensive set of tools, and I think the quality of the video output from an individual prompt is more mature compared to others. There’s less of a chance that you’ll get something completely freaky. It’s similar to Midjourney. It almost doesn’t matter what you write in Midjourney, you get a result that is at least slightly pleasing. Runway has that compared to Zeroscope.”
“If you’re creating something that you want to look beautiful and aesthetically pleasing, Runway’s Gen-2 is the tool to use right now.”
Roope Rainisto
Rainisto contrasts Runway with Zeroscope, the free and open-source software for AI video creation available on HuggingFace, the data science platform that hosts tools for users to create, train, and implement machine learning models. Fully utilizing the former’s model is easier for the average user with a laptop, though it comes with a price — Runway currently has four tiers of plans available for people to use its software, starting at free and moving up to $76 a month.
“If you’re creating something that you want to look beautiful and aesthetically pleasing, Runway’s Gen-2 is the tool to use right now,” Rainisto emphasized.
Tip 2: Use Runway’s Gen-1 and Gen-2 differently (and together)
Rainisto advises that Runway users play around with combining Gen-1 and Gen-2 outputs to obtain a desired result, as both are good at slightly different things.
“It’s worth mentioning Gen-1 and Gen-2 at the same time,” Rainisto elaborated. “If you have a certain thing in mind that you want to tell a story about, it’s actually a good idea to use them both. In those cases where Gen-2 fails, use Gen-1. Gen-2’s results, at this moment in time, there’s not so much movement, it’s almost like a picture that is slightly moving. But in many cases, you want a lot of movement, someone running or something. Gen-2 is not as capable of doing that. It often produces something like slow-motion shots.”
Utilizing a workflow that pulls in other tools, like Midjourney, is a great way to achieve better results. Rainisto often creates images in text-to-image generators to use as visual guides for Runway’s Gen-2. He will then take a screenshot of the video output to use as a seed for Midjourney, repeating the process as many times as needed.
Tip 3: Use Zeroscope for movement (and surprises)
Pointing to one of his most recent AI video creations, Agent Story, as evidence of what Zeroscope’s model can do, Rainisto says that it’s the best tool to use if you’re looking for unexpected results or a model that can handle lots of (literal) moving parts.
“The cons with Zeroscope are that the base-level quality is much rougher, but the pro is that there’s much more movement on screen,” Rainisto said. “Sometimes it even moves around too much, but that’s an easier problem. You can do a cut in the editing process.”
Tip 4: When marketing your video content, know your social platforms
While not directly connected to the creation of AI video, Rainisto underlined the difference in audience interaction with his AI video content from platform to platform. Apart from TikTok’s ability to bring content views and comments in numbers that he just doesn’t see on Instagram, Twitter, or YouTube, Rainisto says the kind of engagement he gets on the short-form video platform for his AI videos is unique.
“My first guess was that TikTok was full of bots, but it’s actually a lot of real people, and it’s really interesting conversation,” the artist said. “Your first guess might be that all the comments would be like, ‘Hey, this looks bad.’ That’s like five percent of the commentary. Most people actually say things like, ‘Hey, this reminds me of my dreams,’ or, ‘This is absolutely horrifying,’ ‘I’m scared, or, ‘I’m excited.’ It’s really not so much about the tech side. When I post something on Twitter, most of the engagements is, ‘Hey, how did you do that?’
When asked why he thinks people, regardless of social platform, are drawn to the perceived “wrongness” or “strangeness” of current text-to-video AI models, Rainisto says the answer boils down to what entertainment really is at its core.
“Would those stories be better if they were 100 percent photorealistic? Would you be more entertained? The answer is not instantly ‘yes.’”
Roope Rainisto
“The first question is this: Would those stories be better if they were 100 percent photorealistic? Would you be more entertained? The answer is not instantly ‘yes.’ When an art piece leaves something for interpretation, this engagement really attracts the mind. This technology will certainly get better and more realistic. But now it’s capable of doing something that no expensive camera is able to do.”
Tip 5: Don’t wait to experiment (especially if you’re a film student)
Lastly, Rainisto emphasizes the importance of capturing these tools as they exist at this point in time. AI models get updated all the time — Midjourney has gone through more than five versions of its model in the span of a year — and that means what users create with these tools now will forever be unique to this particular moment in the development of AI technology.
“It’s really interesting to try to capture the tools as they are now in this month in 2023 because, in a few years, they will be far different. That’s also why these tools are really fascinating right now.”
What’s more, the wider societal benefits of having AI video generators have yet to be fully realized, according to Rainisto. Breaking into the film industry without connections and proper funding used to be something of an impossibility, he notes. Instead of needing to write a and submit a screenplay and hope it gets in front of the right people to be approved and then sent into production on even the most basic level, text-to-video AI tools can help creatives get their vision out into the world in record time.
“These tools will make it so that a writer can write a script, make a short version of that script using AI, and put the video on YouTube instead of spending their time trying to contact 100 producers and hoping somebody will read their material.”
Roope Rainisto
“If you’re a film student, you would certainly want to use this in school just to practice and learn,” Rainisto encouraged. “Writing scripts, submitting them to producers, to competitions, hoping to find an agent, and then breaking into the industry — the traditional routes are extremely hard if you’re not known. These tools will make it so that a writer can write a script, make a short version of that script using AI, and put the video on YouTube instead of spending their time trying to contact 100 producers and hoping somebody will read their material.”
Against the Grain: Stephan Vasement
In Against the Grain, we highlight the artists, platforms, and movements that are building out the foundations of the AI art movement. Get to know the people who, by their very artistic practice, are shaking up not just the world of art but how society views creativity, skill, and human exceptionalism. This week, we are proud to present Stephan Vasement.
Stephan Vasement is a Bangkok-based visual artist with a background in scriptwriting and photography. Starting his NFT journey as a collector on fxhash, Vasement began experimenting with AI art tools with friends when he stumbled across Disco Diffusion, a clip-guided diffusion model for image generation, in the summer of 2022.
Producing work through a combination of Midjourney, DALL-E, and Stable Diffusion and drawing heavily on influences from film directors like Andrei Tarkovsky, Peter Greenaway, and Robert Bresson, Vasement also finds inspiration in music for his pieces.
With the launch of Vasement’s C Series on Objkt in July 2022, he began to catch the attention of art lovers and collectors in the NFT space, and has since seen his work collected by AI heavy-hitters like Claire Silver. To create the pieces in C Series, Vasement tinkered with Midjourney’s programming, allowing him to make significant changes to its algorithm, something that resulted in the unique style the series exhibits.
Vasement continues to experiment with new and different styles, each of which is visually distinct from the other. His many collections — Europe After the Rain, Blueness, and Floral Shapes — are a testament to his ability to artistically shapeshift in ways that few others are capable of doing.
Worth a Watch
In case you missed it, we’ve rounded up some of the most thought-provoking and compelling AI videos (and videos about AI) for you to sink your visual and intellectual teeth into. From Adam Cole’s Zeroscape explorations and Dreaming Tulpa’s breakdown Marc Andreesen and Sam Harris in conversation, there’s plenty to dig into.
