The research company OpenAI introduced the original DALL·E in 2021, but it quickly refined the model as early users experimented with the tool. DALL·E 2 arrived in the spring of 2022, a series of AI systems working in concert. One is OpenAI’s CLIP (Contrastive Language-Image Pre-Training) model, an AI trained to identify hundreds of millions of images and translate them into text descriptions. When a human prompts DALL·E 2 with an image request, CLIP works as a “text encoder,” helping the AI system understand and synthesize elements of the human prompt.
Next, the AI system combs through hundreds of millions of images classified and compressed in “latent space,” a hidden realm where AI groups millions of images according to their similarities. Latent space is a bit like Hanger 51 at the end of “Raiders of the Lost Ark,” a seemingly endless warehouse with artifacts sorted into crates according to some arcane method that’s too complex to fathom. Finally, the AI system applies an “image decoder” to translate its discoveries into a set of 10 different images corresponding to the original human prompt. Users can even modify these AI-generated images with simple text commands through inpainting and editing functions.
DALL·E 2 is only in preview mode right now, open to a small number of “trusted users” who have already created more than three million images. The company employs text filters and automated analysis to flag images that break the company’s strict content policy that only allows “G-rated imagery.” Users can lose access if they create or distribute images that reflect hate, harassment, violence, adult content, illegal activity, deception, or spam.
Beyond the content limitations, OpenAI has an even stricter policy about artists profiting on images they create, forbidding the use of DALL·E 2 to create NFTs or other commercial ways to “license, sell, trade, or otherwise transact” the generated images. “I’m personally very excited about NFTs,” said Future when asked about possible scenarios for the future of AI-generated art. “The closer the relationship we can get between creator and audience, the better. I want my subscribers to be economically incentivized and rewarded for just being a supporter. They can be along with my journey.”
For creators looking for more freedom, a community of artists and coders have been working to offer open-source tools for creators to use without such strict usage rules. This community has developed AI tools like CLIP Guided Diffusion, Disco Diffusion, and Centipede Diffusion to utilize and reproduce the functionality of DALL·E 2. This community has pioneered the art of “prompt engineering,” designing optimal text descriptions to make AI create better art.
Chris Allen is a digital artist and musician obsessed with this new field. He has spent months working with Disco Diffusion, an open-source tool that accepts text prompts and uses OpenAI’s CLIP-Guided Diffusion neural network to produce custom artwork. While the results can’t quite match DALL·E 2’s performance, thousands of artists have joined the Disco Diffusion community, making digital images, video art, and NFTs.
“Eventually, there’ll be a continuum,” Allen told me, looking ahead to a future when DALL·E 2 is available for all artists to use. “There will be corporate-sponsored access with gated communities. For OpenAI and other companies, that will be their business: inventing cool machine learning things and making them available on a subscription or contract basis. Then there will be this long smear of crazy experimental stuff.”
Allen wrote “Zippy’s Disco Diffusion Cheatsheet,” a constantly evolving handbook that introduces artists to this popular open-source AI system. “I love making the art, and I love sharing what I’ve done,” he said. “I love helping other people. It’s a good feeling to know that I’m helping someone achieve their goals, in the same way that someone helped me.”
Allen has created a series of science fiction-themed digital artworks, NFTs, videos, and films. He showed me “Hybridized Crops,” a work of art created with Disco Diffusion, as well as digital editing and animation tools. He has minted 11 editions of this surreal NFT. In the endlessly looping video, the camera moves like a microscope, zooming deeper to reveal microscopic nanobots inhabiting the veins of a plant, organic and inorganic life co-existing at the atomic level. Allen spent hours generating, refining, and stitching together these images on his home computer, tapping into the mighty processing power of Google’s cloud-based computing services. “Lots of digital is very literal. What you put in is exactly what you get out,” concluded Allen. “With AI, you are just steering the horse and hoping it goes the right way. It’s a very liberating experience.”
