☞ World's Fair Posters and DALL-E
Prompt engineering, the holodeck, chess, and more
OpenAI’s new DALL-E 2 tool is now out in beta. I was fortunate to get access to DALL-E about a month ago and I have been playing around with it far too much, from generating images in the style of William Blake, to making art like Thomas Cole (here and here):
As someone on Twitter noted, I am having way too much fun with this. I am enjoying its sheer generative power, as well as dipping my toe into the delights of “prompt engineering.” This is the art and science of what kinds of terms and keywords in a prompt can tip the model into generating what you want. There are tricks here from using terms from photography to prompting a model to use stills from movies (this deck provides a really interesting guide).
Of course, because of prompt engineering’s finicky and esoteric nature, there have been lots of comparisons made to spellcasting in fantasy realms (I’ve previously written about the similarities between code and magic!). And there are many who have noted that this kind of skill, this prompt engineering, is going to be a new ability for which we will require experts.
That being said, I was recently chatting with Matt Clancy, and he noted that perhaps this might be so, but for a far briefer period of time—like a couple years—than we might realize. And then, suddenly, these Large Language Models become good enough that the arcane arts of prompt engineering are no longer necessary. This hypothesis reminded me of how in the years after computers bested humanity in chess there was a period in time where it was the combined efforts of machines and humans—chess centaurs—that were viewed as the best “players” in the world. But this era of of human-machine partnership in chess seems to have been only a temporary period of supremacy.
Perhaps prompt engineering will simply be such a kind of technological parenthesis, and soon enough we will be programming our machines the way people on the Enterprise program the holodeck, asking it for a specific setting and situation and then getting exactly what you want, no finicky prompt engineering required. Of course, a bad prompt can cause unexpected consequences for the Enterprise crew, such as when a virtual adversary was created with the ability to outwit Data, instead of Sherlock Holmes…
In the meantime however, I’ve been trying some prompt engineering-related experiments with DALL-E, including one focused on making World’s Fair posters (original Twitter thread here).
I’ve always been inspired by World’s Fair posters and the excitement and vision that they evoke (some examples here). So I wanted to see what DALL-E could do in this area. And I was not disappointed:
While I wasn’t able to get a lot of meaningful variation based on the city that World’s Fair is set in, I did discover—prompt engineering alert!—that if you vary the year that the World’s Fair is set in, the style changes.
For example, the poster on the left is for 1901 and the right is for 1965:
And here’s 1975 on the left and 1950 on the right:
It’s also very clear that text generation is an issue with DALL-E: “FIIL FAR” and “FAR OF TO FART” are not phrases that should appear on a World’s Fair poster.
And interestingly, if you try to set a World’s Fair in the future, the gibberish text ends up maxing out around the present day. For example, here are some generated posters for World’s Fairs set in 2050:
We also can go much further back in time, imagining what a World’s Fair poster for a fair in the year 1500 might have looked like:
In order to produce location-specific posters, I tried some more exotic places: I asked DALL-E to make me World’s Fair posters for fairs set on the moon and on Mars. And it happily obliged, with the moon setting prominently featuring the Earth in the sky (correct!), and the Mars posters providing the proper red planet vibe:
I did a similar kind of experiment with logos as well, trying to use DALL-E to generate corporate and organizational logos with the styles of earlier eras, using the technique of including years as well. Another fun experiment involved trying to learn what the main characters of Seinfeld might look like according to DALL-E. Super-weird, and I’m not entirely sure what to make of that one…
Is there a big takeaway from all of this? Well, DALL-E is certainly a ton of fun and is going to have some interesting implications for how we think about creativity (I discussed this a bit on a recent Lux “Securities” podcast episode). And I love the following point about these tools, and how it can change how you think about making art, particularly if you don’t think of yourself as that artistic:
“It’s the magic pencil I could never have dreamed I would find.” I strongly agree.
I recently wrote a Twitter thread outlining the Enlightenment 2.0: “A recent trend of software + web tech recapitulating the institutions and ideals of the Enlightenment. Rebooting coffeehouse culture, Republic of Letters, scientific dissemination, and even education, via tech meeting the liberal arts...”
I’m in the process of collecting more organizations and tools that embody the ideals and approaches of the Enlightenment, including thinking more about “tools for thought” here, as well as potentially even some developments in the DeSci space (I think there are also connections here to my ever-growing canon of modern wisdom literature).
I might even end up making an Overedge Catalog-style list for the Enlightenment 2.0. So if anyone has suggestions, please pass them along to me.
A few things worth checking out:
My own idiosyncratic list of “The best books on how science actually works”
When to Design for Emergence: A fantastic essay from Kasey Klimes.
Extraterrestrial Languages by Danial Oberhaus
Until next time.