Simulating New Worlds with Midjourney?

The rise of the smart open metaverse—what I call the crossroads of AI, crypto, and virtual reality—will lead to an explosion of cultural activity over the next few decades. 

That said, one major star in today’s AI scene is Midjourney, whose tech offers some of the most stunning text-to-image capabilities currently possible. 

Already, though, the Midjourney team is also angling toward VR and real-time world simulations, i.e. generating lifelike digital spaces.

These sorts of generative environments have so many potential use cases and can provide a foundation to scaffold around large swathes of a smart open metaverse. 

Let's break it down 👇

When it comes to AI text-to-image models, the big three giants right now are DALL·E 3, Stability Diffusion, and Midjourney. 

I tinker around and try to stay up-to-date with all three of these (frankly incredible) tools, and on the Midjourney side of things, one of my favorite follows is Nick St. Pierre, a creative director and AI wiz who’s a great resource for Midjourney tips, updates, and beyond. 

Having said that, one of Nicks’s recent tweets really piqued my interest. In it, he highlighted the Midjourney team’s upcoming plans with a quote from one of their latest Office Hours sessions:

"We're really trying to get to the world simulation. We're building 3D Midjourney, video Midjourney, & real-time Midjourney where things move really, really fast. 

Put them all together, you have a world simulation. The goal is to build each of those 3 things separately, then bring them all together...

...it will be more of a sandbox. People will make video games in it, people will shoot movies in it, but the goal is to build the open world sandbox."

This is huge news, as it offers us a glimpse into the future where complex virtual experiences will be generated on command via straightforward prompting. This “no-code” UX shift will open up and revolutionize how many virtual spaces are made and experienced, leading to an explosion of possibilities for new kinds of metaversal work and play. 

Text-to-image AI models were the first big unlock here. Text-to-video and image-to-video efforts, like OpenAI’s Sora and Midjourney Video, are the next steps. And real-time world simulation is the ensuing holy grail. So the grand question is, can Midjourney pull it off?

I think so. Midjourney’s open beta started only 20 months ago, in July 2022, and yet the team has already generated +$200M in revenue and racked up over 16 million users. That war chest and traction this early on has given Midjourney great insights and momentum to keep innovating. 

Additionally, Midjourney founder David Holz has considerable experience in the VR and augmented reality fields. He co-founded Leap Motion, a VR hardware play that predated the Oculus headset. This reality, combined with Midjourney’s world simulation plans, suggests Midjourney may eventually release its own Head Mounted Display (HMD) device. 

So while world simulations may become the natural lodestar of all current text-to-image AI efforts, I believe no one in this space is better positioned to go deep and wide than Midjourney right now. If anyone can pull this off, Midjourney can—and I think they will, eventually, and probably faster than most people expect.

Still, that capability is some years away currently. However, that doesn’t mean we can’t already start dreaming about the immersive metaversal possibilities that are to come here. 

For instance, I think world simulations will be a great match with Autonomous Worlds, i.e. games that track their logic and state fully onchain. This architecture supports client-agnostic designs, so anyone can spin up their owner interfaces. Imagine, then, an interface builder that taps Midjourney so you can provide images and then create custom worlds to play through onchain games with real-time visuals. 

This is just one example use case, but the possibilities to come will be limited mainly by our imaginations. Think spinning up an art gallery on command with your favorite NFTs, or creating immersive experiences around real estate in metaverse projects. 

Cinema, education, gaming, virtual events, etc.—everything can be expanded upon in new ways via world simulations, and that’s why I think Midjourney is positioned well to become a pillar of the smart open metaverse. Similar to how AI agents will make it much easier to fill out virtual worlds with human-like intelligence, world simulations will make it much easier to flesh out these spaces with incredible real-time visuals. 

So as the AI x crypto x VR crossroads continues to intertwine and the smart open metaverse comes further into focus, be sure to keep an eye on Midjourney. Their work holds great promise in making the friction of creating large, immersive virtual spaces trend toward zero. The implications here for creators of all stripes are huge, and the entertainment industry stands to be revolutionized accordingly. Take note now, NFTers, and get ready!

Today’s action steps:

• 🎨 Learn about Midjourney: Study the docs
• 📱 Follow Nick St. Pierre: Midjourney insights
• 🔍 Catch up on my previous roundup: Going Multichain