Introduction to AI Prompt Engineering for Midjourney

AI tools blew up the internet and changed our world almost overnight. However, one thing that most users learn just as fast is that AI prompt engineering is a challenging task. You can ask ChatGPT almost anything, and it will understand and offer some answers. But if you want to make AI generate a specific type of complicated text, you’re in for a struggle. It’s even harder when generating images, even with an AI platform as advanced and ‘experienced’ as Midjourney.

Devtorium teams are actively exploring the capabilities of artificial intelligence technologies. As part of this project, our UI/UX designers tested Midjourney thoroughly to understand the limits of this AI service and find the best ways to apply it in their work. Today, they share some insights about AI prompt engineering that they managed to get while studying AI-powered image generation platforms.

Before AI Prompt Engineering: How to Start Using Midjourney

Using Midjourney is quite simple, but it’s not as straightforward as simply downloading an app and entering a prompt. First of all, you need to have a Discord account. Once you set that up, you’re going to use Discord to access Midjourney. Start by joining its channel.

Midjourney takes care of its users by offering a very detailed and easy-to-understand set of docs. There you’ll find step-by-step instructions on how to:

• Set up and use a Discord account
• Join the Midjourney community
• Generate images using commands
• Learn various commands and parameters
• Subscribe and manage your subscription
• Upscale generated images

If you are having any issues, the community of Midjourney users is vast and quite friendly. Therefore, you should be able to get answers to any question fast.

At the moment, a monthly subscription to Midjourney costs $10 for a Basic Plan, $30 for Standard, and $60 for Pro. You can cut the cost when buying an annual subscription. Also, remember that the platform subscription will run out faster depending on the number of images you generate. Therefore, mastering the art of AI prompt generation will actually save you money as you’ll get accurate images faster.

AI Prompt Engineering Tips for Midjourney 

Now that you’ve set everything up, it’s time to generate images. So, you enter the command /imagine and enter your prompt. That’s where the fun starts because the prompt’s detail level and word order define the image you get. Note that the order of words and modifiers in the description matters quite a bit. So, if you want to be understood by AI, you must learn to speak its language.

That said, regardless of how advanced or well-structured your prompt is, you will always get an image. Moreover, the prompt structure isn’t fixed. It means that even if you switch some parts around, you’ll be able to get a beautiful and accurate image. However, as the Devtorium design team discovered, some structures just work better.

Here’s what they suggest using as a standard reference:

[Scene description]. [Modifiers], [Artist or style inspiration], [Technical specifications]

Now, let’s look at those parts and how to make the most out of this formula.

Scene Description

That one’s relatively easy. Your first step in AI prompt engineering is to make a short description of exactly what you want to see. For example, if you want to generate an image featuring pancakes dusted with sugar and surrounded by blueberries with a white background, that’s your scene description.

The most important thing to remember here is that when generating an image, start with a general description, adding specific details after it.

When generating an image with Midjourney, always start with a general description. Then, add details to make it more specific.

Click to tweet

Modifiers

In essence, modifiers are the specifications that explain the machine what you want to achieve. It means that they are the most crucial element during AI prompt engineering. Your selection of modifiers will define the details of your image, so be precise but not overly descriptive.

Here are some general tips specific to Midjourney modifiers:

• Put the words in brackets () if you want the AI to pay more attention to this specific detail. You can also use double brackets (()) to highlight the most important elements that AI must pay attention to.
• Use modifiers to describe important image details such as lighting, style, mood, or specific image elements.
• You can use a structure modifier:1.4 to draw the attention of AI to a specific element. It means that the machine will process it with a 140% value instead of all other words with a default value of 100%.

Types of Modifiers for AI Prompt Engineering

• Type of art: digital painting, watercolor, concept art, portrait, full body, etc.
• Style: epic realistic, fantasy, intricate, vivid, harmonious, lifelike, fantastic grotesque, architectural, balanced, cinematic, hyperrealistic, sophisticated, etc.
• Lighting: warm lights, cinematic lighting, iridescent gold, dramatic lighting, dark, etc.
• Detail complexity: intricate detail, photorealistic, realistic, emphasis on detail, etc.
• Background description: complex background, color background, flowering landscape, etc.
• Character description: appearance details (hairstyle, coloring, etc.), character action or pose.
• Visual effects: art styles (oil painting, charcoal, etc.), photography style (bokeh, Adobe Lightroom, Photolab, etc.), 3D effects (Unreal Engine, Volumetric DTX, etc.), general (medieval, muted colors, pastels, impressionism, Bauhaus cubism, etc.)
• Technical specifications: micro-details, 4K UHD image, RAW photo, extremely detailed CG Unity 8K wallpaper, etc.

Artist or Style Inspiration

This is also a type of modifier for AI prompt engineering. However, adding this specific element will affect the image greatly, so we thought it deserves a separate mention. Simply put, you can make Midjourney generate an image that will imitate your chosen artist’s style in great detail.

AI’s database is extensive so it can do equally good similes of classical painters and contemporary digital artists.

When you select an art style for this modifier, AI will use the works of top artists in this niche as a reference.

How to Learn AI Prompt Engineering from AI

As this topic is trending today, you can find an AI prompt engineering course on almost any education platform. However, the best way to learn is to ask AI itself. It’s easy to do when generating images through Midjourney because it has a command /describe.

When you use this command, you upload an image onto the platform, and AI will describe it for you. Studying the word patterns it uses will allow you to understand the way AI thinks. So, you can use such descriptions as a reference when making your own prompts.

However, you need to remember that artificial intelligence technology is only in the early stages of development. Therefore, if you use the same prompt that AI generates through /describe, you might get some very fun results.

AI Prompt Engineering Limitations: Why We Still Need Human Designers

AI tech available to every user today can do some incredible things. However, it doesn’t mean that designers and artists will be out of work soon. In fact, knowing how well they work with AI is now one of the qualities of a reliable UI/UX design company. This technology is developing fast, but it has quite a few limitations. Starting to work on AI prompt engineering makes them very clear.

First of all, there are quite a few things that AI doesn’t understand. Nuances of meanings and personalized speech patterns are beyond it.

Simply put, if you take two people and have them describe the same image, they might give you two very different texts. It’s because we are guided by our perception, which is defined by our own experiences. So, where one person sees a stunning mountain landscape, someone else might see a desolate clump of rocks.

AI takes all prompts quite literally, so sometimes it’s very hard to make it generate exactly the type of image you want.

Moreover, artificial intelligence struggles with the negatives. So, your number one tip for AI prompt engineering is to never use exclusions. For example, avoid saying ‘the man doesn’t have an arm’. Instead, choose ‘a one-armed man’.

This limitation in understanding human speech is also the reason why prompts must be structured. When we describe something, we are mostly driven by emotions. However, AI always uses logic and math. The order of words you choose will affect how important it considers the details you list. It’s truly that simple.

However, as Devtorium designers commented while working on AI prompt engineering for Midjourney, the model learns extremely fast. So, we might all live to see when it becomes advanced enough to understand and translate into code even the most complex human emotions and fantasies.