The Ghost of a Guinea Pig
Prompt Engineering: Experiments at the intersection of Accident and Control
Using references in Midjourney V.4
With the ongoing discussion of ai art generators, artist rights and artists works being used to educate without artists consent, it has me thinking of an experiment. What happens when I feed my personal art into Midjourney and use it as my foundational reference to create prompts?
How much does it influence the generated works? How far can I bend it while still paying homage to the fed in image?
So that is what I set out to do for today’s experiment in prompt engineering. Take one of my digital illustrations and use that as an image reference in my prompt.
Custom Gig Poster | illustration by Jyn Henzel
Digital Illustration
Tools used:
Wacom Cintiq 13” HD
Adobe Photoshop
Brush set by Frenden.
Used for marketing materials for event:
Digital Social posts
Print Marketing
Large Format Poster and Banner
Ok here we go…
First prompt is always the tester, the proverbial stick in the water, what in the heck is about to happen round. I wanted to simply feed in my image, and keep my asks pretty generic. First thought, what happens when I ask for a photorealistic version of my illustration?
Prompt 1 | https://s.mj.run/hY-GDxe8wrM, photorealistic, cinematic lighting --ar 2:3 --v 4
Okay okay. That is not what I was expecting, but daaaaaang. I am liking the direction. The moody lighting, a beautiful haunting Victorian ghost story vibe.
So…let’s roll that a few times. I want to see how the story unfolds.
Reroll. Upscale. Variation.
I can see the reference image in the way that it is capturing the perspective, angles and curves that are found in the architecture in the poster, and in some examples bringing in how the Nameless Ghouls are being placed as art in the homes. I love how the typography carries the similar decorative look of the Ghost logo and adds great accents to the artwork. Its reinterpretation is wonderful, and one of my favorite topics to explore with MidJourney. Rooms and houses that say so much just with the break of light carefully dodging dark corners. Mysterious figures that blend into the background waiting for you to turn away. The ghost stories of our homes.
Reroll. Upscale. Variation.
Environment Directed Prompts | 24
1x | Original prompt
12x | OG prompt rerolls
9x | Upscale
2x | Variation
Features |
Directional light source
Cloaked figures
Decorative typography
Repetition of tables and chairs, light source (candles & lamps)
Warm earthy glow and colors with cool grays and blues.
Now let’s see if I am able to bring out the figures from the original reference more than just capturing the mood and environment. I add in additional descriptors referencing the figures in the poster I want MidJourney to pull from.
Prompt 2 | https://s.mj.run/hY-GDxe8wrM, photorealistic, evil priest inspired by the band Ghost, a feamale dmeon coming up from the ground in latex body suit, cinematic lighting--ar 2:3 --v 4
Whaaaat is this?!
Let’s address the elephant in the room to all my eagle eye patrons real fast…or to anyone with a reading level about 1st grade…yes I did misspell female demon. Autospell has done me dirty and given me no-long term favors. When I type fast, I most often forget to proof read and just roll with it.
So, no surprise that I am not seeing a female represented here. The epic evil priest surrounded by these gothic architectural details...I am here for it.
Quick image upscale of Image 1 to check out the insanely detailed chest plate.
Upscale
Okay time to adjust the prompt. I wanted to define how many figures to include and loose details of each.
Prompt 3 | https://s.mj.run/hY-GDxe8wrM, photorealistic, two figures an evil priest and a female demon, latex body suit, cinematic lighting --ar 2:3 --v 4
Now we are talking! The figures MidJourney is pulling is interesting, a gothic marriage of horror filled fantasy art. The backlighting and backgrounds are pushed back beautifully, hierarchy of elements laid out in pleasing composition. Let’s roll on this for a minute.
Reroll, Upscale, Variation.
I AM LOVING what the reference image and prompt are creating. High contrast shadows, dual color palette, amazing death metal-esque type treatments. I absolutely will revisit this series and work with it more.
Next experiment!
Am I able to pull out the female demon as my primary character? She was always one of my favorite figures I have illustrated in her vinyl body suit and large curving horns. This could get interesting.
Prompt 4 | https://s.mj.run/hY-GDxe8wrM, photorealistic, female demon with large horns, full body latex body suit, crawling out of the ground, cracked laval, cinematic lighting --ar 2:3 --v 4
Well that is nightmare fuel.
I wanted to direct the style of horns a bit more and added a frame of reference from the amazing character design of Tim Curry’s Darkness in the 1985 movie Legend. Since I was a child, this has been my internal gauge for the depiction of devils and demons. Sultry and sadistic. Thank you to Rob Bottin for the epic makeup design; you have made a lasting impression on my imagination.
Prompt 5 | https://s.mj.run/hY-GDxe8wrM, photorealistic, female with large horns like the devil in the movie Legend, full body latex body suit, crawling out of the ground, cracked lava, cinematic lighting --ar 2:3 --v 4
Reroll, Upscale, Variation.
Next, I wanted to see if I could pull in the throne from the reference material and made a sloppy prompt addition for a black goat head crested throne. It also felt like a good time to adjust the clothing direction.
Prompt 6 | https://s.mj.run/hY-GDxe8wrM, photorealistic, female evil nun with large horns like the devil in the movie Legend, full body latex body suit, sitting on a throne has a black goats head at the top, cracked lava, cinematic lighting --ar 2:3 --v 4
It would be stupid not to roll with this for a moment.
Reroll. Upscale. Variation.
Yep, definitely exploring this one more in the future.
Time to go back one step and push the direction as more succubus and see if I could get a plus size body type.
Prompt 7 | ttps://s.mj.run/hY-GDxe8wrM, photorealistic, beautiful curvy female with large horns like the devil in the movie Legend, full body latex body suit, crawling out of the ground, cracked lava, cinematic lighting --ar 2:3 --v 4
Ha! that is NOT what I was expecting. Looking back on my previous prompt I noticed a playful little copy paste error which failed to include the “h” in the image address. This was a great chance to see how the prompt was functioning in unison to the reference image. You can really see the change in the color palette, style, and level of realism in the results.
Okay, back at it!
Prompt 8 | https://s.mj.run/hY-GDxe8wrM, photorealistic, beautiful curvy female with large horns like the devil in the movie Legend, full body latex body suit, crawling out of the ground, cracked lava, cinematic lighting --ar 2:3 --v 4
SYNOPSIS
Character Directed Prompts | 28
7x | 6 Unique prompts + 1 Misfire
8x | OG prompt rerolls
13x | Upscale
2x | Variation
Features |
Interpretations of teal, gold, and charcoal color palette from the reference
Decorative typography
Gothic overtone
The OG Ghost poster gave more lighting direction than I would have put money on. In the environments it tended to lended itself to creating these deep long shadows through windows and hallways. In the figure compositions we saw it come out as a high contrast lighting with times of backlighting to the main figures.
The drape in the fabrics throughout the prompts brought out the posters portrayal of the Nameless Ghouls and Papa Emeritus III. Dark cloth rippling in shadows. A heavy grounded presence. And the happy accidents were always brilliant. How it chose to interpret background figures was beautiful at times. Mythical creatures surrounding a throne, twisted forms in darkness, robed figures in waiting.
As common features became evident, it was awesome to see the variety achieved with the prompt direction. Using prompt descriptors for gender, style reference, actions, environments; MJ was responding to the adjustments with solid consistency.
Documenting the hows and whys has given me a new perspective on my prompt engineering tendencies. Giving a more top down view in how to navigate the information I was able to see where I had the potential to really fine tune key elements and took the time not to knee-jerk react to prompt direction.
Going into this and not knowing where the journey would take me, I am very pleased with this netted out. It has been interesting to see the interpretations Midjourney made with the elements from the reference. Sometimes it came out as a mood. Other times it came out in gothic architecture. Each time a bit of a surprise and delight with the subject matter. Definitely looking forward to exploring a few of the above pathways and fine tune elements.
Have fun exploring and experimenting!
I have provided my full prompts along with an active url link to my reference image. I give consent to use of the reference image to remix in your own custom AI explorations and prompt engineering. I would love to see what you create; please tag me @ai_sentient_echos
Original reference and all images shown © 2022 Jyn Henzel.
