Daniel Miessler's Blog, page 31

Auto-generating Blog Art Using GPT-3, DALL-E, and Stable Diffusion

I’ve been blogging since 1999 and one of the most challenging parts of finishing a post is coming up with a good image.

It’s been a bit overwhelming, tbh.

Lately—as I’m sure you’re aware—everyone’s been all about DALL-E, Midjourney, Stable Diffusion, and all the other AI-art generation technologies. It’s cool stuff, but a lot of the magic comes down to prompt engineering, which is non-trivial.

If prompt engineering is the hard part, then let’s use AI to write the prompts.

In other words, AI might be able to make you a cool piece of art, but only if you tell it how (and in very specific and strange ways).

Your writing was creative already, so let that do the work.

But what if you could tell it how automatically, just based on the text in what you wrote?

💡🤖 Fun Project: A quick tool that:

1. Takes the text of a blog post
2. Summarizes it into a DALL-E prompt
3. Submits that to DALL-E
4. Now you have art for the blog!

— ᴅᴀɴɪᴇʟ ᴍɪᴇssʟᴇʀ (@DanielMiessler) October 17, 2022

That’s the idea I became obsessed with last night, and I was able to get a working proof of concept in less than 30 minutes! This stuff is just nuts.

How to build it

More on few-shotting.

Ok, as I said in the tweet, there are a few key steps:

Summarize the textTurn that into a promptCreate an image from the prompt

My technique—and be gentle because it’s a very quick POC—is to combine the first two steps into a few-shot learning template. For anyone who just WTF’d, few-shotting is basically showing an AI the right answer a few times and then giving it a similar problem with no answer.

So I wanted the template to both 1) train the AI to write a prompt, and 2) feed it my text to build the prompt from. I did that by mixing those two in the Summary portion of the template.

One of the training shots, showing the mixture of prompt and content summarization

You can tweak this language to make it match your blog.

Specifically, the:

a professional digital painting about…

…was static in all the summaries I trained it with. And so was:

gorgeous digital painting, warm colors, captivating, trending in artstation…

Which means when I ask this tool for the real (non-training) prompt—using my new text—I don’t have to add those things!

Training the prompter

Some guru will be able to show me a better way, no doubt.

So that’s pretty much it. You just need the text to get summarized, which includes a label (I used Text), and then a summary that is your output on a separate line (I used Summary). Then, as your final step, send the thing you want to summarize (which is the completed prompt), and leave the answer blank!

Here’s what the whole promt looks like, with all the “Text:” portions being taken from other blog posts of mine:

The full series of Text/Summary training scenarios

Remarkably solid prompt quality

Using this technique I was able to generate some pretty decent prompts, which surprised me. Here’s the opening of a blog post I did recently, and I wanted to see if this technique could create a decent prompt for it.

It’s common to hear that it’s hard to get into cybersecurity, and that this is a problem. That seems to be true, but it’s informative to ask a simple follow-up: The current cybersecurity jobs gap sits at around 2.7 million people. A problem for who? I think what we’re facing is an instance of the Two-Worlds Problem that’s now everywhere in US society. It’s where the vast majority of people are feeling pain from an issue, but there’s an elite that lives in a reality where the problem doesn’t exist.

That’s not the whole article, of course, but it’s already got some decent stuff to key off of. So how did this trainer do? Here’s the prompt it created:

a professional digital painting about cybersecurity, jobs gap, two-worlds problem, pain, elite, reality, gorgeous digital painting, warm colors, captivating, trending in artstation

Seriously?!?! That’s stunning. So not only did it keep my DALL-E 2 prompt stuff in there, but it also added the summary nested in the middle!

So it extracted these terms from the text: cybersecurity, jobs gap, two-worlds problem, pain, elite, reality, and then it seamlessly blended them into the specific art styling I’ve told it I want for my site.

You can tell which is which by the icons.

The UL Newsletter: Finding the Patterns in the Noise…Get a weekly analysis of what's happening in security and tech—and why it matters.

So I’m pretty excited at this point, but now for the big question. How do the images look based on this prompt? Here are some of the outputs of running this prompt on DALL-E, Midjourney, and Stable Diffusion.

Output from three different AI-engines using the GPT-3-created Prompts

Are you f-ing kidding me. This is ridiculously good.

Oh, and you know what? I’ve not even begun, because I can comb through all the different art styles that these engines can make, and I can add a particular style I like to the prompt training. Actually I have multiple ideas there:

Hardcode the art style you want for your blogHardcode a color schemeIn your custom prompt creator tool, factor in the content for the art style!Add a custom art call to the tool, so you can ask for different art styles based on moodCreating a simple utility, bac

I’m a bit over-stimulated right now, but here’s what I’ve done so far to make this kind of slick and usable. I’ve created a simple utility called Blog Art Creator (bac), which sounds fancy but is nothing but a silly shell script.

I have the training scenarios in a single file, training.txt.I save my new content to another file, content.txt.I have the final prompt in another file, prompt.txt.

The utility just concatanates text, so it combines the training data, your content, and the prompt, and sends all of that to GPT-3. I use this cli tool within my script:

gpt3 –engine text-davinci-001 –temperature 0.75 –freq-penalty 1 –pres-penalty 1

So for me my final command is:

bac

And as you know, you can create some pretty lame looking stuff if you don’t do your prompts right. So here is a brand new prompt from a much earlier blog post.

People are hyped about the metaverse, and it’s honestly understandable. First, we’re going through some shit as a species right now. Social tension, the best part of Star Wars’ last three movies were the previews, and there’s a global pandemic. Part of this metaverse hype is just being excited to be excited, and I get it.Metaverse lore has lots of fun stuff in it—from AR/VR to headsets to avatars to moving between worlds. But I think we can reduce the metaverse to one concept and two axes.The metaverse is about experiencing a better life than in meatspace.Most important to this is content.Second most important to this is interface.ContentContent is life. It’s the source of meaning. It’s what gives provides us with evolution’s nectar of happiness in the form of hormone releases when we struggle and achieve something.

Gaming is Metaverse 1.0

Which produced this prompt:

a professional digital painting about the metaverse, hype, understandable, exciting, better life, content, interface, gorgeous digital painting, warm colors, captivating, trending in artstation

Running the bac command and getting output

And created these images from DALL-E in the first two runs.

In. Sane.

I will absolutely be using this for my own stuff going forward.

Wrapping up

Well I’m stoked. I can’t wait to customize this thing more, and I really hope you enjoy messing with it and finding tons of ideas on how to improve it. And share it out or hit me up if you want to hack on ideas.

NotesNext steps for me would also include automating the submission to the image generation services, but that’s a bit kludgy right now. I’ll add that to the automation once it’s available and/or easier to submit to and get results from.

News & Analysis | NO. 352

window.location.replace("https://mailchi.mp/danielmiessler/uns...

No related posts.

Something is Up With Meritocracy

This is UL Member Content

Subscribe

Already a member? Login

News & Analysis | NO. 351

window.location.replace("https://mailchi.mp/danielmiessler/uns...

No related posts.

Stadia is Google’s Product Strategy

Few things in tech were more predictable than Stadia shutting down.

Here’s what I wrote the week it came out:

Here’s what I said about it in 2021.

And here’s my analysis of why this keeps happening: How I Knew Stadia Would Fail

The overall reason for this is UI/UX in my opinion, which I talked about here in 2020.

I’m not an expert on internal Google politics, but it seems to me—and I’ve heard elsewhere—that they pretty much let engineers run everything. The coding, obviously, but also a whole lot of the product side as well.

If that’s true, it’s not working. Again, maybe there are other factors that I’m not aware of, but to me this reads as not having a clear product vision and solid designers as part of product teams.

As I ranted about with Google’s management interfaces, the interfaces seem completely oblivious to how people will actually use the products. So they fail.

The UL Newsletter: Finding the Patterns in the Noise…Get a weekly analysis of what's happening in security and tech—and why it matters.

Over, and over, and over.

Here’s another instance of me complaining about this from 2019.

To be clear, Google’s strategic goals and large projects are overwhelmingly respectable (self-driving cars, fighting disease, etc.), and I continue to look up to them as leaders in technology for those types of efforts.But when it comes to their basic web services, their mobile platform, their UI/UX, their sense of design and integration, their attention to detail, and 1,000 other things about how they present themselves, they’re third-rate or worse.

I’ve been talking about this for nearly 10 years now. And so have thousands of others.

Billions?

At this point I’ve lost my empathy for them. How in the hell can you continue making the same exact mistake, for like 10 years, which is no-doubt causing the loss of hundreds of millions of dollars in losses in dozens of products—but you don’t adjust?

How?

Whatever. I guess we just slowly wait for someone who knows how to make a GUI to catch up to them in backend competence. Because then they’ll be in serious trouble.

News & Analysis | NO. 350

window.location.replace("https://mailchi.mp/danielmiessler/uns...

No related posts.

5 Things I’m Still Waiting for With the iPhone, Apple Watch, and AirPods

I just camped for the new lineup of Apple products, as I have every year since 2007, and I am pretty impressed.

As many have said, this year was an incremental update. The iPhones look nearly identical. And so do the AirPods Pro 2.

The exception was the Apple Watch Ultra, which is a rather new form factor. Beefier. With some extra features. But utlimately it’s still 90% the same as the Series 8.

I would argue this is a good thing. It means they think they’re getting close to the ultimate forms of these products. And I agree. With a few exceptions that I can’t wait to see filled in.

1. USB-C

Or Thunderbold, but with a USB-C connector.

Yeah, I know, everyone gets this one. It’s just time to unify everything behind a single connector. Nuff said already.

2. A Round Apple Watch

I was a watch guy before I was an Apple guy, and I am bummed about wasted real estate with the round watch faces that sit inside the Apple Watch’s rectangular form factor.

I would love something that looks more like a Submariner, but with the Apple buttons on it. I’d pay thousands for it. Probably. I don’t know why it’s taken this long, but I hope it comes soon.

3. High-Resolution Audio on AirPods

The new Airpods Pro 2 have improved drivers and algorithms that measurably improve their sound, but as a mid-level audiophile (shudder) I am still waiting for the final piece of lossless and high-quality audio. It’ll require a new protocol, but that shouldn’t be too much of an obstacle. f

It’s a small point, and things already sound great, but if you’re into audio it’s pretty annoying to know the source is lossless/high-quality, and the drivers are excellent, but that the protocol is stripping quality from your audio.

4. TouchID Under the Glass

FaceID keeps getting better, but I’d love if it was eventually paired with TouchID that activates when you put your thumb on the screen.

It might not be better or faster when you’re looking at the screen, but there are enough times when you blink, or look away, or at a bad angle, and you don’t land a FaceID authentication. So it’d be brilliant to have both.

5. Unification of The Naming

Finally, just a naming gripe which I don’t really fault them for but can’t wait for them to fix.

The UL Newsletter: See the ideas, patterns, and models that will keep you ahead…Sign up to get a concise summary of what's happening (and why) every Monday morning. Apple Silicon has Normal, Max, and UltraiPhones have Normal, Plus, Pro, and Max

Plus there are the iCloud tiers, which might be too hard to bring in. But can we try to standardize a bit?

I do see the problem. iCloud and other services they’re following the Disney+ naming convention. And with iPhones you have to have a ladder in both the regular line and the Pro line, and the steps on those ladders have to be distinct to avoid confusion.

Maybe the answer is unifying the line and just having more steps. So it’s just:

iPhoneiPhone Plus (larger base model)iPhone Pro iPhone Pro Plus

I think the challenge is differentiating size vs. performance. Those two things need different laddering terminology.

Perhaps Plus can be the size difference, while Pro, Max, Ultra are the performance tiers.

And with Apple Silicon, it just becomes:

M2 M2 ProM2 MaxM2 Ultra

Not sure they should have that many tiers though.

—

Anyway, these are the things I think they should sort out to arrive at the peak of this line of products for the next five years or so.

USB-CA Round Apple WatchHigh-res AudioTouch-ID Under the GlassUnified Naming

Let me know if you have any other ideas.

Using Custom Searches in Safari (in 2022)

I’ve just started using Safari again after being on Chrome for a while, and one of the things I miss most from Chrome is custom searches.

There you can search Amazon directly from the address bar by doing:

You can change the search prompts to be single letters like ‘a’ for Amazon.

a tv lightstrips

At the time of writing (September 2022) there isn’t any way to do this in Safari. You can do this, but it’s too much typing:

amazon.com tv lightstrips

Alfred is like Quicksilver, Omnikey, and Spotlight all in one.

The best solution I’ve found is to just use Alfred, which is great since I use it already for text expansion and many other tasks.

The custom search options in Alfred

As you can see in the screenshot, there’s an entire Web Search section in Alfred that lets you set custom search engines. So you just go into there and set your trigger letter for each, and you’re good to go.

After invoking Alfred and the Amazon search prefix

Then you just hit ⌘ + SPACE to invoke Alfred, and then type a tv lightstrips to search on Amazon.

Your query will be opened automatically in a new tab in Safari.

Hope that helps you!

News & Analysis | NO. 349

window.location.replace("https://mailchi.mp/danielmiessler/uns...

No related posts.

My Mom Died on Saturday

My mom died on Saturday.

My biological mom became unable to function when I was around 5 due to mental illness, which left my dad and me on our own. Sometime after we were blessed with a strong, beautiful soul named Rhonda.

My dad and I were like rescue dogs, and she saved us. She brought so much love into what was a hard, matter-of-fact relationship between my dad and I. She taught him to be kinder with me, and with himself. And she spent her whole life helping me to open up about about my own internal life.

She stayed on me about homework and school, and there is zero doubt in my mind that I would not have graduated high-school were it not for her.

She had a tremendous talent for listening to people, and I only in the last five years or so realized that my enjoyment of helping people came from her. She was talking about her view of how people are just struggling, which is why they need to talk, and why we need to help them.

My daft self was like, “Wow, we’re so similar in that way.” And a few minutes or hours later I remembered that people learn things from their parents, and realized I learned this perspective from her.

People have been coming around randomly, after hearing the news, talking bout how easy it was to talk to her, and how much she helped them. It’s sweet and beautiful to hear.

She died of an aggressive type of cancer that was supposed to take her within weeks or months, that had no chance of remission. But the diagnosis was back in 2020, and the oncologists were able to give us a miraculous extra two years with her. I am forever thankful to them for that time.

She died peacefully in her sleep, with my dad sleeping next to her.

I will miss her.