How to Use Stable Diffusion to Create AI-Generated Images

Artificial intelligence chatbots, like ChatGPT, have become incredibly powerful recently – they’re all over the news! But don’t forget about AI image generators (like Stable Diffusion, DALL-E, and Midjourney). They can make virtually any image when provided with just a few words. Follow this tutorial to learn how to do this for free with no restrictions by running Stable Diffusion on your computer.

Good to know: learn how to fix the internal server error for ChatGPT.

What Is Stable Diffusion?

Stable Diffusion is a free and open source text-to-image machine-learning model. Basically, it’s a program that lets you describe a picture using text, then creates the image for you. It was given billions of images and accompanying text descriptions and was taught to analyze and reconstruct them.

Stable Diffusion is not the program you use directly – think of it more like the underlying software tool that other programs use. This tutorial shows how to install a Stable Diffusion program on your computer. Note that there are many programs and websites that use Stable Diffusion, but many will charge you money and don’t give you as much control.

System Requirements

The rough guidelines for what you should aim for are as follows:

• macOS: Apple Silicon (an M series chip)

• Windows or Linux: NVIDIA or AMD GPU
• RAM: 16GB for best results
• GPU VRAM: at least 4GB
• Storage: at least 15GB

Install AUTOMATIC1111 Web UI

We are using the AUTOMATIC1111 Web UI program, available on all major desktop operating systems, to access Stable Diffusion. Make sure you make note of where the “stable-diffiusion-webui” directory gets downloaded.

AUTOMATIC1111 Web UI on macOS

1. In Terminal, install Homebrew by entering the command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Good to know: find out more about how to use Homebrew in this guide.

2. Copy the two commands for adding Homebrew to your PATH and enter them.

3. Quit and reopen Terminal, then enter:

brew install cmake protobuf rust python@3.10 git wget

4. Enter:

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui

AUTOMATIC1111 Web UI on Windows

1. Download the latest stable version of Python 3.10.

2. Run the Python installer, check “Add python.exe to PATH,” and click “Install Now.”

3. Go to the AUTOMATIC1111 Web UI repository on GitHub, click “Code,” then click “Download ZIP” and extract it.

AUTOMATIC1111 Web UI on Linux

1. Open the Terminal.

2. Enter one of the following commands, depending on your flavor of Linux:

Debian-based, including Ubuntu:

sudo apt-get update
sudo apt install wget git python3 python3-venv

Red Hat-based:

sudo dnf install wget git python3

Arch-based:

sudo pacman -S wget git python3

3. Install in “/home/$(whoami)/stable-diffusion-webui/” by executing this command:

bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)

Tip: get help deciding whether you should buy an external GPU.

Install a Model

You’ll still need to add at least one model before you can start using the Web UI.

1. Go to CIVITAI.

2. Click the drop-down arrow on the download button and select “Model SafeTensor.”

3. Move the .safetensors file downloaded in step 2 into your “stable-diffiusion-webui/models/Stable-diffusion” folder.

Run and Configure the Web UI

At this point, you’re ready to run and start using the Stable Diffusion program in your web browser.

1. In your terminal, open your “stable-diffusion-webui” directory and enter the command ./webui.sh --xformers for Linux / macOS or ./webui-user.bat for Windows. When it’s finished, select and copy the URL next to “Running on local URL”, which should look like http://127.0.0.1:7860.

2. Paste the link in your browser address bar and hit Enter. The Web UI website will appear.

3. Let’s change some settings for better results. Go to “Settings -> Stable Diffusion.”

4. Scroll down and check “Enable quantization in K samplers for sharper and cleaner results.”

5. Scroll up and click “Apply settings,” then “Reload UI.”

FYI: If you need to find an image source, use Google.

Use txt2txt to Generate Concept Images

Now comes the fun part: creating some initial images and searching for one that most closely resembles the look you want.

1. Go to the “txt2img” tab.
2. In the first prompt text box, type words describing your image separated by commas. It helps to include words describing the style of image, such as “realistic,” “detailed,” or “close-up portrait.”

3. In the negative prompt text box below, type keywords that you do not want your image to look like. For instance, if you’re trying to create realistic imagery, add words like “video game,” “art,” and “illustration.”

4. Scroll down and set “Batch size” to “4.” This will make Stable Diffusion produce four different images from your prompt.

5. Make the “CFG Scale” a higher value if you want Stable Diffusion to follow your prompt keywords more strictly or a lower value if you want it to be more creative. A low value (like the default of 7) usually produces images that are good quality and creative.

6. Leave the other settings at their defaults for now. Click the big “Generate” button at the top for Stable Diffusion to start working.

7. Below the “Generate” button, click the image thumbnails to preview them and determine whether you like any of them.

If you don’t like any of the images, repeat steps 1 through 5 with slight variations.

If you like one image overall but want to modify it or fix issues (a distorted face, anatomical issues, etc.), click either “Send to img2img” or “Send to inpaint.” This will copy your image and prompts over to the respective tabs where you can improve the image.

If an image is exceptionally interesting or good, click the “Save” button followed by the “Download” button.

Finding the Prompts Used for Past Images

After you’ve generated a few images, it’s helpful to get the prompts and settings used to create an image after the fact.

1. Click the “PNG Info” tab.

2. Upload an image into the box. All of the prompts and other details of your image will appear on the right.

Tip: use one of these Windows tools to batch-edit images.

Use img2img to Generate Similar Images

You can use the img2img feature to generate new images mimicking the overall look of any base image.

1. On the “img2img” tab, ensure that you are using a previously generated image with the same prompts.

2. Set the “Denoising strength” value higher or lower to regenerate more or less of your image (0.50 regenerates 50% and 1 regenerates 100%).

3. Click “Generate” and review the differences. If you’re not pleased, repeat steps 1 through 3 after tweaking the settings.

4. Alternatively, click “Send to img2img” to keep making modifications based on the new image.

5. Rewrite the prompts to add completely new elements to the image and adjust other settings as desired.

6. Click “Generate” and review the result.

Use inpaint to Change Part of an Image

The inpaint feature is a powerful tool that lets you make precise spot corrections to a base image by using your mouse to “paint” over parts of an image that you want to regenerate. The parts you haven’t painted aren’t changed.

1. On the “img2img tab -> Inpaint tab,” ensure that you are using a previously generated image.

2. Change your prompts if you want new visual elements.

3. Use your mouse to paint over the part of the image you want to change.

4. Change the “Sampling method” to DDIM, which is recommended for inpainting.

5. Set the “Denoising strength,” choosing a higher value if you’re making extreme changes.

6. Click “Generate” and review the result.

Stable Diffusion probably won’t fix everything on the first attempt, so you can click “Send to inpaint” and repeat the above steps as many times as you would like.

Good to know: look through these websites to find images with a transparent background.

Upscale Your Image

You’ve been creating relatively small images at 512 x 512 pixels up to this point, but if you increase your image’s resolution, it also increases the level of visual detail.

Install the Ultimate SD Upscale Extension

1. Click “Extensions -> Available -> Load from.”

2. Scroll down to find “Ultimate SD Upscale manipulations” and click “Install.”

3. Scroll up and click the “Installed” tab. Check “ultimate-upscale-for-automatic1111,” then click “Apply and restart UI.”

Resize Your Image

1. On the “img2img” tab, ensure you are using a previously generated image with the same prompts. At the front of your prompt input, add phrases such as “4k,” “UHD,” “high res photo,” “RAW,” “closeup,” “skin pores,” and “detailed eyes” to hone it in more. At the front of your negative prompt input, add phrases such as “selfie,” “blurry,” “low res,” and “phone cam” to back away from those.

2. Set your “Denoising strength” to a low value (around 0.25) and double the “Width” and “Height” values.

3. In the “Script” drop-down, select “Ultimate SD upscale,” then under “Upscaler,” check the “R-ESRGAN 4x+” option.

4. Click “Generate” and review the result. You should notice minor changes and sharper details.

You can increase the resolution further by clicking “Send to img2img” and repeating the steps while increasing the “Width” and “Height” values further and tweaking the “Denoising strength.”

Frequently Asked Questions

Image credit: Pixabay. All screenshots by Brandon Li.

Brandon Li

Brandon Li is a technology enthusiast with experience in the software development industry. As a result, he has a lot of knowledge about computers and is passionate about sharing that knowledge with other people. While he has mainly used Windows since early childhood, he also has years of experience working with other major operating systems.