Sign In

A Real Business Case with ControlNet

A Real Business Case with ControlNet

If you find our articles informative, please follow me to receive updates. It would be even better if you could also follow our ko-fi, where there are many more articles and tutorials that I believe would be very beneficial for you!


如果能同时关注我们的 ko-fi 就更好了,

那里有多得多的文章和教程! 相信能使您获益良多.

For collaboration and article reprint inquiries, please send an email to [email protected]
合作和文章转载 请发送邮件至 [email protected]

by: John


This article serves as a tutorial for those who have never worked in the design industry but are interested in utilizing AI to monetize their skills. It also provides a practical case study for aspiring designers who are currently studying in the field.


Client's Requirement

One of the biggest challenges in taking on design projects is communication with clients. Most clients are not clear about the desired outcome. The client informed me that the modeling effects produced by a designer they hired for a few thousand TWD were not satisfactory. Limited budgets often restrict the time-consuming process of creating creative scenes from scratch using traditional design tools. AI effectively addresses this issue by achieving client satisfaction in the shortest possible time, surpassing their expectations.


The desired rendering requested by the client

The client's request is to have a car repair shop on a race track at night. It should have a professional and competitive look, as it will be used as a background for live streaming. The client also mentioned wanting an angled perspective.

It seems that the client initially didn't realize the seriousness of the issue until they received the draft and some unrelated sample images. Creating such a specific and unique design can indeed be challenging, and it may not be something that many designers or sellers on certain platforms can easily achieve.

Design for a certain online marketplace merchant

Sketch Generation

When starting a sketch, it's important not to focus on the color tones. What we're looking for are clean lines. We can later use controlnet canny to control and unify the color tones and style.


So the first step is to find suitable reference materials to work with. It's somewhat similar to collage-making. Here, we have several options for sourcing materials, such as:


1. Internet or Stock Image Libraries

There are many perspectives to choose from on the internet. You can directly capture screenshots and use SD (StyleGAN-ADA) to redraw them. Redrawing with an intensity level between 4.0-4.9 will help preserve the visual elements of the original image.

2.AI Generation Software

You can use software like SD (StyleGAN-ADA), Midjourney, or AI effects in Photoshop to generate sketches.

To generate the sketch using AI

the client's requirement is to have a race track and a repair shop, with the race track on the left and the repair shop on the right. 


First, you can search for images or references that capture the feeling of a race track. Once you have selected suitable images, you can collage them together to create a section of the road. 


Next, you can use controlnet to control the lines and add key elements to the sketch. Experiment with different variations by running several iterations. Keep refining the sketch until you find a result that satisfies the client's requirements.

Because the client's requirements were relatively abstract and I didn't have a clear direction, I used the new AI features in Photoshop to enhance the overall effect of the image. Then, in Photoshop, I re-drew some of the edge lines to give them more definition.

Adjusting the composition of the image according to the client's requirements, including the direction of the vehicles and the road surface, etc., in fact, the level of hand-drawing is really casual and doesn't require much skill. It's all about the feelings. After that, I redraw based on the control net to address issues with line recognition. There are some identification problems that may arise, and I make corrections accordingly. This is where I spend more time on adjustments.Collage materials

I need more elements on the field, so please extend the material in the upper left corner of the PS and add relevant keywords related to the field. Then control it with ControlNet.

This technique utilizes preemptive control, which allows AI to think on your behalf while you focus on controlling it.

Extended scene.

Model style

After extending, collage according to the needs, blend with redrawn elements, and adjust before collaging again. Then, use the material or AI-generated image function in PS. Once the composition with the above image is confirmed, activate the text-to-image function and start searching for suitable models to correspond to the realistic style image.

At this step, while finalizing the main composition, unify the overall color tone. It may involve omitting some details, but it will make the overall image look realistic and sufficient.

Up Scale

When using Ultimate Upscale, we first determine the number of segmentation blocks before proceeding with the enlargement. For example, if we divide this image with 300 pixels into three blocks, the optimal effect is achieved. In this case, we would fill in the tile width with 100. As a result, the overall image becomes clearer. However, it would be even better to separately segment and redraw the main subject of the image for a more enhanced effect.The difference is clearly visible, and the overall effect image has been generated. Now, it only requires further adjustments based on the client's specific requirements. Additionally, select and redraw specific areas by using the local selection tool.

The effect after upscaling.

Showcasing the modification process.

The final result.

In conclusion

this was my first experience taking on a project. Surprisingly, it only took two days to complete, despite the client's urgency. At first, I thought it would take at least a week, but I managed to meet their deadline. This experience, as someone who is not from the design industry, allowed me to create a commercially viable design solution and added a practical case to the creative process of incorporating AI in design.