sd.next and sd 3.5 medium

[michelgirault]

Hi everyone,
I'm very new to sd.next but i was wondering if someone was successful running sd 3.5 medium on sd.next?
currently with all my attempt I was not able to do it, i get always some error message

[vladmandic]

hhmm, yes? it's supported and documented.
did you read wiki?
what is the error? did you document it and create an issue?

[michelgirault]

thanks for the feedback, i check and look at the wiki, with the gui i was able to via the network->reference->sd 3.5 medium, my following question, is there any document to do that via the api/cli and not the gui? thank you

[vladmandic]

reference models can be loaded via api like any other model.
pseudo-code: post('/sdapi/v1/options', { 'sd_model_checkpoint': model_name })

[CalculonPrime]

I'm having a problem with SD 3.5 medium as well. I downloaded the safetensors file from CivitAI here. It's a 4.75 GB file. When I try to load it into the web UI, I get this error:

00:41:17-372516 INFO     Autodetect model: detect="Stable Diffusion" class=StableDiffusionPipeline
                         file="c:\AI\sd-vladmandic-webui\models\Stable-diffusion\stableDiffusion35_medium.safetensors"
                         size=4871MB
Diffusers 2013.10it/s ██      33% 2/6 00:00 00:00 Loading pipeline components...
00:41:17-391389 ERROR    Load model:
                         file="c:\AI\sd-vladmandic-webui\models\Stable-diffusion\stableDiffusion35_medium.safetensors"
                         pipeline=Autodetect/NoneType config={'low_cpu_mem_usage': True, 'torch_dtype': torch.bfloat16,
                         'load_connected_pipeline': True, 'safety_checker': None, 'requires_safety_checker': False,
                         'local_files_only': False, 'extract_ema': False, 'config': 'configs/sd15', 'use_safetensors':
                         True, 'cache_dir': 'C:\\Users\\biff\\.cache\\huggingface\\hub'} Failed to load CLIPTextModel.
                         Weights for this component appear to be missing in the checkpoint.
                         Please load the component before passing it in as an argument to `from_single_file`.

                         text_encoder = CLIPTextModel.from_pretrained('...')
                         pipe = StableDiffusionPipeline.from_single_file(<checkpoint path>, text_encoder=text_encoder)


00:41:17-400830 ERROR    Load model:
                         file="c:\AI\sd-vladmandic-webui\models\Stable-diffusion\stableDiffusion35_medium.safetensors"
                         is not a complete model
00:41:17-403806 ERROR    Load model: name="stableDiffusion35_medium" not loaded
00:41:17-405793 INFO     Startup time: 7.51 torch=3.28 gradio=0.77 diffusers=0.08 libraries=1.03 extensions=0.46
                         ui-networks=0.12 ui-txt2img=0.26 ui-img2img=0.06 ui-control=0.09 ui-settings=0.18
                         ui-extensions=0.42 launch=0.25 api=0.06 app-started=0.15

So in SD 3.5, the safetensors file isn't a complete model? It is for SD 1.5, so I guess I have a lot to learn.

I read your WIKI here and it doesn't say manually downloading the file is unsupported, just that it's recommended to use the "reference model." The WIKI says to load the reference model via Networks -> Models -> Reference, but I don't see a Networks page in the web UI. When I go to the Models page and try to search CivitAI, it just hangs and I see a bunch fo black squares, never any search results.

How can I manually download the remaining pieces of SD 3.5 medium that it's saying are missing?

The WIKI says that SD 3.5 consists on these parts:

• Unet/Transformer: MMDiT
• Text encoder 1: CLIP-ViT/L,
• Text encoder 2: OpenCLIP-ViT/G,
• Text encoder 3: T5-XXL Version 1.1
• VAE

Which of these parts are in the safetensors file that I referred to and which are missing?

[vladmandic]

So in SD 3.5, the safetensors file isn't a complete model? It is for SD 1.5, so I guess I have a lot to learn.

safetensors file can be anything - whatever author chooses to put inside it. it could be a full model. in this case, its NOT.

The WIKI says to load the reference model via Networks -> Models -> Reference, but I don't see a Networks page in the web UI.

networks button is on top-right, close to generate button!

[CalculonPrime]

Is there a tool like 7Zip to browse the contents of a safetensors file? In the case of the SD 3.5 medium model I gave the link to above, it states that it's a "checkpoint trained." From looking at the model page, how can I tell that the checkpoint will be incomplete?

Then as far as downloading using Networks -> Models -> Reference, it's not working for me at all. There doesn't seem to be an easy way to search for a reference model with a wildcard string, but I can find it using the browser's text find. When I click on the SD 3.5 Medium model, then at the top of the UI it shows something like "processing 23423.0/0.9s", but the elapsed time just keeps growing, never stops, and looking at my internet activity, nothing is being downloaded. In the terminal window where python is running no activity is shown. What am I doing wrong?

[vladmandic]

In the terminal window where python is running no activity is shown. What am I doing wrong?

yes, but its not really for end-users, its more for developers. see cli/model-keys.py.

In the terminal window where python is running no activity is shown. What am I doing wrong?

search works - its a string match, do not use wildcards.
when you click on it, what does the console say?

[CalculonPrime]

The "black squares" I mentioned when doing a search under "Models" are actually present because the output in the "light-teal" theme is black text on a black background - so completely invisible text. Unlike the other case, this is not just a judgment call but broken. If I attach fixed CSS for that theme will you accept it or do I need to try to submit it via github itself?

Also, it would improve the UI to allow resizing the columns in the search results. As it stands you have to scroll each model title box to see the full title unless you zoom out the browser to make the fonts really tiny.

But the bigger problem is that even though I provided a hugging face token in both system settings and the model search/download page, it doesn't seem to get picked up. Download of the model fails with an access error:

02:52:11-139630 ERROR Diffusers download error: id="stabilityai/stable-diffusion-3.5-medium" model access requires login 02:52:11-144587 INFO Available Models: path="models\Stable-diffusion" items=1 time=0.00 02:52:11-146573 INFO Diffuser model downloaded: model="stabilityai/stable-diffusion-3.5-medium"

I also tried downloading from Networks > Models > Reference but there there's no place to specify the token. Where is it picked up for that case? Also, why is there only a hugging face token in system settings and not tokens for other sites, like CivitAI?

Also, you said I can wildcard search in Networks > Models > Reference with the existing code, but that doesn't seem to be true. If I don't click "Reference" so that the search box (top left) is empty, then I can type any text into it and have it search as expected, but if I click reference, then the box is populated with "Reference/" and so if I type "3.5" after that it doesn't find the 3.5 models. I.e. "Reference/ 3.5" So it's not an intuitive workflow.

Probably what you want to do there is, generate an "and" search of all patterns separated by whitespace in that box. So if the box contains "Reference/ 3.5" then you do a search that requires both "Reference/" and "3.5" to hit on a record to include it.

[CalculonPrime]

Writing this as an update rather than editing the original in case others have the same issue. The access problem was because in addition to setting up the token, you have to go into the SD 3.5 medium page on huggingface and provide your personal details on that page (even though you already have an account on huggingface and access token).

The other questions I asked, though, like why there's no CivitAI token in system settings, remain.

[CalculonPrime]

It did ultimately download SD 3.5 medium, but it's running very slow. I have a 4060 Ti with 16GB VRAM and SDXL models are quick with automatic 1111, but SD 3.5 medium with this web UI is taking 45 seconds per iteration, so the default 20 iterations takes 15 minutes!

During this time, the VRAM is showing full use, but the GPU and CPU are near 0% utilization.

[vladmandic]

If I attach fixed CSS for that theme will you accept it or do I need to try to submit it via github itself?

github pr is preferred, but if you send it here, i'm not going to reject it.

I also tried downloading from Networks > Models > Reference but there there's no place to specify the token. Where is it picked up for that case?

in settings.

The access problem was because in addition to setting up the token, you have to go into the SD 3.5 medium page on huggingface and provide your personal details on that page (even though you already have an account on huggingface and access token).

this is pretty much as expected and fully documented. if model publisher requires it, nothing i can do about it.

why is there only a hugging face token in system settings and not tokens for other sites, like CivitAI?

because nobody asked for it? if you have a need for that, create a feature request.

but SD 3.5 medium with this web UI is taking 45 seconds per iteration, so the default 20 iterations takes 15 minutes!
During this time, the VRAM is showing full use, but the GPU and CPU are near 0% utilization.

this basically tells me that you're out of vram and gpu is swapping to system ram which is 20x slower.
and you need to tune your settings accordingly. but that's not a topic for here.

[CalculonPrime]

I was able to run both SD 3.5 medium and large on my system using ComfyUI, with the default workflows (1024x1024):

• SD 3.5 medium took about 30 seconds to complete generation
• SD 3.5 large took 1 minute 48 seconds to complete generation

So nothing close to 15 minutes. Also, as I saw in the case of SD 3.5 large, overflowing memory doesn't leave CPU+GPU idle as was the experience I recounted above. With ComfyUI I ran about 20-30% GPU usage with 3.5 large and its memory overflow.

Are you saying you don't want to discuss what must be a configuration problem? Or I should start another thread? I think other people using your web UI are going to hit this as well as I didn't do anything strange to the config - I just downloaded the model from inside the UI.

[vladmandic]

I'm saying that is EXACTLY the vram swapping issue.

amd yes, I don't want to discuss it all in a single thread, there are way too many things in this one so far.

Also pls read wiki and enable offloading.