Furkan Gözükara patreon

Auto Installer Bat Files For Automatic1111 & DreamBooth Extension On Windows

🕑 Added 2023-06-18 23:10:38 +0000 UTC

Comments

Furkan Gözükara

Hello. For endpoints sadly we don't have any. It is only useful for companies so I didn't cover it.

Pedro Atencio

Hi everyone, I just saw these tutorials (The END of Photography, Ultimate RunPod Tutorial For Stable Diffusion) and I want to know if there is any documentation explaining how to use automatic1111 and dreambooth in Runpod but using endpoints.

Furkan Gözükara

yes this installs specific version. otherwise new version wont working good

zimzimma bimbimma

Tried following along with your video tutorial with a fresh install but when I tried to train I got the error "not supported between instances of 'nonetype' and 'float'", not sure what this is but I tried using settings I previoulsy used when dreambooth was working and the results are great, thanks for uploading this mate :)

Furkan Gözükara

yes use new folder it should work great

Nenad Kuzmanovic

I have one question? I already have auto1111 installed. Can i keep that checkout, a make new one (in new folder, but on the same partition)? I'm asking cause i had problem with permissions and access tokens on Huggingface with diffusers library, when i had two installs (don't ask why i had two checkouts :-)

Furkan Gözükara

Hello. It is inaccurate path. You need to put it inside : stable-diffusion-webui\models\Stable-diffusion stable-diffusion-webui\models\Stable-diffusion

wamaka kaminyoge

I having trouble with installing Realistic vision V2.0 . I placed the file in /models/dreambooth . However, it does not appear in the sourcecheckpoint dropdown in the interface

Furkan Gözükara

Hello you were right. I just updated Part 2. it will fix that error. thank you

Furkan Gözükara

hello. did you run part1 and then part 2? part 2 fixes errors. i am testing right now.

Hint Fan

Hello! I have this error after launching Launching Web UI with arguments: --xformers Traceback (most recent call last): File "C:\DreamBooth Tutorial\stable-diffusion-webui\launch.py", line 38, in main() File "C:\DreamBooth Tutorial\stable-diffusion-webui\launch.py", line 34, in main start() File "C:\DreamBooth Tutorial\stable-diffusion-webui\modules\launch_utils.py", line 340, in start import webui File "C:\DreamBooth Tutorial\stable-diffusion-webui\webui.py", line 35, in import gradio File "C:\DreamBooth Tutorial\stable-diffusion-webui\venv\lib\site-packages\gradio\__init__.py", line 3, in import gradio.components as components File "C:\DreamBooth Tutorial\stable-diffusion-webui\venv\lib\site-packages\gradio\components.py", line 37, in from gradio_client.documentation import document, set_documentation_group ImportError: cannot import name 'set_documentation_group' from 'gradio_client.documentation' (C:\DreamBooth Tutorial\stable-diffusion-webui\venv\lib\site-packages\gradio_client\documentation.py) Press any key to continue . . . I have tried uninstalling everything and do a clean install but nothing works I have also tried this tutorial of yours at https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/How-To-Install-And-Use-Kohya-GUI-And-Do-Ultra-Realistic-SDXL-Training-Tutorial.md After downloading everything and try to run Kohya, I have the same error regarding this gradio_client Sorry for being a noob and looking forward for your help! Thank you!

Furkan Gözükara

wow this is important will add to the top

Chris

I think i found the error that i made. I tried to run part1.bat on an external drive, that is not ntfs, but exfat. This messed up the way git tries to set permissions on the repo directories or sth like that, i guess. after running the bat on another ntfs drive, all seems to work again. maybe that infomation is sth that you can add to your guide, to use ntfs and not sth like exfat.

Furkan Gözükara

yes. and then run part 2 and then run web ui

Chris

You mean sth like part1.bat > log.txt?

Furkan Gözükara

you are welcome. you can also record cmd output and send me so i can tell better

Chris

Okay, will try again in a few hours and give you feedback then. Thank you!

Furkan Gözükara

hello again. i just did a fresh install here my CMD output. run part 1 until completes. it will give some error. run part 2. then start web ui it is working : https://pasteboard.co/CermNFhzJ43M.png

Chris

Yeah, another drive letter, and fresh folder

Furkan Gözükara

testing right now. installing into a fresh folder right?

Chris

Hello, i am getting errors with a fresh install with the two bat files. It worked a few weeks ago, then i wanted to install freshly on a new ssd, now i get these errors: [+] xformers version 0.0.22 installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [!] accelerate NOT installed. [+] diffusers version 0.16.1 installed. [!] transformers NOT installed. [+] bitsandbytes version 0.35.4 installed. ####################################################################################################### # LIBRARY ISSUE DETECTED # ####################################################################################################### # # accelerate not installed. # transformers not installed. # # Dreambooth may not work properly.

Furkan Gözükara

great

Yar Malik

no stress! i put everything from windows to runpod for faster execution and its working.

Furkan Gözükara

hello. did you do a fresh install? we are installing certain older version for both automatic1111 and DreamBooth extension. which base model you tried to train? i will test for you

Yar Malik

I keep on getting this error, when I create my model in Automatic1111 and I tried 3-4 times. Any idea how I can fix it. Btw I named my model bestRealism2 Missing model directory, removing model: C:\dreambooth_tutorial\auto\stable-diffusion-webui\models\dreambooth\bestRealism2\working\text_encoder In Detail: The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\requests\models.py", line 816, in generate yield from self.raw.stream(chunk_size, decode_content=True) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\urllib3\response.py", line 934, in stream data = self.read(amt=amt, decode_content=decode_content) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\urllib3\response.py", line 905, in read data = self._raw_read(amt) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\urllib3\response.py", line 811, in _raw_read with self._error_catcher(): File "C:\Python310_11\lib\contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\urllib3\response.py", line 729, in _error_catcher raise ProtocolError(f"Connection broken: {e!r}", e) from e urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(377497523 bytes read, 1333043057 more expected)', IncompleteRead(377497523 bytes read, 1333043057 more expected)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\sd_to_diff.py", line 1290, in extract_checkpoint text_model = convert_ldm_clip_checkpoint(checkpoint) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\sd_to_diff.py", line 622, in convert_ldm_clip_checkpoint text_model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14") File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\transformers\modeling_utils.py", line 2432, in from_pretrained resolved_archive_file = cached_file(pretrained_model_name_or_path, filename, **cached_file_kwargs) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\transformers\utils\hub.py", line 417, in cached_file resolved_file = hf_hub_download( File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\huggingface_hub\file_download.py", line 1457, in hf_hub_download http_get( File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\huggingface_hub\file_download.py", line 524, in http_get for chunk in r.iter_content(chunk_size=DOWNLOAD_CHUNK_SIZE): File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\requests\models.py", line 818, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(377497523 bytes read, 1333043057 more expected)', IncompleteRead(377497523 bytes read, 1333043057 more expected)) Missing model directory, removing model: C:\dreambooth_tutorial\auto\stable-diffusion-webui\models\dreambooth\bestRealism2\working\text_encoder Restored system models. Duration: 00:19:24 Exception loading config: [Errno 2] No such file or directory: 'C:\\dreambooth_tutorial\\auto\\stable-diffusion-webui\\models\\dreambooth\\bestRealism2\\db_config.json' Traceback (most recent call last): File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\dataclasses\db_config.py", line 410, in from_file with open(config_file, 'r') as openfile: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\dreambooth_tutorial\\auto\\stable-diffusion-webui\\models\\dreambooth\\bestRealism2\\db_config.json' Can't load config! Traceback (most recent call last): File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1326, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1229, in postprocess_data self.validate_outputs(fn_index, predictions) # type: ignore File "C:\dreambooth_tutorial\auto\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1204, in validate_outputs raise ValueError( ValueError: An event handler (load_model_params) didn't receive enough output values (needed: 10, received: 7). Wanted outputs: [html, html, html, html, html, html, html, dropdown, dropdown, html] Received outputs: ["", "", "", "", "", {'visible': True, 'choices': [], 'value': '', '__type__': 'generic_update'}, "Error loading model params: 'bestRealism2'."]

Furkan Gözükara

Latest version not working. The author left the maintenance of the script. So auto installer installs last working good version. Currently I suggest Kohya SDXL DreamBooth. I plan to search best hyper parameters for SD 1.5 in Kohya too hopefully soon.

Artie Medvedev

It's strange that Training Steps Per Image (Epochs): 201 in attached settings png files. But this does not matter much. But the Class Images Per Instance Image = 0 in the settings may confuse people. Also I see you are using newer version of automatic1111 v1.6.0. But the version from auto installing script is 1.4.0. Do both versions work properly? And the dreambooth interface from your settings snaps is a bit different. So I assume it is also a different version. Do you now if the version of dreambooth you included in auto install scripts already features --shuffle_after_epoch improvement from InB4DevOps? P.S. I had no luck training dreambooth model with latest versions of a1111 and dreambooth. See no convergence. Tried different settings. The instance dataset is perfect. The class dataset was from your patreon. Did you tried the latest version?

Furkan Gözükara

it is not an error you can continue using 100%

taiga

after doing both bats: Launching Web UI with arguments: No module 'xformers'. Proceeding without it. ------------- CUDA SETUP: Loading binary D:\dream\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cudaall.dll... *** Failed reading extension data from Git repository (sd_dreambooth_extension) Traceback (most recent call last): File "D:\dream\stable-diffusion-webui\modules\extensions.py", line 65, in do_read_info_from_repo if repo.active_branch: File "D:\dream\stable-diffusion-webui\venv\lib\site-packages\git\repo\base.py", line 897, in active_branch return self.head.reference File "D:\dream\stable-diffusion-webui\venv\lib\site-packages\git\refs\symbolic.py", line 357, in _get_reference raise TypeError("%s is a detached symbolic reference as it points to %r" % (self, sha)) TypeError: HEAD is a detached symbolic reference as it points to 'dc413a14379b165355502d9f65856c40a4bb5b6f'

Furkan Gözükara

yes after that run part 2 it will fix httpx error

taiga

just tried bat 1: File "D:\dream\stable-diffusion-webui\venv\lib\site-packages\httpx\_transports\default.py", line 275, in __init__ self._pool = httpcore.AsyncConnectionPool( TypeError: AsyncConnectionPool.__init__() got an unexpected keyword argument 'socket_options' Press any key to continue . . . PS D:\dream>

Furkan Gözükara

you are welcome

Andrew Giesler

Brilliant!

Furkan Gözükara

do this. make a new folder. run part 1 until you get error. then run part 2 and it will get fixed

Andrew Giesler

Unfortunately, part1.bat gave me an error toward the end. Immediately after "Launching Web UI with arguments: --xformers", I get a traceback for a "TypeError: AsyncConnectionPool.__init__() got an unexpected keyword argument 'socket_options'". The script terminates with a "press any key" so the webui doesn't load. Is this a known problem with the script, and do you know whether there's a workaround? I do have SD working without Dreambooth in a separate directory.

Furkan Gözükara

this is still best for SD 1.5 until kohya adds EMA which is coming very soon. But Kohya SDXL Dreambooth is far superior than this

San Milano

Is this better than training with Kohya Dreambooth?

Furkan Gözükara

yes you can stop training with dreambooth and resume it. it supports both

Rafał Ryniak

Can I manually stop and resume the training so as to check the trained safetensor and remove unnecessary ones, each has 4GB, memory is running out )))

Furkan Gözükara

Awesome. It is hard at the beginning but you will understand over time very well

Furkan Gözükara

Yes you can convert it to Lora by using possibly Kohya Gui

Furkan Gözükara

Yes you can try

Rafał Ryniak

OK, there was no question, the answers come to me after a while from space, the dreambooth training is so long because it trains the entire safetensor checkpoint, i.e. later I move within the realisticvision model with my character, I did lora training in KoyaSS before, also according to your tutorial but for checkpoint 1.5 it took less than h and was also satisfactory but then i move within lora and i can try other checkpoints, am i right or wrong?

Rafał Ryniak

Last question, if I select the safetensor that suits me, will I be able to convert it to lora somehow, maybe the question is absurd but like I said I'm blind. )))

Rafał Ryniak

I would like to thank you very much, I've been working as a computer graphic designer for 25 years, I specialize in Photoshop, photo processing, I'm burnt out professionally and I got bored with my current job, thanks to AI, and especially thanks to your knowledge, photography has become my passion again, last month I haven't stopped from the computer, my knowledge of AI was zero, now I absorb it like a sponge, I am aware that I am still blind, but thanks to you all these things are easier. Best regards from Poland. P.S. Dreambooth flies on )))

Furkan Gözükara

use previous safe tensor file. maybe last one failed. find the one that is working

Rafał Ryniak

when i set the completed safetensor and try to generate the image, the SD doesn't finish it, is it normal if the training is not finished, to free up space i threw out some initial safetensor, will the training still finish if i start with the last completed safetensor, it took almost 10h

Furkan Gözükara

it is normal. dreambooth method fully fine tunes the model. so they are full model.

Rafał Ryniak

sory english, each of my safetesor is about 4gb, is it normal that it's so big?

Furkan Gözükara

load your last safetensors not original realistic vision to continue dreambooth training. your last safetensor will be used as continuing

Rafał Ryniak

i.e. should I load my last safetensor as a training model? or original realisticvision?

Furkan Gözükara

create a new training model from last checkpoint and you will be continuing from there

Rafał Ryniak

Hey, what if the runpod is full during the night during the training and the training hasn't ended, can I somehow resume it with a command after freeing up space?

Furkan Gözükara

Yep it generates a new venv and install there. Sorry for late reply I am on 1 week vacation

Casper Smit

Is it installing in a venv?

Furkan Gözükara

Just to let other knows. We fixed the issue. Probably it was because of skipping changing python relauncher py and forgetting to restart pod or forgetting to update cudnn after restarting the pod. All written on github file

Furkan Gözükara

indievish

2023-07-08T12:47:48.745387542Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 7.79 GiB (GPU 0; 47.54 GiB total capacity; 33.87 GiB already allocated; 7.64 GiB free; 38.48 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

indievish

Hi - I tried on runpod but I still get the Out of Memory error on 1 x RTX A6000 12 vCPU 31 GB RAM torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 7.79 GiB

Furkan Gözükara

You are welcome. Sorry that I couldn't prepare an auto script for mac since I don't own a mac

Kellan

I tried, but decided just to go with the manual route. Thanks!

Furkan Gözükara

Unfortunately not. I don't have mac so I can't verify if will run or not. Have you tried? But if you raise to gold membership I can connect your computer and try to install on your Mac if you prefer.

Kellan

Possible to run on Mac?

Furkan Gözükara

alternatively you can rent a runpod pod and do exact training with spending lesser than 5$. i have runpod installer script on patreon it is up to date : https://www.patreon.com/posts/runpod-auto-84716845

Furkan Gözükara

For 10 GB your best option is doing LORA training. You can't train dreambooth. I have kohya lora video : https://youtu.be/TpuDOsuKIBo - you can also use this extension for LORA training. hopefully will make a video for that too