Ultimate Batch Image Pre-Processing Gradio APP for Windows and Cloud (RunPod , Massed Compute) With Yolo Models - 1-Click To Install And Use

Furkan Gözükara

hi it is 2 stages. first auto crop and then resize for exact resolution. the logic explained in this video and hopefully i will make a new video soon : https://youtu.be/Fbuyu35TkE4

Joseph Eudave

Hello, im hab=ving issues croping images. the images are exported but with the same resolution and without croping. i tried reinstalling everything, installing in an anaconda env, but its still not working. is there something im missing?

Furkan Gözükara

thank you for explanation. a major improvement and new feature hopefully coming soon. i will make a dedicated video for the app

Dmitry

For those who can't work with the "image resizer" tab. Example: We have an image "img1.jpg"(768x512) and we want to change it to a 1024x1024 image - Create a folder named "1024x1024" and put our file - "img1.jpg"(768x512) there - Important: Let's say your folder is located on the desktop: С:\Desktop\1024x1024 The path to specify: "С:\Desktop\" - this is the path that the app will use to search for the "1024x1024"folder. If you specify the path With: С:\Desktop\1024x1024, the application will not see your images, so it will first search for the folder: "1024x1024" and not the files inside the \Desktop\1024x1024 directory. It took me a few nerve cells to figure out what was going on. I hope this will help someone. PS Many thanks to the author of the app for this awesome tool. It may be correct to explain in detail on the "image resizer" tab how to specify the\path correctly.

Furkan Gözükara

i recommend 1024x1024. it already process all sub folders. just try to understand folder format

Hockey

Which resolutions do you recommend putting into "Aspect Ratios (comma-separated, e.g., 1024x1024,1280x720,1920x1080,...)" box for ultimate Dreambooth training later? I have lots of low resolution and high resolution pictures. Can you choose a folder and it will also work on all the subfolders?

Furkan Gözükara

find_duplicates is still not working yet. please remind me tomorrow hopefully i should add.

Furkan Gözükara

awesome

reaper557

Thank you very much for continuing to work on and update this app! I particularly appreciate the face extraction, which I have just used in a recent model train. Your implementations work so much better than any janky solutions I've coded up in Python with Qwen2.5 (still fun to do). (づᴗ _ᴗ)づ♡

Kevin O'Malley

I replace the find_duplicates function with this to make it work. import threading import queue def process_image(q, folder_path, hash_dict, duplicate_images, destination_folder, hash_size, cutoff, lock): while True: try: filename = q.get_nowait() except queue.Empty: break image_path = os.path.join(folder_path, filename) try: with Image.open(image_path) as img: temp_hash = imagehash.average_hash(img, hash_size) with lock: if temp_hash in hash_dict: duplicate_images.append((filename, hash_dict[temp_hash])) shutil.move(image_path, os.path.join(destination_folder, filename)) else: for h, existing_file in hash_dict.items(): if abs(temp_hash - h) < cutoff: duplicate_images.append((filename, existing_file)) shutil.move(image_path, os.path.join(destination_folder, filename)) break else: hash_dict[temp_hash] = filename except Exception as e: print(f"Error processing {filename}: {e}") finally: q.task_done() def find_duplicates(folder_path, destination_folder, hash_size, cutoff, num_threads): os.makedirs(destination_folder, exist_ok=True) image_files = [f for f in os.listdir(folder_path) if f.lower().endswith(('png', 'jpg', 'jpeg', 'bmp'))] total_files = len(image_files) hash_dict = {} duplicate_images = [] lock = threading.Lock() q = queue.Queue() for filename in image_files: q.put(filename) threads = [] for _ in range(num_threads): t = threading.Thread(target=process_image, args=(q, folder_path, hash_dict, duplicate_images, destination_folder, hash_size, cutoff, lock)) t.start() threads.append(t) for t in threads: t.join() with open('duplicates.txt', 'w') as f: for dup in duplicate_images: f.write(f"{dup[0]} is a duplicate/near-duplicate of {dup[1]}\n") print(f"Found {len(duplicate_images)} duplicates/near-duplicates. Results saved to duplicates.txt.")

Furkan Gözükara

you are welcome it can sometimes happen

Astaroth

Reinstall worked, it seems like the first install was done with missing required tools. Thanks for the amazing work and fast answers !

Furkan Gözükara

hi it should work. can you reinstall and send me install logs : monstermmorpg@gmail.com

Astaroth

I am trying to run the image cropper

Furkan Gözükara

hi in which screen? some screens are not coded yet. i am planning look at them

Astaroth

Hi I get this error while running the script : File "C:\ImgPreProcess_V6\venv\lib\site-packages\ultralytics\models\yolo\model.py", line 26, in __init__ safe_run("/tmp/ultralytics_runner") File "C:\ImgPreProcess_V6\venv\lib\site-packages\ultralytics\utils\downloads.py", line 283, in safe_run os.chmod(path, 0o770) FileNotFoundError: [WinError 3] Le chemin d’accès spécifié est introuvable: '/tmp/ultralytics_runner'

Furkan Gözükara

the logic is simple. the auto cropper will crop the subject from the image but will not crop the subject body parts or any parts. then you need to run resizer. for 1280x1280 to work first crop to 1280x1280 then it will generate that folder and that time resizer will work. resizer will crop subject body parts / parts as well. if it is human it will focus on face and will not crop it

Nicolas Nicolau

Im if i want only use image resizer i get this in console: ERROR:root:Folder not found for resolution 1280x1280. Skipping this resolution. INFO:root:Total images to process: 0 Bat i hae the some pics i use in the previous example

Nicolas Nicolau

Hi. I try crop and resize with your app. The problem is, the person are cropped correctly, but dont resize to the dimmensions we put, the script only crop. No errors in the console. When i go to resize directory created "1024x1024" (for example) the pics are biggest, like the original, but only the person image 1/1 F:\testpic\testpic (5).jpg: 448x640 1 person, 62.8ms Speed: 3.3ms preprocess, 62.8ms inference, 5.8ms postprocess per image at shape (1, 3, 448, 640) image 1/1 F:\testpic\testpic (6).jpg: 448x640 1 person, 41.5ms Speed: 3.1ms preprocess, 41.5ms inference, 1.5ms postprocess per image at shape (1, 3, 448, 640) image 1/1 F:\testpic\testpic (7).jpg: 640x448 1 person, 1 bench, 57.5ms Speed: 2.6ms preprocess, 57.5ms inference, 2.1ms postprocess per image at shape (1, 3, 640, 448) image 1/1 F:\testpic\testpic (8).jpg: 448x640 1 person, 36.3ms Speed: 2.5ms preprocess, 36.3ms inference, 1.5ms postprocess per image at shape (1, 3, 448, 640) image 1/1 F:\testpic\testpic (9).jpg: 448x640 1 person, 58.9ms Speed: 2.6ms preprocess, 58.9ms inference, 1.0ms postprocess per image at shape (1, 3, 448, 640) Cropping images: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:41<00:00, 1.38s/image] Processing complete! Processed: 30, Skipped: 0, Total: 30/30 images in 00:00:41.

Furkan Gözükara

hi some functions are not coded yet. only crop and resize working perfectly atm.

s s

Furkan Gözükara

we have it. you can use this app just upgraded to yolo v11 for you : https://www.patreon.com/posts/112126955

楠 陈

Can you create a standalone YOLO installation package? https://huggingface.co/Ultralytics/YOLO11

Furkan Gözükara

sure

Brian Skipper

Thanks, yes I had them in a folder named 786x1024 that was generated by your cropping script. They are the right aspect ratio but they wouldn't resize with the script. It did resize a few other images that were in the same folder. I'll try to reproduce this and see what is shown in the terminal when I get time.

Furkan Gözükara

you need to enter resolutions that you want and also give parent folder path that will have folders named as them.

Brian Skipper

Having a strange issue with the resize tab on 3 images. The cropped fine under 768x1024 but when I got to resize them I get the message "No images found to process in the specified resolution folders." I have resize without cropping checked. I'm glad it's only 3 images so I can manually do it for now, but do you know what is going on? The cropped images resolutions are: 1590x2121 2261x3014 2109x2812

Furkan Gözükara

1 : kohya has no limit i saw people does millions of images 2 : i prefer all 1024x1024 as a beginning - best is compare cropped vs bucketing enabled 3 : my gui has certain objects others will be center cropped 4 : it works 5 : when you train multiple objects with FLUX they get mixed, currently testing de-distilled models to see if this can be improved 6 : SD 3.5 may work better will full research that too hopefully soon

대훈 조

I would like to fine-tune on various objects, not on a specific person or item, but on a wide range of different objects. I have a few questions regarding this situation: 1. In Kohya, how many images can I include for fine-tuning? For example, can a large dataset with tens of thousands of images (like 10,000) be processed within the Kohya interface? 2. Is it necessary to crop and resize the images? If I have a dataset with various image sizes, do I need to ensure that all images are of the same size? 3. If resizing is required, does your GUI provide a way to crop & resize images of various objects? 4. Does fine-tuning work well even when there are mixed image file formats, such as PNG, JPG, and JPEG?

Shepard4k

jesus that was fast, thank you!!

Furkan Gözükara

added that feature as v3

Shepard4k

Is it possible to save the cropped images in jpg format instead of png? i need to crop 2700 jpg images und would like to save them in jpg again.

Furkan Gözükara

ye not easy. maybe you can try claude 3.5 and modify code and see if it can do something that will work for your case

Goldwaters

It would be extremely useful if you could extract all detections instead of just the highest confidence one. Somewhere in this code you are taking the best detection and looping over it. Couldn't you tell it to loop over each detection and use the same code each time? https://freeimage.host/i/2JAU2YQ (Maybe not simple, but just thinking, as I'm not a programmer myself exactly)

Furkan Gözükara

not that i know. probably you should manually crop such images

Goldwaters

Is there a way to specify which detection to crop? For example, an image with multiple people - it seems like YOLO just crops the image to the first person it detects in the image. Even just letting it crop the image multiple times for each person would help and then you could just delete the extras.

Furkan Gözükara

perfect

abaj006

Yes, thanks for that. I followed your tutorial video for cudnn and now works.

Furkan Gözükara

yep i just tested with fresh install cuda 12.4 and cudnn 8.9.7 works perfect - they are my system default as shown in video

Furkan Gözükara

please follow this video you are missing cudnn : https://youtu.be/DrhUHnYfwC0

abaj006

I believe I have all the dependancies installed, but getting this error on Windows 10 PC: [WinError 126] The specified module could not be found. Error loading "C:\Users\UserName\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\lib\cudnn_cnn_infer64_8.dll" or one of its dependencies.

Furkan Gözükara

It is very likely to c++ tools please follow this post and reinstall after that https://www.patreon.com/posts/111553210?utm_campaign=postshare_creator&utm_content=android_share

Error_404_unknown

Hi I keep getting this error W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found 2024-09-17 21:39:27.818229: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine." I tried updating tensorboard in the venv but that didn't work either. Is this version not compatible with CUDA 12.3? I don't really want to downgrade my system CUDA and risk breaking everything else.

Furkan Gözükara

delete olders use this one. this one is faster better more optimized

Dmitry

Hello, a little confused, I have a version of resizer_v3, if I understand correctly, this is the same resizer_v3 but with the ability to install on RunPod and other external resources? Ie, the old folder resizer can be deleted, I want to restore order in the files, trying to figure out what and how

Furkan Gözükara

ye others are not ready yet :)

Training AI

Thank you for this! Cropping & resizing tabs work perfect. Got the following error when trying the Find Duplicates tab: ``` Traceback (most recent call last): File "C:\Python310\lib\multiprocessing\queues.py", line 244, in _feed obj = _ForkingPickler.dumps(obj) File "C:\Python310\lib\multiprocessing\reduction.py", line 51, in dumps cls(buf, protocol).dump(obj) AttributeError: Can't pickle local object 'find_duplicates..process_image' ``` Edit: OOPS I just noticed your note that the other tabs are still under development. Thank you sir!