DAINAPP
patreon
New Application. Clip-App
🕑 Added 2021-07-06 17:01:58 +0000 UTC
This is the new "secret" application, its pretty simple for now, but I plan to keep slowly improving it for some time.
Ok, what is this?
This is a GUI that combine two AI Architectures, CLIP + VQGAN. It let you write a text and it will generate a image based on that text.
There is a reddit for those type of images, you can check it out here:
https://www.reddit.com/r/deepdream/
This GUI is based on a colab that can be found here:
https://colab.research.google.com/drive/1go6YwMFe5MX6XM9tv-cnQiSTU50N9EeT
Other than some modifications I did to the code, you can use the colab if you like.
How to use it:
You write whatever you want the the text input box, and it will try to generate a image based on that. Each line will be a new image. There is some tricks the deepdream community found during testing. Like adding Unreal Engine at the text might create more crisp results.
Using the special character | it will let the AI separate the word tokens, but usually I don't like the results using this character.
Usually you want something between 1200 and 1600 loops to generate a decent image.
First time using it might download a ~500mb model file.
Memory Usage:
This is where it get tricky.
There is a option that affect the VRAM that the app need. But even so it's require a lot of memory.
4gb VRAM:
- Can generate 200X200 images using the Resample Image Model
- Cannot generate images using the Konia Image Model
8gb VRAM:
- Can generate 400X400 images using the Resample Image Model
- Can generate 200X200 images using the Konia Image Model
Konia model use more VRAM, but it also produce better results. I will try to improve Vram usage, but can't guarantee it.
Will try to post some results later on, but is very similar to the ones found in deepdreams.
Comments
Buda Dangol
Happy to help
GRisk
Hi there, I will read about it and try to find a fix for it. I might need your help with some tests, alright?
Buda Dangol
I finally got my NVLink but it still runs out of memory with the same size image files. not sure how to get it to address memory on both cards?
GRisk
Thanks! Nice to see the output as video, once the next Rife update is ready I will update the Clip-App
sure, here's one: https://www.youtube.com/watch?v=YD0JuyoasZ8 obvs i do some other stuff to it, not just raw output.
GRisk
Hi Graham, glad you are liking, the credits go to CLIP creator and the nice fellows that made the colab. I'm just trying to make it more accessible to everyone while doing my own upgrades on the code. Actually, I plan to export the PNG files to mp4 inside the app, just need a few more updates.
dear GRisk, this little thing is incredible. i'm realizing now you did not intend for it to be used to create videos. but the videos it creates are amazing! please make it spit out image files with the same number of characters so that RIFE can make sense of them when flipping the images into a video. frankly it seems like you could "Export to RIFE" from Clip-App. if anybody else is doing this alchemy
Buda Dangol
i am trying 800x800
Buda Dangol
will see what the maximum I can do is
GRisk
If you test it and get some cool results, please share with me. Mine can only handle 200X200 lol
Buda Dangol
My dual 3090s thank you.
