The Hated One
patreon
How to use ANY AI privately
🕑 Added 2024-11-15 22:32:00 +0000 UTCIntro
I want to teach you how to use AI privately. Not just how to run an LLM locally on your device, but how to use any AI out there privately.
By now, there are more than enough tutorials on how to run AI on your laptop, which is private but insufficient. Not everyone is gonna have the hardware to run a capable enough model on their device. You are gonna need a dedicated GPU and plenty of RAM in order to run something like a 7-billion-paramater Llama3, which is a very small model compared too GPT-4 or Gemini.
And even if you do have the hardware, sometimes these small open source models are just not capable enough. And in any case, you probably want to hop between multiple models that are gonna excel in different tasks.
So what can you do if you are in that situation?
In this private AI tutorial, I am gonna show some really cool things you can do take control of your data while using cloud based LLMs. And I am gonna teach not only how but mostly when you should hop between local and cloud-based services.
You are gonna learn what to do and what to avoid. There is so much to show you so be sure to stick till the end where I’ll share my secret methods I use to bypass all forms of identification with these online services.
Okay, ready? Let’s begin.
Data practices
Before I can teach you about the privacy techniques, you need to understand why you need to protect your privacy from AI. And the short answer is – because the situation is really bad.
AI companies like OpenAI, Microsoft or Google have really went all in on the race to the top. To the point where all safeguards went to the side. Which includes any basic protection of your messages and interactions. This is a major step backwards in privacy.
Anything you say or upload to ChatGPT, Copilot or Gemini will be collected, stored and retained for a long time. [0 – 2] Your chats will be tied to your identifiers, such as your account details and your IP address. Even if you request deletion of your data, they won’t actually delete the data sets. They’ll only remove your identifiable credentials from your profile. But your profile will remain theirs indefinitely. [2]
And what do they do with your private data? They allow human reviewers to process and annotate your conversations. Which means any confidential information you might share with these models, will be read by an indefinite amount of involved parties. And they will share your information with third parties without giving you a prior notice. [2, 3]
But there is a much deeper problem with AI that don’t respect your privacy. They use your prompts to train their models. And research has shown that it is indeed possible to extract training data from the language model by a range of malicious attacks. [4]
To keep this focused, I am gonna leave links to these research papers. But in short, LLMs are not supposed to memorize training data. But apparently they do and it is possible, very easy and cheap for anyone in the world to attack these models and get them to spill out their secrets. [5, 6] Which means whatever you tell these cloud-based AI tools could potentially be extracted by attackers from around the globe. And the larger the model, the more vulnerable it is to this data extraction. [4, 5]
Privacy protection
Hope that makes you understand why privacy invasion with AI is a disaster in the making. Now let’s move on to what you can do about it.
In simple terms, data collection by AI companies revolves around two major categories:
what data is collected
who it is collected from
There are different privacy techniques you need to implement to protect either of these individually.
In some cases, it might be enough to just protect your identity, while letting them steal your chats.
In other cases, you might want the reverse – that your identity will be revealed to the service provider, but the provider will not know what prompts you are sending their way.
The most extreme technique is the last combination – that is to prevent any data collection and identification at all.
So how do you navigate this?
Let me show you what no one else on this platform is gonna teach you. How to use a privacy invasive AI privately?
De-identification
In case you need to use a service like ChatGPT for whatever reason, you are in the first category of privacy techniques. Because all of your chats will be collected no matter what, focus all of your privacy techniques on de-identification. And it starts before you even create an account.
ChatGPT is an app you can download on Android and iOS but it also works in the web browser. And unless you are on GrapheneOS, your Android or iOS app will collect deep data from your phone through a range of invasive permissions. On top of that, Google and Apple will also collect your usage data about ChatGPT. Using a website doesn’t immediately expose you to this level of surveillance.
So using a native mobile app is out of the question, unless again you are on GrapheneOS. I will get to that later.
But before you go ahead and create an account through the website, you must do this first. Because your IP address will be collected and tied to your chats, you have to obfuscate it. The best way to do this is either with a VPN app or Tor. If you are new to VPNs, I recommend you start with the free version of ProtonVPN which you can use without an account. [7] If you are willing to pay for a VPN service, I recommend Mullvad above anything else. [8] As I usually say, do not use any of the popular YouTuber sponsored VPNs. Remember to run full-device VPN before doing anything else.
Now with the VPN up and running, you are still not ready to go to ChatGPT. You need an alias. An alias is a fake email address that will obfuscate your real one and thus protecting your identity from the provider. I recommend that you download SimpleLogin, create an account and generate a new alias. I am gonna assume that you use a password manager and do not reuse your passwords on different services. If you don’t, what are you even doing? Get Bitwarden. [9]
Now with a fake IP address and a fake email address, you are ready to go to the ChatGPT website. Copy your email alias and use your password manager app to generate a strong and unique password.
Now you can congratulate yourself. You have a pseudonymous account that will still collect your prompts but your identity will be separated from them. But your work is not done yet. Because what if you need to be more sensitive about your prompts and don’t want them recorded permanently?
Data minimization
This is when you need to leave services that make you their product and find those whose business it is that you are not. I’ve found three options that attempt to solve this problem. One of them is gonna run your prompts through a proxy and the other two will erase your prompts upon fulfilling the request. They all have of their pros and cons.
Venice.ai is a private permissionless cloud-based AI provider. [12] You can use it with or without an account, for free or with a premium. And while Venice will collect your metadata depending on the type of your account, they will route your prompts through a proxy and will not store or retain your conversations. They also do not require identifiable information and if you use a VPN and an email alias, they will not learn that much about who you are.
There is also a new chat and image AI from HuggingFace called HuggingChat. [10] This one works with multiple open source models again with or without an account. HuggingFace doesn’t share your conversations with anyone and you have the option delete them.
Brave browser also has their own AI service called Leo. [11] As long as you use the Brave-hosted models, your records will be purged once a response has been generated. It only works inside the Brave browser but you don’t need an account unless you are after the more advanced features. Brave also promises not to log your IP address or other identifiers, but I’d still use a VPN because why not at this point.
Local AI
Congratulate yourself some more because now you know how to de-identify yourself and minimize your AI footprint. But you should not rest here because there are scenarios where neither of these solutions are sufficient.
If you want to brainstorm a business idea or have an intimate one-on-one about your mental health, you want absolute certainty that none of your information ever leaves your device. Because you don’t want to just run a generic model. You want to be able to create your own AI character to have a conversation with or analyze your confidential business documents. In this case you are gonna need a laptop and a tool that will help you run AI locally, even without internet connection.
There are many tools that do this, like Ollama with OpenWebUI. [13] I have a tutorial on these tools that I did for Naomi Brockwell, which is already public on her channel. To use OpenWebUI, you first need to install Ollama. [14] Use Ollama to download models with command prompts. Then you need to install Docker. And then using a Docker command in the terminal install OpenWebUI. I will leave links to all of these commands in the description. [15]
OpenWebUI will run from a Docker container as a local host which you can open up in your default web browser. OpenWebUI allows you to customize your models or feed them your documents for the AI to analyze for you.
But for most users, I think a lesser known tool would be easier to navigate. That tool is called Jan, and you can install it from Jan.ai. [16] Jan will work with models downloaded from HuggingFace. To get started with a local LLM, choose a model that Jan recommends for your device. Jan can also connect to cloud based AI, but for maximum privacy, stay with the local ones instead. Even if you don’t have a beefy machine, you should still be able to run one of the smallest models relatively smoothly. You can also use the HuggingFace website to find more models and import them to Jan manually.
Secret sauce
But what if you need even more privacy. What you can’t make any compromises whatsoever and you might be a target of an advanced hacking group? In that case, you need something that even Edward Snowden recommends. It’s called GrapheneOS. [17]
I have GrapheneOS on my Pixel phone and I use it every day. But whenever I need to install an app I don’t trust, I put it in a separate user profile. [19] Which is this isolated space that is completely removed from the rest of system. This profile is running a full-device VPN before I even start doing anything on it. I download isolated Google Play Store [18] and create an anonymous Google account – anonymous because it doesn’t have my phone number, my IP address or my real name.
Then I download the app I don’t trust, in this case – ChatGPT, and use my fake information to sign up. And to pay for premium services, I use anonymously purchased gift cards for Google Play Store instead of using PayPal or regular debit cards.
I know my information is still collected but I don’t care because it’s fake and isolated and doesn’t lead to any identifiable information about me. This is only possible on GrapheneOS so I included this at the end of this tutorial, but if you want me to go through the process in detail let me know and I will make a video about it.
End
That’s it for now. Take control of your privacy because it matters. And support me on Patreon because I am not affiliated nor sponsored so this is just a pure advice to the best of my ability. Thank you!
SOURCES
[0] https://privacy.microsoft.com/en-US/privacystatement
[2] https://support.google.com/gemini/answer/13594961?hl=en
[3] https://openai.com/policies/privacy-policy/
[4] https://arxiv.org/abs/2311.17035
[5] https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html
[6] https://royalsocietypublishing.org/doi/10.1098/rsta.2018.0083
[7] ProtonVPN https://protonvpn.com/
[8] Mullvad VPN https://mullvad.net/en
[9] Bitwarden password manager https://bitwarden.com/
[10] Hugging Chat AI https://huggingface.co/chat/
[11] Brave Leo AI https://brave.com/leo/
[12] Venice AI https://venice.ai/privacy
[13] OpenWebUI https://openwebui.com/
[14] https://ollama.com/download
[15] https://docs.openwebui.com/getting-started/quick-start/
[16] Jan AI https://jan.ai/download
[17] https://x.com/Snowden/status/1175430722733129729
[18] https://grapheneos.org/usage#sandboxed-google-play
[19] https://grapheneos.org/features#improved-user-profiles
Comments
The Hated One
Yes, this is really good. Although the host OS you use this setup is not gonna be as secure as a mobile OS because of better MAC policies and containment of the system on a mobile security design. Just to keep that in mind. But kudos to your setup!
Jose Vanduka
No issues or limitations at all. I got the idea from how GrapheneOS works using the profiles. I manifested that idea on the computer.
The Hated One
Oh sorry about that. Will reupload.
rohitkumarankam
Audio is only playing on left channel
The Hated One
This is even better if you are okay with the potential VM limitations on your hardware. Or is this not an issue with your setup?
The Hated One
If a phone number is required, I first check to see if a temporary number works (you can buy these with crypto) or if I can bypass the phone number requirement (by changing a VPN server, changing device/browser and creating an account there, etc...). If needs a real phone number then I put the app in a separate profile with no other apps.
Eight-Legged Dj
Awesome video, really something much needed is in this of AI. Question: how do you handle profile anonymity when services like ChatGPT use phone numbers for verification. Moreover, is there any way to obtain anonymous access to the pro versions?
Jose Vanduka
This is really good and much needed. You packed a ton of info into that 11 minutes. I would also add, that I go one step further on cloud-based and local AI. I have a clean and locked-down host system that I never use for anything other than running VMs. I create a master VM image and keep it up to date. I then make on demand clones of the master and run local AI or cloud-AI in these separate and isolated VMs. Of course the host is running a VPN, etc. For cloud-based AI accounts, I use the clone one time and delete it. I am so happy that you produced this video. Great job.
