Are you fascinated by LLMs like GPT by OpenAI, and at the same time wondering if you will compromise your privacy, IP leak, Security, or Model Inaccuracies? well, the answer is PrivateAI!
Check out how you can install and configure LLM like LLAMA 2 on your Windows laptop, and overcome the concerns listed in the above paragraph!
Before we start, I highly recommend all of you see my earlier blog which lists the difference between AI, ML, DL, and GenAI
Pre-requisites: Windows 11, minimum 10GB RAM, Nvidia GeForce MX570 GPU
Manual Steps
1) Download and install Visual Studio Build Tools, use the defaults like the following screenshot.
3) Go to the Start menu and launch "Anaconda Powershell prompt" Run as Administrator
> conda install PyTorch
> conda install cuda --channel nvidia/label/cuda-11.7.0b
4) Visit the
Meta AI website and register to download the model/s. After registration you will get an email, please don't delete this email.
> mkdir llama2
> cd llama2
> git clone https://github.com/facebookresearch/llama
run the download.sh using Git Bash
Provide the URL that you get from the email
Choose 7B-chat
It will take a while to download the model (it took nearly 22 min for me)
5) Once the model is installed, you can use it programmatically and interact with it, for which you need some GUI.
Automated way: If you are finding it difficult to follow manual steps, you can try following steps
- Using the command prompt, clone the repo "git clone https://github.com/oobabooga/text-generation-webui"
- From the folder, double-click on install_windows.bat
- This will install the required software on your laptop and start the UI http://127.0.0.1:7860/.
- Create an account at https://huggingface.co/ using the same email address that you have used to download the LLAMA 2 from the Facebook website.
- In the local UI, in the model tab search for meta-llama/Llama-2-7b and click download
- For the download to start, your request should have been approved by Meta and you need to set HF_USR and HF_PASS environment variables in your local machine by generating a token in the Huggingface > Settings > Access Tokens.
- Once downloaded, you can refresh, select the LLAMA, and load it (Please refer the following screenshot)
- Then navigate to the Chat tab and start using it without worrying about privacy, IP leaks, Security etc.
- If you are facing issues in downloading Meta LLAMA, then try VMware/open-llama-7b-v2-open-instruct
- Enjoy your Private AI LLM set!
References:
Also, please check out my other posts related to this subject