My Private AI setup on laptop - Step by step guide to install LLM on your local Windows Laptop!

Are you fascinated by LLMs like GPT by OpenAI, and at the same time wondering if you will compromise your privacy, IP leak, Security, or Model Inaccuracies? well, the answer is PrivateAI!

Check out how you can install and configure LLM like LLAMA 2 on your Windows laptop, and overcome the concerns listed in the above paragraph!

Before we start, I highly recommend all of you see my earlier blog which lists the difference between AI, ML, DL, and GenAI

Pre-requisites: Windows 11, minimum 10GB RAM, Nvidia GeForce MX570 GPU 

Manual Steps

1) Download and install Visual Studio Build Tools, use the defaults like the following screenshot.

2) Install Conda / Anaconda, Git, wget

3) Go to the Start menu and launch "Anaconda Powershell prompt" Run as Administrator 

> conda install PyTorch

> conda install cuda --channel nvidia/label/cuda-11.7.0b 

4) Visit the Meta AI website and register to download the model/s. After registration you will get an email, please don't delete this email.

> mkdir llama2

> cd llama2

> git clone https://github.com/facebookresearch/llama

run the download.sh using Git Bash

Provide the URL that you get from the email 

Choose 7B-chat

It will take a while to download the model (it took nearly 22 min for me)

5) Once the model is installed, you can use it programmatically and interact with it, for which you need some GUI.

Automated way: If you are finding it difficult to follow manual steps, you can try following steps

• Using the command prompt, clone the repo "git clone https://github.com/oobabooga/text-generation-webui"
• From the folder, double-click on install_windows.bat
• This will install the required software on your laptop and start the UI http://127.0.0.1:7860/.
• Create an account at https://huggingface.co/ using the same email address that you have used to download the LLAMA 2 from the Facebook website.
• In the local UI, in the model tab search for meta-llama/Llama-2-7b and click download
• For the download to start, your request should have been approved by Meta and you need to set HF_USR and HF_PASS environment variables in your local machine by generating a token in the Huggingface > Settings > Access Tokens.
• Once downloaded, you can refresh, select the LLAMA, and load it (Please refer the following screenshot) 
• Then navigate to the Chat tab and start using it without worrying about privacy, IP leaks, Security etc.
• If you are facing issues in downloading Meta LLAMA, then try VMware/open-llama-7b-v2-open-instruct
• Enjoy your Private AI LLM set! 

Local LLM in action! 

References:

https://github.com/facebookresearch/llama
https://blogs.vmware.com/explore/2023/09/21/top-ai-ml-sessions-explore-2023/?utm_source=rss&utm_medium=rss&utm_campaign=top-ai-ml-sessions-explore-2023
https://git-scm.com/download/win
https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f058
https://github.com/oobabooga/text-generation-webui

Also, please check out my other posts related to this subject

• Power of Multi-Cloud DBaaS 

• MLOps Made Easy with Kubeflow on vSphere: Streamlining Machine Learning Workflows

• Artificial Intelligence, Machine Learning and Deep Learning for IT Operations

• Exploratory data analysis using Python

• Cloud Computing and Data Analytics fundamentals

• IT Operations Management

• MTTIC

• AIOps for Tenant and Platform Operation