ChatGPT requires a connection to the internet to use its AI assistant. Nvidia now does this locally with Chat with RTX, provided you have a compatible graphics card.

Everyone now knows ChatGPT and knows what you can do with it. All you need is an internet connection. You can ask the smart assistant anything, but you automatically sacrifice privacy. Nvidia wants to solve that with Chat with RTX, a tool that lets you create your own AI chatbot that runs locally on your PC.

The technology uses Retrieval-Augmented Generation (RAG), Nvidia TensorRT-LLM software and Nvidia RTX acceleration. For this you need a current GPU. Anyone who has an Nvidia GeForce 30 or 40 series graphics card in their laptop or desktop today can get started with the free tool on Windows 10 and Windows 11. Click here to download. The graphics card requires at least 8 GB VRAM.

In the demo we see that you have to load a data set each time, after which the tool does the rest and finds answers to your questions. Everything stays local to your PC. The only thing that requires an internet connection is analyzing a YouTube video. The question you ask and the answer generated remain local to your PC.

With Chat with RTX, Nvidia now supports various file types including .txt, .pdf, .doc/.docx and .xml. All you need to do is navigate to the folder that contains the data. The tool does the rest.

The manufacturer also encourages developers to play with Chat with RTX. There is a base project available on GitHub that you can build on. Anyone who creates an AI-supported application or a plug-in based on TensorRT-LLM RAG by February 23rd can win, among other things, an Nvidia GeForce RTX 4090. Further details can be found here.