Google is releasing a new “variant” of the Gemini model as a preview. Gemini 1.5 Pro can accurately process up to millions of tokens.

Gemini 1.5 Pro is a new and improved version of the Gemini model that Google launched in December last year. It is an “intermediate” LLM that is at a comparable level to Gemini 1.0 Ultra but requires less computing power. All Gemini variants have multimodal capabilities, meaning they can handle different types of input, from text to images to audio.

Gemini 1.5 Pro should excel at processing long documents. Google boasts that its latest model has the longest context window of any large-scale Foundation model to date. The preview version can handle inputs up to a million tokens, but Google writes in the technical paper that the limit of Gemini 1.5 Pro is ten million tokens.

For comparison: Claude 2.1 processes up to 200,000 tokens, GPT-4 Turbo is limited to around 130,000. In the demo below, Google Gemini 1.5 Pro delves into the Apollo 11 mission transcript, a 402-page document.

The devil is in the details

This means that Gemini 1.5 Pro can process enormous amounts of information at once. One million tokens is equivalent to approximately 1 hour of video, 11 hours of audio, code files with more than 30,000 lines of code or more than 700,000 words. The model can also filter very specific information from these long token strings.

This achieves 99.7 percent accuracy for up to a million tokens, and for ten million it is still 99.2 percent, according to benchmarks Google provides in the paper. Chatbots based on the model can therefore have long conversations without forgetting details, even with complex tasks or many follow-up interactions. According to Google, it is also possible to personalize the model without having to fine-tune it.

Google will offer all of its Gemini models through Vertex AI. There, Google Cloud gets access to pre-built models as well as APIs that you can use to create your own Gemini bot. The general public will likely learn to spot twins through the chatbot Bard, which now also bears the name of the underlying model.