A researcher managed to transfer the GPT-2 model to an Excel file. The experiment serves to provide insights into the structure of the LLM and the underlying architecture.

Does an LLM fit in an Excel spreadsheet? Anyone who has ever asked themselves this question now knows the answer. At a conference in Seattle, researcher Ishan Anand showed how he managed to put OpenAI’s GPT-2 model into a spreadsheet. He now also shares his work on GitHub.

The GPT-2 model is a predecessor of the current version GPT-4, although the two models are hardly comparable with each other. GPT-2 consists of “only” 1.5 billion parameters, compared to 1.76 trillion(!) parameters for GPT-4. The GPT-2 model can also only process a maximum of ten tokens or single words at a time, while GPT-4’s limit is 128,000,000.

Anand also had to use a “smaller” version of the model to transfer it to Excel’s XLSB library, which can accommodate 124 million parameters. Due to these limitations, Anand has not yet been able to integrate GPT-2 into a Google Spreadsheet.

Prediction of tokens

GPT-2 doesn’t allow you to chat like ChatGPT. The first version of the popular chatbot was based on a later version of GPT, GPT-3.5. GPT-2 works according to a prediction mechanism in which the model supplements the input with the most likely continuation. The experiment is primarily intended to gain more insight into how an LLM works and the underlying Transformers architecture in a more accessible way.

OpenAI launched GPT-2 in 2019: the source code and neural network are available for research on the company’s website. It is noteworthy that they initially did not dare to do this for fear that the application could be misused “to generate misleading, biased or offensive language on a large scale”. Five years and several GPT versions later, the same concerns about generative AI are still being raised, but it has now become an unprecedented hype in the technology sector, thanks in part to OpenAI.