Cloudflare wants to bring AI inference closer to the user through Workers AI. The company is launching a new developer platform to run AI models on the network instead of in a data center.

Every company today is looking for ways to integrate some form of artificial intelligence into their service offering. The rapidly increasing and opaque costs of maintaining AI models and the explosion of new tools and vendors are just some of the challenges companies face. Cloudflare is stepping into the breach with a new developer platform that allows companies to build AI applications without having to manage infrastructure.

The Cloudflare platform consists of three components. Starting with Workers AI, which Cloudflare calls the first serverless AI. Finally, Cloudflare operates the GPUs on its global network. This means customers no longer have to walk around with “suitcases full of GPUs,” CEO Matthew Prince told SiliconAngle.

AI closer to the user

With this offer, Cloudflare is responding to a sensitive aspect, namely latency. Because the GPUs are in the network, you can take the AI ​​models with you anywhere, so to speak. Data for inference does not need to come from a data center, making it available for wordloads more quickly. Among other things, Workers AI is intended to make it possible to bring large AI models to the edge.

Cloudflare also launches the Vectorize vector database for AI models. From generating embeds for the built-in models and indexing them to querying and storing the source data in R2. Vectorize ensures that this all happens on the same platform.

Finally, there is the AI ​​Gateway, which helps developers and business managers stay on top of things. Today, there is little insight into the cost of AI infrastructure or how many queries are being executed and from where. AI Gateway aims to bring more transparency to AI traffic and also includes measures such as caching and rate limiting to keep costs under control.