With the increasing popularity of artificial intelligence, companies’ IT capabilities are reaching (and over) their limits. Is the future of AI in the cloud or in the edge?

KubeCon 2023 couldn’t escape the current topic of conversation: artificial intelligence. Everyone seemed to agree that (generative) AI is the technology of the future during the big crowd for cloud-native development. But alongside optimism, attention was also paid to the many headaches that the AI ​​race is causing for many IT companies.

Of course, an AI model doesn’t just come into being. Just look at GPT-4: the language model behind the hugely popular ChatGPT tool. The model was trained with 170 trillion parameters. This number alone may not say much, but it is ten times as many parameters as GPT-3 and makes GPT-4 by far the most complex model to date. The more parameters, the more a model can do, so to improve the AI, more parameters must be added.

This requires huge amounts of data in the terabyte range and the necessary capacities to store and process them. Microsoft has already spent hundreds of millions of dollars on GPUs to keep ChatGPT alive. That might not be money for Microsoft, but the average IT company doesn’t have the budgets and data center capacity to do it. Even OpenAI CEO Sam Altman warns that ChatGPT’s growth spurt is no longer sustainable.

As a result, small businesses are marginalized and AI falls into the hands of a select group of tech giants. But even the big behemoths can’t keep building data centers to house all these algorithms forever. During KubeCon, experts from the cloud industry try to find the pieces of the puzzle to solve the capacity conundrum. The edge is also viewed with great interest. Where is the future of AI?

Is Edge the new cloud?

Edge computing has long been recognized as a technology with a potentially significant impact on the IT industry. Today, around 10 percent of enterprise IT data is generated and processed outside of traditional data centers. According to Gartner, it should be 75 percent by 2025. Over time, the edge computing market could grow up to four times larger than the public cloud.

For these reasons, Edge is sometimes referred to as the “new cloud,” though we should be careful with such generalizations, Intel’s Arun Gupta warns. “It’s true that Edge is taking on more and more workloads that would previously have taken place in the cloud, but the two are too different to be considered the perfect replacement.” Red Hat’s Daniel Fröhlich prefers to describe Edge as “all computing applications that don’t take place in the cloud or a data center”.

Both gentlemen hit the nail on the head. First of all, the data flows at the edge and in the cloud are completely different. The idea of ​​cloud computing is that devices send data to the cloud where it is processed and managed from a central platform. Edge computing turns this around. The data stays on the device itself as much as possible, pulling the computing infrastructure towards the data, so to speak.

On diet

Edge computing thus offers many advantages for organizations with more limited resources. First there is the latency. Since cloud computing sends data to the cloud provider’s data centers and then back to the device, the process is delayed.

This delay is only a few milliseconds and is barely perceptible to the naked eye, but it is still there. In an edge environment, you experience little to no latency as the data stays local. In the context of autonomous vehicles, for example, these few milliseconds can decide whether or not a collision occurs. After the intervention, the data can still be moved to the cloud for more in-depth post-analysis.

In addition to speed, edge also offers advantages in terms of (cost) efficiency. Basically, workloads in the edge require less powerful IT equipment, because the workloads must also be able to run at locations where no data center can be set up. Or, as Fröhlich illustrates in his lecture: “In the marginal area, platforms can be put on a diet”.

By this, the German engineer means that edge platforms are usually “lighter” than their cloud counterparts, but still contain full distributions. For example, cloud-native principles can also be applied in an edge environment where less pure computing capacity is available. Platforms like Red Hat MicroShift, KubeEdge and Azure Edge then build the imaginary bridge between cloud and edge.

data dilemma

Do you need to cancel your cloud subscriptions immediately and move all your workloads to the edge? Just wait a while with that. There are some factors that can definitely be a hindrance in connection with artificial intelligence.

Whichever way you look at it, AI requires computing power. Lots of computing power. The advantage of the cloud is that storage and computing capacity can be scaled more flexibly if required. The (many) data required to run models can therefore be obtained from a central data pool. Gigantic models like GPT-4 are currently too overwhelming for the Edge.

Another potential problem that Marino Wijay (Solo.io) points out is the lack of a stable network. Even at the edge, servers and devices must communicate with each other. However, edge networks can sometimes be in remote locations where Wi-Fi, 4G or 5G are not available. How do you solve this? This is where a service mesh comes into play. A service mesh is an infrastructure layer built on top of an application to manage traffic between applications.

When it comes to security, there are pros and cons to both cloud and edge. Organizations dealing with sensitive data may prefer to store it at the edge. After all, you do not have to entrust this to any external party and you have control over whether the data is stored in compliance with data protection regulations.

On the other hand, edge security is a complex matter. Basically, the main security risks for edge servers are no different from the risks for the cloud: Malware, routing attacks and DDoS are common threats, and physical damage must also be considered. However, due to the distributed nature of Edge, there is a need for a more robust security strategy based on advanced authentication methods.

“It’s true that Edge is taking on more and more workloads that would previously have taken place in the cloud, but the two are too different to be considered the perfect replacement.”

Arun Gupta, Intel

No black and white story

The choice between cloud or edge that many organizations face today does not have to be an either/or story, but rather a both/and story. Because edge is so different from cloud, it should be viewed as a complement rather than a replacement. One workload will perform better on a public cloud server, while other workloads will feel better on an edge server. As a result, organizations are increasingly turning to a combined, hybrid approach to maximize the benefits of cloud and edge.

Even during KubeCon, the experts keep the ball in the middle. “There is no one size fits allsolution,” concludes Gupta. Artificial intelligence comes in many shapes and forms. Organizations that want to get into the technology should therefore first ask themselves what exactly they want to achieve with AI and adapt their infrastructure accordingly.