Microsoft’s “Hybrid Loop” Shows the Basics of AI-Powered Client/Cloud Computing

Foresight: The most intriguing idea to come out of Microsoft’s Build conference this year is a forward-looking concept the company calls the “Hybrid Loop,” which defines a set of hybrid applications that bridge the gap between the cloud and edge client devices. At the basic level, hybrid applications will be able to run both locally on a client PC and in the cloud and, more importantly, dynamically switch between them.

If there’s one place you’re likely to hear where computing and apps are headed, Microsoft’s developer-focused Build conference is probably the place. True to form, this year’s Build introduced several interesting new concepts that managed to give a hint of where Microsoft’s vision for software is headed, while simultaneously hinting at some key technologies (notably CPUs). Arm-based) that they believe will be needed to power that future.

For hybrid applications, the cloud is meant to be viewed as an additional computing resource, much like a GPU can be used in addition to the central processor powering a typical PC. For example, these types of applications would be able to make run-time decisions for AI inference on client devices or on the Azure cloud.

Additionally, since most of these new applications are likely to incorporate some level of AI, they should also be able to take advantage of NPUs (Neural Processing Units) or other types of AI hardware accelerators. Thus, hybrid applications and the hybrid loop concept can be written to leverage the CPU, GPU, and NPU of a local device and, as needed or instructed, also utilize cloud-based computing resources.

It’s a fascinating theory, but how well it can be translated into actual capabilities remains to be seen. After all, the concept of cloud-based processing for client devices has been around for quite some time. It helps power virtual desktops and thin clients, among other applications, and forms the basis of Microsoft Windows 365 and its notion of the Cloud PC. For certain types of environments or situations, these solutions can work very well. However, performance limitations, scaling issues and more have forced them into niche deployments thus far and they only represent a tiny percentage of the overall client computing market.

Still, taking some of the principles of hybrid cloud environments and applying them to the customer certainly seems like it should work. Dynamic resource allocation has been a key part of cloud computing architectures since the early days, so it makes sense to want to apply some of these principles to client computing devices. Plus, one of the huge benefits of running in the cloud is that it makes platform-specific application issues much less of a concern. By their very nature, cloud-based applications are cross-platform and hardware-platform independent.

A big challenge is that until now apps have not been specifically written or optimized to work in these types of environments. Part of Microsoft’s vision is to use cloud-native development practices to build these AI-powered hybrid client/cloud applications. In fact, several Build announcements have focused on new tools to help developers start getting used to these ideas. Microsoft talked about leveraging its OnnX Runtime and Azure ML tools along with a new prototype AI toolchain to enable developers to build apps that fit into this hybrid loop model.

A practical challenge is that most PCs today don’t have dedicated AI processing. Intel and AMD have both talked about it and will likely have more dedicated AI silicon in future games.

A practical challenge is that most PCs today don’t have dedicated AI processing. Intel and AMD have both talked about it and will likely have more dedicated AI silicon in future games. Additionally, Nvidia’s CUDA efforts show that GPUs can be used for AI applications, but most of them only work on servers. The only current AI processing option for Windows PCs are the Qualcomm Snapdragon SoCs used in Windows on Arm-based devices, including Microsoft’s Surface Pro X.

Despite very modest shipments to date for this entire category, Microsoft strongly believes in the potential of Arm-based PCs and has made this clear through several announcements during Build.

First, Microsoft previewed Project Volterra, which is a tiny desktop PC/dev kit powered by Snapdragon. Project Volterra leverages Qualcomm’s new neural processing SDK for Windows and can be used to start building these kinds of AI-enabled hybrid apps. The Qualcomm tool leverages Snapdragon’s built-in Hexagon DSP, Kryo processor, and Adreno GPU that mobile developers have been using for several years to build AI-powered smartphone apps. Recognizing that the vast majority of PCs still have x86 processors, Microsoft also said it would bring NPU-specific support to all of its other Windows developer tools.

Speaking of developer tools, Build’s other big Arm-related announcement was the release of a full set of native Arm developer tools, including Visual Studio 2022, VSCode, Visual C++, Modern .NET6, and Java , and more. The company also said it is working to get a number of open source projects, including Python, node, git, etc., to natively target the Arm architecture. It’s all part of the company’s effort to make Arm an equal x86 citizen in the world of PC clients.

Even with the potential growth of Arm-based PCs and dedicated AI processors on PCs, it remains to be seen whether current client PC architectures can effectively leverage cloud resources. Although nothing has been announced, it looks like Microsoft is working on new kinds of cloud PC designs and architectures that might provide a better way to do this. We will see.

Microsoft doesn’t expect the Hybrid Loop concept to take off overnight, so many of these efforts are expected to span several years. Still, it’s quite interesting to see the direction in which the company sees the IT world moving. There are clearly big hurdles to overcome, but bringing together cloud computing, edge computing, client computing, AI and more into an organized whole that works across a very heterogeneous set of hardware resources is a vision that I think we’re going to be thinking about for a while.

Bob O’Donnell is the founder and chief analyst of TECHnalysis Research, LLC, a technology consulting firm that provides strategic consulting and market research services to the technology industry and the professional financial community. You can follow him on Twitter @bobodtech.

Sherry J. Basler