The term ‘machine learning’ was first coined in the late 1950s. Since then, the application of artificial intelligence has been at the heart of the global commercial and public sector vision for leveraging technology to predict outcomes with greater efficiency and automate previously human-reliant activities.
Microsoft has been at the core of this movement, with the firm’s work to explore how AI and other intelligent technologies could be used to achieve these goals dating back to the foundation of Microsoft Research (MSR) in 1991.
“From the earliest days at MSR, one of our main goals was to create more natural user experiences and to do this we used speech, language understanding and vision to model how people think and communicate,” says Lili Cheng, corporate vice president of Microsoft AI and research. “We have been committed to democratising these technical innovations by partnering with universities and the open-source community, publicly sharing our research for over 30 years.
“All Microsoft products have incorporated innovations which began at MSR including the Azure cloud, developer tools, Xbox, Windows and Office. Today, given the vast amount of information people need to access and interact with for work, they can benefit from using AI as a co-pilot, helping them to prioritise their time so it is spent on the things that matter most.”
Microsoft has collaborated with many organisations around the world to continue driving its research and delivering AI products that empower public sector and commercial businesses to achieve more.
“MSR’s commitment to making the latest technological advances openly available means that more people can create innovative solutions for the scenarios that matter for their communities,” says Cheng. “Our partnerships with the Massachusetts Institute of Technology and the Barcelona Supercomputing Center, as well as open-source organisations like OpenAI, allow these technologies to be accessible to broad communities.”
The use of AI is now widespread, from simple text predictions to complex data analytics. One of the major ways it is reaching users is via the cloud and augmented reality technologies, such as Microsoft Azure and HoloLens.
“We build our HoloLens experiences and platform on top of the Azure cloud,” says Cheng. “This includes all our AI cognitive services and machine learning tools, communication services, low-code tools, as well as our cloud and edge platform components. The Azure services we leverage in Microsoft products can also be used and extended by any developer. This allows for a full range of solutions – from out-of-the-box solutions to highly customised low- and pro-code developed solutions.”
These technologies are also creating more tangible business use cases for the metaverse. Though the concept – now well-known due to the relaunch of Facebook as Meta – has seemed vague and abstract to many, Microsoft is using its augmented reality solutions and HoloLens to create the industrial metaverse.
“The metaverse can be used in many different ways, for example, with people gathering in an imaginary world or meeting together for a virtual work meeting,” says Cheng. “Our focus on the Industrial Metaverse allows organisations and their increasingly mobile frontline workers to overlay relevant information in real-world environments to complete service tasks, get help from remote experts, or receive training.”
Workers can also use HoloLens or a mobile device – alongside Microsoft solutions such as Dynamics 365 Field Services, Remote Assist and Guides – to receive step-by-step instructions and remote assistance from experts. In this scenario, the expert can see what the worker sees and annotate the physical world to instruct workers in real time.
On top of this, Microsoft is enabling partners and customers to make use of the Dataverse. “It provides a single location to store all your data, improve interoperability and run intelligence across data from various applications,” says Darren Bennett, general manager of mixed reality applications at Microsoft. “Imagine your mixed reality data is stored in the same place as data for your inventory, work orders or internet of things signals; now you are looking at the shortest path from insight to action in the real world.”
With the business value of augmented reality and AI so clear, Microsoft’s ecosystem of partners is building on the technologies to deliver further industry-specific solutions.
“Many customers have very unique problems that they are using holograms to address,” says Bennett. “Fortunately, there are some very savvy independent software vendors that will build a bespoke experience that a general toolset cannot easily address. This might offer an intense level of accuracy or pushing the limits of the device that might not be a good trade off in a general toolset.
Another way that partners are driving the establishment of these technologies in industry is with content. “For example, 30 per cent of words used in a paper manual are used to describe where something goes,” says Bennett. “A holographic arrow pointing to the exact placement removes the need for any of these words, thus simplifying and speeding up the whole process. While we have invested a lot in making our authoring application very easy to use, we have partners who are experts in the creation of training materials. They help customers get started and leave them with a foundation to be able to grow their library of guides to encompass all their procedures.”
The work Microsoft is doing with AI and HoloLens is key to helping the firm to achieve its mission of empowering every person and organisation on the planet to achieve more.
“We have a number of automotive companies using HoloLens to help with training, doing work on the factory floor and giving remote assistance,” says Cheng. “One example is Toyota – it uses HoloLens with Guides in its factories to train new employees from day one.”
Mercedes-Benz is another loyal Microsoft customer that is making the most of the latest augmented reality and AI services. It uses the HoloLens Remote Assist feature in its service bays to help technicians deal with its ever-evolving fleet of vehicles. “Using Microsoft Teams, an expert technician can see what the service centre technician is seeing,” says Cheng. “HoloLens senses the surface of the vehicle they are looking at and shares a 3D description of that surface within the Teams application. The expert can then draw on that surface from wherever they are based; they don’t need to travel to offer real-time feedback in the physical work area. The scenario feels so intuitive that you can forget that advance computer vision is at play.”
In an ever-changing technology landscape where new approaches are being taken to improve the way businesses use AI, the cloud and augmented reality solutions, Microsoft has high hopes for what its solutions might enable in the future.
“Mixed reality, Azure and AI, along with the apps and services – like Field Service, Remote Assist and Guides – that empower frontline workers, are all important to deliver the Industrial Metaverse,” says Cheng. “This new iteration of the metaverse will enable organisations across the world to rethink how they design and optimise their processes – and we are proud to be part of it.”
A variety of Microsoft partners also contributed to this feature: Augury, Anywhere365, Automated Intelligence, Axis Communications, AVEVA, Blue Yonder, CyberProof, Infosys, Johnson Controls, Mazars, Metafile Information Systems, Orbus Software and Teradata. Read about how they are using Microsoft’s technologies to help customers access the benefits of AI and ML.
This article was originally published in the Winter 2022 issue of Technology Record. To get future issues delivered directly to your inbox, sign up for a free subscription.