Following the announcements of Copilot+-enabled AI PCs at the Microsoft Build developer event on May 20, Microsoft released new developer tools, improvements to Microsoft Azure AI, and new enterprise options for Copilot. GitHub Copilot received a long list of new capabilities enabled by first-party and third-party services.
Meanwhile, reactions to AI Recall's memory feature include some backlash against its observation of all user activity. Announced at Microsoft Build on May 20, Recall enables search across any activity on a PC with Microsoft AI, allowing the user to ask questions in natural language and receive answers to all their activity on the device.
Team Copilot, AI agents and Copilot Studio open business opportunities
Microsoft, on May 21, offered three new ways to work with its AI assistant Copilot: Team Copilot, Copilot agents, and Copilot Studio on the Microsoft Power Platform.
“Driving AI-accelerated PC use cases in the enterprise makes a lot of sense for Microsoft,” said Olivier Blanchard, research director at The Futurum Group. “It's a huge opportunity for Microsoft, PC OEMs and silicon suppliers, from Qualcomm to Intel and AMD as an entire ecosystem.”
“By taking the lead from the operating system, support, software, and developer ecosystem angles, Microsoft positions itself as the primary orchestrator and facilitator of AI PC adoption in the enterprise, taking a lot of pressure off PC OEMs and silicon suppliers,” he said. Blanchard.
Team co-pilot
Team Copilot adds more initiative to the AI assistant, allowing it to function, as Microsoft said, as a kind of teammate or facilitator. Team Copilot can take notes, manage meeting agendas, flag important information or unresolved issues, and manage projects. Customers with a Microsoft Copilot license for Microsoft 365, which costs $20 per month, will be able to try Team Copilot in preview later in 2024.
Co-pilot agents
With co-pilot agents (which Microsoft spells in lowercase when referring to this specific offering), you can ask AI to take on custom roles based on your business needs. For example, a co-pilot agent could handle requests, automate processes, add context to processes and meetings, or learn based on user feedback. Agents are available to customers in Microsoft's Copilot Studio Early Access program, with wider availability expected later this year.
Copilot Studio on Microsoft Power Platform
To create copilot agents, you can use Copilot Studio, the no-code, message-based platform, to design and test what actions the copilot can perform. Developers can ask their co-pilot to perform specific tasks and then provide the AI with text or other resources it may need to have context about that specific business process. Microsoft Copilot Studio is free with a Microsoft Copilot for Microsoft 365 license in limited private preview.
Copilot stack and Snapdragon development kit for Windows unlock Copilot+ PCs
For developers preparing to work on the new Copilot+ PCs, Microsoft introduced the Copilot stack in Windows, additions to the operating system that leverage and facilitate the development of more AI models and capabilities. The Copilot stack includes:
- Windows Copilot Runtime, with its library of AI-powered APIs on the device.
- Windows Semantic Index, the operating system capability behind Recall. In the future, Windows Semantic Index will connect with the Vector Embeddings API to help developers create vector stores and perform augmented retrieval generation within their applications.
- Phi Silica, a small language model tailored for the neural processing units in Copilot+ PCs.
- Native support for PyTorch on Windows with DirectML.
- Developer preview of web neural networks for Windows in DirectML.
- New AI-powered productivity features in Dev Home.
Qualcomm announced the Snapdragon Dev Kit for Windows on May 21, allowing developers to access the NPU on Copilot+ PCs. The development kit includes:
- 3.8GHz 12-core Oryon CPU with dual-core boost up to 4.3GHz.
- 32 GB LPDDR5x memory.
- 512GB M2 storage.
- 80 watt system architecture.
- Supports up to three simultaneous external displays.
AI comes to Microsoft Fabric
Microsoft Fabric now includes real-time intelligence powered by AI, which can analyze and discover data. The real-time hub allows users to view their organization's data in real time and set up automatic alerts. Real-time intelligence is now generally available and in public preview.
More updates to Microsoft Fabric are detailed on the Microsoft site.
GitHub Copilot can communicate with Docker and other services
GitHub Copilot, the generative AI coding assistant, has received updates to allow it to communicate with first-party and third-party partner services, including as Copilot Expansions. Of particular relevance to Microsoft Build is GitHub Copilot for Azure, which allows GitHub users to deploy to Azure using prompts.
Copilot expansions are only available to invited users for now. The full list of first-party and third-party services that GitHub will be able to extract data from is divided by availability date.
Guest users can currently access Copilot Extensions that allow them to pull data from the following in GitHub Marketplace to GitHub Copilot:
- DataStax.
- Stevedore.
- Try Lambda.
- LaunchDarkly.
- McKinsey and company.
- Octopus implementation.
- Pangea.
- Pineapple.
- Product science.
- Read me.
- Sentinel.
- Equipment Tool Kit.
The following extensions will be available to all users “in the coming weeks” via the Visual Studio Marketplace for VSCode, according to GitHub:
- Microsoft, including Teams Toolkit and Microsoft 365.
- Stripe.
- MongoDB.
Azure AI Studio is now available, with additional security barriers against hallucinations and cyberattacks
At Build 2024, Microsoft announced the general availability of Azure AI Studio, the pro-code platform for generative AI development. Azure AI Studio can be found at ai.azure.com for free. An Azure account is required to create a co-pilot.
Microsoft expanded Azure AI Studio's suite of responsible AI tools and security features, including filters for specific content, rapid shields to fight rapid injection attacks, and additional precautions against hallucinations.
Some of the new Azure features will take advantage of NVIDIA's Blackwell AI accelerator.
UK casts doubt on Recall privacy
After Microsoft announced the Recall AI search feature on Copilot+ laptops on May 20, some praised its novel approach to searching on a full PC. However, the “snapshots” that AI takes of user activity have prompted the UK Information Commissioner's Office to contact the Redmond giant for more information.
“We are consulting with Microsoft to understand the safeguards put in place to protect user privacy,” the ICO said.
The ICO believes companies should “rigorously assess and mitigate the risks to people's rights and freedoms,” the ICO told the BBC.
SEE: Our definitive guide to defend against common threats to companies
On May 20, Tesla CEO and former OpenAI backer Elon Musk compared Recall to the horror TV show. black mirror.
Microsoft said recovery snapshots are stored only on the local PC, a feature enabled by the on-device AI accelerator. Users can pause recovery, set recovery to ignore individual websites and apps, or delete individual snapshots and time ranges.
Blanchard said the ability to delete records or selectively choose what appears in Recall means “users are in control of the experience and their own privacy. Additionally, unlike search engines, that data will be stored on users' PCs, adding an extra layer of data privacy and security to the feature,” he said.
However, Blanchard said: “What Microsoft will have to figure out is what happens to that data when PC users replace or upgrade their Copilot+ PCs in a few years, but there's time to find out.”
