The biggest news from Microsoft Build 2024

An upcoming preview for Microsoft Copilot in Azure. Team Copilot’s ability to bring generative artificial intelligence to entire organisations and departments. And upgrades to Defender for Cloud to better protect AI apps.

These are some of the biggest news and coming features the Redmond, Wash.-based tech giant unveiled during its annual Microsoft Build event, being held Tuesday through Thursday in Seattle.

Based on Microsoft’s announcements, the vendor seeks to expand its domination of the nascent GenAI market by giving developers more access and more control while creating copilots, chatbots, workflow automations and other outcomes of this emerging technology.

Microsoft Build 2024

Build 2024 is expected to have about 200,000 people registered for in-person or online attendance, with 4,000 people attending in person in Seattle, according to Microsoft. The event features more than 300 sessions, demos and labs.

Multiple news items from the conference focused on opportunities with Microsoft Fabric, an AI-powered analytics platform that the tech giant revealed can more than quadruple ROI, with users seeing a payback period of less than six months.

All the advancements come shortly after Microsoft Chairman and CEO Satya Nadella issued a memo to employees urging them to prioritise security over new feature releases when necessary, a memo prompted by concerns over a 2023 Microsoft cloud email breach.

Indeed, as part of Build 2024, Microsoft said that within its Azure AI offering, it has 20 “responsible AI” tools with more than 90 features.

On the vendor’s April quarterly earnings call, Nadella pointed to the early opportunities with AI and to systems integrators adopting tools for fine-tuning artificial intelligence models as a sign of growing market maturity.

Here are more of the biggest news items from Build 2024.

Copilot in Microsoft Fabric, AI skills

Microsoft has made Copilot in its AI-powered analytics platform Fabric generally available.

Copilot in Fabric aims to help data professionals increase productivity and let business users more easily search through data. Users can leverage conversational language to create dataflows, generate code, generate functions, build machine learning models and visualise results, according to Microsoft.

This copilot is on by default for eligible tenants, with experiences in Data Factory, Data Engineering, Data Science, Data Warehouse and real-time intelligence, which are all still in preview.

The copilot experience for real-time intelligence will allow users to type questions in conversational language. Copilot will automatically translate Kusto Query Language (KQL) users can execute.

Also in preview is AI skills, a capability meant to allow any user to have a conversational question-and-answer experience with data. Users select the data source in Fabric they want to explore and can ask questions without configuration, according to Microsoft.

The experience will show the query generated to find the answer to the user’s question. And users can add tables, set more context and configure settings for a better experience.

AI skills honor existing security permissions. And users can configure this capability to respect unique organisation nuances. Abilities to enrich Microsoft Copilot Studio-made copilots and interact with Microsoft 365 Copilot are “coming soon,” according to Microsoft.

More Microsoft Fabric updates

The preview of Fabric’s real-time intelligence is one of the biggest reveals at Microsoft Build 2024. Real-time intelligence is meant to help users take actions on high-volume, time-sensitive granular data.

Analysts can leverage low-code and no-code experiences with real-time intelligence, but professional developers might also benefit from this new feature thanks to the code-rich user interfaces, according to Microsoft.

This workload combines Synapse Real-Time Analytics and Data Activator for event streaming data and offers a new real-time hub for discovering, managing and using that data. Users can transform and route events in the hub to any Fabric data store and create new streams.

Users can also trigger actions like alerting a production manager when equipment is overheating or rerunning jobs when data pipelines fail with real-time intelligence.

Microsoft also launched a preview for a new Fabric Workload Development Kit, which aims to allow developers to extend applications within the platform. Apps built with the kit will appear as a native workload in Fabric. A workload hub is coming soon in Fabric, according to Microsoft, adding a spot where users can discover, add and manage kit-made workloads.

API for GraphQL, also in preview, allows data professionals to access data from multiple sources in Fabric with a single query API, according to Microsoft. Users can turn to API for GraphQL to streamline requests, potentially reducing network overhead and accelerating response rates.

Fabric user data functions, in preview, work across notebooks, pipelines, event streams and other data services.

Inside Fabric’s Data Factory experience is a new feature, data workflows. Powered by the open- source Apache Airflow, data workflows promise users the ability to author, schedule and monitor workflows or data pipelines using Python, according to Microsoft.

Users can leverage data workflows to define Directed Acyclic Graphs files for complex Fabric data workflow orchestration, according to the vendor.

Fabric OneLake enhancements

Along with these Fabric updates, Microsoft has expanded shortcuts for the platform’s unified, multi-cloud OneLake data lake to connect to data from on-premises and network-restricted data sources beyond Azure Data Lake Service Gen2.

An on-premises data gateway means users can now make shortcuts to Google Cloud Storage, Amazon S3 and on-premises or otherwise network-restricted S3-compatible storage buckets, according to Microsoft.

Fabric also expanded integration with Snowflake and Azure Databricks, according to Microsoft. Users gained more interoperability between Snowflake and OneLake. OneLake will also support Apache Iceberg in the future.

Azure Databricks Unity Catalog tables integration with Fabric will allow users to create and configure new items in Fabric in a few clicks, among other use cases, according to Microsoft.

The ability to access Fabric lakehouses and other data items as a catalog in Azure Databricks is coming soon, according to Microsoft.

Copilot, Copilot Studio expansions

Microsoft said that later this year it will launch a preview for a way Copilot for Microsoft 365 license holders can bring AI to a team, department or whole organisation.

Team Copilot promises to facilitate Teams meetings, manage the agenda, track time and take notes, according to Microsoft. It can moderate Teams chats by summarising information and responding to group member questions.

In Planner, Team Copilot can act as a project manager by creating tasks, assigning tasks, tracking deadlines and notifying team members when they need to give input.

In the meantime, early access program members have gained the ability to make custom copilots in SharePoint for quick information retrieval. A public preview of this capability will come later this year, according to Microsoft.

Now generally available is SharePoint Embedded, allowing developers to build file- and document-centric apps that can be integrated with custom copilots, allowing a headless, API-only way to deliver Microsoft 365 capabilities through enterprise and ISV apps.

Of interest to solution providers is a preview allowing for copilot extension publishing through Partner Center. Users can access these extensions to install from the store in Copilot and app stores in Microsoft 365 products like Teams and Outlook.

More Copilot advancements

A new capability for Copilot Studio early access program members allows users to make agent custom copilots that can automate business processes. A preview is scheduled for later this year, according to Microsoft.

The agents reason over user inputs and system actions, use memory to bring in context, and learn and act based on user feedback, according to Microsoft.

Use cases include IT device procurement and customer concierge for sales and service, according to the tech giant.

Copilot Studio now has connectors that will allow developers to more easily build extensions— more than 100 customers and partners are building Copilot extensions with Copilot Studio and Teams Toolkit for Visual Studio.

Copilot Studio connectors for Microsoft Fabric are coming soon, according to the tech giant, joining Copilot Studio’s more than 1,400 Power Platform, Microsoft Graph and Power Query connectors.

Those connectors will allow copilots to use public websites, SharePoint, OneDrive, Dataverse tables, Microsoft Fabric OneLake, Microsoft Graph, third-party apps and other data sources. Developers will have the ability to curate and manage available Copilot connectors.

Coming soon are new agent capabilities allowing Copilot Studio to orchestrate defined tasks and functions behind the scenes. Copilot Studio agent capabilities are available in the early access program. Public preview should come later this calendar year, according to Microsoft.

Use cases include giving a copilot the authority to reason over actions and user input, leverage memory, learn based on user feedback, record exception requests and ask for help for unfamiliar situations.

This could help automate long-running business processes in customer on-boarding, with copilot offering training and relevant resource guides, seeking feedback for future enhancements and responding promptly when customers voice concern, according to Microsoft. Copilot will recall past conversations for context and personalised experiences while following guardrails.

Teams news

The premium version of Microsoft’s popular Teams collaboration app received AI and data protection updates.

Becoming generally available in June for Teams Premium and Copilot for Microsoft 365 is intelligent recap support for meetings with transcription enabled but recording not enabled.

Going generally available next month is an additional transcription control for meeting organizers. Now generally available is the ability for administrators to specify which meetings are enabled to share content with external participants, according to Microsoft.

Microsoft pushed more Teams features to custom apps and website experiences built with Microsoft Azure Communication Services, including Microsoft PowerPoint Live, Call Diagnostics Center, noise suppression during a video call, picture-in-picture for iOS and Android, closed captions, real-time transcription using Azure AI Speech and file sharing during a meeting.

Next month a preview becomes available for custom emojis and reactions in Teams. Now in preview is the ability to use slash commands in the Teams compose box.

Rolling out is the ability for developers to insert code into a Loop component or convert a native code block into a Loop component shareable across Teams chat, Teams channels and Outlook.

Coming soon to Outlook is the capability to directly insert third-party Adaptive Card-based Loop components from services like Jira, Trello, Confluence Cloud, Lucid Software, Mural and Priority Matrix.

Azure AI advancements

Azure AI Studio is now generally available, according to Microsoft, and promises the ability to develop custom copilot apps.

Users can explore AI tools, orchestrate multiple interoperating APIs and models, ground models, test and evaluate them and then deploy at scale with continuous monitoring, according to Microsoft.

A prompt flow feature will support workflow orchestration for multimodal models, including the use of images as inputs and outputs in conversations and models. That includes Llama 3, Mistral Large and Cohere Command R+. In preview are tracing as well as debugging and monitoring for generative AI apps features.

Azure AI Studio’s model catalog has more than 1,600 models with frontier and open large language models (LLMs) and small language models (SLMs) for users to compare benchmarks.

Microsoft-backed OpenAI’s newest flagship model, GPT-4o, is now available in Azure AI Studio and as an API. GPT-4o brings together text, image and audio processing for generative and conversational AI experiences, according to Microsoft.

Azure AI Studio users also gained a preview for a new model deployment route—AI Toolkit for Visual Studio Code. In this toolkit, developers can acquire, run and fine-tune models before Azure AI Studio deployment.

The AI Studio and Azure OpenAI Service now have previews for prompt shields and groundedness detection. These shields aim to mitigate indirect and jailbreak prompt injection attacks on LLMs. Groundedness detection is meant for finding ungrounded materials or hallucinations in generated responses, according to Microsoft.

Azure OpenAI Service also gained a generally available update for assistants API, allowing for users to make virtual assistants and chatbots with nuanced understanding and responsiveness.

A new OpenAI Service preview allows users to interact with AI guides inside Microsoft Mesh 3D experiences.

Azure AI Models-as-a-Service

Azure AI’s Models-as-a-Service (MaaS) is coming soon, according to Microsoft, and has previews for Nixtla’s TimeGen-1 and Core42 JAIS. Coming soon are models from AI21, Bria AI, Gretel Labs, NTT Data, Stability AI and Cohere Rerank, according to Microsoft. Microsoft’s Phi-3-mini is now GA in MaaS.

Microsoft has made its Phi-3-vision multimodal model—sized at 4.2 billion parameters—in the Phi-3 family of AI SLMs available in Azure.

Phi-3 models are aimed at personal devices, and the new vision model allows users to input images and text and receive text responses, according to Microsoft. Users can ask questions about charts, open-ended questions about specific images and perform other tasks.

Developers can experiment with the models in Azure AI Playground and build with the models in Azure AI Studio, according to Microsoft.

In preview for Azure AI Speech are speech analytics and universal translation for voice-enabled apps.

Users can bring together transcription, summarisation, speech recognition, speaker diarization, sentiment analysis and other audio and video data for insight and translate video files into several supported languages, uploading one or a series of videos to translate and generate video content in other languages automatically.

Microsoft also revealed that its Edge browser is set to get AI-powered real-time video translation for YouTube, LinkedIn, Reuters, CNBC News, Bloomberg, Coursera and other websites, with initial options for Spanish to English and English to German, Hindi, Italian, Russian and Spanish.

Azure AI Search’s storage capacity has been increased to up to 12 times in vector index size for no additional cost to help scale retrieval-augmented generation (RAG) workloads.

AI Search previews include built-in image vectorisation through Microsoft Azure AI Vision, new platform integration with Azure AI Search’s OneLake connector for files and enhancements to vector and hybrid search, according to Microsoft.

Azure AI gained general availability for an Azure Database for PostgreSQL extension meant to ease bringing AI capabilities to relevant data.

In preview is Azure Database for PostgreSQL in-database embedding generation, with a promise of single-digit millisecond latency, predictable costs and the ability to keep data compliant for confidential workloads, according to Microsoft.

Other Azure updates

The new Azure Cosmos DB for NoSQL, powered by the DiskANN algorithm library, makes Cosmos DB the first cloud database with lower latency vector search at cloud scale without server management, according to Microsoft.

A preview for Azure SQL DB’s Microsoft Copilot capabilities allows users to convert queries into SQL language so developers can interact with data through natural language.

Azure Database for MySQL will receive Copilot capabilities that allow for summaries of technical documentation in response to user questions, but Microsoft did not say when these capabilities become available.

Azure AI Content Safety now has a preview of custom categories, which allow users to make specific content filters deployable within an hour to protect against emerging threats, according to Microsoft.

Azure Kubernetes Services (AKS) Automatic is a new managed Kubernetes experience in preview. Users can leverage AKS Automatic to move from container image to deployed app in minutes while still accessing the Kubernetes API.

Now generally available are new Microsoft Azure reference architectures and implementation guidance for private chatbots. This includes reference architectures for Microsoft Azure OpenAI Service, landing zone accelerators, cloud guides for chunking strategies and service guides for machine learning, according to Microsoft.

A new model type called custom generative is coming soon to preview, according to Microsoft. This model will allow users to process complex documents with a variety of formats and templates, starting with a single document. It will guide users through schema definition and model creation with minimal labeling.

The model will leverage LLMs for extracting fields. Users only need to correct output when the model makes a field error. The model will adapt to samples added to the training data set, continually improving after deployment.

Microsoft made an Azure API Center generally available to help with managing API sprawl. Users can turn to API Center for discovery, consumption and governance of APIs regardless of type, life-cycle stage or deployment location, according to the tech giant.

GitHub Copilot for Azure

GitHub Copilot, which now boasts of having 1.8 million subscribers, has a private preview for its first set of first- and third-party extensions, including a GitHub Copilot for Azure. Access is limited to offers directly from Microsoft.

Developers can use Copilot Chat to explore Azure resources, manage Azure resources, troubleshoot issues and locate logs and code. They can ask Copilot questions about choosing an Azure service, running a React app and selecting the best Azure database to use with Django, among other use cases. The Copilot will also guide developers through deployment.

The earliest extensions will come from DataStax, Docker, LambdaTest, LaunchDarkly, McKinsey & Co., Microsoft Azure, Microsoft Teams, MongoDB, Octopus Deploy, Pangea, Pinecone, Product Science, ReadMe, Sentry and Stripe.

Microsoft will expand the ecosystem through hundreds of partners in the Copilot Partner Program over the coming months, according to the tech giant.

Organizations also will have the ability to create private Copilot Extensions for their homegrown developer tooling, according to Microsoft.

Microsoft Copilot in Azure

Microsoft’s latest AI companion is Copilot in Azure, which promises to ease operations management across cloud and edge environments. Microsoft said it will open Copilot in Azure preview over the next couple of weeks.

Users can ask why apps are slow, how to fix certain errors and other questions, prompting Copilot guidance through possible causes and fixes, according to Microsoft.

Organisations will have the ability to grant access to specific users or groups within a tenant. Copilot in Azure can align with existing organisational operational standards and security policies, according to Microsoft.

More AKS skills have been added to Copilot for Azure to simplify common management tasks, according to Microsoft. These tasks include configuring backups, changing tiers, finding YAML files and more.

Copilot users can also now ask questions about data in plain text and receive the corresponding T-SQL query, speeding up the coding process. Copilot also now includes Defender for Cloud, prompting capabilities for risk exploration, remediation and code fixes.

Microsoft has launched a preview of Cobalt 100 Arm-based virtual machines. These VMs are the first generation to feature Microsoft’s new Cobalt processor, custom-built on an Arm architecture and optimized for running general-purpose and cloud-native workloads, according to the tech giant.

These VMs have shown up to 40 percent improved performance versus comparable Azure VMs, according to Microsoft.

Microsoft has also made generally available the Azure ND MI300X v5 VM series optimized for Azure OpenAI Service and other demanding AI and high-performance computing workloads.

The tech giant also launched a new service called Azure Compute Fleet to simplify capacity provisioning across VM types, availability zones and pricing models for easier scaling and improved performance and cost. Users can control VM group behaviors automatically and programmatically with this service, according to Microsoft.

Security updates

Microsoft has upgraded Defender for Cloud to protect AI apps from code to cloud, according to the tech giant. The tool has AI security posture management features for finding AI services, AI tools and vulnerabilities.

Defender for Cloud has a native integration with Azure AI Content Safety so teams can monitor Azure OpenAl apps for direct and indirect prompt injection attacks, sensitive data leaks and other threats, according to Microsoft.

A preview for Copilot Trust Platform is coming later this year, according to Microsoft. Users will have the ability to enforce enterprise data protection, scale inference and retrieval across enterprise data and app estates and run responsible AI checks.

.Net enhancements

Microsoft’s free, open-source, cross-platform framework .Net now has the .Net Aspire cloud-native suite of tools and libraries for building cloud-native apps, which is generally available, the tech giant said.

Users can leverage preconfigured common resiliency patterns and a built-in dev-time dashboard to speed up app building, according to Microsoft.

.Net 9 Preview 4 has enhancements across runtime, libraries, SDK, ASP.Net Core, .Net MAUI, C#13, Entity Framework Core, .Net Aspire and more.

Now generally available is a MongoDB Provider for Entity Framework Core, allowing .Net developers to more simply incorporate MongoDB in EF-based apps.

Power Platform

During Build 2024, Microsoft revealed that it has pushed model explorer in Power BI to GA.

Model explorer promises a view of all semantic model objects in the data pane to speed up data item searching. Users can create calculation groups and reduce the number of measures by reusing calculation logic and simplifying semantic model consumption through this tool, according to Microsoft.

Power Automate gained more AI-powered automation tools, including AI flows for outcome-based automation in the early access program. AI flows allow Power Automate to interpret process objectives, generate automation plans, autonomously select actions and validate outcomes, with users able to make adjustments and guardrails.

Also in the early access program is a new AI recorder feature that allows users to create robotic process automations (RPAs) through screen sharing and voice coaching. The Power Automate Recorder captures users’ desktop processes through voice and video and makes user interface automation in minutes instead of days, according to Microsoft.

The recorder can detect user interface changes in app interfaces, and users can make edits with natural language in Copilot.

Power Apps now has native Git integration for synchronizing environments with Git repositories. Power Apps Studio now allows for source code viewing and use in a readable YAML format. Users can create templates for screens, reuse code and store files in a code repository.