This week (23th-25th May) we had the pleasure to attend the Microsoft Build 2023.
Microsoft Build is an annual conference event held by Microsoft, aimed towards software engineers and web developers using Windows, Microsoft Azure, and other Microsoft technologies. The conference features a variety of presentations, workshops, and interactive sessions on Microsoft’s platforms, tools, and services, with keynotes from senior leaders of Microsoft.
This year, Microsoft has announced many revolutions regarding the modernization of its various products. You did not get the chance to attend? Do not panic, you are about to travel back in time! MDW, which is specialized in these technologies, has been your eyes and ears. We have summarized the main announcements for you.
Microsoft Fabric
As per official documentation, Microsoft Fabric is « an all-in-one analytics solution for enterprises that covers everything from data movement to data science, Real-Time Analytics, and business intelligence. It offers a comprehensive suite of services, including data lake, data engineering, and data integration, all in one place. »
It offers a single unified product, integrating Azure Data Factory, Azure Synapse Analytics, and Power BI.
Microsoft Fabric features seven core components:
Azure Data Factory (Preview) combines the straightforwardness of Power Query with the robustness of Azure Data Factory. It offers over 200 native connectors on-premises and in the cloud for data sources.
Azure Data Factory (Preview) combines the straightforwardness of Power Query with the robustness of Azure Data Factory. It offers over 200 native connectors on-premises and in the cloud for data sources.
Synapse Data Engineering (preview): this feature offers a great Spark platform with quality authoring experiences, allowing for large-scale data transformation and democratization of data.
Synapse Data Science (preview): Provides a complete workflow for data scientists to build sophisticated AI models, collaborate easily, and train, deploy, and manage machine learning models.
Synapse Data Warehousing (preview): This feature offers industry-leading SQL performance and scalability. It separates compute from storage, allowing for independent scaling, and natively stores data in the open Delta Lake format.
Synapse Real-Time Analytics (preview): Allows developers to handle data streaming from the Internet of Things (IoT) devices, telemetry, logs, and more and analyze large volumes of semi-structured data with high performance and low latency.
Power BI in Fabric: Offers leading visualization and AI-driven analytics that enable business analysts and users to extract insights from data. The Power BI experience is deeply integrated into Microsoft 365, providing relevant insights where business users already work.
Data Activator (coming soon): Provides real-time detection and monitoring of data and can trigger notifications and actions when it finds specified patterns in data—all in a no-code experience.
OneLake
Microsoft Fabric introduces OneLake, a built-in, multi-cloud data lake automatically available to all Fabric users. It is an intuitive data hub, organized and indexed for easy discovery, sharing, governance, and compliance, benefiting developers, business analysts, and end users.
Much like OneDrive, all Fabric workloads are directly connected to OneLake.
Each tenant has access to a single instance of OneLake, which offers a unified view of the filesystem across different users, geographical locations, and even multiple cloud environments. For ease of management, the data within OneLake is segmented into manageable units called containers.
OneLake aims to eliminate the chaos of data silos created by different developers working with their own isolated storage accounts. It provides a unified storage system where data discovery and sharing are simplified, and policy and security settings are centrally enforced.
OneLake offers us a feature called “Shortcuts” that makes it easy to share data between users and programs without making extra copies of the data. It can work with data storage in ADLSg2, Amazon S3, and, soon, Google Storage, making it a multi-cloud system.
Data analytics for the era of AI
OneLake can handle structured and unstructured data, giving users much flexibility.
Fabric provides a simplified data system using OneLake as the central place for storing data and Delta and Parquet as the standard format. This means users don’t need to keep different copies of data for different uses. Instead, one copy of the data in OneLake can be used for all tasks. All compute engines can instantly work with the data stored in OneLake: There is no need to load in DWH or refresh in Power BI.
If needed, we can synchronize the OneLake with our local computer as we could do with OneDrive. And thus, explore the files and even add new files directly from here. Changes will be instantly reflected to the Fabric.
Image source: Data analytics for the era of AI
Governance and Security across Fabric is also addressed, with Fabric offering a unified SaaS platform integrating Power BI, Synapse, and Data Factory. This enables data teams to collaborate on a single workspace, on the same copy of data, with centralized administration, governance, and compliance tools. These capabilities include data lineage and impact analysis, data protection with sensitivity labels, data endorsement, admin monitoring, and more.
Image source: Data analytics for the era of AI
Copilot everywhere and for everyone!
With Copilot inside Microsoft Fabric, users will benefit from substantial AI assistance allowing them to describe their problems and get immediate answer.
Thus, Copilot will help them turn words into dataflows and data pipelines, create machine learning models or generate SQL queries.
Users will also have the possibility to create custom Natural language tools, in the form of plug-ins, that combine Azure Open AI and the organization’s data.
1. Create the Plug-in based on data
2. Define the process/steps
3. Get the answer with detailed information and source:
Moreover, they will also be able to ask questions through Teams, for example, and get answers based on data in Fabric:
Data Activator
Another interesting feature that Microsoft Fabric brings us is the “Data activator”. This new no-code tool will empower the users to drive actions automatically from the data.
Image source: Data analytics for the era of AI
Data Activator offers the following:
1. Data Connection: The Data Activator can link to various data sources within Microsoft Fabric, such as Power BI datasets, Event streams, etc. It consistently scans for patterns that warrant action upon establishing a connection with your data.
2. Recognizing Actionable Scenarios: Data Activator offers a centralized platform to determine actionable trends in your data. It could be a KPI crossing a certain threshold or a more complex analysis involving temporal changes, like a downward trending value.
3. Prompting Actions: When Data Activator identifies an actionable pattern, it initiates an action. This could be sending an email or alert via Teams or triggering an automatic process, such as a Power Automate flow or an activity within your organization’s enterprise apps.
Microsoft Power BI
As part of the new Fabric platform, the latest developments for Power BI provide users with a range of new features and functionalities. These enhancements transform data analysis and visualization, boosting productivity and efficiency.
Build and edit reports using Copilot
Among these exciting innovations, Copilot, integrated with Power BI, brings large language models to help users create and manipulate data visuals and insights effectively. It will assist users in the following tasks:
– Report creation
– Generation and modification of DAX calculations
– Creation of narrative summaries
– Asking questions about their data
All of that in conversational language.
Let’s look at some examples.
You can describe the report you need, by providing detailed information
…and it will be built in seconds!
You will be able to adjust it using a user-friendly interface:
Want to make your report look like an existing one? No problem! Just give Copilot an example of a report, and it will transform it in seconds:
For example, by applying same theme.
You can as well add additional graphs, or data summarizations:
Power BI Desktop users will also have the possibility to be assisted by Copilot. Using natural language, they can ask questions and get answers in form of a DAX query. Indeed, a new view called “DAX query view” (future update) will allow them to create and test measures, see data preview, as well as get help from the AI model to write DAX code.
In addition, Power BI’s new storage format based on Delta Lake and Parquet aims to reduce data duplication and management. This feature, referred to as Direct Lake mode, allows for impressive performance with no data movement necessary, thereby reshaping big data consumption.
Git integration
Collaboration is also enhanced with Git integration for Power BI datasets and reports. Developers can now easily track changes, revert to previous versions, and merge updates from team members into a singular reliable source.
Image source: Empower every BI professional to do more with Microsoft Fabric
Bing + ChatGPT
Bringing Bing to ChatGPT as the default search experience
ChatGPT will no longer be limited to its knowledge before 2021, which it was trained on. The integration with Bing will provide more timely and accurate responses. It also allows answers to be grounded by search and web data, including citations, so you can easily navigate to the sources.
Plug-ins
Microsoft announced the extensibility model for Microsoft 365 Copilot with plug-ins, allowing developers to integrate their apps and services into Microsoft 365 Copilot. These plug-ins enhance AI systems by enabling interaction with APIs from other software and services for real-time information retrieval, data incorporation, and new computations.
The three types of plug-ins for Microsoft 365 Copilot include ChatGPT plug-ins, Teams message extensions, and Microsoft Power Platform connectors, enabling developers to leverage their existing software resources and skills.
Microsoft also talked about interoperability allowing developers to build and submit plug-ins that work across various platforms including ChatGPT, Bing, Dynamics 365 Copilot, Microsoft 365 Copilot, and Windows Copilot.
Bing is also increasing its support for plug-ins. In addition to previous integrations with OpenTable and Wolfram Alpha, new partnerships have been established with several companies such as Expedia, Instacart, Kayak, Klarna, Redfin, TripAdvisor, and Zillow. These plug-ins, built into the chat, provide relevant recommendations to the users.
Some examples:
– Expedia: Users can enjoy a conversational trip planning experience for seamless travel planning.
– Instacart: Bing will help you plan dinner menus, create a shopping list, and place orders for delivery from your preferred grocery retailer.
– Kayak: The plug-in will be a virtual travel assistant, conversationally presenting popular features.
Furthermore, other companies are also enabling plug-ins for Bing Chat. In the coming weeks, these plug-ins will be available within Bing chat or the Edge sidebar across desktop and mobile platforms. This move will significantly enhance user experiences and offer personalized, intuitive assistance.
Azure AI Service
Microsoft’s Azure AI Service has announced several updates to enhance productivity, efficiency, and content safety.
Some key updates:
Azure AI Studio will allow developers to Build, train, customize, evaluate, and deploy models. They will be able to ground AI models with structured and unstructured data.
Prompt flow will provide a streamlined experience for prompting, evaluating and tuning large language models. Users can quickly create prompt workflows that connect to various language models and data sources.
Azure AI Content Safety: an AI-powered service that classifies and scores harmful content, understands text in various languages, and complies with regulatory standards. It leverages advanced image recognition and real-time detection of unsafe content.
Azure Cognitive Search is introducing a vector search feature coming soon in the preview. This feature will enable developers to store, index, and search by concept in addition to keywords using various forms of organizational data. It will empower developers to build apps that can generate personalized responses, provide product recommendations, detect fraud, and identify data patterns, among other capabilities.
Azure Cognitive Service for Language is introducing new features, currently in preview, that allows developers to customize summarization in addition to existing features like entity recognition, text classification, and conversational language understanding. Furthermore, integrating everyday language understanding within Power Virtual Agents streamlines the creation of intelligent chatbots and virtual agents.
Windows Copilot
Copilot integration to Windows 11 will provide centralized AI assistance and will help people in their daily tasks.
It will assist you with things such as:
• Personalize and navigate your PC
• Work with documents, such as explain, rewrite, summarize
• Complete tasks, such as switch to dark mode
• Simplify workflows with plug-ins, like launching a song on Spotify
Conclusion
Although these announcements are extremely recent and businesses have not yet been able to adopt these new features, we recommend being particularly attentive to the very positive impact that these solutions can have on your business and digital transformation.
The adoption of Microsoft Fabric can lighten the development times as well as the lifecycle of your products and their continuous integration. The centralization of data services within a homogenized interface indeed represents an advantage on your time to market.
The Synapse Data Warehouse solution provides the ability to differentiate between storage and compute expenses. This feature can help large corporations accurately allocate costs to respective departments. The added benefit of autoscaling ensures that users won’t be restricted by hardware that’s insufficient for their needs. Despite this, proper cost management strategies are crucial to prevent some costs from unexpectedly escalating.
Synapse Real-Time Analytics offers the possibility of implementing a real-time data streaming solution that is fully integrated with the rest of the suite, which was not the case before. Concretely, this simplification offers many possibilities for data engineers to meet strong business needs for real-time analytics.
Data Activator promises to be able to trigger events in real time based on a detection of real-time data changes. To date, no consolidated tool offered this possibility, and each company had to implement ‘Change Data Capture’ mechanisms and APIs to respond to it. This very promising technology seems to us to be a new significant pillar in the establishment of a data-driven organization.
Microsoft introduces OneLake, a feature that allows for the virtualization and consolidation of multiple Data Lakes from Azure, AWS, and soon also from Google. This allows for the establishment of a platform to govern them all and enables the opening of the Microsoft ecosystem to more heterogeneous architectures.
The inclusion of Copilot in Data Flow, Power BI, and Windows signifies a notable leap forward in technology, poised to boost both developer and end-user productivity. It’s our belief that organizations should start gearing up for change management now, considering the substantial return on investment this solution is expected to generate. This change will inevitably affect the daily routines of employees, so it’s crucial to prepare for this transformation in their workflows.
If you’re looking to modernize your solutions given the major advancements these new AI capabilities offer, it is important to work with experts who have experience in implementing these types of solutions.
At MDW, we understand that the success of any implementation project lies in careful planning and execution, and we can help improve solutions with best practices in mind. Our team of experts can work with you to identify your specific business needs and recommend the most effective solutions to meet them.
Being experts in Microsoft Data, AI and Analytics solutions, we recommend and follow the best standards. This certifies your organization uses the right policies defined to implement corporate solutions at scale with control on cost, security, delivery, and operations.
You want to know more about us? Feel free to subscribe to our Linkedin page, to contact us at contact@mdwpartners.com.
