Learning Notes - Azure Synapse Analytics

Everything gets evolved into the next level. Earlier it was SQL DW, Now it has evolved into MPP with ML components.

Azure Synapse Analytics – Azure Synapse Analytics is a new offering available on Microsoft Azure. It’s a combination of SQL Data warehouse (MPP offering), Apache Spark, pipelines, and a workspace to manage this entire ecosystem

What is dedicated SQL pool (formerly SQL DW) in Azure Synapse Analytics?

• Dedicated SQL pool (formerly SQL DW) stores data in relational tables with columnar storage
• PolyBase uses standard T-SQL queries to bring the data into dedicated SQL pool
• Dedicated SQL pool uses PolyBase to query the big data stores.

Reference - Link

Architecture - Link

Dedicated SQL pool (formerly SQL DW) uses a node-based architecture.

Applications connect and issue T-SQL commands to a Control node. The Control node hosts the distributed query engine, which optimizes queries for parallel processing, and then passes operations to Compute nodes to do their work in parallel.

Similar to Map Reduce here you see distributed parallel processing. Hope to experiment few more examples in the next posts.

Building real-time enterprise analytics solutions with Azure Synapse Analytics 

Key Notes

• Dedicated SQL pools
• Serverless consumption pools
• Azure Synapse Analytics

Workspace Features

• SQL Pools
• Spark Pools
• Pipelines - Integration and Orchestration
• All resources governed by common security model
• Connected service to expand synapse
• Linked services for Data Integration

Demo 1

• Synapse Analytics workspace

Demo 2

• Azure Synapse and Azure ML
• Synapse Notebook
• Hummingbird generates in onnx format
• Connect to AzureML Workspace

Demo 3

• Use the model in Synapse workspace

Happy Learning!!!