Azure is the new mainframe
Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.
Start free todayEnhanced monitoring capabilities and tags/annotations in Azure Data Factory
Play Enhanced monitoring capabilities and tags/annotations in Azure Data Factory
Description
Gaurav Malhotra and Scott Hanselman (@shanselman) explore tagging support and enhanced monitoring capabilities, including dashboards and improved debugging support in Azure Data Factory. Data integration is complex and the ability to monitor your data factory pipelines is a key requirement for dev ops personnel inside an enterprise.
Now, you can tag/annotate your data factory pipelines to monitor all the pipeline executions with that particular tag. In addition, Data Factory visual tools provide dashboards to monitor your pipelines and an ability to monitor your pipeline execution by the Integration Runtime (IR) upon which the activities execute.
Jump To:
[01:45] Demo Start
- Alert and Monitor data factories using Azure Monitor
- Azure Data Factory overview
- Azure Data Factory docs
- Azure Data Factory pricing
- Create a free account (Azure)
Follow @SHanselman Follow @DataAzure
Don't miss an episode: Follow @AzureFriday
Download
The Discussion
-
RichieBzzzt I'm liking annotations. Turns out they accept dynamic content as well, which is really handy if you are executing pipelines from other pipelines. Would be great if the "Add Dynamic Content" was shown in the UI to make this fact obvious. -
Gaurav Malhotra Currently you cannot parameterize annotations which means you cannot accept dynamic content for them.. Thanks
-
Ankur How do you monitoring multiple data factory at a same time in same dashboard?
Also how do you monitor pipelines running on linked IR from shared IR dashboard?
More episodes in this series
Code-free data transformation at scale using Azure Data Factory
Related episodes
Create dependent pipelines in your Azure Data Factory
An overview of Azure Integration Services
Azure Instance Metadata Service updates for attested data
Anomaly detection using machine learning in Azure Stream Analytics
An overview of Azure Blueprints
Monitor your Azure Data Factory pipelines proactively with alerts
Using HashiCorp Consul to connect Kubernetes clusters on Azure
Run Azure Functions from Azure Data Factory pipelines
Interning in Azure Engineering and the Visual Studio Code extension for ACR Build
Learning Azure: Part 3—A quick tour of Microsoft Learn
