Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.

Start free today

Monitor your Azure Data Factory pipelines proactively with alerts

Play Monitor your Azure Data Factory pipelines proactively with alerts
Sign in to queue

Description

Data Integration is complex and helps organizations to combine data and complex business processes in hybrid data environments. The increase in volume, variety and velocity of data has led to delays in monitoring and reacting to issues. Organizations want to reduce the risk of data integration activity failures and the impact to other downstream processes.

Manual approaches to monitoring data integration projects are inefficient and time consuming. As a result, organizations want to have automated processes to monitor and manage data integration projects to remove inefficiencies and catch issues before they affect the entire system.

In this episode, Gaurav Malhotra joins Scott Hanselman to show how organizations can now improve operational productivity by creating alerts on data integration events (success/failure) and proactively monitor their Azure Data Factories.

Embed

Download

The Discussion

  • User profile image
    RanjithPc

    Hi,

    This is RanjithPC, Have two questions.

    1) I have followed above mentioned steps but i not received any mail and dot see any list of pipeline process run history.

    2) Email notification if I want to send mail to more than one people then how it will support, because i have add second mail id it throws validation message "Invalid email"

  • User profile image
    Monica Frintu

    Hello RanjithPc,


    There is a known delay between the time the alert is created and the time when the alert becomes active in Azure Monitor. You can also check the status of your alert in the Azure portal(https://portal.azure.com/).
    If you want to send email to a group of people, you need to create a distribution group and use that in the email notification.

    Thanks,
    Monica

  • User profile image
    Shakira

    Hi,

    Could you perhaps inform me what permission is needed for users to be able to create these alerts?
    I have a user who wants to create an alert but he gets the following error:

    The client 'client email address' with object id '************' does not have authorization to perform action 'Microsoft.Insights/metricalerts/write' over scope '/subscriptions/4e3a0dbd-45b1-46f1-8ce2-1fb5e3d2a811/resourcegroups/********/providers/Microsoft.Insights/metricalerts/ADF' or the scope is invalid. If access was recently granted, please refresh your credentials.

    I tried going through all of the permissions to the Data Factory but no luck figuring it out.

    Many thanks,
    Shakira

  • User profile image
    MonicaF
    Hello,

    You need to have "Monitoring Contributor" role to be able to set diagnostic settings for a resource and create alerts.See this for detailed information : https://docs.microsoft.com/en-us/azure/azure-monitor/platform/roles-permissions-security.

    Hope this helps,
    Monica

Add Your 2 Cents