Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.

Start free today

Event-based data integration with Azure Data Factory

Sign in to queue

The Discussion

  • User profile image
    JohnN

    Thank you for this demo, and to the ADF team for adding this much-needed functionality.

    But even in your demo it shows how slow Data Factory can be. At 7:32 in the video the stats show that 355 bytes took 44s to transfer!!??

    Is there any way (as azure Admins) we can see a more granular breakdown as to what that 44s was spent doing? How can we speed this up for real-world data loads?

  • User profile image
    gauravmalhot

    Yes, you can see the granular breakdown using our visual tools by doing the following steps:

    a. Click 'Monitor' tab on left bar

    b. Identify the pipeline run for the copy operation

    c. Click on the 'View Activity Runs' icon under the pipeline run

    d. Click on 'Details' icon under the 'Actions' column for your activity run

    e. You can see how much time was spent in queue, in actual transfer, throughput, duration etc.

    The time taken is not linear with the amount of data being transferred. You can transfer data from blob to SQL DW which was the case that I was showcasing at a throughput of 1 GB/s meaning you can transfer 1 TB worth of data in close to 20-25 mins. Please give it a try and let us know if you see any perf issues.

Add Your 2 Cents