Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.

Start free today

How to prepare data using wrangling data flows in Azure Data Factory

Play How to prepare data using wrangling data flows in Azure Data Factory
Sign in to queue


Gaurav Malhotra joins Scott Hanselman to show how wrangling data flows in Azure Data Factory empower you with a code-free, serverless environment that simplifies data preparation in the cloud and scales to any data size with no infrastructure management required. It uses the industry-leading Power Query data preparation technology (also used in Power Platform dataflows, Excel, and Power BI) to prepare and shape the data. Built to handle all the complexities and scale challenges of big data integration, wrangling data flows enable use Apache Spark execution to help you easily prepare data at scale.

[0:02:00] - Demo



The Discussion

  • User profile image

    The "Done" button after creating Merge Queries does nothing. Upon debugging receiving Sink dataset filepaths cannot contain a file name. Please remove the file name from 'WranglingFile1'. Pretty much in preview stage as it says :)

  • User profile image

    Why is it called "wrangling data flow"? To wrangle means to quarrel or collect.

  • User profile image
    The "Done" button is just an indicator for UX to know that user is done with doing the transformation. I agree that this is confusing and we are working on improving the experience.
    Regarding the sink dataset, as the error message says, the sink dataset can only contain the folder (and no file). The idea is once the transformations are run on source dataset, the ADF will create the output files in the folder specified in the sink dataset. Please remove the file name from sink dataset.
  • User profile image
    Mr TA

    This is poor design. Now I have to create 2 datasets for each blob: one without a filename for the sinks, and one with a filename for everything else. What were you thinking when you did this, Microsoft?

Add Your 2 Cents