Build with an Azure free account. Get USD200 credit for 30 days and 12 months of free services.

Start free today

Azure Data Lake Storage Gen2 overview

Play Azure Data Lake Storage Gen2 overview
Sign in to queue

Description

James Baker joins Lara Rubbelke to introduce Azure Data Lake Storage Gen2, which is redefining cloud storage for big data analytics due to multi-modal (object store and file system) access and combining the best qualities of a dedicated analytics file system with those of a cloud-scale object store.

For more information:

Tag:

Azure

Embed

Download

The Discussion

  • User profile image
    Joel Mamedov

    I like the content but, not so much delivery. This guy is too casual for internet scale presentation.

    I cannot hear him well even though my speakers are maxed out. His phrases fade out at the end. Too casual.
    I suppose the outfit must match the presentation.

  • User profile image
    robcaron
    Hi Joel - I'm sorry you're not happy with the informality of Azure Friday. The format of the show is an engineer to engineer conversation, so we don't script the show and guest performance varies as people do. If you're having difficulty with sound level (not able to repro here) you can turn on captions, which also make the content accessible for many languages.
  • User profile image
    nwomnii
    cmon guys, he is an Aussie, doesnt get more formal
  • User profile image
    VikrantJain
    Azure SQL data warehouse Polybase with Azure Storage datalake Gen2 is not working with MSI at folder level however it is working with role base access.

    I am getting below error "This request is not authorized to perform this operation using this permission" with MSI where access has been given to MSI at folder level.


    Appreciate your response
  • User profile image
    robcaron
    From James Baker:

    I am able to confirm that Azure SQL Data Warehouse’s Polybase capability does indeed work in being able to reference data in ADLS Gen2 using the system-assigned MSI when that MSI only has ACL access and not RBAC access.

    There are 2 things to consider when applying ACLs in ADLS Gen2:

    1. Irrespective on where the MSI is granted read ‘R’ and/or write ‘W’ access, the principal MUST have execute ‘X’ permission from the root directory all the way down to the data directory. This allows the MSI to traverse the directory structure and will fail without this permission.

    2. The ‘Principal ID’ of the MSI must be the guid value specified in the ACL. This can be obtained using the Powershell ‘Get-AzSqlServer’ cmdlet.
  • User profile image
    Niner269763
    this was exactly what I was looking for. A great presentation!
  • User profile image
    axcarlin
    There is still a missing point in the discussion of ADLS Gen2 security , the fact that a user needs RBAC reader role over the Storage Account in order to access the folders in the filesystem, and he will have the access ( as viewer ) to all the folder without respect to ACL , that only controls read/write access.

    It is not yet clear to me how to achieve a very simple goal, i.e. being able to access only the folders for which I've been included to their ACL ( by user or group id ), through a web application, like Storage Explorer.
    It's not really a good idea to been able to see the whole hierarchy of folders, even if I cannot actually look into the content.

    Anyway, it looks very promising
  • User profile image
    robcaron
    @axcarlin:

    From James Baker:

    The requirement for RBAC reader role no longer applies to the variety of tools & frameworks that integrate with ADLS Gen2. This means that you can choose coarse-grained access mechanisms via RBAC or fine-grained access via ACL or a combination of both.

    Access control via ACLs-only does require special handling is some tools (eg. for Azure Storage Explorer you need the v1.9+ to ‘mount’ an ADLS Gen2 container as the user will not be able to browse to that account).

Add Your 2 Cents