PolyBase in SQL Server 2016

Play PolyBase in SQL Server 2016

Description

This week's episode of Data Exposed welcomes Sahaj Saini, Program Manager on the SQL Server team. Today, Sahaj joins us from our southern California office to show us an exciting new feature of SQL Server 2016 called PolyBase. PolyBase simplifies management and querying of both relational and non-relational data using T-SQL.

Sahaj starts by providing the background of why PolyBase was introduced in SQL Server 2016 to support the increasing volume of non-relational data that land in cost-effective storage systems and high-scale data processing systems such as Hadoop. The goal with PolyBase in SQL Server 2016 is to expand the reach of SQL Server to these external data stores using known technologies and languages.

Sahaj then walks us through the architecture and key capabilities of PolyBase in SQL Server 2016, including the ability to do ad-hoc queries on Hadoop data and the ability to import data from Hadoop and Azure Storage to SQL Server for persistent storage.

We wrap up the show with an excellent demo from Sahaj illustrating how easy it is to work with data in Hadoop through SQL Server 2016. Great introduction to PolyBase in SQL Server 2016. More videos to come on this topic!

Embed

Download

Download this episode

The Discussion

  • User profile image
    Ilya Geller



    SQL is obsolete. Why?
    For instance, there are two sentences:
    a) ‘Pickwick!’
    b) 'That, with the view just mentioned, this Association has taken into its serious consideration a proposal, emanating from the aforesaid, Samuel Pickwick, Esq., G.C.M.P.C., and three other Pickwickians hereinafter named, for forming a new branch of United Pickwickians, under the title of The Corresponding Society of the Pickwick Club.'
    Evidently, that the ' Pickwick' has different importance into both sentences, in regard to extra information in both. This distinction is reflected as the phrases, which contain 'Pickwick', weights: the first has 1, the second – 0.11; the greater weight signifies stronger emotional ‘acuteness’; where the weight refers to the frequency that a phrase occurs in relation to other phrases.

    SQL does not see and cannot produce the above statistics – SQL is obsolete and out of business.

  • User profile image
    Jon

    Ilya nailed it. He came up with some kind of bizarre edge case that SQL Server has trouble with. We'd better extrapolate that to SQL is obsolete. I'm going to delete all my databases now and switch to using whatever the current fad of the moment is.

  • User profile image
    sokhaty

    Besides predicates, what can be pushed down into Hadoop as a YARN job? Joins, single row functions, how about aggregates and window functions? I suppose if you are generating MR jobs for submission it can be pretty much anything, if your query planner is sophisticated enough.

  • User profile image
    Yoshihro​Kawabata

    Thank you Sahaj Saini and Scott Klein for Excellent episode.
    I'm looking for the next episode.

    and, I hope step by step to try PolyBase on Azure SQL Database / Azure HDInsight, for "Push-down computation to Hadoop" functionalty.

    Regards,
    Yoshihiro Kawabata

  • User profile image
    asharovarov

    I was trying to create External Data Source on SQL Server 2016 CTP3 for RDBMS type, but it gives "Incorrect syntax near 'RDBMS'" error. Even though I copied query from them MSDN  page:
    https://msdn.microsoft.com/en-us/library/dn935022.aspx?lc=1033

    Is RDBMS type supported by SQL Server 2016? or it is just supported by Azure?

     

  • User profile image
    TorstenG

    @asharovarov: The RDBMS type is only supported for Elastic Database query in Azure SQL Database v12. It is not part of SQL Server 2016. We know that the MSDN page is currently not clear about this. We are iterating over an improved version of the MSDN topic to fix this.

    Thanks for pointing this out!

    Torsten

  • User profile image
    Dennes

    Hi,

    The example used hadoop on linux (port 8050). I was trying to do a pushdown example with hdinsight (hadoop on windows) but had no success, it seems the name node doesn't accept external connections to ports 8020 and 8032 as expected.

    Is it possible today ? How to configure this ?

    Thank you !

  • User profile image
    ravonmsdn

    Microsoft msdn polybase analytics will carry your enterprise to the ends of the earth.

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.