Anders Hejlsberg - LINQ
- Posted: Sep 13, 2005 at 11:27 AM
- 354,431 Views
- 87 Comments
Loading User Information from Channel 9
Something went wrong getting user information from Channel 9
Loading User Information from MSDN
Something went wrong getting user information from MSDN
Loading Visual Studio Achievements
Something went wrong getting the Visual Studio Achievements
Right click “Save as…”
Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums,
or
Contact Us and let us know.
Follow the Discussion
Oops, something didn't work.
What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in. You need to be signed in to Channel 9 to use this feature.What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in and view them all on your notifications page.sign up for email notifications?
now I know what the WinFS docs were talking about when they said C# 3.0's integrated query would be supported for querying WinFS stores.
Wow
Does ths not look like FoxPro? Where you can intermingle SQL with structure code in your source file? And get SQL extracts right into local vars?
Is it just me or are we back some 15 years' time?
No, FoxPro was 15 year ahead of it's time.
A lot of the functions remind me of Ruby blocks and iterators. Don Box did a series on implementing Ruby continuations in C# 2.0 a while back that was fascinating.
Heey does this mean we can pull data from database or xml quicker?
BTW, what was the "var" class (i.e. "var q = new...")? I expect the Monad folks wish they had this a year ago. I suppose they must shred this into their code now for all the object query stuff they do.
Now all anyone has to do is implement an LINQ Object Provider (LOP) around their data and Bingo, instant user query access.
Hmm...: fat, ntfs, AD, exchange, winfs, machine config, registry, perf counters, net config, hardware info. The list is endless.
Something like this would be way cool to be able to do some day:
var q = From Microsoft.LINQProviders.AD
Select WindowsIdentity
Where Username == "billg" && Domain == "MS"
WindowsIdentity wi = (WindowsIdentity)q[0];
WindowsPrinciple wp = new WindowsPrinciple(wi);
if ( wp.TryLogon("password") )
Console.WriteLine("Logon good.");
else
{
Console.WriteLine("Could not authenticate.");
return false;
}
Console.WriteLine("billg is an Admin:"+wp.IsInRole("Admins"));
using(Impersonate i = new Impersonate(wp))
{
File.Delete(@"c:\Linix\*.*"); //
}
Very cool indeed. Pizza and beer for the team!
--William Stacey [MVP]
so:
var i = someArray.Length; //i is an int
but
var i = 1.0; //i is a double
var i = {Name = "BillG", Title = "Chief Architect"}; //i is an anonymous type with string's Name & Title
pretty niffty if you ask me. very similar to python actually where you dont have to declare the type because it just gets inferred by how its used. I like it!
anyway back to w\tching video, i hope this give me a clue wtf happened to object spaces
Steve “Lightning” Trefethen posted a link to Anders Hejlsberg's LINQ video on Channel 9 Forums, so I was intrigued and took a look. After all, Anders was the father of Delphi.
In the video, at about 7 mins 50 seconds into it, Anders was talking about Language INtegrated Query, and he was explaining to the audience where “Where” and “Select” came from, “because an array doesn't have a select or a where guy” and that C# 3.0 implements a new thingy known as extension methods, which is the “ability to extend an existing type with new methods”.
Sounds like he's talking about Delphi's class helpers concept. You can take the man out of Borland, but you can't take Borland out of him... ;o)
Class helpers or extension methods have been available in the Delphi language since Delphi 8 came onto the scene nearly 2 years ago.
And the "var" thingy? Is he sneaking parts of the Delphi language into C# 3.0? ;o)
That this is moving forward is great. This will change programming and development forever.
That this is moving forward is great. This will change programming and development forever. Cant
That this is moving forward is great. This will change programming and development forever. Cant wait to
Wonder how long it will be before we get this in a product.
Will this be the death of DataSets? DataSet has not been able to fit into the web service world anyway (no type information in wsdl).
Very cool! Even those words simple rather wimpy, this has solved one of the major shortcomings in the langugage as to dealing with data regardless of type. Big, huge, gigantic leap for .NET!
Oh.
My.
God.
This is one of those "change everything" things.
By itself, this is impressive, but combine this with WinFS and Web service classes... yeeow, the possibilities are staggering! No wonder they're so excited about this.
One quibble - when Anders was talking about the reason why the "from" clause had to be first, it made a lot of sense, but then I went to the VB 9.0 whitepaper and saw this:
Dim SmallCountries = Select Country _
From Country In Countries _
Where Country.Population < 1000000
For Each Country As Country In SmallCountries
Console.WriteLine(Country.Name)
Next
The VB.NET syntax for LINQ looks more like SQL than the C# syntax. However, Anders' point about the C# syntax being better for XML querying still holds and is demonstrated by this VB.NET 9 snippet later in the whitepaper:
Dim CountriesWithCapital As XElement = _
<Countries>
<%= Select <Country Name=(Country.Name)
Density=(Country.Population/Country.Area)>
<Capital>
<Name><%= City.Name %></Name>
<Longitude><%= City.Longitude %></Longtitude>
<Latitude><%= City.Latitude %></Latitude>
</Capital>
</Country> _
From Country In Countries, City In Capitals _
Where Country.Name = City.Country %>
</Countries>
There's a lot of XML cruft before you get to the "from" and "where" of your LINQ code. It's easy to imagine how nasty it could get with a more complicated XML document. The C# syntax clumps all the LINQ syntax at the beginning and leaves the XML cruft for the end.
By the way, note the ASP.NET-like syntax in VB.NET 9!
Also, in the NorthWind example is LINQ doing a full table scan every time? Or does it take advantage of existing indexes? But you're not duplicating the SQL Server engine inside the CLR ... are you?
Also, will LINQ warns me of malform cross joins? Where I'm about to return 200 billion rows? Who of us haven't done that, right? Am I right? Hello?
That was, having calmed down a bit, pretty much exactly what I was going to ask.
SQL Server has been developed over a long time and is very very good at doing things like getting the two records out of 100 million that you want very quickly. I hope DLinq takes advantage of this - I mean, does it as Minh asks do a full scan of the table or does it parse your DLinq query in to T-SQL?
I find it slightly disturbing how this is presented and everyones comments here. There are interesting comparaisons to be made. There is a technological context.
This is VERY different than ORM. Typically with ORM you define a class and then map that to the database. And then you have some routines that can pull the data ouf of the DB into instances of that class. This here is completely different. Yes, it doesn use classes to define the mapping between the DB and the CLR type system, but you will hardly ever use instances of those classes. What you rather do is use the rich projection system to e.g. construct anonymous types on which you will then run your business logic. VERY, VERY different from any ORM approach I know about.
Yes, everything feels very familiar, BUT have a closer look. This is NOT just introducing some of the ruby etc stuff into C#. That too, but look at the database integration for example, that is just completely new ground. They use a lot of the constructs that are around in languages like ruby, but they use them for completely new things.
Don't worry. When you query a database via LINQ, the query expression that you construct in C# is translated into the corresponding SQL query and executed against the database. The whole thing does NOT first pull all rows from the DB into memory to do the filtering on that! Download the preview and read the overview document. It is absolutely amazing how they did that from a technical point of view!
I think, the answer is how do they implement the IEnumerable<T> interface. It's possible to open the SQL query within the GetEnumerator() implementation, then it can build the correct and exact sql query which returns the requested result only from the server. So, if you don't enumerate on a LINQ construct whatever is that, you don't contact with the sql server. A rude sample:
class SampleClass : IEnumerable<string>
{
IEnumerator<string> IEnumerable<string>.GetEnumerator()
{
using(SqlConnection connection = new SqlConnection(ActiveScribaDatabase.ConnectionString))
{
using(SqlCommand command = new SqlCommand("select CustomerName from Customers where zip=345", connection))
{
connection.Open();
using(SqlDataReader datareader = command.ExecuteReader())
{
while(datareader.Read())
{
yield return datareader.GetString(0);
}
}
}
}
}
}
Ahhh... That makes sense. You wouldn't have the home page for LINQ, would you? Google returns too many pages.
kdb = KSDatabase.new(dbh)
kdb.map_all_tables
kdb.select(:Users) { |user| user.company == 'Northwind' & user.age > 25 }
This is not quite 'VERY VERY VERY' different. It's using lambda functions like Anders does in the same context for the same purpose. No need to define classes. Don't get me wrong what Anders is doing is great. It has graceful syntax too. It's just a great evolution not an earth shattering revolution.
my jaw is on the floor!
I didn't think about using generics to extend existing types!!It didn't quite sink in yet... is this something we will be able to do in 2.0 ?
Scoble: It's not free.... we pay for windows!
bill makes money by selling windows, office, exchange, sql server.....
developers help him sell those thing by developing great apps.
(.... I know u know that.....)
You are right and I was wrong. I didn't know that things like that existed, I was only thinking about the typical ORM available for .Net.
Is at least the fact that all of this is done with strong typing something new?
Question. What about locking? How does locking syntax work when selecting over a collection? Is it just:
int[] numbers = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 };
lock(syncRoot)
{
var lowNums =
from n in numbers
where n < 5
select n;
}
Console.WriteLine("Numbers < 5:");
foreach (var x in lowNums) {
Console.WriteLine(x);
}
TIA
Comment removed at user's request.
Ooooooooo ..... uuuuuuuggggg ...... eeeeeeerrrrrrrr ..... eeeeeeeeek!
I've had a cow!
How good is this! I want a T-Shirt with the words LINQ on it!
For sure. I'm not huge on strong typing myself but I'm happy to see this for C#.
It's ALL NEW! OMG! CHANGES EVERYTHING!
</not>
http://msdn.microsoft.com/netframework/future/linq/default.aspx
For those who didn't make it to the end, here's the link for the bits.
I just wanted to quickly respond to some of the questions
Will LINQ support querying something besides IEnumerable in the future? How would I tell LINQ to query a Binary Space Partition? (requiring LINQ to know about the data container)
You can implement the LINQ pattern (your custom implementation of the LINQ standard query operators like from, where, select, etc) for your own data domain.
Also, in the NorthWind example is LINQ doing a full table scan every time? Or does it take advantage of existing indexes? But you're not duplicating the SQL Server engine inside the CLR ... are you?
When querying against Northwind, you are using DLinq, which is an implementation of the LINQ pattern for databases. What DLinq does is translate a LINQ query into a SQL statement, send it to the database and convert the results back into strongly typed objects for you. It uses deferred execution which basically means you don't have to pay a price until the last possible time form an execution standpointl. We'll be posting more information on how execution works soon.
This is not quite 'VERY VERY VERY' different. It's using lambda functions like Anders does in the same context for the same purpose. No need to define classes. Don't get me wrong what Anders is doing is great. It has graceful syntax too. It's just a great evolution not an earth shattering revolution.
In my opinion, the biggest change is that we're abstracting out the data store, meaning you can program against objects, relational data, and XML using a single, unified programming model. For example, you could use the VSTO (Visual Studio Tools for Office) API's and write a LINQ sample to query your email, like find all emails where:

1) the subject contains "LINQ"
2) sent in the last month
3) that were marked "urgent".
The Ruby libraries on the other hand are specifically for relational data. As an astute C9'er pointed out earlier, you can even imagine implementations of the LINQ pattern for Active Directory or WinFS where you can reuse the same syntax (from, where, select) without having to understand each domain. With the LINQ Project, you have one programming model across all domains. IMO, that's huge
I hope the VB team implements this stuff into VB.NET. This is just an awesome feature set!
LINQ will absolutely be in both VB and C#. In fact, you can download the VB Tech Preview compiler at: http://download.microsoft.com/download/2/a/4/2a405b66-1b1c-4fca-bfbf-007aad63d307/LINQ%20VB%20Preview.msi
Thanks everyone for the great feedback so far!
-Dan
In any one particular space, Microsoft has some really good competitors (eg. Flash is great for vector graphics today, PDF is the ubiquitious standard for transferring documents, java is a productive language for writing server side code), but its the sum of your parts (especially when you bring them together) that puts the MS platform well out in front (in my mind at least).
I should probably just read the documentation, but will LINQ be able to join data from 2 seperate data sources - eg. SELECT a.customer, b.product_photo FROM <sql data source> a JOIN <xml data source> b? ON (a.productid = b.productid)?
Thanks for the video...it was very interesting and I wish you guys would put up more videos from Anders. I am somewhere in the middle on the LINQ phenomenon, however, and again I keep going back to my old comment:
Why now? Why not a few years ago? Were there performance issues? The underlying technology has certainly been there for a while. Or was it an internal structuring/organizational issue? I can see reasons for both. Conceptually, however, only some of this is "radical".
Yes, this is big leap forward, and I'm all for it. I started using a bit of C# last month, and I am toying with the idea of building a language parser with it. However, relational databases have their limits, too, although the model has generally carried the industry a long, long ways.
Yes, it will, in fact in the Jim Allchin keynote, Anders and Don wrote a LINQ query that joins the results of Process.GetProcess() to a database table and then displayed the results as XML.
Thanks,
-Dan
This is great, I can't wait to use it. I noticed that the VB.NET syntax is more similar to a standard SQL SELECT statement. I understand the reason why in C# is arranged differently, but I really wish they would go the extra mile and tweak the C# compiler.
Just kidding. I did hear the words "strongly typed" used repeatedly.
But it's interesting to see the productivity people had with the older VB (where you did not have to worry about data types) combined with the robustness of C# (where you do have to worry about data types) -- but now it's the compiler that does the worrying for you.
That in itself is going to be a huge win. All those null vs. dbnull vs. empty string bugs... GONE.
The fact that it's universal... wow. Now maybe when I code against Outlook I can stop messing with collections and enumerators and retrieve the items I want in 5 lines of code instead of 50... using the same syntax I use to query databases...
... Select clause?
For instance, If we intend to build a screen that allows the user search and display customers using customers' Name, Phone #, City, and etc...
Today, a constructed SQL will look apporx like this:
string sqlStmt = "SELECT * FROM CUSTOMER WHERE ";
if(txtCity.Text.length() > 0)
sqlStmt += " CITY = '" + txtCity.Text + "' "; // London ; )
...
What will be an equivalent in LINQ to accomplish the above shown sql?
If C# (.NET) could support at least MAC and LINUX! ("MONO" is too slow and too far way from a preofessional grade lavel!)
Thanks
I'd like to know that as well!
Very exciting stuff, although I can't help thinking about LISP a little...
Aside, is there any way that we can get a clearer video of what's on the demonstrator's monitor? Half the time I can't make out what's being typed. (Sorry if this has already been covered a thousand times over; I have been gone from #9 for a while...)
Are they becomming absolete or is there still a specific use for them?
In other words, will LINQ give you as much control as those classes do?
The DataSet always seemed to me like te ideal place to put relational data, because of it flexibility and genericity, but in my mind was troubled by large memory use and inability to extend it.
Will LINQ bring the same flexibility to ordinary objects and will there still be a use for the DataSet (in this or in a changed form)?
Marcel
LINQ for Mono.
I hope there wouldn't be a way to do it.
This is more or less VB6 way; but wait, VB6 had parameterized queries for ages but noone used them.
Concatenating SQL queries is evil. Search Google and MSDN for "SQL Injection Attacks" to see how dangerous it is.
Besides manually generating WHERE clauses is wrong from the proper design prespective.
-DashNY
This depends. XQuery and SQL both provide for both static and strong typing on declarative expressions. However, the integration into the programming language type system is well-done (and thus could be considered "novel").
DashNY,
I thought for some time how to reply to you… you criticize but do not provide a relevant solution.
You focus on details in VB, which is like focusing on trees while we are talking about the forest. In any case VB is not used for large enterprise level applications.
It is arrogant to assume that others understand less or are less intelligent then you.
Lastly, please focus on the discussion at hand and make relevant posts, keeping your advices to yourself, unless asked for.
I am new here, and begginner in C#,
To me its like chinese,
WHat the heck is IEnumerable of T for some T mean?
Can you put it in simple english for me?
Read in some place though that some of the drawing code was slower, but that probably is in the works on improving.
True, maybe it performed not well enough. [C]
But hey, be honest to yourself. What you (probably everyone) actually want is just to persist objects! :O
Why not skip all the fuzz and move on to an object database!
And of course, query the database using new query mechanism of C# 3.0.
Does anyone agree?
I love the poster behind Anders:
B#
C#
J#
I#
Stay #
Gabor
I want an Anders Hejlsberg autograph. The guy was my hero for C#, my hero for C# 2.0 and now my hero for C# 3.0.
He did say in the video that there was a technological context and that this was not unprecedented stuff. Did you watch/listen to the whole vid?
Its just a generics thing. Look up "generics" and ".Net 2.0" and you should find some stuff. Just basically saying that it implements the IEnumerable interface for type "<Any Type>", or type "<T>" as it is commonly referred to. Generics is some pretty neat stuff.
say:
var MyCustomType customType = {LINQ/DLINQ/XLINQ}
or:
var<MyCustomType> customType = {LINQ/DLINQ/XLINQ}
Thanks,
Frisky
"So can an somehow assign a name to my anonymous type to use elsewhere in my application?"
There is no way currently for an anonymous type to leak outside of the method it's used in. If you're going to do that, you should just create a type normally and give it a real name.
Normally, I'd agree about the dynamic SQL generation, but have you actually tried using DLinq?
var customers = from c in db.Customers select c; // base query
IQuery query; // final query
if (filterByCity)
{
string cityName = "London";
// modify the base query
query = from c in customers where c.City == cityName select c;
// or query = customers.Where(c => c.City == cityName);
}
else
{
query = customers; // use the base query directly
}
Console.WriteLine(db.GetQueryText(customers)); // base query
Console.WriteLine(db.GetQueryText(query)); // correctly modified query
I have a couple of questions.
In the overview document I could not find any references to how to define a primary key for a given relation when querying over a list of objects stored in, say, ArrayList? Or does the query perform a linear search?
Are there plans to support implicit parallelism for executing queries?
With strongly typed databases there has always been a challenge responding to the changes to the data schema. How do the new language extensions address changes to existing object schema? For example, if the layout of a class that is being queried has been changed, what is going to happen? Am I going to get a compile exception or a runtime exception when attempting to execute a query?
Typically, when quering over millions of records it would not be desirable to load the entire contents into memory. The question is if it is a requirement for the entire searchable contents to be loaded into memory before querying the data?
What types of joins the new extension is going to support?
How does the extension support the variable binding?
Appreciate your response
Thanks!
Was there a bit of product placement in this video? I was expecting hear the Coca Cola slogan at the end of this video.
LINQ looks interesting I liked reading this article
http://msdn.microsoft.com/netframework/default.aspx?pull=/library/en-us/dndotnet/html/linqcomparisons.asp
And when I get over the FROM SELECT thing I might give LINQ a try
I agree, I watched the video too. We offer an architecture for .NET 2.0 called EntitySpaces that is strongly typed, spoon feeds you via intellisense, has nullable types, and so on, it also offers natural language queries that run without recompilation on many DBMS systems with more on the way, for example, here's a dynamic query:
AggregateTestCollection aggTestColl = new AggregateTestCollection();
Our NUnit tests run against Microsoft SQL, Access, Oracle, and MySQL, same exact code, same exact binaries, only a different connection string. LINQ is cool, but sometimes I think folks at Microsoft think it's only real or invented if they do it ....aggTestColl.Query.es.CountAll = true;
aggTestColl.Query
.Select (aggTestColl.Query.IsActive,
aggTestColl.Query.DepartmentID)
.Where (aggTestColl.Query.IsActive.Equal(true))
.GroupBy(aggTestColl.Query.IsActive,
aggTestColl.Query.DepartmentID)
.OrderBy(aggTestColl.Query.DepartmentID.Ascending,
aggTestColl.Query.IsActive.Ascending);
aggTestColl.Query.es.WithRollup = true;
aggTestColl.Query.Load();
Mike Griffin
EntitySpaces LLC and MyGeneration
http://www.entityspaces.net
http://www.mygenerationsoftware.com
And a lot of people seem to think their solution is already the best. Your syntax looks quite a deal different from LINQ. When companies pour developer time into a language or even just a construct, its usually not just to get the basic idea coded up, but to also come up with a good "syntactic logic". As it was explained in the video, SQL has been done for a long time as "SELECT * FROM TABLE etc". They turned that around and worked in a different direction.
Also, compare your code to something like
IEnumerable<string> expr = from s in names
where s.Length == 5
orderby s
select s.ToUpper();
The query statements look like part of the language instead of being methods being run on new sub-query objects.
My two cents,
Aditya
Actually with N-tier apps do not have a need for ADO.NET and it should be hitting the classes anyways.
They are just wrapping reflection around the queries. Which is what many world class tier apps already do. It's just easier now.
Hell I havent even seen someone use a DataSet in a year or 2 now. And if they do I call them a noob.
Object databases have been developed for the last 5 years and a few are .net open source projects. Some already handle indexing.
After all, you just inject queries. It still uses the SQL engine.
This qualifies: Web Service Software Factory from p & p.
It isn't LINQ, but it is released...
Anders, you have done so much for software development!
Maybe VS will get there in Do you remember subclassing
get the object
name it
drop it into a new or existing class
Job Done
Try the on Pocket PC!!!!!!!
You cant even change the disabled forecolor so that you can see a disabled textbox!!!!
Richard JJS
LINQ obviously abstracts the act of retrieving data away and allows for a declarativ programming style.
But almost any application also needs to update existing and insert new persisted data in databases.
I guess that developers for now still need to interact directly with ADO.NET? Are there any researches going on in the field of declarativly adding/updating data, for example as extension methods on Collection<T> and Table<T>?
>> sqlStmt += " CITY = '" + txtCity.Text + "' "; // London ; )
I really hope you are protecting your database more than this in reality and this was just for the simplicity of the example. A user could do SQL injection if they wanted very easily. While it's more popular to do SQL injection in web apps, it can happen in Win apps as well.
--
The LINQ stuff is really great, the comments about performing joins across multiple sources is intriguing. I could select email from outlook, have it join where the from address matches values in ActiveDirectory for a specific group... so you could pull out all emails from administrators, for a simple example.
The danger I see, however, is that it could be very easily abused and not completely understood so that some very slow queries would be written.
In the above example, I'd imagine that since the two pieces of results are coming from different sources (one from AD and one from your email source) that each would be returned and joined in memory. That means that I may have 1000 emails and only one of them is from an admin but all 1000 emails are returned before the joining can occur. The compiler wouldn't know that the list of admins is small enough to execute it first and pass it in as a param to the email engine so that it would only return that one email.
When first watching, I had the thought "ok, now so many things are so much simpler that it allows amateurs to do what normally required a professional" maybe now the differentiator will be that I will understand enough of how it is all working that my code will designed to run significantly faster.
It's all great stuff. I love that through the type inferencing and dynamically generated classes, it has the ease of non-typed languages (Ruby) without losing the performance and benefits of strong typing.
LINQ for SQL (previously called DLINQ) will hook up to property changed events of the objects that are retrieved using LINQ for SQL.
They will then track any changes you do to the objects, and when you instruct the DataContext to submit the changes back it will do so intelligently, i.e. topological sort so that deletes, inserts and updates are performed in valid sequence.
I remeber six months back i was talking to friend on the objects and implementation and i mentioned that it will be awesome to have SQL like queries for various operations inside managed code... and now i am glad to hear that i can see that kind of implementation in .NET soon.
Its really a great future to have.
I heard that this feature is not available in Java....True?
It would be nice to see more demonstrations of Table join examples.

I've written several opject dumping utils over the years, is there anyway we could get a copy of yours it looks like a keeper
Remove this comment
Remove this thread
close