Expert to Expert: Brian Beckman and Sam Druker - Deep Entity Framework

Very interesting stuff. I have faced the problem of constructing a view from multiple sources on multiple occasions, while maintaining the ability to update and/or insert new data. This framework really sounds like a solution for this problem. Or am I on the wrong track here? Like it would be great if could composite data from multiple database sources and/or other kinds of data objects/structures, because this would greatly reduce the cost and effort to be put into connecting different (software)systems together.

I never understood the entity framework, but now I do.  It seems very interesting.

Currently, I have a data access class I wrote called "StoredProcedure".  I then have code generators to create the stored procedures, and the VB.NET classes.  It's specific to the type of operation, such as insert, update, delete, select as output params, select multiple, etc.

Although I think it's very clean, replacing all of that with the entity framework, if it's clean and is performant, would be great.

However, the stored procedure class I created offers quite a bit and has some intelligence in it.  For example, formating the sproc name, validation, not adding certain parameters if they don't pass validation and that parameter has a default, etc.

Also, I plan on adding possibly some type of load balancing to it, possibly based on the stored procedure name (or parameter specifiying read only or data is modified by this sproc), and also user based specific connection strings.  (This way, for example, you can setup mirroing database, and have the read-only stored procedures executed on the mirrored database.)

I'd be curious to see if the entity framework would be able to do this as well, and\or whether or not you can inherit and modify certain objects that are responsible for this.

Thanks to all.  Very informative.  However I am still wondering where the business logic goes?  My brain wants it to go inside the EF so I can code it in 1 place using c# (for example) and not have it sprinkled all over the client, BLL, and sprocs in the server.  Is this the case?  Do I use code-behind to add my BL inside the EF and it travels with the EF?  If so, great, I would like to see a video on it.  Also, personally, I would like to never have to switch into tsql again as I don't like moving between DSL and keeping those all fresh in my head, plus all the added complexity.  Will we be able to code our sprocs inside the EF using .Net?

Also, as you hinted to.  It seems to me that the model I really want is have the EF be exposed as a service on the server side.  Then my client can connect to it (using Linq) using a client side EF provider.  So it will look and feel like DLinq, but will be talking to EF front end instead of ADO/DB directly.

And what I "really" want to do someday is this from the client:

Future<Customer[]> f = Future.Create(() =>
    {
        return DB.Customers.Where(c=>c.Active==true);
    });

ObjectDumper.Write(f.Value); // Wait and print active Custs.

So this "ships" the Func to the Server (i.e. does not convert to TSql).  And the server processes Func directly (like a dynamic sproc). So it is like directly passing a sproc right into the server and getting results.  Objects are not created or serialized until rehydrated on the client, the stream from server to client would be byte[] (i.e. not xml).

Thanks much.

Cool video. Charles can you get an interview with the guy Microsoft just hired that created SubSonic? I want to know if he is going to move SubSonic over to EF.

I'm looking forward to more videos from Brian

Charles wrote:

What do you think of this format: expert to expert?
C

I love the new format expert to expert is a great idea

littleguru wrote:



Charles wrote: What do you think of this format: expert to expert? C

AWESOME! I had a "brain wave" (if you want to call it that way) a few hours ago. Why not throw a few experts together and let them defent their technology against each other? I don't know if that is doable but sounds very cool to me...

haha like "expert death match"

Charles sort of already did it with functional vs. imperative at JAOO. I think that could be revisited because all those guys seemed pretty set on functional languages. I want a diehard imperative guy to take on a diehard functional guy haha

Looks like ADO.NET Entity Framework Beta 3 was released today.  Looking forward to having a play!

Great stuff, data modelling is definitely more interesting than most people think. I hate it when people think of DB's as black boxes, and have no understanding of all the layers (yes donkey like an onion) in between a conceptual I want this customers info and the DBMS spitting out the data.

I hope one day to have the variety of projects under my belt Brian does I too am a physicist (superconductivity and protein folding) that have landed myself into IT/programming role. I definitely find the similarities in the problems that need solving, however one pays better than the other

Charles wrote:

What do you think of this format: expert to expert?
C

I like the format very much.  Thanks

Charles wrote:



Cyonix wrote:  littleguru wrote:  Charles wrote: What do you think of this format: expert to expert? C AWESOME! I had a "brain wave" (if you want to call it that way) a few hours ago. Why not throw a few experts together and let them defent their technology against each other? I don't know if that is doable but sounds very cool to me... haha like "expert death match" Charles sort of already did it with functional vs. imperative at JAOO. I think that could be revisited because all those guys seemed pretty set on functional languages. I want a diehard imperative guy to take on a diehard functional guy haha

I'll see what I can do
C

Throw in an OS expert and I'll fetch the popcorn!
Ding ding!

Is there interview/blog or anything that compares the Entity Framework to NHibernate or other ORM mappers? I'd like to get a some detail into the differences between that and the EF, EF's advantages/disadvantages/etc...

I just finished my part of a project and I used ORM for the first time, the Castle Project's ActiveRecord witch is built on top of NHibernate, and I really liked it for the most part.

Nomenclature comment first, since I get myself confused. ADO.NET is basically at the third major version as of this imminent (not-quite-Orcas) release. Much of the Entity Framework parts of ADO.NET (like the mapping/entity provider, the object facade that builds on top of that to provide ORM API's, bridge, eSQL and LINQ support are the "v1-ish" boxes. Some boxes, the updated .NET Data Providers, command query trees, and updated DataReaders, are a bit of both.

The EF v1 doesn't do multiple data sources. Fundamentally, it does "reshape" data in a queryable and updateable way, based on the rich model described in the EDM and a set of mapping corresondences. Our core scenarios are around core data access/independence. We will bring the reshaping capability to "disconnected" (don't want to confuse with "offline") programming ala DataSet soon (in a release to be named later, barring acts of force majeure, etc, etc.).

That's an important step along the way to compositing data from multiple sources in the way you describe. Then we'd have the part of the problems that let's a programmer explain what span of data they want, when it shoudl be refreshed, how to push changes, etc. Multiple sources gets complicated with further details like cache coherence and isolation levels and transactions and all that, um, database stuff.

MetaGunny wrote:

I never understood the entity framework, but now I do.  It seems very interesting.

Thanks, me too.

MetaGunny wrote:

Currently, I have a data access class I wrote called "StoredProcedure".  I then have code generators to create the stored procedures, and the VB.NET classes.  It's specific to the type of operation, such as insert, update, delete, select as output params, select multiple, etc.

Although I think it's very clean, replacing all of that with the entity framework, if it's clean and is performant, would be great.

However, the stored procedure class I created offers quite a bit and has some intelligence in it.  For example, formating the sproc name, validation, not adding certain parameters if they don't pass validation and that parameter has a default, etc.

The EF engine and generators have hooks to enable common validation scenarios. I can't speak to the sproc renaming off the top of my head--let me see if I can get Pablo or Tim or Steve to stop by for an answer.

MetaGunny wrote:

Also, I plan on adding possibly some type of load balancing to it, possibly based on the stored procedure name (or parameter specifiying read only or data is modified by this sproc), and also user based specific connection strings.  (This way, for example, you can setup mirroing database, and have the read-only stored procedures executed on the mirrored database.)

I'd be curious to see if the entity framework would be able to do this as well, and\or whether or not you can inherit and modify certain objects that are responsible for this.

One nice side effect of doing a bunch of metadata plumbing under the hood is that we can start to regain some of the flexibility with conn strs.

staceyw wrote:



Thanks to all.  Very informative.  However I am still wondering where the business logic goes?  My brain wants it to go inside the EF so I can code it in 1 place using c# (for example) and not have it sprinkled all over the client, BLL, and sprocs in the server.  Is this the case?  Do I use code-behind to add my BL inside the EF and it travels with the EF?  If so, great, I would like to see a video on it.  Also, personally, I would like to never have to switch into tsql again as I don't like moving between DSL and keeping those all fresh in my head, plus all the added complexity.  Will we be able to code our sprocs inside the EF using .Net?

In my own view of the "state of the art" app model BL is specific to a tier [Don't ask me about tier-splitting yet--I'm an engineer not a scientist]. Yes code-behind for mid-tier or client logic but your sprocs are still in your data-tier with that dev environnment. Architecturally speaking, I do anticipate being able to write "BL" type sprocs in the database with Entities. Of course, you can write sprocs inside SQL Server using VB or C# today. For certain scenarios that may be a good head start.

In my experience I find schemes for"traveling code" or "code shipping" a big red flag. Distributed deployment is hard enough; adding a self-modifying aspect can get crazy pretty quick.

staceyw wrote:


Also, as you hinted to.  It seems to me that the model I really want is have the EF be exposed as a service on the server side.  Then my client can connect to it (using Linq) using a client side EF provider.  So it will look and feel like DLinq, but will be talking to EF front end instead of ADO/DB directly.

And what I "really" want to do someday is this from the client:

Future<Customer[]> f = Future.Create(() =>
    {
        return DB.Customers.Where(c=>c.Active==true);
    });

ObjectDumper.Write(f.Value); // Wait and print active Custs.

So this "ships" the Func to the Server (i.e. does not convert to TSql).  And the server processes Func directly (like a dynamic sproc). So it is like directly passing a sproc right into the server and getting results.  Objects are not created or serialized until rehydrated on the client, the stream from server to client would be byte[] (i.e. not xml).

Thanks much.

Sprocs are the "material in the room", as Brian would say. But I can imagine a world where the queries are shipped over directly as canonical trees. This would be like the very first SQL QP which was a new front end basically grafted into the existing QUEL pipeline. The EF architecture was inspired by that example in fact. The EF design was faced with needing to support multiple query syntaxes from day 1.

On the serialization format, note that TDS takes care of all that, just like in pre-EF ADO.NET, ODBC, JDBC, and OLEDB. Whether or not you get XML scalar types.

Small soapox moment, diving into the opening: 

Of course "select c as customers where c.active" is itself not a func. It's an expression. Further, it's an expression whose evaluation can be manipulated in very interesting ways by an optimizer. Even better, it can also be composed while maintaining that set of properties. And finally, the monad itself can be treated by the language as a unit of remotability. All 4 of those things are very very good for high-performance, low-obscurity database programming.

Charles wrote:



I'll see what I can do
C

Pass. I'm already overdue for a dunk tank (sorry Jenn).

Charles wrote:

What do you think of this format: expert to expert?
C

ummm.... not for everyone :D