Concurrency Safe C# from TSI/Midori team: Joe Duffy etc.

Today I found a very interesting paper from Colin Stebbins Gordon, Joe Duffy and several people from MSR Cambridge

Colin was a team member of TSI.

Microsoft: Technical Strategy Incubation - Intern, Program analysis work (Summer 2011)
Microsoft: Technical Strategy Incubation - Full Time, Kernel developer on an unannounced systems project. Worked on kernel debugger implementation, interrupt handling, post-mortem debugging, platform abstractions, all work interacting with other subsystems. (August 2008 - September 2009)

The paper is: Uniqueness and Reference Immutability for Safe Parallelism
http://www.cs.washington.edu/homes/csgordon/papers/oopsla12.pdf

Abstract: A key challenge for concurrent programming is that side-effects (memory operations) in one thread can affect the behavior of another thread. In this paper, we present a type system to restrict the updates to memory to prevent these unintended side-effects. We provide a novel combination of immutable and unique (isolated) types that ensures safe parallelism (race freedom and deterministic execution). The type system includes support for polymorphism over type qualifiers, and can easily create cycles of immutable objects. Key to the system's flexibility is the ability to recover immutable or externally unique references after violating uniqueness without any explicit alias tracking. Our type system models a prototype extension to C\# that is in active use by a Microsoft team. We describe their experiences building large systems with this extension. We prove the soundness of the type system by an embedding into a program logic.

Basically there are 4 new modifiers in C#: readable, writable, immutable and isolated.

IMHO 'readable/writable/immutable' are similiar to 'const' in C++, the interesting one here is 'isolated'.

writable: An "ordinary" object reference, which allows mu-tation of its referent.
readable: A read-only reference, which allows no mu-tation of its referent. Furthermore, no heap traversal through a read-only reference produces a writable ref-erence (writable references to the same objects may exist and be reachable elsewhere, just not through a readable reference). A readable reference may also refer to an im-mutable object.
immutable: A read-only reference which additionally notes that its referent can never be mutated through any refer-ence. Immutable references may be aliased by read-only or immutable references, but no other kind of reference. All objects reachable from an immutable reference are also immutable.
isolated: An external reference to an externally-unique object cluster. External uniqueness naturally captures thread locality of data. An externally-unique aggregate is a cluster of objects that freely reference each other, but for which only one external reference into the ag-gregate exists.

And, Implicit converting and recovering !

No global state !

No threads ! No thread for 'async' block ! Actor-based concurrency !

Destructive read: 'consume' !

Polymophism ! Generics for permissions !

Interesting  ...

The 6 segment 'Evaluation' is the most interesting part, described the actual language used in TSI projects, and history of the evolution, very informative read !

This is the most recent deep techninal sneak peek at what's cooking inside the mysterious TSI department !

I hope Joe Duffy will present this at YOW! 2012.
http://www.yowconference.com.au/general/details.html?speakerId=2991

This whole "ag-gregates" feels very similar to CQRS EventSource Aggregates which I heavily use in my apps these days... I can't live without my aggregates, and eventsource for that matter! Very interesting!

http://cqrsguide.com/doc:aggregate

It always facinate me how you manage to dig up all these cool research projects. Great find.

@magicalclick: I was just googling for "Technical Strategy Incubation" and this is on the second page. Actually I haven't google this for many months, so its pure luck.

Hmmm.... actuall Joe Duffy has talked about a lot of the ideas in this video
http://channel9.msdn.com/shows/Going+Deep/Joe-Duffy-Perspectives-on-Concurrent-Programming-and-Parallelism/

And maybe we can read these again
http://www.bluebytesoftware.com/blog/default,date,2010-07-11.aspx
http://www.infoq.com/articles/Interview-Joe-Duffy

Now if we can just find MSR-TR-2012-79 ...

Was doing some reflecting on past interesting videos and "Wolfram Schulte" came up a couple of times in regards to comments he made of future "parallelism" work (was from a SPUR) video in 2010 ( Inside SPUR - A Trace-Based JIT Compiler for CIL )

So checking out his research page seems like in the July 2012 there are 2 recent updates

1. Waiting for Godot: The Right Language Abstractions for Parallel Programming Should be Here Soon

2. Maine: A Library for Data Parallel Finite Automata

Seems like MS may have cracked how to deliver 'parallelism', it was hinted as far back as 2009/2010 and only now being realized (in felix's uncovered paper )

Interesting times ahead ...

@LiquidBoy: Nice find. How do you download a tech report from Microsoft Research?

All this parallelism/concurrency discussion also remninded me of a great channel 9 video i watched 2 years ago of an interview of Corrinne Yu, shes a great graphics programmer and helped developer and chair some of the Direct3D api's .. Currently works as architect at 343 studios working on the latest HALO...

She made some pretty impressive all encompasing quotes on future directions of GPU programming and programming in general that resonated with me.

She talked about in 2010 how D3D had reached a level of maturity and that they were at a juncture where new paradigms in coding and parallelism and possibly new languages would emmerge, even calling out possible semantic attribute based changes like we see in the above mentioned papers.

http://channel9.msdn.com/Shows/TechWithTina/Corrinne-Yu-Principal-Engine-Architect-Halo-Team-Microsoft-Part-One

Its worth watching this video, she gets very excited and talks a million miles an hour BUT its worth listening to what she says, shes very very smart.

Im hoping that all this "unmentioned" work that these papers mention is being done in the graphics space and hopefully its in XBox vNext, maybe throwing in some midori ...

EXTENSIBLE DATA PARALLEL SEMANTICS (submitted 2010 , published june 2012)

http://www.google.com.au/patents?id=XJUeAgAAEBAJ&lpg=PA8&zoom=4&ots=FRdUDQtXUU&dq=directx&pg=PA1#v=onepage&q=directx&f=false

Whats interesting is the mention of taking General Purpse languages (C/C++/C#/JavaScript etc) and spraying them with data parallel features ...

Added to this is mention of DirectX 12 etc.. I really hope we see all this in XBox vNext ...

this paper is published on MSR site as MSR-TR-2012-79

http://research.microsoft.com/apps/pubs/default.aspx?id=170528

OK, now we know the name of this language, M#

https://careers.microsoft.com/jobdetails.aspx?jid=111551&pp=VR

The Technical Strategy incubation team is seeking an exceptional developer to build the user interface platform for next generation UI applications. Your responsibilities will include the design and implementation of core UI services such as composition, input, window management, and a controls framework for both native and HTML5 applications. All development is in M#, a C#-like language for writing asynchronous and parallel operating system components with strong guarantees of correctness.

@felix9: Wow, another UI framework ... not sure what to think about that, but M# sounds interesting overall.