Software Transactional Memory: The Current State of the Art

Posted By: Charles | Dec 29th, 2008 @ 12:19 PM | 70,272 Views | 6 Comments

Formats:

A few years ago I got the chance to learn about Software Transactional Memory for the first time while visiting MSR Cambridge. The great Simon Peyton-Jones and Tim Harris explained to me the thinking behind STM and how it might evolve. It was a tremendously interesting conversation. If you haven't watched that interview, I highly recommend it as a precursor to this one. Today, STM is no longer only a research project. The Parallel Computing Platform team is incubating and extending the technology, finding that it may in fact work in the real world...

Of course, there is no silver bullet to solving the Concurrency Problem, but STM may be an important part of a larger solution (you've leraned a great deal about what Microsoft is up to in the concurrency and parallelism space here on Channel 9 and it should be somewhat clear by now that many of the technologies we've presented to you may end up as pieces of a broader solution...)

Here, STM Program Manager Dana Groff and STM Principal Developer Lead Yossi Levanoni discuss the current state of STM and outline the work their team is doing to craft this incubation/research technology into a practical real-world solution (STM is not available yet for experimentation. It's in incubation. It's not known if or when STM will become a viable product.). So, how has STM evolved over the past two years, anyway? Tune in.

Enjoy.

Tags: Concurrency, Parallel Computing, Programming, Software Composability, Software Engineering Research, STM

Rating:

vasilep

Finally we hear a more pragmatic story on STM. No more Holy Grail! Thanks Dana and Yossi.

Too bad we did not hear much about the “The Current State of the Art” on Hardware TM. Maybe we could have heard how CISC architecture is saving our (like in “we, the software guys”) pride (read “behinds”) once again. Smiley

dot_tom

Another great interview Charles. It's very encouraging to see Microsoft openly approaching the parallelism 'problem' from multiple angles. I'm really hoping that in 2009 you'll be putting up videos with the folks doing the 'side effects / mutation aware' type system evolutions that David Callahan et al referred to with a nod and wink at the PDC...

Taking a step back for a moment.

As Microsoft encourages us to elevate our thinking about concurrency and parallelism out of the quagmire of explicit threads and locks, and as more and more managed technologies enter this spaces (Tasks, PLINQ, STM etc) I wonder if it isn’t time to revisit the root namespace. Krzysztof Cwalina and Brad Abrams have long championed the notion of putting ones lower level types for ‘advanced’ or seldom encountered scenarios in a sub-namespace. Today with .NET 4 we have:

System.Threading
System.Threading.Tasks
System.Threading.Collections

And who knows, if it isn’t baked into some language maybe the future will see:

System.Threading.TransactionalMemory

However if we want to encourage folks not routinely stumble upon Thread and Mutex etc types then perhaps a better organisation would be:

System.Concurrency
System.Concurrency.Tasks
System.Concurrency.Collections
System.Concurrency.Threading
System.Concurrency.TransactionalMemory

In fact one might go so far as to effectively promote the Tasks sub-namespace types to be prime candidates for the root System.Concurrency. At least then as we move forward we can remove the bug-factory-esc lower level primitives from being the first thing developer’s encounter. It strikes me that it should be possible to enhance the Fusion binding system to allow code to safely migrate towards a reorganised namespace schema. There are echoes here of Rico Mariani’s Pit Of Success, i.e. ensuring the developers encounter best practice (types) first.

Charles
Welcome Change

In reply to vasilep

#Dec 29th, 2008 @ 9:54 PM

Indeed, software tooling innovations (languages, runtimes, libraries, design patterns and guidance) to support many-core architectures needs to catch up to hardware advances and you can bet (as mentioned in this interview) that software crafters and hardware engineers are working together very closely as we march into the age of thousands of cores per chip. It's all about meeting in the middle given the complexity of the problem on both sides. We're making real progress, but have a long way to go.

C

Charles
Welcome Change

In reply to dot_tom

#Dec 29th, 2008 @ 10:24 PM

Good thinking. Indeed, the basic idea is to abstract away Threads from explicit thought (just as .NET and Java do in terms of memory allocation and lifetime management of objects via garbage collection). In fact, this is exactly what the TPL does (you program Tasks, not Threads, explicitly). This seems to be the direction the Parallel Computing Platform team is heading, but you the programmer will have to help us figure out if this is the right approach. Are threads evil? Of course not. The platform that we've built around them over the years was not designed for many-core processors so the current semantics of explicit threaded programming are, well, lacking for the general purpose programmer who simply wants to solve problems in object oriented and imperative way (sequential code composition). Rather than forcing you to learn a completely new way fo thinking, the goal is to fold concurrency and parallelism into the tools and compositional patterns already burned into your mind. Obviously, this is a very hard task, but a worthy and noble one.

See you in the middle,
C