Entries tagged with concurrency - Channel 9

Wes Dyer and Stephen Toub: Rx and Px - Working Together

Charles — Thu, 19 Nov 2009 21:56:00 GMT

Reactive Extensions for .NET (Rx) released this week during PDC09. Rx uses Parallel Extensions for .NET (Px) for all of it's concurrent and parallel computing needs. How is it using Px, specifically? What's going on here and why?

Stephen Toub, PM on the Px team, and Wes Dyer, developer on the Rx team, tell us all about this partnership the experience of collaborating on two very compatible technologies that, taken together, create something beautiful. Some many xs, so little time.

Enjoy.

E2E: Erik Meijer and Burton Smith - Concurrency, Parallelism and Programming

Charles — Tue, 03 Nov 2009 16:19:00 GMT

The great Burton Smith, Microsoft Technical Fellow and an international leader in high-performance computer architecture and programming languages for parallel computing joins functional programming purist and language design guru Erik Meijer to discuss several major themes of parallel computing and distributed programming. As always, you will get a lesson in history, present trends and future possibilities. This is simply an awesome and deeply wonderful conversation. Burton is a treasure.

Erik shows up for the conversation only after Burton begins to talk about a potential definition for functional programming. Right on queue, Erik arrives!

Burton will be presenting his thinking on parallel and concurrent programming at PDC09. He will also be a panelist on the Future of Programming panel (and Erik will be the panel moderator - you won't want to miss the panel if you are attending PDC!).

Visualizing Concurrency: VS 2010 Beta 2 - Parallel Performance Profiling Advancements

Charles — Mon, 19 Oct 2009 17:21:00 GMT

In Visual Studio 2010 Beta 1, you were introduced to new analysis and profiling capabilities (Parallel Profiling and Performance Tools) designed to make concurrency understandable and, ultimately, debuggable. Today, with the release of Visual Studio 2010 Beta 2, we introduce an updated and significantly more capable concurrency visualization and profiling tool which is available with other profiling features in Visual Studio 2010 Premium and Ultimate. What does it do, exactly? How does it work?
What's new?

Here, Architect Hazim Shafi, Dev Lead Sasha Dadiomov and PM Bill Colburn tell us all about the Concurrency Visualizer Profiling Tool, including a demo. So, fire up Beta 2, spin up some threads and visualize concurrency. You should profile an already-existing application that employs concurrency and, perhaps for the first time, get to see what your concurrent code is actually doing at run time.

Parallel visualization tools team blog: http://blogs.msdn.com/visualizeparallel/

The parallel computing dev center: http://msdn.microsoft.com/en-us/concurrency/default.aspx

Hazim's blog: http://blogs.msdn.com/hshafi/default.aspx

Parallel Performance Tuning for Haskell

Charles — Sat, 12 Sep 2009 19:30:00 GMT

Very interesting work with implications for integration into more mainstream runtimes... In general, runtime support for parallel tuning is necessary going forward. The Many Core age has only just begun... This paper is a great read.

Parallel Haskell programming has entered the mainstream with support now included in GHC for multiple parallel programming models, along with multicore execution support in the runtime. However, tuning programs for parallelism is still something of a black art. Without much in the way of feedback provided by the runtime system, it is a matter of trial and error combined with experience to achieve good parallel speedups.

This paper describes an early prototype of a parallel profiling system for multicore programming with GHC. The system comprises three parts: fast event tracing in the runtime, a Haskell library for reading the resulting trace files, and a number of tools built on this library for presenting the information to the programmer. We focus on one tool in particular, a graphical timeline browser called ThreadScope.

STM.NET: Who. What. Why.

Charles — Fri, 14 Aug 2009 17:39:00 GMT

Software Transactional Memory is no longer a pipe dream or the stuff of academics. STM.NET, as it's called, is ready for your experimentation.

The goal of STM.NET is to be able to exploit concurrency by using components written by experts and consumed by application programmers who can then compose together these components using STM. Transactional memory provides an easy-to-use mechanism to do this safely. STM.NET is of course not a concurrency silver bullet and this is an experimental rrelease of the .NET Framework that allows C# programmers to try out this technology, specifically a particular implementation of STM.

The STM team really needs your feedback to understand if they're doing the right things to meet your needs. Traditionally, using STM for simple trasactional tasks didn't make sense. The overhead was too high. Is this still the case? What needed to change in the .NET Framework to enable STM.NET? Remember, this is a .NET Framework experiment to enable STM for managed code.

Here, we meet most of the team responsible for STM.NET: Chris Dern‎, Yossi Levanoni‎, Sasha Dadiomov‎, Weirong Zhu‎, Sukhdeep Sodhi‎ and Lingli Zhang.

Tune in, meet the team and get a good sense of what this very small team has accomplished with STM.NET and learn about some of the paths taken to get there. This represents really great engineering. Congratulations to the STM team! Now, Niners, go get the bits!

Enjoy.

Expert to Expert: Erik Meijer and Michael Isard - Inside Dryad

Charles — Tue, 21 Jul 2009 15:55:00 GMT

Microsoft Research recently announced the availability, under Academic Licensing, of Dryad, an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs.

A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming.

That's a pretty heady statement. What does Dryad do, exactly, to enable this level of abstraction, shielding programmers from the incredibly complex world of distributed parallel computing? Does the level of abstraction impact the degree to which sophisticated programmers can interact with and control some of the low level mechanisms of the Dryad runtime? What is it about LINQ that made it the no-brainer managed programming abstraction for Dryad?

Simply, how does Dryad work? This is the core question that Erik and I had after our conversation with Roger Barga (part one of this E2E mini-series on Dryad and DryadLINQ - perhaps we should focus just on DryadLINQ next time, but for now, all the information in this conversation is certain to keep you very busy and answer many questions you may have after learning about Dryad in part one...).

Lots of whiteboarding here. Put on your thinking caps!

Enjoy.

Expert to Expert: Erik Meijer and Roger Barga - Introduction to Dryad and DryadLINQ

Charles — Thu, 16 Jul 2009 16:43:00 GMT

Microsoft Research recently announced the availability, under Academic Licensing, of Dryad, an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs. A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming.

DryadLINQ is the managed high level programming abstraction used to compose Dryad vertex topology graphs that the Dryad infrastructure uses to partition parallel computations. Here, Erik Meijer and Dryad team member Roger Barga discuss Drayad and DryadLINQ at a high level so that most of us can understand the implications, history and future of Dryad. This is an introductory piece. Erik and I will dive deep into Dryad with one of the scientists behind it in the second part of this Expert to Expert mini series on Dryad. UPDATE: The Going Deep episode on Dryad is now live.

Enjoy! This is incredible and important technology for simplifying the inherent complexity of distributed computation in the cloud. In essence, DryadLINQ enables a sequential programming experience over what will execute across potentially thousands of machines (depending upon the computational complexity of the program) concurrently. Much to learn here. Channel 9 will help teach.

Editorial note: When we discuss native code and the implementation of Dryad, the focus is on DryadLINQ not the Dryad infrastructure and low level vertex APIs, which are written in C++. Just to be clear...

Useful links:

Connect site: http://connect.microsoft.com/site/sitehome.aspx?SiteID=891

ER Website on Academic Use: http://research.microsoft.com/en-us/collaboration/tools/dryad.aspx

MSR Info: http://research.microsoft.com/en-us/projects/dryadlinq/

Sebastian Burckhardt - Data Race Detection with CHESS

Peli de Halleux — Fri, 12 Jun 2009 18:31:00 GMT

Sebastian Burckhardt gives a short tutorial of some of the new features of CHESS: data race detection and ChessBoard. CHESS is a concurrency testing tool takes a concurrent unit test and executes it with different thread schedules. Sebastian explains us how CHESS can detect data races, a very subtle kind of concurrency bug. You'll also learn how to drill into concurrency issues using the ChessBoard, a little application designed to drill and investigate concurrent tests.

CHESS home page: http://research.microsoft.com/en-us/projects/chess/
CHESS forums: http://social.msdn.microsoft.com/Forums/en-us/chess/threads/

The Research in Software Engineering team (RiSE) coordinates Microsoft's research in Software Engineering in Redmond, USA.

MetraTech: Polymorphic Iterators, Concurrency, CRM and MetraFlow

Charles — Wed, 20 May 2009 17:45:00 GMT

Donald Trump recently praised Metratech’s business model embraced by their customer, ACN, who appeared on the Celebrity Apprentice.

Metratech’s own David Blair, Director of Product Architecture, discusses how Microsoft is both a partner and a customer of Metratech. David shares how they leverage the “Infobus Pattern” architecture and apply a micro-architecture approach with polymorphic iterators to provide dynamic CRM and other valuable tools. David gives a sneak preview of MetraFlow their product for addressing highly concurrent systems with a cluster enabled model. MetraTech Corp. offers an innovative approach to charging, billing, settlement and customer care that enables organizations to embrace change without compromise.

The flagship product, MetraNet, is deployed globally, enabling diverse customers to build innovative and successful business models, securely collect revenue, and manage their entire customer lifecycle in some 12 languages, 17 currencies and 52 countries.

Axum Published! Tutorial: Building your first Axum application

Charles — Thu, 07 May 2009 17:45:00 GMT

Not familiar with Axum? Here's a C9 interview with the Axum team to refresh your memory (it's a domain specific language for concurrent programming, formerly known as "Maestro", developed by the Parallel Computing Platform team).

Here, Axum PM Josh Phillips walks us through building a simple Axum application in just over 5 minutes. Josh builds a simple “math library” on agents and shows how easy it is with Axum to focus on your code and get parallelism and safety implicitly.

Axum Team Blog: http://blogs.msdn.com/maestroteam

-->Get Axum<--

Intelligent Light: Computational Fluid Dynamics and High Performance Computing

Charles — Wed, 08 Apr 2009 20:21:00 GMT

Before a plane flies for the first time, in the sky, it has flown many thousands of virtual miles in distributed clusters of computation cells, calculating non-linear differential equations of fluid dynamics.

Intelligent Light, with its Fortran and Python writing programmers, represents a typical ISV in the Microsoft HPC partner community with their flagship application having long been available on UNIX and Linux HPC clusters. Intelligent Light provides an application called FieldView that takes massive data from Computational Fluid Dynamics (CFD) applications and visualizes that data for engineers who design F16 fighters and Formula One cars. Because of the long compute times required, FieldView is often run in parallel on High Performance Computing (HPC) clusters to return quicker results. In this video, Intelligent Light founder Steve Legensky demonstrates the complex mathematics used by CFD engineers and talks about how HPC has evolved in his industry over the past 20 years.

Steve is awesome.

Check out the Microsoft ISV site for more information about ISVs working with Microsoft.

Concurrency and Parallelism: Native (C/C++) and Managed (.NET) Perspectives

Charles — Mon, 30 Mar 2009 17:24:00 GMT

Parallel Computing Platform team members Stephen Toub, Rick Molloy, Don McCrady and Dana Groff join me for a chat about the differences and similarities in their conceptual approach to designing and building concurrent programming abstractions targeting .NET developers and native (C/C++) developers.

Besides the obvious semantic (and runtime) differences between purely managed (.NET) and native code (C/C++), how does the Parallel Computing Platform team develop technologies for each domain and how are these technologies different? Surely, system level developers need system level tooling support that can improve their experience with writing native code that can effectively (and safely) scale to 8 cores (and that's nothing. How many cores will be the norm in 5-8 years? Mostly likely significantly more than 8....). There’s no PLINQ for C++, for example. That said, the fundamental problems the Parallel Computing Platform team are trying to solve span languages and runtimes, but are the differences only in implementation details and programming abstractions? What's the same? What's different? How? Why?

This is another great conversation with some of the folks designing and building technologies that will ultimately, in one form or another, converge into tools (or components of tools) that will help software developers effectively, efficiently and reliably compose applications and services in a Many Core world.

Enjoy!

Scott Guthrie: Inside Silverlight 3

Charles — Wed, 18 Mar 2009 19:12:00 GMT

Scott Guthrie, Corporate Vice President, leads the teams that create the .NET developer and designer technologies and tools. Silverlight 3 has arrived in beta form. Expression 3 also ships today in Preview form. Grab the bits and play. Provide feedback. OK. That's the housekeeping stuff. Now, on to this conversation with Scott. We talk about many things in this interview and weave the deeply technical view on Silverlight 3 with more surface level perspectives. So, if you're looking for bite size marketing messages, this video isn't for you :)

What are the significant changes/improvements/inventions in Silverlight 3? Silverlight 2 brought the world a cross platfrom CLR. How has the CLR changed in Silverlight 3? Silverlight 3 enables Silverlight to run outside the context of a browser. What does this mean, exactly? Silverlight and Many-Core? What has the significant real world feedback been like regarding Silverlight 2 and how has it helped shape Silverlight 3? How do you work with the Moonlight folks, exactly (see Miguel's perspective on this as well)? And much, much more. This is a typically long Charles interview and we travel around The Gu's globe of knowledge (which is vast - he's an engineer at heart).

Thank you, Scott, for spending so much of your valuable time for a Channel 9 interview that should prove most interesting to many folks who watch and listen.

Enjoy!

Expert to Expert: Erik Meijer and Anders Hejlsberg - The Future of C#

Charles — Tue, 03 Mar 2009 20:16:00 GMT

It's always a pleasure to get a chance to sit down and geek out with Anders Hejlsberg. Anders is a Microsoft Technical Fellow (a Technical Fellow is the highest ranking technical position at Microsoft) and programming language design master. He's the creator of C# and one of the founders of .NET. Anders is an expert language design craftsman. C# is one of the most popular languages Microsoft has created and certainly the most widely used language by developers who target the .NET platform.

Erik Meijer, Expert to Expert host, programming language designer and occasionally-radical category theoritician, has spent many years working with Anders and the C# team. As you may know, Erik has been a key contributor to the addition of functional constructs to C#.

Here, Erik and Anders wax on topics ranging from the design of C# 4.0's dynamic keyword (what's the thinking behind the thinking) to the potential near and far future of the C# language (and general purpose imperative programming, generally). Anders also spends some time at the whiteboard explaining C# 4.0's support for covariance and contravariance. Of course, we can't forget about concurrency and parallelism, so we don't.

As you might expect, the conversation takes some interesting jaunts into various programming language design rabbit holes. For example, Anders discusses the notion of creating a new language to support new problem domains versus extending current languages to meet the needs of developers who need to express solutions to complex problems (so, how do you make a language like C# more dynamic in the sense that it can readily help developers solve problems that the language was not initially designed to solve?). We talk about the work being done on a service-oriented C# compiler (compiler as a service), C# as an ESDL container (or as an EDSL itself to be hosted in other environments...) and much more. This is a fantastic conversation with some of Microsoft's true visionaries. Enjoy.

Expert to Expert: Inside Concurrent Basic (CB)

Charles — Wed, 25 Feb 2009 23:09:00 GMT

"Concurrent Basic extends Visual Basic with stylish asynchronous concurrency constructs derived from the join calculus. Our design advances earlier MSRC work on Polyphonic C#, Comega and the Joins Library. Unlike its C# based predecessors, CB adopts a simple event-like syntax familiar to VB programmers, allows one to declare generic concurrency abstractions and provides more natural support for inheritance. CB also offers open extensibility based on custom attributes."

Code Sample:

Module Buffer

Public Asynchronous Put(ByVal s As String)

Public Synchronous Take() As String

Private Function CaseTakeAndPut(ByVal s As String) As String When Take, Put

Return s

End Function

End Module

OK. Sounds great. There are new keywords, Asynchronous and Synchronous. Conceptually, these are easy enough to understand. How do they work, exactly? What's the thinking behind the current design? Why was VB.NET chosen as the language to extend? Wouldn't any CLI language suffice? Who thought this up, anyway? What's the thinking behind the thinking?

Enter C9 celebrity host Erik Meijer, who leads yet another great conversation with fellow software experts Claudio Russo (MSR Researcher and co-creator of Concurrent Basic) and Lucian Wischik (software developer and current VB.NET Czar).

Obviously, we've been focusing a lot of attention of Concurrency and Parallelism over the past few years. We talk about the library versus language approach quite a bit. In this case, concurrency constructs have been baked into the language to form a different variant of VB, CB (Concurrent Basic). CB is a research project and therefore a research language. It has no ship vehicle and is not available for trial at this point. Microsoft makes no committment to shipping VB with these concurrency constructs built in. CB is a research language. CB is being shown in action at this year's MSR TechFest. Be sure to check out Laura's coverage of TechFest 2009.

geekSpeak recording - The PowerThreading Library for Silverlight with Jeffrey Richter

Brian Johnson — Fri, 20 Feb 2009 10:47:00 GMT

In this geekSpeak webcast, Jeffrey Richter talks about his AsyncEnumerator class that allows programmers to use a simple synchronous programming model to perform asynchronous operations, thereby writing highly scalable and responsive applications and components. The class is useful for console applications, Windows Forms, Windows Presentation Foundation (WPF), Web Forms, Web services, SQL, Microsoft Silverlight, and more.

The geekSpeak webcast series brings you industry experts in a "talk-radio" format hosted by developer evangelists from Microsoft. These experts share their knowledge and experience about a particular developer technology and are ready to answer your questions in real time during the webcast. Your hosts for this geekSpeak are Glen Gordon and G. Andrew Duthie. To ask a question in advance of the live webcast, or for post-show resources, be sure to visit the geekSpeak blog.

Guest Presenter: Jeffrey Richter, Cofounder, Wintellect

Jeffrey Richter is a cofounder of Wintellect, a training, debugging, and consulting firm dedicated to helping companies build better software, faster. He has written several best-selling programming books about Microsoft .NET and Microsoft Win32, including CLR via C# (Microsoft Press, 2006). Jeffrey is also a contributing editor to MSDN Magazine where he authors the "Concurrent Affairs" column.

Expert to Expert - Joe Duffy: Perspectives on Concurrent Programming and Parallelism

Charles — Tue, 17 Feb 2009 17:39:00 GMT

Joe Duffy spends a lot of time thinking about the future of concurrent programming and parallelism. In his role as Lead Developer in the Parallel Computing Platform team, Joe is the creator of PLINQ and a key contributor to many of the managed (.NET) concurrency incubations happening in and around his broader team. He's also an author (check out his latest book, Concurrent Programming on Windows)

You've met Joe many times before on C9 and the concurrency topic should be quite familiar to you by now (There's a lot of very innovative thinking going on in the parallel computing platform team (and it's not just about the managed world, as you know)).

We've spent a lot time discussing library-based approaches to enabling parallelism in a readily understanable, predictable, safe and scalable way for .NET programmers. We've also spent time on language level approaches to the problem (new constructs in C# that make it easier to compose in a semi-functional way (lamdas, LINQ, etc) or purely in a hybrid-functional way in F# or with experimental DSLs like Maestro).

Erik Meijer, Expert to Expert host, programming language designer and one of the high priests of the lamda calculus spends a great deal of time thinking about the problem of software's capability to scale effectively (as efficiently, safely, and as composable as possible) in the Many-Core age. So, we add Joe + Erik and we get many excellent, insightful questions and answers. Of course the notion of side-effects plays a big role here and we even debate the merits of Haskell in the real world. This is a great conversation. It goes deep, but not so far into the rabbit hole that you won't be able to find your way back. :)

Enjoy!

Dave Probert: Inside Windows 7 - User Mode Scheduler (UMS)

Charles — Mon, 02 Feb 2009 20:11:00 GMT

Here, we continue our exploration of the morphology of Windows 7 on Going Deep with windows kernel architect Dave Probert. You may remember him from an early four part episode of Going Deep where he teaches us about general purpose operating system architectures and history: Part 1, Part 2, Part 3, Part 4

That was a great conversation from a few years ago and it's been way too long since we returned to Windows kernel world to converse with and learn from Dr. Probert. Not surprisingly, Dave has been busy innovating the Windows core.

Dave and team, working very closely with the Parallel Computing Platform People, have created a very compelling new user mode thread scheduling/management system in Windows 7. In a nutshell, the User Mode Scheduler provides a new model for high-performance applications to control the execution of threads by allowing applications to schedule, throttle and control the overhead due to blocking system calls. In other words, applications can switch user threads completely in user mode without going through the kernel level scheduler. This frees up the kernel thread scheduler from having to block unnecessarily, which is a very good thing as we move into the age of Many-Core... Speaking of Many-Core, remember the piece we did on the Concurrency Runtime (ConcRT)? ConcRT is built on top of UMS and is the best way to most effectively utilize this new user mode thread scheduling model in Windows 7.

Make yourself comfortable and spend some time watching and listening to Dave make all of this crystal clear.

This is another great conversation with a fantastic OS architect and Windows kernel professor. Lots to learn here. Enjoy.

Expert to Expert: Meijer and Chrysanthakopoulos - Concurrency, Coordination and the CCR

Charles — Fri, 16 Jan 2009 20:56:00 GMT

In this episode of Expert to Expert, programming language designer Erik Meijer chats with CCR creator George Chrysanthakopoulos. We've spent a good deal of time on Channel 9 addressing the Concurrency Problem and the various approaches Microsoft is taking in an effort to help solve it. George's CCR is a piece of managed technology (.NET) that provides an unusually high degree of concurrency for developers targeting Windows. The Coordination and Concurrency Runtime has been around for about five years. How are people using it today to build scalable concurrent systems? What's the current state of the CCR and what's it's future? Why is the CCR a better approach to scalable distributed concurrent programming than other technologies out there? Is concurrency the real issue? George believes that it's all about coordination(the other C in CCR) and concurrency is really just a side effect of coordinating systems. If you get distributed coordination right, then you have a concurrent system that can scale. Really? Do explain, dear George (oh, and he does and as passionately as you'd expect from him). This is a fantastic conversation. Classic Channel 9.

Enjoy!

Madan Musuvathi - Getting started with CHESS in Visual Studio 2008

Peli de Halleux — Wed, 14 Jan 2009 18:40:00 GMT

Today we are sitting with Madan Musuvathi for a quick tutorial on writing concurrency unit tests, powered by CHESS in Visual Studio 2008. Madan goes through a simple bank account example that contains a subtle concurrency bug. He shows how to turn a unit test into a concurrent unit test in a snap, find concurrency issues and debug them using CHESS.

CHESS is an automated concurrency testing tool for .NET and Win32 that finds and reproduces heisenbugs.

Ask questions in the forum

The Research in Software Engineering team (RiSE) coordinates Microsoft's research in Software Engineering in Redmond, USA.

Parallel Computing in Native Code: New Trends and Old Friends

Charles — Mon, 12 Jan 2009 19:48:00 GMT

We've covered a lot of ground on both C++ and Parallel Computing on Channel 9 over the past few years. For C++ in particular, we've gone deep on many fronts with some of the main players in Microsoft's native programming world. Damien Watkins is one of these players and he's the brains behind most of the interviews you've seen on C9 (he thought them up and set them up). But who is Damien and what does he do?

Rick Molloy (PM) and Don McCrady(Development Lead) have been on Channel 9 before and they are both members of the native side of the parallel computing platform (PCP) house. It's no surprise that most teams who ship Microsoft software work closely with the C++ team given that most of our products are written in native code. The C++ team produces the de facto compiler that most teams at MS use. The PCP team is no exception.

We figured it would be fun to get a C++ player (Damien is a PM on the front-end native compiler team) and some Parallel People together in a room to discuss the native side of the Concurrency Problem (and possible solutions) and get a feel for the synergy between teams. The next version of C++, C++0x, will undoubtedly contain new language constructs that will make it easier to program many-core algorithms. We dig into some of these here as well as reveal for the first time on C9 some new members of the C++ language that you may not have heard about yet....

Enjoy. This is a great conversation among key thinkers who live in and innovate the native world.

Software Transactional Memory: The Current State of the Art

Charles — Mon, 29 Dec 2008 20:19:00 GMT

A few years ago I got the chance to learn about Software Transactional Memory for the first time while visiting MSR Cambridge. The great Simon Peyton-Jones and Tim Harris explained to me the thinking behind STM and how it might evolve. It was a tremendously interesting conversation. If you haven't watched that interview, I highly recommend it as a precursor to this one. Today, STM is no longer only a research project. The Parallel Computing Platform team is incubating and extending the technology, finding that it may in fact work in the real world...

Of course, there is no silver bullet to solving the Concurrency Problem, but STM may be an important part of a larger solution (you've leraned a great deal about what Microsoft is up to in the concurrency and parallelism space here on Channel 9 and it should be somewhat clear by now that many of the technologies we've presented to you may end up as pieces of a broader solution...)

Here, STM Program Manager Dana Groff and STM Principal Developer Lead Yossi Levanoni discuss the current state of STM and outline the work their team is doing to craft this incubation/research technology into a practical real-world solution (STM is not available yet for experimentation. It's in incubation. It's not known if or when STM will become a viable product.). So, how has STM evolved over the past two years, anyway? Tune in.

Enjoy.

Maestro: A Managed Domain Specific Language For Concurrent Programming

Charles — Mon, 22 Dec 2008 19:46:00 GMT

Josh Phillips(PM), Niklas Gustafsson(Architect), and Artur Laksberg(Developer) of the Parallel Computing Platform Team spend some time with me to discuss a managed (.NET-based) DSL (Domain Specific Language) for concurrent programming, Maestro. Maestro incorporates well-entrenched language patterns (imperative, OO, C style syntax, etc) and language constructs (channels, agents, domains) in a compelling way to make concurrent composition more accessible and familiar to the legions of sequential code composers.

Here we dig into the architecture and design of the Maestro language and compiler as well as discuss the philosophy behind this incubation project (at this point in time there are no plans to release Maestro as a product - it's a research project, an incubation...). Why create another language to help solve the Concurrency Problem? What's the advantage over implementing a library (this is .NET after all -> CLR + BCL = most of the power of the platform)? There's obviously good reasons for implementig Maestro as a language, but you'll need to watch and listen to find out.

Enjoy.

Note: After this interview was conducted and posted to Channel 9, the Maestro team has renamed their technology to Axum. So, they are now the Axum team and the managed DSL for concurrent programming they're incubating is called Axum. :)

CHESS: An Automated Concurrency Testing Tool

Charles — Wed, 10 Dec 2008 21:46:00 GMT

CHESS is an automated tool from Microsoft Research for finding errors in multithreaded software by systematic exploration of thread schedules. It finds errors, such as data-races, deadlocks, hangs, and data-corruption induced access violations, that are extremely hard to find with current testing tools. Once CHESS locates an error, it provides a fully repeatable execution of the program leading to the error, thus greatly aiding the debugging process. In addition, CHESS provides a valuable and novel notion of test coverage suitable for multithreaded programs. CHESS can use existing concurrent test cases and is therefore easy to deploy. Both developers and testers should find CHESS useful. The CHESS architecture is described in this technical report.

Here, we meet some of the researchers behind CHESS, Madan Musuvathi and Shaz Qadeer. Joining in the conversation are two software test engineers extraordinare, Chris Dern and Rahul Patil. Chris and Rahul use CHESS as part of their daily routine of finding bugs in the various technologies that power Microsoft's Parallel Computing Platform. Tune in and learn about this great technology from the folks who know it best.

Brian Beckman: The Zen of Stateless State - The State Monad - Part 2

Charles — Thu, 20 Nov 2008 21:38:00 GMT

Concurrency is a problem that faces all developers as we move to the age of ManyCore processor architectures. Managing state is an important aspect of programming generally and for parallel programming especially. The great Brian Beckman demonstrates three ways of labeling a binary tree with unique integer node numbers: (1) by hand, (2) non-monadically, but functionally, by threading an updating counter state variable through function arguments, and (3) monadically, by using a partially generalized state-monad implementation to handle the threading via composition. Of course during this lesson from one of the masters of mathematical programming, we wind through various conversational contexts, but always stay true to the default topic in a stateful monadic way (watch/listen to this piece to understand what this actually means :))

This is another great conversation with astrophysicist and programming master Brian Beckman. Brian is one of the true human treasures of Microsoft. If you don't get mondas, this is a great primer. Even if you don't care about monadic data types, this is worth your time, especially if you write code for a living. This is part 2 of a 2 part series.

See part 1 here.

Below, you will find several exercises for generalizing the constructions further. Here are the source files you need for playing with these algorithms in visual studio or your favorite Haskell environment. Brian will monitor this thread so start your coding engines!!

Exercise 1: generalize over the type of the state, from int
to <S>, say, so that the SM type can handle any kind of
state object. Start with Scp<T> --> Scp<S, T>, from
"label-content pair" to "state-content pair".

Exercise 2: go from labeling a tree to doing a constrained
container computation, as in WPF. Give everything a
bounding box, and size subtrees to fit inside their
parents, recursively.

Exercise 3: promote @return and @bind into an abstract
class "M" and make "SM" a subclass of that.

Exercise 4 (HARD): go from binary tree to n-ary tree.

Exercise 5: Abstract from n-ary tree to IEnumerable; do
everything in LINQ! (Hint: SelectMany).

Exercise 6: Go look up monadic parser combinators and
implement an elegant parser library on top of your new
state monad in LINQ.

Exercise 7: Verify the Monad laws, either abstractly
(pencil and paper), or mechnically, via a program, for the
state monad.

Exercise 8: Design an interface for the operators @return
and @bind and rewrite the state monad so that it implements
this interface. See if you can enforce the monad laws
(associativity of @bind, left identity of @return, right
identity of @return) in the interface implementation.

Exercise 9: Look up the List Monad and implement it so that it implements the same interface.

Exercise 10: deconstruct this entire example by using
destructive updates (assignment) in a discipline way that
treats the entire CLR and heap memory as an "ambient
monad." Identify the @return and @bind operators in this
monad, implement them explicitly both as virtual methods
and as interface methods.