Andy Ayers: Understanding the Phoenix Compiler Framework
- Posted: Apr 09, 2008 at 12:19 PM
- 24,604 Views
- 19 Comments
Download
How do I download the videos?
- To download, right click the file type you would like and pick “Save target as…” or “Save link as…”
Why should I download videos from Channel9?
- It's an easy way to save the videos you like locally.
- You can save the videos in order to watch them offline.
- If all you want is to hear the audio, you can download the MP3!
Which version should I choose?
- If you want to view the video on your PC, Xbox or Media Center, download the High Quality WMV file (this is the highest quality version we have available).
- If you'd like a lower bitrate version, to reduce the download time or cost, then choose the Medium Quality WMV file.
- If you have a Zune, WP7, iPhone, iPad, or iPod device, choose the low or medium MP4 file.
- If you just want to hear the audio of the video, choose the MP3 file.
Right click “Save as…”
- Mid Quality WMV (Lo-band, Mobile)
- MP3 (Audio only)
- WMV (WMV Video)
The compiler gurus over in C++ World (we spend a lot of time in building 41) have just released the latest
CTP of the Phoenix Compiler Framework. It's been a while since we've learned about Phoenix here on Channel 9 so we thought it necessary to find out what's going on with the soon-to-be de facto compiler technology
for all of Microsoft's products.
Andy Ayers is an Architect on the Phoenix team. He sits down with us to dig into the details of Phoenix and address it's current state, history and future (no whiteboarding here, but this is still quite deep. There is a cool demo at the end of the interview.)
. It's very interesting to note that the Phoenix team has tested the scalability and power of their pluggable (extensible) compiler framework on the likes of Windows. Turns out Phoenix handles such a large task (compiling Windows) very well. This new compiler
technology has been designed with many-core in mind. This means Phoenix is our most parallelized compiler which adds up to shorter compilation time. As usual, the C++ team is ahead of the curve when it comes to preparing for the concurrent future.
Tune in and learn all about the future of Microsoft's compiler technology from one of the minds behind it all. The future is very bright indeed!
Enjoy!
Low res version here.
Comments Closed
Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums,
or
Contact Us and let us know.
Follow the Discussion
Sadly, I'm not really in the C++/native word, but in the Java / Managed World.
I like that too. And ability to Step backwards and reverse the state would be so cool.
Look forward to new compiler in the product. Thanks much folks.
BTW - small correction. The managed class is "StringBuilder" (not stringbuffer).
One of the cool things about Phoenix is that it blurs the line between managed and native code in a number of interesting ways.
So, if you're into managed code, you'll be happy to hear that the dynamic slice tool works on managed code too.
Reverse debugging is indeed really cool, but there's a sizeable gap between what we can do with slicing and actually being able to run the program backwards. Still, the idea is compelling...
We should be dislosing release plans sometime in the not-too-distant future. But you don't have to wait until then to have fun with Phoenix -- download the SDK and you'll get a version of Phoenix that plugs right into the VS2008 C++ toolchain.
Thanks for the correction.
Binary translation is certainly doable, and the machine-specific part of Phoenix is extensible (though this is not yet as easy to do as we would like it to be). Sounds like a fun project for somebody to try...
And it would be great with more static IL optimizations. On the other hand, in the parallel world of the future, there should be enough cores to continuously GC, profile and analyze code, so one wonders how much Phoenix can adapt to dynamic compilation and analysis demands.
Anyway, cool stuff.
Thanks Andy. I was hoping that was true. Phoenix may be a better way to "plug" in things like Volta in the compile pipeline and it reads the attributes and does its thing. I think that would allow a base compiler with N number of plug-ins. Disable the plug-in and the attributes are ignored and the program compiles without volta majic.
Wonder about things like Services plugins. Write a standard console app, and enable a Service plugin and it will atomatically "wrap" the code in a service or wcf service or silverlight service, or MS Sql hosting service, or VM package, or Tee'd off setup package, or Powershell-to-c# converstion (or visa-versa), etc. I like that c# to Sql native idea. I smell some dynamic deployed Linq sprocs coming on - so I could deploy c# lamdas as native code methods and hang em off SQL to be called like simple (remote) functions (ala volta like). Some interesting ground to travel.
I've always wanted to do an IL-to-IL optimizer. While the jit does a good job it doesn't have time for in-depth analysis, and there are a number of things you can do upstream to boost performance.
As far as what all those cores will be doing -- I expect we will find good ways to employ them to directly address user problems. Phoenix itself can profitably use 6-8 cores, and with a bit more work we should be able to scale even higher. It may be that the world of code is more dynamic in the future, but I thought that 10 years ago when I worked on a big static compiler and things haven't really changed that much.
And that's what all optimization freaks are craving for. And to get conceptual simplicity whilst not sacrificing performance too much [declarative programmer, imperative compiler].
I haven't studied IL and IR, but would the IR actually be a better IL? It sounds like it could be.
If IL is at a lower level than IR, and if an IL-IL optimizer has to maybe abstract up to IR, then wouldn't it be better to just stay with IR - depending on the effort required to go IR->IL.
I wonder if the TCPA could be used to secure highly optimized snapshots of compiled code [in encrypted files] so the JITr could effectively be relieved of a lot of up-front work. Of course there's NGen which might do some optimizations up-front.
Sounds great.
I didn't mean in the sense of dynamic languages (necessarily), more in the sense of JIT'd bytecode.
MSIL/CIL/IL is the way it is for a couple of reasons -- it has a compact encoding, it is relatively straightforward to translate to machine code, and its semantics are carefully specified so that verification is possble. Phoenix IR has a much different set of design parameters and so has different attributes: a representation that is somewhat redundant but can represent rich relationships among the instructions in a program, the flexibilty to represent programs at several different semantic levels (eg HIR, LIR), and the expressive power to describe most of the popular machine architectures.
A bytecode that has some of the attributes of our IR makes a lot of sense -- the ability to specify a register set, the ability to annotate the IR with useful derived facts (perhaps, as you note, provably correct ones), the ability to mix semantic levels (as Phoenix IR allows LIR/HIR islands in HIR/LIR). If you can have all that and retain the benefits of MSIL then it would be something really interesting.
And I'm with you on that last comment -- around 1998 or so I was convinced jitted code was going to take over the world.
Develop marketplace for embarking parallel computing in real business scenes, build parallel process enabling compiler business likewise Microsoft sold Visual Basic for computer beginner’s to PC worldwide. Catchy Microsoft business message might be "eco friendly parallel computing starts from Microsoft platforms compiler technology"
Well, the initial perception about Phoenix was that it's going to be a common backend for both register based and stack based frontends, which, once plugged into different frontends, supposed to allow both managed and native targets for the entire language family under the generous umbrella of MSVC linker. few practical examples:
1. Generate 100% native (register based) code from c# and vb (finally allow software in a box / commercial software manufacturers to use languages beyond c++) - even allow the .net framework itself to have a native (register based) incarnation.
2. Generate hybrid application from any language (in the family) - which would allow natural evolution of existing native and managed application, for any languages in the family (today only MSVC++ allow that)
3. Allow 3rd party compiler manufacturers to expose their frontends results to a "standard" backend, with clear dual intent, without having to worry about backends, linkers, etc
Now, I do happen to know that this is not a walk in the park. it never was, that's why Phoenix was born in MS Research, but still, I see obvious (long term strategic) advantages in implementing the initial plan, the first beneficiaries being:
1. The Microsoft compiler teams
2. 3rd party compiler manufacturers (Borland, RemObjects, etc)
3. At last but not at least, the Customers, compiler end-users, developers all over.
I can only hope that static analysis framework is only a first step into this direction and that Visual Studio 2010 will include some of these ideals, empowering the Microsoft Platform and Developer Tools experiences even more!
Best Regards,
Daniel
Hi there,
I would like to know about your initiatives (if any) of sharing Windows Kernel source code and why not, other subsystems (.net framework, managed and native compilers, office itself, sql server).
What can be done, if I, a Microsoft dedicated developer, wanted to build all these products in-house, of course, for cognitive purpose ONLY, under STRICT NDA. Is there a chance to use your source code, obviously in good/mutual faith, from outside Redmond? I know that some of your kernel guys work remote, but they are full time Microsoft employees. What do you do to cover the (relatively small, yet important) segment of customers who have enough knowledge and skills to benefit directly from your source code, obviously beyond the regular benefits coming from consuming binaries. Do you have (strictly NDA-ed) private repositories where developers can go, be able to branch, check-out, build, understand, extend and check back in Microsoft core source code. Same question about special access to your bug tracking systems. Same question about white (or at least gray) box testing your products, extending your unit testing, automation, propose fixes, etc - my question targets kernel and compiler engineers in first place.
Do you have a program which states the precise conditions an individual Software Engineer or an Organization have to meet in order to be allowed to see, build from, extend the source code of Microsoft core technologies?
I'd really appreciate your feedback, either way. Thanks!
Regards,
Daniel
I'm glad to see that some of our vision resonates with you. There's a lot I would like to say about where we are headed and what might be possible in future product releases, but I'm going to have to leave things up in the air for now. Let's just say that there are a lot of cool things that can be done -- which ones of those happen and when is still being sorted out.
For educational purposes you can download the Windows Research Kernel which gives you kernel source for a good part of the Server 2003 kernel. You can build this with Phoenix if you want to experiment with OS/Compiler interactions (though if you do so in the near future, I recommend using the VS2005 based June 2007 SDK, since the newer April 2008 SDK exposes some source issues in the WRK code). And the SSCLI will give you a pretty clear idea of what a good part of the .Net 2.0 code looks like.
As far as building Phoenix yourself, or building other microsoft products, from what I know it's really up to each product group to decided if/when/how to provide access to source. I don't know if there is an umbrella program that applies to the entire company but I suspect there isn't any such thing.
As is typical with such things there are many factors and stakeholders involved. For example, we have thought about including portions of the compiler test suite with the SDK but a goodly number of test cases have been licensed from third parties or are based on code given to us with other sorts of restrictions. Just sorting out what we can give out is a major effort.
Remove this comment
Remove this thread
close