ARCast.net - Non-Code Artifacts Approach Zero with Scott Hanselman | ARCast with Ron Jacobs

rojacobs
Ron Jacobs

Johnny:	It's Thursday December 21st and ho, ho, ho, you're listening to ARCast. [music]
Ron Jacobs:	Ah, yes you are, and thank you Johnny, see I told you that holiday season thing would work, you know [chuckles], it's another great day on ARCast as we approach the holiday, Christmas holiday here in the United States and whatever holiday you like to celebrate, I hope that you're having a good one, I just got up early this morning and checked my email, found a nice gift certificate from my boss in the email, so you know, I went online and ordered me a nice new computer game to play over the holidays, I'm going to have a lot of fun with that, hope you're enjoying your time with your friends and family and maybe squeezing in an ARCast because today we have another pod caster, yeah, that's right, today we have Scott Hanselman, who is on the show from Barcelona where we did a pre-conference session together called "Introduction to Software Architecture" so let's go back to Barcelona, and welcome.
Johnny:	Scott Hanselman! [clapping]
Ron:	Welcome back to ARCast, everybody, this is Ron Jacobs coming to you from Tech Ed Barcelona...
Scott Hanselman:	Barcelona!
Ron:	[in Spanish accent] Barcelona! I would say it like the locals would say it.
Scott:	Si.
Ron:	Yeah. [laughs] And you're listening to the voice of fellow pod caster Scott Hanselman, welcome, Scott.
Scott:	Hola. [Ron laughs]
Ron:	And your show is called "Hansel minutes"
Scott:	Yes, Hansel minutes.
Ron:	At Hanselminutes.com.
Scott:	Yeah, the name was actually, a guy at my work named Travis Illig came up with this name because I'm kind of notorious for giving estimates that aren't very accurate... [Ron laughs]
Scott:	So I would say, "oh, yeah, totally, I'll do that in ten minutes"...
Ron:	Yeah.
Scott:	...and after a while, you know, when ten minutes turns into two weeks, Travis said "OK, well, wait a second, when you give an estimate is it in minutes or is it in Hansel minutes?"
Ron:	Oh, there you go.
Scott:	So that is where that, and that actually turned into another Hansel metric which was the number of Hansel lines of code... [Ron laughs]
Scott:	So Travis set that up for me and then Carl Franklin's like, "Oh, you've got a show right there" so we kind of keep the show every week down to 20 minutes.
Ron:	Yeah.
Scott:	Twenty Hansel minutes. [Ron laughs]
Scott:	And we were actually the only pod caster that encourages people to listen to it in double speed so they won't waste their time.
Ron:	OK, I was going to say, I thought it was longer than that. [laughs]
Scott:	Sometimes we overflow but if you squish it, if you actually listen to it in double speed, it will always fit into 20minutes.
Ron:	Oh, very good.
Scott:	Even the longest shows run 20 minutes.
Ron:	I like that, OK, excellent, well, so I'm delighted that you're here because you're here to help me out for the pre-conference session that we did together on Monday, "The Introduction to Software Architecture", and pre-conference sessions are always hard because they're so long, you know, it's five hours of standing up in front of a crowd and speaking if you do it by yourself, so I was happy when you agreed to come and do half of it so I didn't have to stand up there the whole time because I get really tired doing that.
Scott:	Yeah, it was pretty hard work and I had to put a lot of work into thinking about all of the different American colloquiums I would use and different American words and phrases that maybe wouldn't translate perfectly so I think about 15, 20% of the talk was me explaining exactly what I was trying to say...
Ron:	Yeah.
Scott:	...when I said whatever English colloquium didn't work...
Ron:	Right.
Scott:	...so it was very interesting, I felt the scores indicated that people felt they got their money's worth, you know, again, this is the thing with me, I'm kind of neurotic about not wanting to waste people's time...
Ron:	Yeah, yeah.
Scott:	...so when you're going to sit up there for five hours, it sounds like a long time, but at the same time, you want it to be five valuable hours because it's five hours that they're not at work.
Ron:	Right.
Scott:	Right, so what can we do to pack the maximum amount of information into that five hour period?
Ron:	Yes, so one of the things that I, that you said during your talk that I thought was fantastic was you talked about how you want the number of non-code artifacts to approach zero. Could you tell me a little bit more about that?
Scott:	Yeah, and as much as I would like to take credit for that, but I cannot, that is, I'm not exactly sure who the original person was but the person who told me that was one of our architects at Carillion, the place where I work, we do ebanking software, his name was Bryan Wyntime, and he feels very strongly and has kind of been indoctrinating the rest of us and it's something that I felt strongly about but haven't put it in such an eloquent way as that, and Bryan says that "the number of non-software artifacts should approach zero" and it's kind of a throw away statement but if you really drink it in and realize that what he's saying is that, like what I used to say years ago, the award document has no teeth, writing prose is great, but are those keystrokes moving us closer to the goal. So we're working on a project right now that is all about automating the deployment and management of our platform and Bryan is heading up that project and he keeps pushing us every time we want to create a document, he's saying, "Well how can I make that something declarative? How can I make that code, if there's a requirement that a business person wants?" Well, like we talked about in the pre-conference, the home page should load in less than two seconds, you can write that in a word document all you want but when that doesn't happen, it's not the word document that's going to jump up and say, "Hey, the home page didn't load in time, stop, agghhh". That doesn't happen, right, so we need to have that bit of information, that expression of programmer intent, rather than being expressed in terms of prose, be expressed in code. So then if we express it in code or an XML file or something that is at least readable by the business person then we don't need the word document...
Ron:	Right, right.
Scott:	So we took what was a many, many tens of, I think it was upwards of 70 pages word document, distilled it down into an XML file and then we can turn around and generate the word document later if it makes the business person happy, but express everything that you want to do, and then we look at things like In spec, even though right now we're using everything in that unit, tools like In spec to better describe the specification.
Ron:	OK.
Scott:	It's all about expressing intent, right, this is kind of a bigger topic but when thinking about selecting a programming language or environment, what best expresses our intent in a way that is helpful for the business person and helpful for the computer?
Ron:	Well, you know what I love about is that there are so many times when people have these big word documents and they say stuff and then it's their intent, they fully intend to do it and they wrote it all down and everybody agreed to it and then it goes into the source control and no one ever looks at it again until months later when someone realizes we didn't do any of those things or most of them we didn't do.
Scott:	Word documents are fundamentally flawed and very, very sad things. [Ron laughs]
Scott:	May the word document provide you comfort when the application is crashing around you and the suit is waving the word document and complaining about how some specification or some crucial line he thought was important that was not expressed in code. Bryan, on the team at Carillion, really feels very strongly about asserting every assumption you possibly can, this isn't just something that is a way to write tests or a way to write specifications but it's kind of a philosophy of developing software. If there's any assumption, assert it. A guy who I used to work with at Carillon who's no longer there, would always jump on us about parameters that come into our methods and would he say, "Could that ever be no?". Well, yeah, it could be no. "If it ever were, would that be a valid state?" Well, no. Well, assert that. There's so much implicit things going on, implicit parameters right, when you have some global variable or some variable with wider than local scope that needs to be a certain way. This happens a lot in ASP.net applications because people use the query string or the forms or the cookie or HTP.context.current.items -- these are all implicit parameters. Someone takes a perfectly harmless method like "do it," takes no parameters, maybe does two or three things and then exits and then they look like the climatic complexity of that particular method, they look at that and it says, "Oh it has a complexity of one" but it accesses 14 internal intrinsic variables and makes a lot of assumptions, that the system will be in a certain state, so the idea is to set the system up such that the only state that is valid is the successful state.
Ron:	You know, that reminds me of one of my early mentors talking about me, he said "grasshopper, always assert your assumptions".
Scott:	He started out with that, he actually said "grasshopper"?
Ron:	Yes, yes, he did. [laughs] Well, you know, in fact when I think about, it's appropriate that the guy that's pushing you guys on this is involved in deployment because deployment is an area that is often really under served, shall I say, or overlooked maybe is a better way. I had to work on a project once where I got the short straw so I had to do the deployment. You know, it's the job that nobody wants. I had to start analyzing what exactly has to go in this deployment. This team was not very good about tracking this. So, it turned out that I had to search through code to find everything, every little assumption they were making about a file will be in this certain location, a registry key will be set, and all of these things. We had to kind of reverse engineer it. Eventually, though, what I ended up doing, was writing my own - this was long in the days before things like EndUnit - so I had to write my own little test harness though to basically do the install and run a whole series of tests on the install environment.
Scott:	Yeah, you wrote check disk for your install environment.
Ron:	I did, yes.
Scott:	You know, a sanity checker.
Ron:	But you know what, we could never get it right until we did that because it was just too hard. There was too many things to manually check.
Scott:	Yeah. And that's one of the things that's fundamentally missing in Windows. I think if we watch for this in the next few years we are going to see that there are people deep in the bowels of the beast at Microsoft who are thinking about these problems. Our cries are not falling on deaf ears. I know for a fact that they are working on some stuff to make this kind of thing easier. I don't know what that will look like. I know it won't look like Application Center. I know it won't look like Systems Management Server but it will be something that will be able to say, "is this system healthy." Now this guy Brian one time on the team at Corillian that has been thinking about this, has an interesting way to deploy our application. We have a large application and in its most-installed form can take, you know, a dozen MSI installers to configure. It is kind of a hassle. It is a great run-time environment. It's a fantastic development-time environment. It just happens to have a mediocre install-time and we are trying to make it so much easier. It takes about a day to a day and a half to install. It's kind of like Exchange, right; you don't just bang out an Exchange install in 15 minutes. It takes thought. So, one of the things that we are doing, which is a little unconventional, is we'll actually take an installed, completely configured system - not like a virtual disk, but just the folder, the Program Files directory - and we'll check that into Subversion. So we basically freeze-dry the system. Check that into Subversion and label it; call that "default". And then all of the registry changes, decomp configuration, ackles, database, all that information, all that kind of implicit parameters, all that extra out-of-band information is then written up in PowerShell script.
Ron:	Oh, OK.
Scott:	So then I can take a brand new, fresh, naked machine; do a checkout; just say, "I want that one"; run the post install script which sets all of the different registry keys and what not; and fire it up. And we've taken what was once about a one day, sometimes two day, install and taken it down to 15 minutes.
Ron:	See I like that. That's taking a fresh look at the problem. I like that approach.
Scott:	Yeah, I had had the idea initially to check this thing in on the web side because I am an ASP.net guy. I really understand how ASP.net works. It seemed to me like if you wanted to do a large distributed deploy, because this is interesting but it is not fundamentally interesting on a single box install. But, when you are working with a bank that might have 25 web servers that all need to look the same, you can do MSI installers and silent installs and Systems Management Server and stuff. But what I really wanted to be able to do was say make those 25 boxes look the same. I didn't want to use cloning or changing the SIDs and all that kind of stuff. I've got perfectly good boxes that are all set up. I just want to put a web site on that box. I could have used DFS, distributed file system, that would have gotten the files over there but it wouldn't have got all the changes that were there. What Brian allowed us to do was, he said, well why don't we just check in the whole damn thing. That's where I said I don't think that is going to be possible. Of course he proved it two hours later that it was totally possible. And it works great. What we ended up doing was using Power Shell. One of the things that is not included in Power Shell 1.0 was a remoting interface-the ability to invoke Power Shell commands on a remote machine.
Ron:	Oh, right.
Scott:	Power Shell does include the concept of a run space, where you can take the Power Shell pipeline, the list of commands, and you can hold it up in process. It's an engine; it's an execution engine. So we wrote a service, and we put Power Shell instances in pools and services on these different remote machines.
Ron:	Ah...
Scott:	So you bring up the naked machine, you have that one service running on the machine, and then you say, "Power Shell script lock, " which usually consists of "left curly brace, two plus two, right curly brace" and hit enter. That would do two plus two, right?
Ron:	Yeah.
Scott:	So we go "left curly brace, two plus two, right curly brace, dot remote invoke, quote machine name" and then that work happens remotely. So then we use some background jobs processing code that's very similar to the Unix background job processing to do distributed asynchronous work. We basically said "On these 25 machines, go off and set up that website."
Ron:	Yeah.
Scott:	So we fan out using Power Shell, each of those machines does it get from subversion, does the set up check to see if things are healthy and then reports back.
Ron:	Cool.
Scott:	So we can go and set up a web farm in an hour versus in a week.
Ron:	That is great. Yeah.
Scott:	Yeah.
Ron:	Power Shell is pretty amazing, and a lot of people don't know about it.
Scott:	The thing about Power Shell is: how do you explain it in an elevator speech?
Ron:	Yeah.
Scott:	Like, a guy gets in the elevator with you and you have five floors to explain to him why Power Shell is the greatest thing since God talked to Moses. It's kind of a difficult thing to do.
Ron:	Yeah.
Scott:	It literally puts Windows on a string. Wmi, com, adsi,.net-it is a scripting language on top of all of that stuff. It's not about the Shell, it's not about the command line. It's about that run space, that Power Shell pipeline.
Ron:	Yeah.
Scott:	The fact that you can hold it and host yourself...you can script enable any application in five or six lines of code.
Ron:	Wow.
Scott:	Not just Hansel lines of code, but actual lines of code. There's a really great demo where a guy wrote... do you remember "Logo?" When we were kids there was a language they would let us play with on the Apples, Apple one or Apple 2, where it was called "Logo." You would say, "turtle-dot-right."
Ron:	Well, I was actually a teenager then, but it's OK. [laughs]
Scott:	I think I was eight. OK, so there was this thing called "Logo." Basically, you would draw things like fractal patterns, nano bots, you would learn about forward loops and stuff in very simple language, and you control this turtle.
Ron:	Yeah.
Scott:	So this guy wrote a winforms application with this turtle object. You say, "Turtle-dot-left, turtle-dot-right, " the turtle moves around. It looks like the little triangle that you saw in asteroids, and it just draws lines.
Ron:	Uh-huh.
Scott:	If you go and do looping patterns and make the turtle turn ten degrees, you can make a nice sunflower or fractal pattern or whatever. So he added to this Power Shell. There's a text box where you type in the form loops and the different patterns. The way that you do that is you make a run space, and then you hand an instance of your dot net object, in this case the turtle object, and you hand that to Power Shell.
Ron:	Oh, OK.
Scott:	Then, any script that has access to that particular variable, you can say, "This object is called 'Turtle' when it is inside Power Shell."
Ron:	OK.
Scott:	So then any objects within your application, anything within your own document, object modal, or anything that you want to automate, you just hand in to this run space and say, "Hey, Power Shell, you have access to this, this and that" and you can enable it.
Ron:	Ah...
Scott:	So I'm going to take his application for my next "Coding for Fun" article and attach an actual robot to it.
Ron:	Oh, nice.
Scott:	Then you'll be able to use Power Shell to not only make the turtle move around on the screen but also in real life.
Ron:	Yeah!
Scott:	Like, a felt pen or a Sharpie, attach it to the robot so if it draws designs on the screen, it will draw them on pieces of paper.
Ron:	Nice.
Scott:	Power Shell is going to make all that happen. How else would it I do it? I would have to go write a scripting thing or buy a language or something.
Ron:	You know what? This opens up an opportunity that has been a problem for a lot of people used to using VB script or Java script to provide extensibility to their customers. They go, "Hey, you can add this behavior, just write a little VB script to do it."
Scott:	They would embed VBA in their application.
Ron:	Yeah, exactly.
Scott:	When all they wanted was a little scripting, a little something.
Ron:	Yeah so I am wondering if you could just do PowerShell instead. Would that be a good replacement, you think?
Scott:	It would be possible. I don't know enough about VBA. I do know that most managed applications don't want to include VBA in their stuff because it is kind of old school. But if there was an application and you have a nice clean object model, I think it would be really cool to be able to write macros for Outlook, from within Outlook. Certainly Outlook has a lovely OLE automation layer and I can certainly automate it out of process. I can put Outlook on a string from outside of PowerShell but when I want to write a macro, I got to do it in on the inside. I have to do it in VBA. So, yeah, I think PowerShell would be a very clean way to do that.
Ron:	Well at least I can imagine it being super useful for kind of like the management surface of the app. So like if you said -- when we shut down this app there is a standard set of things you do to shut it down or pause it.
Scott:	And then a script that would run.
Ron:	Yeah you could add your script to shut down my custom service also.
Scott:	The trick there would be to figure out when is it appropriate to use PowerShell on the outside of my app versus on the inside. PowerShell being able to talk to WMI, being able to talk to COM and.NET so easily, in the example you gave, you are saying, on the event of my shutdown, run this little tiny PowerShell script. Other kinds of things that you might want to automate, if you simply put stuff in WMI, or made things available as a.NET interface from the outside, you can automate as well. Like we've automated some things in VirtualServer because VirtualServer puts all this information inside of WMI. I had no idea. Like information about how hard the CPU is working, how full the virtual disk is. So you can go and say -- for every machine on this virtual server if any of them has a CPU greater than 50%, email me. Or whatever, you can go nuts.
Ron:	Oh cool.
Scott:	Or some of these newer tools, like PowerGadgets? Have you seen this? PowerGadgets is a third-party tool that takes PowerShell output, the PowerShell objects in the pipeline. So you could say -- get process pipe select dash first five, so maybe get the first five processes, and then pipe that into get chart. It will actually build a chart in real time.
Ron:	Oh nice.
Scott:	So you can take anything at all like an example of an application you wrote. If you write an application that puts health and monitoring information in a perfmon or WMI using PowerShell and PowerGadgets, you can get a free dashboard.
Ron:	Very cool.
Scott:	So we always ask ourselves, if I was going to make this application a well-managed, easy-to-manage, healthy application, what would that dashboard look like? You know, you've said this yourself. This is the gauges that you would put on your dashboard. Then PowerShell is that rubber band that's helping you lash them all together.
Ron:	What I like about that, if you did it that way, the gauges visualize the raw output, but the raw output is also available to whatever other management tools that people want to use.
Scott:	Exactly. So why would I have to build a management interface? I really just need to publish the information and then have tools that are rich enough to give me that stuff. So with something like PowerGadgets I can say, for example, I do banking -- number of calls to get accounts per second. I go and do a little one-line ad-hoc query, pipe it into a gauge, pick that gauge up drag it to the window's sidebar and dock it. Then I get to see the health of my system.
Ron:	That way you can give people the capability to do these very simple ad-hoc things.
Scott:	They can get pretty complex though. You can say -- do this every five seconds. Like I saw one guy, speaking of complicated queries, he said -- for every machine in my domain, check the number of batteries that it has, retrieve the serial number for that battery and check it against a list of bad Dell batteries...
Scott:	[laughs]
Ron:	.. and let me know the machine names. So he was able to find people who had Dell laptops within a certain range because the data was sitting within WI.
Scott:	Nice.
Ron:	So he could make a gauge - number of dangerous Dell batteries on my domain.
Scott:	[laughs] I like it. I like it.
Ron:	This is the thing, right. Everything that you can think, why not express your intent like that? I mean, how else would you do that? Probably with McGyver and VBScript in the past.
Scott:	Well you know this came up in my "Patterns of Anti-Pattern" talk on Wednesday. I was talking about the notion of the uber-service. A service that just takes in XML and returns XML that says like "execute this."
Ron:	Yeah, I call those third-class services.
Scott:	I keep running into people who want to do this. Since I started giving this presentation now I get emails from people around the world that are like -- "my company wants to do this; please help me to talk them out of it".
Ron:	They just want to pass around POX, plain old XML.
Scott:	Yeah, exactly. So I actually think your argument about non-code artifacts applies here because what do you do when you have an interface like that and somebody says well what XML do I send you? They say, oh we'll send you the Word document that describes the XML that you can send us. So then your interface is not really testable or that intent is not testable.
Ron:	Well that gets into the whole discussion about early-boundedness. Like some of the stuff that is going on with implicit parameters. Like what is a dynamic language versus what is a non? C# 3.0 really starts to blur the line a little bit. I mean, what is a dynamic language? What is a dynamic interface? Is var or object a reasonable thing for something to take as a parameter in return? In C# 3.0 even though the language feels like it is kind of dynamic and early-bound, is it really? Is it the compiler that is doing this? Is it the jitter that is doing this? Is it the IL? Languages like Ruby are dynamic because they are interpreted. Languages like C# 3.0 are dynamic because of some of the funky stuff we are supposed to do just-in-time. So we are starting to blur the line. Then as it starts to blur the line in the underlying compilers, what does that mean for us as designers and what we can, and should get away with, in our methods? Just because you can do something, does that mean it is the right thing to do?
Scott:	Yeah, exactly. I'm thinking when it comes to services then there is a much more human consideration about setting an expectation about "this is what this service does".
Ron:	So what if I gave you a service that takes a data set and returns a data set?
Scott:	I don't like that for the same reason.
Ron:	Is that better than taking a string and returning a string?
Scott:	No, no.
Ron:	So then if that's silly -- if we laugh at that, and go oh, no, of course not, don't send me a data set and return a data set -- why do we see so many functions that take something and return a data set? I have had this article up on my blog. I think it is called "Data Sets are the Apawn of Satan and All That is Wrong with the World".
Scott:	[laughs] Exactly.
Ron:	I'm not really against data sets but I was just trying to make the point that a data set isn't an object of type "whatever is inside it", it's an object of type bowl. Right? You can put an orange in a bowl or an apple in a bowl but it is still just a bowl.
Scott:	[laughs] Exactly.
Ron:	And then having to look inside to ask it, I find a little frustrating. I don't like to go and say, "if is typeOf".
Scott:	Right. I'm with you. I'm with you. I have to agree with that. All right we have to let you get going so that you can prepare for your talk this afternoon. Thank you so much for joining me on ARCast today.
Ron:	Scott Hanselman, ladies and gentlemen, with just a kind of a grab bag of cool architectural stuff -- you know, PowerShell, dynamic languages -- you never know where Scott is going to go and that is why he has a very interesting podcast show at Hanselminutes.com. Speaking of that seminar that we did together, we had the seminars videotaped in the US and in Barcelona. I am releasing little clips of those in about 15 minutes chunks. on ARCast.TV. Now if you haven't seen the video version of our podcast show at ARCast.TV, let me encourage you to go download an episode and check it out. In fact, I think I may just feature the audio on a bonus episode sometime. Hey, we'll see you next time on ARCast.
Announcer:	ARCast radio is a production of the Microsoft Architecture Strategy team: www.ARCast.net.

ARCast.net - Non-Code Artifacts Approach Zero with Scott Hanselman

Links