ARCast.net - The Agile Architect with Jeffrey Palermo

Announcer: It's Thursday, July 7, 2007. You're listening to ARCast.

Ron Jacobs: Hey welcome back friends to ARCast. This is your host, Ron Jacobs. Today we are going to ticket Orlando 2007, where I was up on the virtual Tech Ed stage as people were kind of wondering by, and recorded a nice little chat with Jeffrey Palermo.

Now, Jeffrey Palermo is an MVP. He is a legend for his "Party with Palermo," the legendary Tech Ed party. He had like over 400 people there the night before Tech Ed, and so everybody is wearing these little tags like "I Party With Palermo," or whatever. It was a lot of fun. Jeffrey is a really fun guy, and also kind of an expert on Agile stuff. So, Jeffrey and I had a nice talk about what it means for an architect to understand this whole trend about being "Agile."

It's something that I'm paying a lot more attention to now, because I really like a lot of the Agile practices. If you're an architect, or just thinking about it, you need to understand what it means to be an Agile Architect. But first, let's have an ARCast Rapid Response.

[siren]

Announcer: Your post on the MSDN Architecture Forum has been selected for an ARCast Rapid Response.

[siren]

Ron: This is Ron Jacobs, and welcome back to an ARCast Rapid Response. Today I am joined by, Patrick Lowendall. Patrick, you're an MVP there in Sweden?

Patrick Lowendall: Yes, that's correct.

Ron: Yeah, and a C# MVP. OK, well that's great! I'm glad to have you today on ARCast Rapid Response. Patrick, I've been looking at this message that's posted by Nella -- oh boy, I don't know how to say this -- Nellie Manning -- I guess that's it. It's on the MSDN Architecture General Forum.

He's building a "Smart Client" application that's in.NET and he's going to connect to a WCF Service. His issue is really thinking about the contract -- there's kind of two parts to the issue. He's thinking about the contract of his WCF Service. "What should it look like? What kind of data should we pass back and forth." Then he's thinking, "DataSet, Custom Entity, XML, which one?"

He's kind of leaning against DataSet because he wants non.NET clients to be able to use this service. But then he's also thinking about, "How do you track the changes that you made on the client side?" So if you maybe connected to the service, got an array of customer objects, and you bind these to a grid -- somebody goes off and makes a few changes -- how do you track those changes, and make sure they get propagated to the back-end system? That's the question. What kind of answers do you have for this guy?

Patrick: This is actually a common question when it comes to communication, and it's rooted into that. People try to do API style communication, where the service and API to their client, consumer, whatever.

The thing that we want to try to do is, you want to try to separate the contract of what they say you want to send. You really want to narrow down on the date they are sending, and have like only the date of full operation.

So the first thing you should think about is, what dates am I actually needing for the sending it down to the server? Many people when they use the API style, they send like huge datasets back and forth, or they do active dates or object parities. But that means that there's actually a lot of datasets coming down--might be coming down to server that's not needed.

So for the sake of simplicity, and for being efficient on the wire, you should really look at in this scenario using WCF's data contract feature to create those kind of contracts. So, that's the first...

Ron: OK, let's inject something in here then. So rather than saying, "We're going to pass a bunch of data back and forth," what you're saying is, "The client side really needs to keep track of what's been changed. Then when it gets a change and wants to update the server side, it's going to send only what's been changed. The shape of the data that it sends back, might be different than the shape that you retrieved from the server."

Patrick: Yes, that's correct. That's because, like in the scenario I talked about right before this, if I have questions that I sent to the client and the sort of text and type, and a lot of information about that question, that's not a question that's really interesting, when answering that question.

So when I answer that question, I only want to know what's the ID of the question I answer. So the next operation will only have the question ID and the answer value. So you shape the data for the operation you actually do, to be efficient.

Ron: OK. That makes a lot of sense. Tell me about the data contract. What is that about?

Patrick: Data contract in WCF is a way to extend contracts and tell you what data you actually are sending. So in HMX services, you had contracts that were more like web methods, and you could create XML documents from objects or something similar.

But in WCF you're more explicit. You show intent by defining these classes with a lot of attributes, telling you exactly what you want and what data a certain operation is expecting, or is expected to set.

Ron: OK. Well, thanks so much, Patrick, and that's been an ARCast Rapid Response.

[siren]

Announcer: This has been an ARCast Rapid Response brought to you by ARCast TV and Architect MVPs worldwide.

[siren]

Ron: Welcome back to ARCast, everybody. This is your host, Ron Jacobs, here at Tech Ed in Orlando, Florida. It's just a beautiful place; kind of hot and sticky for a guy from the Northwest.

I'm joined today by Jeffrey Palermo, who is a Microsoft MVP, an independent consultant who's into Agile coaching. I like that. Also, a user group leader, an I-Net rep, an Eagle Scout [laughs], an Iraq veteran, and the host of the Party with Palermo event. Wow! You are doing it all, aren't you, Jeff?

Jeffrey Palermo: Doing a little bit. How you doing?

Ron: How was that party? So you had this big party on Sunday night, the Party with Palermo. Tell me about that.

Jeffrey: I did. OK. Two years ago at Tech Ed, I realized there's nothing going on on Sunday night. There's absolutely nothing; people are getting into town. So we had a small gathering; the first Party with Palermo had about 35 people at a restaurant. We had it last year, and then we had one at the MVP Summit, where I got sponsors.

Then I did it up larger, so Sunday night at the Globe Lounge, I rented out the entire bar and got corporate sponsors to get involved. We attracted 437 people at a party on Sunday night!

Ron: [laughs]

Jeffrey: I didn't do much advertising. I did it through word-of-mouth, I posted it on my blog. So whoever read my blog, they told their friends and they told their friends and 437 people showed up.

Ron: Who says that geeks don't party, man? Party with Palermo! I love that. That's hilarious! You also have a blog at codebetter.com. I like that name, codebetter.com.

All the good domain names are snapped up, aren't they? I was just glad I got ronjacobs.com. Seven or eight years ago I snagged that one up, so I was happy to do that.

Jeffrey: And I have jeffreypalermo.com. It's always good if you run for president.

Ron: There you go. Who knows? Who knows what good happen, right? So let's talk a little bit about the Agile architect. A lot of people are like, "Agile? I thought that was something like Extreme Programming. That's something for developers; Architects sit in their ivory tower and create use/case documents and write functional specifications." So how do we think about Agile architects?

Jeffrey: Well, first of all, it's important to understand what Agile actually is, because some people may have a misperception. If someone has said, "Hey, we're going to do Agile," and then they've misrepresented, or maybe they didn't do a very good job; they were irresponsible. Extreme Programming, Scrum, Lean, these other practices, are not principles, OK?

Agile is a set of principles, and there's practices that support the principles. There are four main principles, so that you understand what Agile is. Pretty much, if you talk to a lot of folks that are doing Agile, we all agree on the four principles, but from that it all depends on your situation. Because you will adopt different practices depending on your situation. It's just like with technology.

Yeah, OK.

Jeffrey: So the four principles are: Individuals and interaction over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; and responding to change over just blindly following the plan.

Ron: And these things come from the manifesto for Agile Software Development?

Jeffrey: Agilemanifesto.org. I'm sure a lot of folks in the.NET world are familiar with Dave Thomas from The Pragmatic Press. He was one of the signatories on this Agile Manifesto as well as Martin Fowler and many others. These four principles are the root of it all. The practices depend on the situation.

Ron: Right. OK. So what are some of those practices?

Jeffrey: Well you have Extreme Programming which is a set of core practices that all fit together and all complement each other like test driven development, continuous integration, having an onsite customer, the team sitting in the same place and it defines a very structured set of practices that says "If you do all these, we have found these together, to be successful. We have project after project that have implemented these twelve practices and we've shown success."

Ron: Right.

Jeffrey: There are limitations. For instance when you get teams above ten to twelve then you start getting coordination issues. How do you get all those people in the same room. So, it depends. You can't say cookie cutter across the board, "Do this. Do this. You will be successful."

Ron: Hence the need for the Agile coach. You're not the first one I've met. I've met other guys that do Agile coaching and I think that's important because it's not like "I'll just go buy a book. It tells me what to do and we'll just do it." Because, like you said, it has to be adapted to your particular situation.

Jeffrey: And for folks who are maybe just getting into the.NET world, does management expect their people who have doing maybe Delphi or BB6, to jump into.NET and say "Hey, go read a book and come and we're going to do a.NET project and I want you to go just as fast as you've always gone and I expect you to be a success."

Or, do you get a little bit of help and say "OK, here's someone who has experience in.NET and let's hope that they will get the team get up to speed."

Ron: OK. So when you said it depends. OK you mention you mentioned one factor, the size of the team, what are some other factors you think that influence the kind of practices you'll adopt?

Jeffrey: Let's go from one side of the continuum to the other. Are you in a small department where maybe there are three developers and there are some tools that you need and there's a customer that says, "Hey, I need a tool for this. I have a report that I need to run and I'm doing it in Excel and it's just not working out."

So we say, "OK. Gosh, well, this should take maybe four weeks and you know what we'll just put one of our developers on it. It's a small tool. Let's crank it out in four weeks." Well you don't need a whole lot of process to make sure that that project is successful. So that's one side of it.

And you can be completely Agile as long as that one developer is concentrated solely on delivering working software to that customer.

Ron: So would you say then would you not write unit tests for that or not do peer programming or things like that?

Jeffrey: What peer? If it's just one person. So obviously that throws it out. But unit testing, well, it depends. If it has some complex calculations in one particular place and then the rest of it is just configuring the SharePoint or something else then you may find it appropriate to write some unit tests around the calculations or if it's just a report where you're grabbing data and formatting it, then there may not be an appropriate place to write any automated testing.

Ron: So I guess the bottom line is we don't have to be dogmatic about this and be like "No, no, four weeks but we're going to put two guys on it so they can peer program and we're going to get 90% test coverage out of our unit testing on this simple little report thing, " and it ends up taking a lot long than maybe it should.

If we say maybe there's not a big risk around this, it's not a mission critical piece of software, it's something we are going to do, it's OK maybe to compromise a little bit on that.

Jeffrey: Well I don't see it as compromise. I see it as doing the exact appropriate thing for the situation.

Ron: So are you not a true believer then? Is that your problem?

[laughter]

I'm just kidding with you.

Jeffrey: A lot of Agile -- it forces you into an environment where you have to tell the truth. You can't hide behind anything. You have to be completely honest. So if you look at this tiny little project then the honest viewpoint is "Gosh, you know what, in this situation, we need a very, very small process." Whereas if we have a project that's going to be taking orders for a large e-commerce company and it's big a endeavor and it's going to take maybe a year to complete. Well, we need a little bit more process than we'll throw one guy on it and let him work with the business partner.

The two situations are extremely different. In that case is where I think Extreme Programming is really where it excels because it takes these serious projects and brings us a better success rate, a better predictability. It forces us to be honest every step of the way and it forces discipline into the mix.

Ron: OK. So I've read a couple of Agile books, read many Agile articles, and typically I don't even see the word architect anywhere in those books or articles. So, you know, a lot of people say, "Oh, well, you're doing Agile, you don't need an architect." Or, "There is no architect." Or, I mean, what is the role of the architect in the Agile team?

Jeffrey: Microsoft clearly has architect roles and developer roles and a lot of large companies in traditional project management and they have architects and developers and they keep them separate. They are actual job titles. Now, on a team that I would coach, I wouldn't find it necessary to have a person with the job title of architect but if someone from the outside was to say, "Hey, I need to have a meeting with your architect." Well, there's always a guy that you say, "Here we're having a meeting with him."

His job title may not be architect but it's the person in the leadership position. It's the person who also has the global view of the system who is always keeping his head up and looking around to make sure that the entire system is going to come to fruition. If there's web services to talk to or external systems, there's someone who has to coordinate with that. So the role is fulfilled. The duties, yes, you have to have someone doing those duties. Is the job title so important? Well, not really, as long as the job gets done. So, maybe they're in the company structure or they may not be there depending on the company. But when I coach I don't really care if someone has the job title because there's always someone who fulfills the role.

Ron: OK so one of the Agile practices is, you know, you take user stories -- you do this little kind of planning game where you work out what the business, here's the user stories we're going to address and then iteration. One thing that always concerns me when I see this is sometimes I think people are focused very narrowly scoped on "This is all we're thinking about. This is all we're doing. Those user stories. That's it."

Maybe that's OK but I think maybe when they do that they might kind of paint themselves into a corner or they may not be thinking about the next set of user stories that are coming or some other thing that is coming down the road because they're saying we're only going to build what is essential to right now. Are they somehow blocking themselves in because they have that narrow focus?

Jeffrey: Well that's a reaction against the other extreme. Often companies will be at the other extreme. They will say, "We need to plan and make all the decisions right now because if we don't then we're going to be painted into a corner and if we don't make all the decisions right now, one of them is going to cause us problems."

Well when I coach, I say, "We need to make decisions at the last responsible moment. Not all the decisions up front, but we need to make the decision at the right time." And so, if we're doing an e-commerce app and we need to submit an order, well, what does that mean to submit an order? Well there's a whole lot of things from the ground, you know, green fill development, there's a whole of things we need to do before we submit the order.

We need to show what products we're going to sell. We need to have the shopping cart experience. There's a whole lot of other things that may take six months before we're ready to submit an order, maybe it's a web service, or other back end systems.

So, it's OK to delay some of the details of submitting the order, because there's a lot of questions that need to be answered before that. So, the extreme is we need to know everything, and then when I coach to say, "Well, let's break it up. What are we going to deliver in two weeks? Are we going to show products to the customers? Let's hash out all these details. Let's have the development team focus on delivering those two weeks."

Now the architect, or the team lead, the person in the architect role--yes they always have to keep their head up--they always have to be looking forward to make sure that everything is going to come in line. That's part of their responsibilities. So, their job is to make sure that everything falls into place.

But the development team doesn't need to be distracted with submitting an order, when they have a lot of work to do before we even get to that point. We want the development team focused completely on delivering the scope of that iteration.

Ron: OK. So, this is a good point when you talk about iterations and sort of Internet development. A lot of people have a hard time kind of breaking down their work into this sort of style.

In fact, I find Agile teams often have what most people would consider pretty short iterations. Like you talk about a couple of weeks, or maybe a month or something, and they go, "Gee. We've got a project that we thinks going to take a year, and you're telling us...

Jeffrey: Right.

Ron: ... and you're telling us we're going to deliver something two weeks at a time. It's hard to imagine." The old way of doing things people were like, "Well, we're going to spend the first six months just building an infrastructure."

Jeffrey: Right.

Ron: They'll build it all kinds of layers, and...

Jeffrey: So, what does the customer have after six months?

Ron: Nothing.

Jeffrey: Exactly!

[laughter]

Jeffrey: Exactly!

Ron: Exactly!

Jeffrey: They're paying money, and they have absolutely nothing.

Ron: Yeah.

Jeffrey: "Oh, but look at all these pretty documents." That's great, but that can't take a customers order.

Ron: [laughs] Exactly. So, the thing is when you say, "OK. We're going to break this up, " and like you said "submit order, " can only come maybe way out here, because there's so many things we've got to do before.

So when you're kind of thinking through this process, you've got this long list of things you intend to do, it seems to me that one of the big roles of the architect is to say. help to guide the team through what has to be addressed in this iteration vs. this one, vs. this one.

Jeffrey: The team doesn't have to make that decision. That's really the product owners job.

Ron: Oh, OK.

Jeffrey: Who's paying the money?

Ron: Yeah.

Jeffrey: Who has the budget?

Ron: Yeah.

Jeffrey: They're the boss. They say, "We need all this work done." If it's a brand new project they may come out and say, "Well, I need it all done."

Ron: Yeah.

Jeffrey: But, that's just talking about major features. You may have some things that are like--browse the products, do a shopping cart, submit the order, whatever. If those features aren't there, the project is a failure--very clear.

Ron: Yeah.

Jeffrey: But then there's other things such as, "When I browse the product I want a little flashy pop-up that tells me the inventory level, or whether the product is back-logged." Well, if that feature is not there, or maybe not there right away, you would not consider the project a complete failure.

Ron: Right.

Jeffrey: So prioritization--the architect often has to help with prioritization if there's technical details that come into play. Now, as a coach I would coach the product owner as well, so that the architect doesn't have to be distracted dealing with business analysis issues.

Ron: Well like--so here's an example--like I'm the customer and I say, "Look, I want people to get on my website, buy stuff, and submit an order." You say, "OK, that's it, " right? Then I say, "Well, how are the products going to get in the database so that you can see them to order them?" all right, so the architect answers and he goes, "Oh, gee. I never thought about that."

Jeffrey: Right.

Ron: Are just going to let anyone log into the database to order them, or are you going to need a security system that says who can get in there and order them, and assign roles.

Jeffrey: Yeah.

Ron: So, there's probably some level of interaction...

Jeffrey: Oh, definitely.

Ron: ... that the architect has to do to draw these things out with the customer. Right?

Jeffrey: The architect definitely has the logical reasoning skills to look at some requirements, talk with the product owner and say, "Well, here's a detail that's not defined. Tell me about this." "Well, just take the order." "No-no-no, let's talk a little more about this--it's kind of vague."

Ron: Yeah.

Jeffrey: So definitely the architect is thinking of technical issues, and the product owner or customer, they just want their system. So there's a lot of interaction that has to happen, and the architect is often the one where it's appropriate to do that, instead of taking the entire development team, and taking all of their time to talk with the customer.

Now testers can often play a helpful role in this, because in order to test the system, they have to know what the system is supposed to do, and every little detail of its behavior. So the architect has some help. He doesn't have to do it all by himself.

Ron Jacobs: Well, and testers are great at that. I always say that testers are weird people, because they're so detail-oriented, and they'll insist on knowing every last little thing. I remember once when I was writing a specification for WCF.

I was working a feature, and I said, "Well, I think this feature will allow a header to be added to the message at every hop that will help to trace it." And the tester came to me and said, "Well, how many of these headers can we add?" I said, "I don't know; in most cases it will be less than ten." He goes, "Is there a limit?" I said, "No, I don't see a limit." He goes, "OK, I'm going to add a million."

[laughter]

Jeffrey: Exactly! There's a limit! Oh, just put a number in that textbox. What's the range? Just let him type any number. Well if you use 'int' as the data type, there's clearly a range.

Ron: Well, that's the kind of thing a tester can help you think about, because I was only thinking about a normal, every day case, but a tester is going to think about all of the other bizarre relationships there.

Jeffrey: Exactly. So the architect is not alone. You definitely want testers on your team to help out with that, because they're great business analysts, as well.

Ron: And it's funny, because when you read Agile books, they don't often mention testers until some people say, "Oh, well, we're writing tests, and we do test stream development; we don't need testers."

Jeffrey: That's another good point. Let's talk about Waterfall testers, and Agile testers. Waterfall testers are normally called Quality Assurance Specialists, or a QA Department. A separate team that after the complete software package is "coded" (and I'm quoting here for those who can't see), then it's delivered to them, and they verify.

An Agile tester is different. An Agile team is different. They don't wait until the software is finished - they test along the way. And, they're not just manual testers. I would classify it as the dev types on the team, their job title is really "Developer/Testers." Because they're doing a lot of the automated testing. Unit testing, integration testing, maybe some fit testing, storyteller, or other testing types. And then there are "Tester/Developers." They have the testing mindset. They know how to break the system. They know how to put a million characters in the textbox.

But at the same time, they're writing automated test scripts. They're writing tools if other tools aren't available. They are not just manually poking the system, but they're deciding on their test plan, and then they automate it. So it takes development in order to do that, whether it be a scripting language, or using INUnit to do it, or union at forms, or from the top down. So whether it be developer or tester, they both have to code.

Ron: That's a great point. You can't hardly emphasize this enough, that unit tests alone are not sufficient to know that you have everything covered, so we have to have that mindset there. Wow, there's like a million things I could talk to you about. We're just about out of time here, so I just wanted to say Thanks Jeff, so much, for popping in and chatting with me on ARCast today.

Jeffrey: You're very welcome.

Ron: I appreciate it.

Jeffrey: Well thanks for the opportunity.

Ron: Thanks a lot, you guys.

Jeffrey Palermo, ladies and gentlemen, from Tech Ed 2007. Talking about an Agile Architect. Jeffrey's just a fun guy to hang out with. In fact, we had a big party night there at Universal Studios, Orlando, and I hung out with Jeffrey and David Clatt. Talk about two really fun guys to hang around with - we had a blast. We rode around together, went and had some Thai food... we just enjoyed the rides.

TechEd is a very educational experience. I really encourage it for that. But it's also a lot of fun just to come and meet up with people, hang out with them, just get to make new friends, and it's part of the fun. If you've never been to one, you ought to make plans to try and go to one in your country, or come to the US one, or whatever. But sign up early because it sells out. We'll see you next year at Tech Ed and on ARCast.

Announcer: ARCast radio is a production of the Microsoft Architecture Strategy Team, www.arcast.net.