|
Ron: |
All right so those are the anti-patterns. They are kind of fun to talk about what not to do but people sometimes get annoyed at that. [laughs] One guy said on the eval comments for this thing, "All he did was talk about what not to do!"
So here we are going to talk about what you ought to do. In one of the podcasts I recorded some time ago, it was more than a year ago, I sat down with two very smart guys -- Clemens Vasters, who is here this week, he's been speaking, and another guy who is an architect from the UK, Arvindra Sehmi.
We were talking about services and thinking about service oriented architecture. By the way, this podcast, I republished it on my show so it's called "Humans As Services". You can go listen to it.
We were talking though about how did people do business in the days before we had computers. How did they do it? You go all the way back to 1880 and Zurich had a stock exchange. They were doing stuff. They were trading stocks.
They were exchanging money. Somebody was getting this all done and they didn't have a single computer to do it. How did they accomplish that?
Well I'll tell you how they accomplished it. They did it with paper. [laughs] OK. My very first job out of college was to create one of those -- it was one of those projects where they said, "We have a paper system we'd like to put this on computer." I'm sure a lot of you have done these.
It was kind of fun to go back and think about how that system worked. I was working at the University of Oklahoma and their college of continuing education.
They put on things like this, a little seminar -- they'd have a professor, some people attending, they'd have catering, chairs, rooms, facilities, what not. So if you wanted to put on one of these little events, you had to fill out this paper form.
Now it was a nine part form so you had to press really hard. The form had little boxes on it with labels and certain links. It was kind of like a schema.
You would fill out this form and you'd say what class you wanted to have, who the instructor was, what the date was, and had they had all these checkboxes and things so you wouldn't forget some vital piece of information.
Then you would take this nine part form and you would give it the router, or the person who would tear the pieces apart, and they would send one copy to the professor, one copy to the custodian who would arrange the chairs, another copy to the registrar so that students could register for the event, another copy to the facility so that chairs and screens were set up.
It was a very cool system. In fact, when you think about it, this was all done asynchronously.
Nobody took that form and handed it to somebody and said, "I'll wait."
[laughter]
They didn't do that, OK? There were well defined process boundaries. So if you took that form over to the registrar's office, who was going to handle billing and what not, the person who took that form over there doesn't know that Sally is going to enter this on the computer and Sam is going to accept payments from students.
So they don't go and go, "Where's Sally?"
They don't do that, right? There was a box that said Inbox, right? They put it in there and they walked away.
They didn't know the first thing about what that department did when that form once it got there. What they knew was what kind of business result to expect, which is that students would be able to register for this event and pay for it.
That's what you really want to have well defined process boundaries. Know the explicit behavior. So, I like to start with a process. First of all you've got to decide what process you are going to expose with services and you want to decompose the process down into a set of activities.
They might all happen on the same server. Some of them might happen on this server and on another server and the mainframe and other things.
You want to break it down and understand the process really well. In fact, I think it's a great idea to describe it as a work flow. We actually have a whole methodology for doing this called MOTION.
I did an ARCast episode, it's actually a two parter, with Rick Merrifield, who is the MOTION director for Microsoft. So you can listen to that and there's also a white paper on MSDN about MOTION now, if you are interested in knowing more.
Then what you want to do is create your contract. Don't begin with the database, begin with the messages. The messages define what's going on back and forth between your services. Windows Communication Foundation supports the notion of a message contract.
It's specific to my service so I have my update Customer Message. I like to actually call them request Messages and response Messages so I like to have really verbose names on them but I think it makes things clearer.
You mark the different parts of the message. You can have things in the message body and things in the message header if you want. These little types that I've described here can also be described as data contracts. So if I have a customer type and the customer type is used in a lot of different services, I would want to create a data contract for my customer type.
We are being very explicit about what is included in this customer type when it is sent on the wire so I decorate attributes about which parts of this thing will get sent on the wire. I begin by defining the messages.
The next thing I am going define - operations. Like what can you do? You can update a customer. Thinking about moving the state of whatever thing I'm dealing with from one state to another in an atomic fashion, remember the one message, if we can possibly do that.
Third we are going to group these operations into services. It's a good idea to make your services an interface. You describe it as a service contract.
Each operation on the interface that is going to be published on the wire has to be explicitly called out. This is part of boundaries are explicit thinking. So you have to opt in.
There's a tip here. Of course, you want to use portable types. Sometimes when I say this, people think I am against datasets. I love datasets. Datasets are my best friend.
Datasets are really cool -- you can go to a database, you can suck out a bunch of stuff in a dataset, you can use them and massage them and do all you want. But, datasets are not great for people who are not running.NET. OK?
Sometimes people will get a dataset and just shove it on the wire. It's not a very interoperable thing. It's a platform specific type. I think it's a good idea to make my service use platform portable types as much as possible.
Secondly, I like to decouple the internals from the externals of the system. This is one of the beautiful things about data contracts because it really makes it very visible when I'm using a type on the wire.
Let's take that example of the customer. Say I want to have an interface called get Customer. Deep down inside my business logic I have a type called Customer. It makes perfect sense right? We're going to just return this customer type.
The thing is my implementation has a lot of churn to it. I'm fixing bugs; I'm adding new things. Implementation tends to churn. Churn causes instability. Instability makes people unhappy especially when it causes breakage between links of systems. It increases friction.
What happens is, if I take that customer type, I hand it out on the wire, I have now linked the churn of my internal system to my external interface, causing more pain. Our goal ought to be less pain, OK?
If I'm using a customer type like this and it's not decorated with data contracts and what not, some poor programmer who doesn't even realize that type is being used on the wire might accidentally change it and break everything.
Windows Communication Foundation and the message contract and the data contract model causes us to explicitly decorate these messages. So we have a customer message. What we do is bring that customer right up to the wire.
The internal type and then we pour it into the customer message. Ok, and now we send that out on the wire, the great thing about that is that now I can change my customer type any day of the week, I can feel free to change it as long as I'm not breaking my internals its ok.
And I'm not constantly turning my external surface area. Another really fun thing, when you're designing a contract is to always think about the consumer.
Now, I have a couple of friends who are really into test driven development - Peter Provost on the patterns of practice, I think has spoke about it here. And Jim Newkirk, the inventor of n-unit. These guys have taught me a lot about this.
So one of the things about test driven development is that you write the test first. Why? Because it causes you to think about the poor schlob that has to consume your service. And you start thinking. What does it feel like to consume this?
So I was doing this, I was writing a test for a service that I was doing, and here in this one I'm getting some customers from a particular country. I had a load test database, and I put a couple of asserts in there, and it hit me that this first assert is asserting that this particular customer is always first.
Ok, it's a test database, and we're not adding new customers, it's probably going to work. But as I was thinking about this, it hit me. If I was consuming a service that's returning me a collection of customers. And I made the assumption that they were always going to come back in the same order.
That's a dangerous assumption, isn't it? It's like relying on an implicit behavior. Because the contract doesn't say what order these records are going to come back in, right?
Tomorrow the guy who implemented the service, could change his mind, and they could come back in exactly the opposite order, or random order, or who knows what order. But if I'm relying on the order of the service, I'm relying on implicit behavior-which is evil, you must not do that.
But if you said, but the order is really important, I have to what order they come back in. And they have the database and databases are much better at sorting things, why don't we ask them to give me a certain order-great.
Then make the order a part of the contract. Right, you could add an enum that says order by city, order by last name, order by zip code, or whatever you want to do, and then when I request the particular customer question I could ask what order I want. Now I'm relying on explicit behavior, which is always better. Oops, wrong way.
Now, my good friends and patterns of praxis. And I used to be on that team, and their still doing great work have taken a lot of the things that I've said today, and have built a very cool thing.
If you've heard of talking about software factories as kind of an abstract concept, their now beginning to come a reality. And their building a thing called the service factory. What's cool about the service factory is you might be listening to this today going.
Oh this is great, this is so cool, I'm going to write great services, but then when you get back to your company all those other guys who didn't come here and didn't learn all this stuff will keep doing those dumb antipatterns.
Right, so what you want to do is kind of get consistency about the way that everyone builds services.
So if you take the service factory, it's a collection of guidance, like written guidance about what's the right thing to do. But more than that, it's a bunch of stuff that's added to visual studio. So you create a new project and it walks you through this process about creating your message, creating data contracts, creating something else which I didn't talk about today- fault contracts.
And you know doing the right thing in terms of architecture. So we have an opinion about how things ought to be done, the service factory will help you get there. Now its still, you know under development. It's supposed to be finished by the end of July, but you can download a preview release today. And it's not just for windows communication foundation.
We also have a version that does plain old ASP.net web services if you want to use that one. So I highly recommend that.
Now the cool thing is that if you use the document processor module, you're going to get people thinking about sending and receiving business documents. Which is really what I want them thinking about. Don't think about a service as a remote method on an object.
That's the wrong picture, think about it like that in the old days. When we had paper forms and there was an inbox, and you dropped the paper in the inbox. Then at some later point you checked your mail box, and you go. Oh, they got back to me. And you got the paper out of your mailbox, and you got things done.
In fact if you can do it asynchronously, the entire better. Keeping the interface and implementation decoupled allows you to have stable links, while implementation continues to churn, and if you stayed away from platform specific types, your easy to consume from any platform-which is always a good thing.
Ok, one last pattern-reservation. One of the first thing that we happened when we did web services, and at the time I was working on system dot enterprise services in the dot net framework. So I showed up to tech Ed and people went.
Hey, this web server thing is really cool, and can we do a transaction over web service. And I said, no. And they go, but we really want to because it looks really cool, and we did transactions with dcom and com+. All you had to do was check a box, can't we do that?
So I thought, well there ought to be a way. So I went back after tech Ed and I worked out this kind of way to do it, it was kind of a kludgey thing and I was going to write a paper and post the sample, and all this. And then one of the architects on the indigo team said to me. You know I'm not sure that's a great idea.
So I begin thinking about why isn't it a great idea. Well let's imagine the scenario. You got two organizations, maybe their in the same company, or they could be in different companies. And you're doing some kind of an update between the two companies.
Both sides have a database, and you want everything to be consistent in the end of this update. So the question is who controls the transaction.
Somebody has to control, but the thing is, maybe organization a does not trust organization b. And they say, well we want to control it, and the other guys say, well we want to control it. It's a big problem because really distributed transactions assume a degree of trust -between systems that's usually not desirable.
They introduce a tight coupling because the rules of distributed transactions say that if I enlist in a transaction inside my country, my autonomous system, I am now promising that I will do whatever I have to do to ensure that I can commit that transaction.
For as long as it takes until some other foreign transaction manager tells me it's ok. So that might mean that I lock rows, that I lock indexes, I lock pages; I might lock the whole database to comply with the rules of this transaction. And, if I don't know who's controlling the thing, and I don't trust them, why on earth would I want to do that.
Now sometimes when people say this, they say, hey no problem just does a compensating transaction, I've heard this. And it goes like okay, a compensating transaction means ok I'm going to have the update customer method or it's not a method - it's an interface, a service.
So you call update customer and then you go off to do whatever else you want to do, and if something goes bad call the undue update customer. And that'll undue it. Sounds good, but there's a lot of problems with this approach. One problem is what if you call update customer. Something goes wrong, but you never call undue update customer. How bad would that be?
I don't know but it depends on the architecture of the system. But it could be very bad; maybe it means that my database is now out of sync with your database. Or here's another problem. You call update customer, ok.
Time passes, somebody else calls update on the same customer, and now you call undue update, so now when you undue that update you've rolled back both changes. And somebody else is very unhappy, right?
These problems result from the fact that I'm relying on somebody else to fix the problem. So I started thinking, what the airlines did. You know, remember back in the days when you used to actually talk with a person at the airline.
I would say, hey I want a flight. I'm going to go from Seattle to Boston on this particular day, you know, what do you got? They give me some options, and I might say hey you know that one sounds pretty good, I think I might take that one. So they get all of my information, and then they give me a confirmation number, and then they say, hey, by the way, would you like to purchase that ticket now.
And I might say, hey you know I have to check out a few things, you know check with my wife, Ill get back to you. And they say, ok, great, but make sure you call us back, because if you don't that seat will be empty from Seattle to Boston, and wed be really unhappy about that. Is that what they say?
No. Why not? Because they don't trust me, they know I probably wouldn't call back. [laughs] In fact, I'm glad I don't have to call back, right.
What do they say? They say, "Oh no problem, if you don't get back to us by Tuesday at midnight, the whole thing cancels automatically." You see, what are they doing? They are asserting their autonomy over a very precious resource to them, this seat.
So they will give me the ability to get a temporary claim against that resource, which actually doesn't mean a lot.
I've learned, the hard way, that a reservation doesn't mean anything until your tail end is in that seat, the door is closed, and you are going down the runway. Then it means something. Up until then, anything else could happen. But, they don't trust me.
It seemed to me that we can do the same thing in interaction between systems where we don't trust. It's not that we don't trust people, I mean you are nice people, right. But, I don't trust your system. It might have bugs or might be running bad today or something.
So what I would want to do is think about all the states of the interaction. And, by the way, a state driven work flow would be a great way to do this. I would think about all the ways the states that this interaction could go through and the events that cause it to transition.
Sometimes the event is you sent me a message and so I transition it. Sometimes the event is time passed and nothing happened so we need to cancel it. This is very doable. The cool thing about this is it puts all of the control over the consistency of my system and my data, in my autonomous country, right where it ought to be.
So if I had a supply chain app. I might, for example, get a message where somebody wants to reserve a part. Great. I might check my system and say "yeah, I'll give you a reservation, sure I'll do that for you."
I'm going to put a time box around this. I'm going to say that reservation expires at this particular time. If I don't hear back from you, the whole thing goes away.
At some later point, it might be seconds, hours, or days later, I get a message back. By the way, if it's days later, I can't hold a transaction open for that long.
So I can't be really using a distributed transaction to do this, right? This is why work flow is a very cool way to do this because when I get that message back, I can correlate it with a work flow, re-hydrate the work flow instance, and move to the next state because that message came in.
I confirm the reservation. I get the message saying "yup, we want to buy that." I could at that point be like the airlines and say "oh, by the way, sorry, but I overbooked this reservation, you can't have it." I could do that. Or I could say, "yeah, you could have it and here you go."
Now this is a pretty simple example and this is one among many techniques for trying to do data consistency. Often times people will say to me, hey, but I thought there was a standard called WS atomic transactions that allows me to do distributed transactions over web services.
To which I say, yes there is and Windows Communication Foundation supports that standard which is both a blessing and a curse.
It's a blessing because when you need it, now it will be real easy to do a transaction between heterogeneous system in a very inter-operable way.
It's a curse because I think people will use it in places where they clearly should not because they are sharing a transaction and violating their autonomy in places where it would be a very bad idea. Keeping these principals in mind will help you.
HP is an interesting example. In the "World is Flat" book, they talk about how HP went from 50 supply chains down to five. They were driven to do this because of the relentless cost competition in the PC business. They had to learn how to make their systems collaborate horizontally. That meant opening them up.
Instead of the big stovepipes that did everything from beginning to end, they had to open up these systems to collaborate horizontally which is exactly the kind of thing that we are all being drive to do in architecture today.
It's a good thing to get a group on these principals and keep them in mind as you are designing the architecture for where you are going.
There's a whole bunch of resources here that I just wanted to make available for you. Of course, the ARCast show. Actually a lot of people listen. How many of you have heard an ARCast, anybody? Woo, nice, hey, a lot of people here. That's great. I encourage you to do that.
We have a nice site called skyscraper that just launched about a month ago and it has blogs and my artcast show is moving on to that site.
Also a new thing we're putting together: Kind of a training resource for people who want to learn how to become an architect. There's my site at ronjacobs.com, the Service Factory down there at the bottom, and some architecture forums if you want to ask questions and help people out.
Okay, so we have some time for questions. If you have a question, let me encourage you to come to the microphone so people can hear you. If you need to go and have dinner, great - thank you very much for coming.
[audience applause]
Yes, over here. |
