ARCast.net - Replication Run Amuck

Announcer: It's Thursday, June 28, 2007, and you're listening to ARCast.

Ron Jacobs: Hey, welcome back friends. This is your host Ron Jacobs with another episode of ARCast. Coming your way today, we are talking to Tom Fuller who is an ARCast listener who happened to contact me one day. He's like, "Hey Ron, I'm an ARCast listener and I want to be on your show." [laughs] So you know what? Tom was coming out to Redmond for some kind of design review thing or whatever, and I said, "Great! Why don't you pop into my office? We'll sit down and talk about whatever it is that you want to talk about."

Now Tom has been working on a lot of SOA stuff and looking at some of the problems that he sees in the various customers that he consults with in his architecture business. And so today we're going to be talking about an antipattern about replication gone amuck. That's like silos of data but getting replicated all over the place and what kind of problems that causes. Stick around. We'll have more for you in just a minute.

[siren]

Announcer: Your post on the MSDN Architecture forum has been selected for an ARCast Rapid Response.

[siren fades]

Ron: Hey, this is Ron Jacobs, host of ARCast and ARCast TV, and today we are proud to bring you the first episode of ARCast Rapid Response.

Today I'm joined by Jeffrey Palermo, who is an architect MVP. And Jeffrey, I've been looking at the MSDN Architect forum, and I saw this thread that says, "When to use Delegates." Now this is written by a guy named Jamming Jay and he says he has a logger component. It logs at different sources like a text file or event log and he has two methods that, depending on where he caused his logger from, one of them would be called. So maybe if he's calling it from an exception he gets one method. If he's logging for something else, it goes to the event log. And so he's wondering whether it makes sense to use a delegate for each method or to call methods in his Windows service. I'm not sure what he means by that. What do you think? What do we, what can we say to Jamming Jay in about three minutes?

Jeffrey: OK, well let's clarify the problem statement real quick. This is an example. Suppose I've created an interface in my application called ILogger. And it has a method called Log Debug, Log Encode, standard stuff, Log Error, and I want to call that throughout my application. So perhaps he in some places, he's getting, he's getting an exception and you and I of course know the code, and he's calling ILogger.LogError. And then throughout the rest of the code, he's calling ILogger.Encode, and in the one portion that he's having problems with in his UI [?], he wants that to go to, not only a text file, but to the event log. And everywhere else he wants the messages to just go to the text file.

Well, how do we do that if we're calling, really, the same, pretty much the same line of code? How does the source dictate where I write that log message? You could roll your own logging infrastructure to that. However there are plenty of logging framers out there. The basic stuff is in systems that diagnostics with the trace listeners. The particular feature you're looking for, I think you'd do well to look at log for net, where you can specify that log messages coming from certain main spaces will go to completely different places than log messages from the rest of your application. So you can actually say, "Log messages coming from this particular class I want to go to a specific place independent of or as well as grouped with the other messages." So it's real configured one. I'd recommend checking that out.

Ron: Yeah, in fact, the other response to this thread was from a guy named Guarav? I hope I said that right. And he said, yeah, look at log for net, or enterprise library, both of which have kind of logging things, and so highly configurable.

And I guess the problem, like a lot of these threads on the forum, is that the question really isn't clear. So I'm not quite sure exactly what he was getting at, but hopefully Jim and Jay, that helps you. So thanks for the ARCast Rapid Response, Jeffery Palermo.

Jeffrey: Thank you, Ron.

[siren]

Announcer: This has been an ARCast Rapid Response brought to you by ARCast TV and Architect MVPs worldwide.

[siren fades]

Ron: Hey, welcome back to ARCast, this is your host Ron Jacobs. I'm joined today by Tom Fuller, of -- Tom, where are you, where do you work at?

Tom: I'm a consultant working for Blue Art Solutions, it's an architectural training and consulting company out of Tampa, Florida.

Ron: OK. And you have kind of an interesting email address. Is that your, you have a web site that you do?

Tom: I got a couple different things going. The email address with my company is tom dot fuller at blueartssolutions.com, but I also have a community site that I've maintained for a couple of years now. It's called SOAPitStop.com, and that's the one you maybe have been thinking about.

Ron: Yeah, yeah.

Tom: Lots of people mispronounce it as soapitstop all the time.

Ron: [laughs]

Tom: So there's plenty of people who've had fun with me on that one.

Ron: [laughs]

Tom: But yeah, it's SOAPitStop.com was the original intent.

Ron: Yeah, yeah. OK, I got it. You know, there's a bunch of those really funny URLs that where people meant one thing and they ended up with another. You know, so, there was one, I shouldn't say it. [laughs] I recently heard a comedian kind of going through a bunch of these and they're hilarious.

But anyway. We're going to talk about some -- one of my favorite things, which is antipatterns. I like antipatterns because they're stuff that people do, we know it's a bad idea, but they keep doing it anyway. And a lot of times people just aren't aware that the thing they're doing is a bad idea.

So, tell me about this antipattern that you've been thinking about.

Tom: OK. In the, in issue eight of the Architecture journal which was an issue really focused and centered around data, I took a long look at what it meant to have data in a big enterprise and what kind of antipatterns you could see dealing with data when you looked at trying to build things into a solo strategy.

Ron: OK.

Tom: So what we saw and what I've looked at in many cases -- and I want to mention I wrote a, I wrote this with a partner of mine, Shawn Morgan, also. Also I worked with Blue Arts Solutions, we wrote this based on some real field experience where -- where we see is a lot of cases where the kind of the de facto approach for making data available to various lines of businesses is to replicate.

Right, it's this idea that when I need to create new systems, I need to create kind of everything, and I need to create my own castle, and I need to bring data over, I need to put it in my environment and make sure I can get right at it.

And then, you know, when you start to look at those kind of things and how they actually create so much data redundancy in the enterprise. And then when you look at trying to build an architecture that's primarily predicated on exposing services, you find a lot more redundancies. So this antipattern is all about making sure that if you're replicating data, it's for some very key edge scenarios like data warehousing. There's some cases where I in the article list, that data replication can be a pattern as well. Like anything else, right?

Ron: Yeah.

Tom: It's never -- it's never a one shoe fits all when you're talking architecture. If you do those kinds of things, make those kind of assertions, you'll just get flamed.

Ron: OK. So this is a double-edged sword. All right, let me be the Devil's advocate here. Oh, come on, disk space is cheap, I can go buy huge disks, I can slap them in servers, so I'm not worried about that. And I could, I could very quickly have my app up and running just getting the data I want. What's wrong with that?

Tom: So, you're right, I mean, disk space is cheap. It's not necessarily from a hardware and resource perspective. It's not necessarily cost prohibitive. But the problem is this kind of idea that you're fragmenting the truth of some of these core data entities.

All right, so as you replicated it from point to point, you've created this scenario where changes occurring to that data could really create a real problem, right? So if you're creating -- if you're copying and changing, and then you're trying to figure out which services you're supposed to be calling to actually have the right version of that actual entity, it creates a lot of data integrity and confusion.

And so that's probably a bigger problem in my mind. And the thing is, it seems like there's a lot of scenarios where playing it where it lies, that kind of concept, right, the data lives somewhere. There's complications at times, especially since a lot of core system data lives maybe in legacy platforms, right?

Ron: Yeah.

Tom: So you've got to figure out ways to integrate with those legacy platforms and expose that data out, and be able to work with it in an efficient manner. Sometimes that's not possible. Again, another opportunity for data replication as a pattern, you could move data if you have major, major technology constraints.

Ron: Sure.

Tom: But if you don't, right. I mean, and again, I keep going back to if you don't, playing it where it lies gives you the ability to really optimize how data, how core data is maintained, and how it's fetched. And actually, you know, versioning it in one place and dealing with those kinds of things is, is huge. Especially when a big enterprise.

It seems like so much of the time and money and effort is spent in the maintenance cycles of these applications that, you know, if you create that same data or any levels of redundancy, it's just going to balloon your costs.

Ron: Oh, OK. all right. So, I mean, I think a lot of people can understand, you know, that if we got a customer database, let's say, you know. And this other business wants the customer data, and they say, well, let's just replicate it over there. We're going to have to have a strategy for dealing with updates and changes to the data, and collisions of updates, that kind of thing.

So, I mean, this isn't a panacea, it's not like, you know, oh, you just make a copy of it and everything and all our problems go away. It introduces a whole 'nother set of problems to deal with, right?

Tom: Absolutely, absolutely. And the thing is, there's always, there's a lot of very, very sure mechanisms for doing these kind of copy and reuse tactics, as there's a lot of tools out there that make data replication, and even some of the things you just described, right, merge conflict resolutions.

There's tooling out there that helps you do some of those things, but using it as kind of a very, very core approach to building various lines of business systems in big enterprise is, in my mind, an antipattern. Hence the article title.

Ron: Yeah. OK, so, all right. If we set all to the default position as, let's not replicate it, let's create services that make it possible to get at this stuff. Let me explore, though, when you say, "Hmm, service isn't the right choice." When would you say that?

Tom: Well, I mean, services -- so we talked about one, we talked about the issue of legacy integration. So if you're dealing with, say, an older legacy host system that has limitations on the number of, let 's say, incoming real-time connections that it can handle. Incoming concurrent connections.

Let's say you've got some limit, it's just, the system wasn't built for that. It was built around everything local, it wasn't built to be part of a cog of a huge distributed ecosystem.

Ron: OK.

Tom: all right, so. OK, there's one case. Another case is where your execution patterns on that data is going to be drastically different. I think we covered this briefly when I said, let's go ahead and replicate it to our data warehouse and do, and do the analysis, you know, those types of operations that run these big, heavy analytical reports. Let's keep those things separate, and let's make that data replicated there.

What I think -- I think what you can see in -- I mean, the first case not so much, but definitely with the second case, you're copying data over for reporting capabilities, you're copying data over for re-owning capabilities. If you're going to update it and deal with that situation where you have got to figure out how to do merge conflicts, I really don't think there's too many great cases where you want to do it.

I've seen scenarios where the thing that drove it was trying to get a lot of, let's say, data residency benefit, out of being able to do cross-joining and merging with key core data, and all kinds of third party systems. So, I've seen those cases where you really just can't get around the situation. You have got core data, it needs to be re-shaped and stuffed into some system so you can do integration. Those are cases where you can do it as well, but again you have to be very careful where you are going to actually make edits and changes, I think.

Ron: One of the things that really helped me to get my head around this was some work that Pat Hellen, who's one of our colleagues that used to be at Microsoft. He left, and now he's come back actually. He just recently came back. Pat wrote an article about data and thinking about data, and different styles and types of data. So, if we said like, I'm a manufacturer. I have got maybe a product catalog. A product catalog of things I build and sell, or whatever. That's relatively stable. It changes maybe once a month or once a quarter, or something like that. It doesn't change that often.

Then I have heavy transactional data. These are the orders that people are making -- the POs, receipts, and invoices, and all that kind of stuff. That's just the work of the business, day in and day out.

If I could think differently about these kinds of data, like I could say "Product catalog -- relatively stable, it's reference data." And if you say, "Hey, I have this system I'm going to write. I'm getting all this transaction data, and because we're normalized here, I have pointers to product information, so I need this product information." I say, "Great. Replicate the product catalog to your database. You can join to it there. It's great." Because the last thing you want to do is write a join in your web service. [laughs]

Tom: Well, it's interesting that you would mention that. Right, a join in the web service sounds incredibly heavy. But, there are cases where, I think, your joins, and the way you're pulling information in can be optimized. Say, for example, loading in via those IDs and doing small lookups, and doing kind of lazy-loading approaches from your views gives you the ability, I think, to keep the services your main access point into the data. And this isn't necessarily... I mean, on the SOA side, it sounds an awful lot like we're talking just exclusively about data-oriented services. That's kind of a whole different topic to get into, whether or not these services are purely just on top of my ADO.NET transactions, and they're letting me just get in and out of databases.

I think that's probably the wrong thing to be thinking about. It's probably that these are a bit at a higher level, and you're able to extend those things, and do all kinds of other interesting operations through the services endpoint.

So joining across these different services, and pulling those things back up, I guess the counterpoint would be that the performance could be a major, major issue at some point if you're doing large, large join operations. How much of this data are you potentially consuming from a user-based application? There seems to be a lot of cases where you can, effectively, do joining through multiple service calls, and not really cause too much of a performance degradation.

Ron: OK. I guess if you said, "Well look, I'm joining three records." OK, fine.

Tom: Right, sure.

Ron: But if you say I'm joining 3,000 or 3,000,000, I'm going to get worried about what you're doing on their service.

Tom: Sure. But what was our purpose for pulling that kind of data, and doing a join across 3,000,000 records? If it's OLTP type systems, who's consuming that amount of information from a screen? Those are the kind of things that... Be pragmatic about what you truly need for your users to be able to interact with this data. And if it's smaller chunks of data and it's data that can be fetched in a lazy -- in a lazy pattern, then I think services are still a viable approach to gelling that information.

You know, make sure you can get they keys so you can do the lookups in an efficient manner, right? You maybe don't want to be doing huge kind of table scan searches across those two services. I can see that being a major problem. But, you know, maybe you just replicate keys, right? Maybe you just have ways to do lookups. And some of those keys you describe product catalogs.

I mean, there's basic kind of data entities that are able to be natural keys all over the place. all right, I mean, depending upon the vertical. So, retail's a good example. You've got SKUs, you've got things that are truly identity-based things that you can actually go and do lookups with, so. You know, I mean that kind of stuff for replication, also, I think is acceptable.

Ron: Well, and the one thing you have to keep in mind whenever you get in this kind of replicated environment for reference data, there's two things you have to keep in mind. One system has to be the source of truth, right? I mean, this is the main one, that one's a copy. OK, we know that. And everybody needs to know what that is.

And then secondly, you have to have versioning of the data. So this is the product catalog as of, you know, March 1st, 2007.

Tom: Sure.

Ron: And so, whatever you're dealing with, that you know which version of the reference that you're working against, because it will always be a little bit out of date from the source of truth.

Tom: This is if you did -- this is if you did start to replicate.

Ron: Yeah. Yeah. If you did start -

Tom: Which is the complexity that starts to kind of leak in to all the other systems that are being built using the replicated reference data. Right?

Ron: Yeah.

Tom: This is -- again, this is what I think causes that additional, that additional pain. And in big enterprises, in big companies, that pain is, I think, amplified. Because I think the replicas for that reference data, especially if it's kind of core data to your vertical, you might not see one or two replicas, you might see six, seven, eight -

Ron: Yeah, right.

Tom: And that's, that's a lot of versions. That's a lot of balls in your area you're trying to juggle. And I think in the end, just trying to figure those things out and contain those things, is going to -- is going to really, really overwhelm the maintenance cycles and how you're going to actually be able to build applications efficiently over time.

Ron: OK. So let's say somebody goes, oh my gosh, we have that. We've been doing all this. What's your advice to them? What do you tell them?

Tom: So I guess the first thing to look at is exactly how pervasive it is, is it the type of pattern that really comes in and it's the, it's exactly how your main dove leads, or your main, your main designers are trying to solve -- you know, they're trying to solve eighty percent of the problems by replicating the data. And then start, and start cutting into that.

Proof of concept work, I mean, you've got to do something, you've got to prove it, right? I mean, in the end, you can have all the theories on the planet, but if you don't sit down and write a proof of concept and show exactly that this join, this join issue we just described can actually be satisfied by going to the service and doing a secondary lookup, and you don't have to do all the work down in the database with a join to get an efficient user experience.

You know, if you can prove that, then you can get some traction. And then get some success stories. And then, you know, it's going to take time. So first of all, I guess, I guess it's kind of a search and destroy type of a -- type of a mentality.

Go out and find where it's at, you know, build kind of some good case studies around how you could actually start to migrate things away from it. And then absolutely, your new applications, stop doing those things. Right?

As soon as you discover it as an antipattern, stop doing those things within your applications unless it needs one of those very small subsets of anomalies that require it. And then as far as retro? I mean, that's always a pretty, that's kind of a, that's kind of a sticky issue.

I mean, it's usually very difficult to think about how you're going to go out there and actually get, you know, some type of an approval, or some type of a justification for. Complete retro in your enterprise. I mean, I don't see those things happen very often, so it's probably the next time you get back to that application, start thinking about maybe taking a couple of those major kind of batch replication jobs, and cut them off at the knees, and then finding more and then it's got to be a road map type of an effort. You got to take some time and just start to improve the health, as you start to look at your application portfolios and start building new stuff in your enterprise.

Ron: Yes, but you know, I can just hear these guys in the IT management saying, 'Hey, what are you talking about? We have been doing this replication stuff for a long time now. We know how to manage it. It's fine.'

I think when you are fighting those battles, you need some examples to point to and say, "You think it's fine, but what about this?" Or, "You have been experiencing this kind of pain. So what kind of..." You know in antipatterns, we often talk about symptoms. The patient is sick, they have got some symptoms. What are the symptoms of this sickness that you see in this replication gone amok? In my web service in C# code, how is that better?

Tom: So, I guess it really kind of limits the landscape of the change. We have different ways to build systems today. I mean, there are still a lot of fundamental best practices from building applications and laying out data and making sure the things perform and actually serve the right role in your application architecture. But now that you have got these really easy mechanisms for connecting all these kind of disparate heterogeneous systems, you are able to do a lot more from a lot of different things, right.

I mean, it's just there's ways to reuse this stuff a lot more effectively, I think today, and it's largely because of web services ACTP, Text Transports. There is a lot of ways to get into this data and reuse it. So why not centralize? Why not keep these things under tighter lines of control?

Ron: You know, when I think about the reasons why the changes are better in SOA, than they are in data replication, I think about it in terms of isolation, encapsulation and contracts. I mean, you think about, in data replication you are kind of exposed to just raw data.

Tom: Sure.

Ron: Right. And raw data meaning, apart from its meaning, its interpretation, any business logic related to it, you just kind of expose, "Here it is. Just raw data." Right.

Tom: Right.

Ron: And so, means you have to kind of understand all about that data that you are getting, somebody spewing at you and you have to know its meaning, and be able to interpret it. And really, we think about it, the interpretation of the data brings its meaning. Raw data on its own doesn't mean anything, right? But you interpret it.

So, now if I got misunderstandings about the semantics of the interpretation of data, if they crop up in different applications, if this app interprets in one way, that app interprets in another way, then we get into a lot of confusion. But if I have a service with a really clear contract, we add data plus interpretation, gives us the semantic meaning that you want, now we are protected.

Tom: Those are all great comments. I would definitely not have said that by myself. It is absolutely true. What you really want to be able to do is be very explicit with how that data is being worked on and exactly how it is changing. And when you start to let it leak all over the place, you just loose the ability to control and, of course, you loose the ability to adapt and change in version and do all those other nice things that you see. You have the capability of doing, when you start looking at web services and start looking at lining that up with your machine critical data.

Ron: Yes. You know, it seems to me, that, that's always been the goal that we wanted with integration, is lot of stability in the integration links. People are always wanting to change stuff at the same time, and it's like I heard somebody on my show once said that, it's like trying to change the wheels, trying to change the tire on a vehicle that is moving down the highway 60 miles an hour. You know it's silly. If you can just hold still for a minute, we can change this, but we do not have that luxury. It's amazing, though, the things we've learned to live with. I think that people are expecting more now.

Tom: I agree, absolutely. If you don't learn from your mistakes, you're doomed to repeat them, right? I know it's somewhat a clich. But it's so true in this space, especially when you start talking about the fact that these are very mature and accepted ways to deal with getting data to various systems.

You're right. They have been done. They are mature. There are a lot resources and a lot of technologists that know how to make those systems actually get built.

How do you know how to maintain it when you start having a bit more of a broader enterprise scope, to manageability and health of the architecture of your enterprise? I think that's the key in the long run.

Ron: Before you came in here I was listening to some oldies eighties music.

Tom: Right.

Ron: [laughter] 1982, man. That was the year I graduated from high school. That gives you a clue how old I am. I was reflecting back and thinking, "Wow, that was 25 years ago! Oh my gosh!" I was thinking about some of the data integration that we did when I first started in my career, and very often it was CSV files, big huge dumps, or fixed-length records. Often in the mainframes we'd be like "Here's a fixed-length record file." Somebody would say, "Here's the meaning of all this stuff and go write some..."

Tom: FTP it over there.

Ron: FTP, or you'd get an RPG program and spilt it all up, and do stuff with it. I was thinking, "Man. Yeah, that stuff worked," and many applications are still doing that today on a regular basis.

But we all know the pain of, "Man, this is so error prone. Just one little thing goes off." Very difficult to evolve and maintain, especially when you're the new guy and you come in and you get like all this crud. "Go decipher that COBOL program, and go figure what it's doing to this thing."

Tom: I do a lot of speaking, and I've talked about this topic with a lot of developers through the Southeast. It really does resonate with a lot of them. They really do see these patterns still used quite a bit, even though, I think the technologies have evolved to the point where you really do want to try to either fix the major issues with your legacy integration, or come up with strategies that can allow you to keep that stuff central.

I think that's what I keep hearing from a lot of people. In the bigger enterprises, it does seem to be amplified with other folks that I talk to, but I don't think it's the only place it applies. These kinds of patterns can start to creep up... Sorry, antipatterns. These kind of antipatterns creep up maybe without you even recognizing it.

Having standard kind of architectural strategies and review processes for seeing how applications are being delivered early is another great way to try to, at least, intercept this antipattern before it becomes... You're trying to figure out how to plan a major 18 month strategy effort to actually correct it.

So, there's probably a lot of it in the enterprise today. I think, maybe, as a final thought to really look at maybe some really good core case studies where you'll think you'll get a lot of bang for the buck by actually going in and rewriting and building services that are reused by the applications that actually currently copy that data.

Ron: One of the talks that I often give is called "Integration Patterns," which is a talk where I look at different styles of integration. Service oriented integration is just one of the ways. We've been talking about data integration really as a replication. I think one of the areas where I looked at data integration being valuable, you cross a certain threshold when the volume of data exceeds really what services are great at. You know, so if you said, well, I've got system A over here. It needs, you know, five million records from system B. And I'm going to call service five million times to get those records.

Tom: Yeah, that doesn't sound great.

Ron: Yeah.

Tom: [laughs]

Ron: This is a bad idea. But I think you could take some of the same ideas of contracts and encapsulation and leverage them in this kind of data replication. So if you thought about the data replication as being yet another service, it's just choosing -- it's not using -- you're just using it from protocol, but -

Tom: OK.

Ron: It's another service. And so I say, well, maybe I might want to, for example, denormalize some of the data that's going to be replicated out. Or I want to, you know, precalculate some stuff that adds more semantic meaning to the data that's going out. Or I'm going to push it into a temporary table before we replicate it or something so that it isolates it from changes in the rest of the data structure.

I think maybe those kind of things might ease the pain of the -- these kinds can conceptually think about -- this is yet another contract we're going to honor.

Tom: That's a fair statement. Yeah. I guess I don't want to come off as saying that every time you see any type of ETL process you've got a major problem.

Ron: Right.

Tom: I don't think that's -- I don't think that's -- I'm not being that bold about this. I think there's just a lot of cases where you should attempt to avoid it. I think the cases you brought up are very interesting, but they all sound like, I'm probably either preprocessing some type of an aggregation, or I'm doing something where I maybe am giving -- I'm giving some type of a code. Some type of a set of code written and preprocessed to, I guess, maybe get some performance benefit, potentially, or some other things that -- you know, I wonder if those things aren't better suited inside of the services themselves that are interacting with that data.

Right, so it's just, I think you want to be extremely pragmatic about pushing too many of those things into the replication models. Integration's a good -- I think data integration is an interesting case where you've got to really kind of transform the data in a format in which it has to seamlessly plug itself into some other third party tool. I see that all the time. I don't think you can get away from that.

But the one you mentioned about actually running aggregations ahead of time, I can make a case or take issue with that in some cases. It just depends on what that code's doing.

And now if you're running through, you know, again, your five million row example, and you're preprocessing something so that it can be reported against real time in a much more efficient manner, OK. But if it's logic that can actually be still efficiently run through the services themselves, via reporting services and things like that, then I wouldn't want to push it down too far.

Ron: OK. OK. So, you know, like all things architecture, there's a -- it's one of those -- there's no right answer.

Tom: No.

Ron: [chuckles] I always loved it when my professors would say, well, there's no right answer for this, they just want to know what you think. I loved that, you know? [chuckles]

Tom: Was that a philosophy class? [laughs]

Ron: [laughs] No, those were architecture classes. [laughs]

Tom: Boy, I guess the lines are blurring between the two, huh?

Ron: Yeah, that's true. Well, you know, I think the word you used is pragmatic, and that's really, I mean, what works. Let's just get to what works and that's what's great about our cast, we're always talking about things that work, things that don't, patterns, all type patterns, and so.

Tom, thanks so much for joining me today. I appreciate it. Thanks.

[music]

[cheering]

Ron: Hey, Tom Fuller from the SOA Pit Stop, and Tom has some good insight there into, you know, many of the ways in which we kind of put band-aids around these silos of information we've had for so long and kind of replicating, doing a lot of ETL sort of work.

And you know, sometimes it's time to just make the leap, you know? Do some service oriented kind of stuff. And there's -- but there's also times when it's not the right thing to do. And you know, there's no formula for this. This is why we need architects who are experienced people who have listened to ARCast, maybe, and have learned what other people are doing in what situations.

Hey, I hope you enjoyed that. I hope you enjoyed the ARCast Rapid Response. Send me a note to arcast@microsoft.com.

Announcer: ARCast Radio is a production of the Microsoft Architecture Strategy Team, www.arcast.tv.