ARCast - Architecting CommSee (Bonus Video)

Description

CommSee is a simply amazing application written by the team at CBA. This application is visually stunning, architecturally interesting and on top of that it has delivered solid business value to the bank stakeholders, users and customers. Wouldn't you love to be a part of a project like that? After hearing about it I decided I had to learn more so I spent a day chatting with the team.

Links

-Ron

Transcript

Speaker

Text

Start Time

Ron Jacobs

Well, this is Ron Jacobs and I am here at Commonwealth Bank of Australia, where I am joined by the team who is responsible for kind of putting this whole thing together. .. and  guys I am delighted to chat with you today about the CommSee Project. And just so that we can get this off the table to begin with, what does CommSee mean? What does that  stand for anyway?

0:00:16.52

Stuart Johnson

First part Comm is Commonwealth Bank and last part See is ‘Service Excellence Everyday’

0:00:39.04

Ron Jacobs

Ah! OK. It’s really a project that’s focused on increasing your customer service aspect of the bank. [Stuart: Absolutely]Alright! So tell me a little bit about the history. I mean, where did this whole project begin? Where did it come from?

0:00:44.26

Stuart Johnson

OK. Back in 2004 we had an existing VB application that was servicing the premium part of the business. Back then it was…we had to build an application separately to the rest of the bank in that we had a relationship model. Our business was focused on knowing more about customers and their total holdings with the bank. The previous products in the bank had focus more on an account basis, So that’s where our initial project was.

0:00:57.48

Stuart Johnson

Back in 2004, the bank made a decision that that concept was going to be rolled out across the whole bank. So the initial architecture was, like I said before, a VB application that is predominantly batch focus. So we are getting feeds from backend, mainframes systems putting into our backend database. We only had a small number of users,  so that was OK. It didn’t matter if the balance was a day out of date . We had one or two online services, but it was predominantly batch.

0:01:31.27

Stuart Johnson

Once the decision was to roll this across the bank, the whole world changed for us. We had to switch to more online, integrated application. We end… we are going from couple of thousand users to now we have up to 30,000 users. So… we then have to rethink how we are going to do the architecture and that was early 2004.

0:01:59.77

Ron Jacobs

So one of the things, I think is interesting about this is the comment you made that, previously the bank kind of thought of accounts [Stuart: Yeah] and now you are switching to thinking about people who have accounts, who might have lots of  different accounts through different parts of the business that may be these accounts live on different systems and that need to be integrated together?

0:02:22.83

Stuart Johnson

And that was the reason the bank launched a project called, ‘Which New Bank’ and it was about one of the CommSee was a part of that and it was about a single view of customer.

0:02:45.04

Ron Jacobs

Well, I think that’s really important and a very subtle distinction yet when you think about, from the customer’s point of view, They don’t want to come in and deal with somebody who only knows part of their of story they want to deal with their… all of their accounts, they think of their world in that way, why shouldn’t you?

0:02:58.35

Ron Jacobs

You know but let’s move on to think a little bit more about the scope. You mentioned going from 2,000 roughly user to about 30,000 and I imagine a then lot of locations as well.

0:03:16.80

Stuart Johnson

We have over 1100 branches all around Australia. And the original scope was actually less than 30000… the original scope was to roll out um… replace the branch network. Since that time we have… it’s now then rolled across all the call centers. So it’s up to about 1700 sites now. At any one time we have 18,000 connected users.

0:03:28.53

Ron Jacobs

Wow! So guys when you were architecting the system, there probably were lot of decisions you had to make up-front about how you are gonna connect all these users in these vast different locations  and make it work with reasonable performance, what kind of things did you think about early on?

0:03:50.33

Edward Gallimore

We had a very slow network initially. A lot of our bank branches even had sort of 64K link through to the bank branches and there were probably 5 people working on a connection. So it was very important that we had a very efficient communication between client and the server. We subsequently increased the size of the pipe but we still had to ensure that we were very miserly about how we communicated up the wire.

0:04:10.05

Dan Green

[Ron: Yeah] And I think that ties in also with the choice of a rich client application.  it  enabled us to think about how we could potentially cache data to help improve responsiveness, it also enabled us to think about certain techniques for designing front end to allow the drip feeding of information, so that, you know, our users could see certain parts of a customer’s information before other parts might have been available whether that be because the network was running slowly at that time or whether it be that certain systems we were integrating with were slower than others, by designing UI in a certain way we could keep that perception of responsiveness even when networks might be tight or backend systems might be slow.

0:04:38.02

Ron Jacobs

Now, you’ve kind of come to the first big fork in the decision tree of the architecture that lot of people will make in an application like this which is, Should it be a web application or a smart client application and a lot of people are, to tell you the truth, a lot of people sort of default to web application, because it seems like its simpler, no deployment and these kind of things, yet you opted for smart client. Why did you choose that?

0:05:25.82

Edward Gallimore

We control the desktop. So the big down side of a smart client traditionally have been the deployment issue. But given that we do control the desktop, given that we have environment where we can push out the client, that wasn’t such an issue. So we tended towards a smart client implementation. Because it gave us the benefits of the caching, it gave us so much richer client experience.

0:05:52.25

Dan Green

It gave us the ability to do visualizations of data that can be very difficult to do on the web. [Ron: Yeah] That something we definitely exploited in certain parts of the application Then we are sending less data down the wire because the GUI is already there. We are just sending small packets of data.

0:06:15.99

Edward Gallimore

It can also tend to be a nicer development model as well, in terms of not having considerable code goes in the browser and what code goes in the server so…

0:06:32.13

Ron Jacobs

So, did you also have in mind… um… an offline functionality, so if for some reason the network was down that the branch could function, people could function offline? Does that work?

0:06:39.74

Edward

It was considered. Some of the predecessor code, which was prior to CommSee, did perform in that way. But that was in time when the network was very unreliable. The network to the bank branch is now comparatively reliable so we don’t need it there. The other place where it would have been advantageous was for the relationship manager. But it was deemed that the cost of developing an environment, which worked in a disconnected manner wasn’t justified relative to a GPRS sort of mobile type solution.

0:06:53.42

Ron Jacobs

OK. Well that’s a reasonable thing to do. I mean, you did analysis, you said “Well It would be nice but it’s not worth the cost.”

0:07:25.08

Dan Green

Yeah. And it certainly, I think, continues to be considered. There are parts of the application where real time nature is just inherent in requirement. But for other parts of the application where it might just be form entry onsite with a customer, if you are originating a loan or something like that, consideration is  still being given to potential of embedding offline capabilities in the application, you know in storing that data and forwarding it on to our back end systems when they connect up to the network. We know we have that potential.

0:07:32.15

Ron Jacobs

OK. So if we are thinking about the services that you guys decided to. Well, first off, you decided to use a service oriented architecture type of approach with this. Um… how did you think about creating the services, from a tech…? Let’s just begin with the technology. What did you do?

0:08:04.57

Stuart Johnson

OK. So originally in the old days when we had the VB model, we had some web services at that time. But what we tended to build was, we built more of an application that happen to expose web services. When we were moving to the new model, we knew we had to support a lot more developers and a much more agile, flexible approach. So we initially sat down and went through requirements about what we needed our service framework to do.

0:08:24.44

Stuart Johnson

So we kind of listed the requirements we thought, initially we thought we gonna build this or… Because there wasn’t anything in the marketplace at that time that we could go and buy. We happened to come across the work that was being done in the patterns group, EDRA, And we did an evaluation and we felt that it did about 80% or more of what we thought we needed.

0:08:51.72

Stuart Johnson

We took that as a base and we built on top of that. And that’s being largely… very successful and one of the contributors to the success of the project. Once… we what we… one of the reasons of the success is once we had a stable framework for services, we haven’t then built, we haven’t built it out to become a bloated framework. It’s kept… it stayed largely as it is for last two years.

0:09:13.54

Ron Jacobs

Now, you went with web services as really the backbone of the way that the client is communicating with the backends. A lot of people have said, “Ah! You know, web services’ performance is really going to be a concern.” Did you find that to be the case or do they perform adequately for you?

0:09:41.28

Dan Green

One of the things people might talk about with regards to web services and performance is the bloated potential for XML. So, we have certainly built some SOAP extensions and through the various interception points are exposed to you in the web services pipeline, we’ve got some compression algorithms, for example in there to help us out on that front. So in terms of the performance impact based on the bloat of  XML the web services might result in, we haven’t necessarily seen that. We have put it through the perf tests and we are getting the results that we need.

0:09:59.80

Dan Green

Coming back to the discussion before we got into services, I think it is also interesting to look at how we actually partitioned the application up and internally we talk about the idea of having private services and public services and so our rich client tier talks to our application servers through a layer we call private web service, private web services, I should say. Because when you talk about Service Oriented Architecture and web services, there are various tenets you want to follow. One of them, you know, is contract first and how do you design your contracts with those services.

0:10:37.10

Dan Green

Now between our rich client and our application server we can only have the same developers working on that functionality, so we want to enable them to be as productive as possible. And therefore we don’t want necessarily to constrain them in terms of how they design those services to may be support other consumers. They’re going to be the only consumer of that service. So private web services are really designed to support a form or a screen. Or in our terminology, a WinPart. And so that flexibility is with the developer there. They don’t have to get to enterprise focused in how they design those services. So it’s just like a normal NTier app before we were using DCOM in the past or whatever. It wouldn’t necessarily have to get to worked up about the contract that they come up with. It’s the same thing now. 

0:11:13.22

Dan Green

They can design it to support themselves. But then at the application server tier, that code is then going to talk to backend systems or to a database for example. And when it talks to backend systems, that’s  when we plug-in to the services Stuart was talking about before. That has  gone through this process of the service oriented architecture approach making sure that the security is right at that level, making sure that the way that the contracts are designed, the APIs support an enterprise focus.

0:11:55.33

Dan Green

So we actually have these different approaches. A rich client talks to an application server through a web service that’s generally the domain of a particular developer and they can design that to suit themselves. There is some prescription, prescriptive areas that we force them to follow through frameworks and code generators. But then, when they want to talk to back end systems, they go through these public services that uses the EDRA framework etc that Stuart just spoken about.

0:12:23.92

Ron Jacobs

OK. Now we are joined by Jon, who is a, who has been working on the data tier of the environment here. So John, tell me little bit about what’s the data environment look like in this application.

0:12:51.81

Jon Waldron

Um… part of the whole problem we are trying to solve with respect to data is being sourced from many backend systems, pretty much all over the place and that’s the legacy of a having an organization that’s been around for a hundred years with the systems built in the sixties and all that sort of stuff [Ron: So we have some file cabinet somewhere… and that sort of things] Oh we got all sorts of things.[Laugh]

0:13:02.96

Jon Waldron

So and there’s multiple ways to deal with that problem and the decision that we made was, we were pretty much going to leave the backend system where they are and augment them with their own database. Rather than try and take off, a sort of, bite of a big chunk and try and reorganize all of that and put that into new open system modern sort of database. Or anything of that, we will do that iteratively as time goes by. But we were augmenting that, with their own database. So we’ve got a mainframe, which basically calls the customer information file along with various product systems, which as… as Stuart and Ed were talking about before, the product systems are focused on an account by account basis product like a credit card system or demand deposit account system, very much very a siloed based kind of data model. And then also scattered around the bank elsewhere you will find out this sort of systems as well too.

0:13:23.28

Jon Waldron

So, we basically decided to leave that as they were and we’re going to using the web services, sort of Service Oriented Architecture, just pull in the data as we needed and augment with our own data. So our database that we built as part of CommSee, it includes a bit of sort of glue type data to hold only sort of identifiers together, relationship information, a lot of what you call customer soft data, things like your interactions and your tasks and bit of workflow stuff and then another component all together, which was the systems being used for, is product origination as well too. So, It’s now storing a lot of the information for the new home loan applications and personal lending and credit card applications etc.

0:14:12.73

Jon Waldron

So, it’s become equally as important as any of the other backend systems, the mainframes and anything else we have, and it’s there all kind of used together, so the whole thing needs to hang up together for everything to work together, but we have certainly got ways of making them more resilient and more redundant as well too.

0:14:52.46

Ron Jacobs

So you know it’s interesting that you mentioned, you have this problem whenever you bring legacy environments together, that they don’t necessarily store something you need to store and so you have to introduce this other database, like you mentioned, to bring it together. The other problem that we often see is the problem of, kind of, identifiers that link together things, maybe a customer is referred to with one ID in one system but an entirely different ID in that system so you have the… that was the glue that you were talking about.

0:15:09.05

Jon Waldron

That’s exactly right. We’ve got at least, off of the top of my head I can think of, at least  4 or 5 different types of customer identifiers in other backend systems and in our database we link all those together, so we try and keep it together so we can find everything about the customer in one query.

0:15:39.48

Ron Jacobs

And did you have that ,a lot of people have this problem where, the are not even sure which customers are the same ones between the systems, you know, and the address looks the same but the names slightly different and that sort of thing

0:15:54.09

Jon Waldron

Yes. So we had that happening all the time, so first of all from the mainframe every night we get a nightly feed of customers which are merged together because they were found to be actually the same even though they are reported as two different entities but they have been merged, so we’ve some batch process going on that cleans all that up and then intraday as well too whenever a user actually comes across a client that really is actually in the system twice for whatever reasons that they can actually put an entry into a queue and say overnight could you go and clean this up please and actually merge all together overnight.

0:16:06.53

Ron Jacobs

Ah! That’s fantastic. Now, when we think about the Service Oriented Architecture in the constraints of the environment you guys have mentioned, you mentioned caching of data down at the client side, how did you think about kind of efficiently getting this data over to the client side only what’s necessary so that you are not clogging up these very restricted pipes with a lot of unnecessary push of data? How did you do that?

0:16:36.20

Edward Gallimore

The services are built as generic services, which sort of abstractly bring back all client information and the private service system, Dan was just talking to, are built to basically service the GUI component. So we have a component that just shows name and address there will be a private service which sheerly returns name and address back to the client, that private service may in turn, may be calling a public service which brings back the entire client information, but it’s filtered and constrained down across the wire.

0:17:03.40

Ron Jacobs

OK. So at the application server level, really the thin pipe, if you will, is between the application server and the client.

0:17:32.29

Dan Green

Absolutely. It does. Services are designed to make life easy for a developer and how they design things but it also optimize in terms of there is not really redundant data being passed down the wire.

0:17:40.13

Jon Waldron

So there is also a very close correlation between each of these private services all way down to the database to an actual stored procedure of the database who returns data and is very much what you see on the screen is what’s been got from the database we’re not… the way mid tier aggregation perhaps brings things from other data sources as well too. But we only really get back the data we need to service just that particular component.

0:17:51.04

Ron Jacobs

OK. But I am curious though still about the caching aspect.

0:18:11.40

Dan Green

So. On that front each of our private web services, as we call them, can be flagged to be the response within to be cached. So if you are retrieving just salutations what titles Mr.,  Mrs. etc. for some, you don’t have to retrieve that every single time. So we have a web service designed for certain reference data. We have variety of services designed to support reference data and we flag those as being cached. And then on the client you get the response back from the web service, and then within our infrastructure, if you will, like the proxies we call them service agents. But the service agents have the smart to short circuit the request to the application server for those services that have been flagged to be cached. Then obviously its complex with any caching mechanism the complexity lies in when you invalidate the cache, so we have smarts in there about these to set it to invalidate after certain amount of time, or do you persist it to disk.  Those are some of the challenges you face with caching.

0:18:14.02

Ron Jacobs

And do you use a database at the client side as well or files or what?

0:19:11.50

Dan Green

We don’t,what happens is, our services response cache is persisted to the disk. They are encrypted and compressed.

0:19:16.57

Ron Jacobs

Yeah, OK, Well so that sounds fantastic and then now there is, then kind of account transaction sort of data. Are you caching that at all or for some period of time or not at all?

0:19:21.48

Dan Green

Generally not. No if it’s a user specific data we don’t generally cache it because the invalidation of the cache then can become extremely tricky. If you start caching user specific data or I should say customer specific data and the user interacting with the application were to change that data and then move to another screen or something like that, is that other screen going to use the cached version or is it going to go to the source and get… some of there’s complexity there. So no, for user data we generally retrieve that.

0:19:34.70

Ron Jacobs

But the performance you’re getting is good enough, so you’re not worried about.

0:20:07.67

Dan Green

Indeed and again, I mean, some of that performance that we are getting is because of the way our screens are designed to be partitioned up into areas. And so we can drip feed information down so you might be waiting, we have our equivalent of an hourglass, it’s a little fancier, but you can be waiting for a certain part of data in one quadrant of the screen while the others are completing. So that perception of responsiveness is important there as well.

0:20:11.36

Ron Jacobs

So you guys have really, for the user interface, you’ve gone with a composite style interface, it’s very similar to what the patterns and practices team built later, the Composite UI Application Block, which is actually based on some of the work you did. Which I love that model, right, because in… in… we both had a little interchange here that we gave some ideas for your framework that you built the services with and you guys likewise became the seed for an idea that we later put into this application block. But the idea of these little parts that kind of independently function so that like you were saying we can feed this one and while this one is populating this one is doing something useful so you don’t end up with that kind of very serialized wait, now I can do something. Right? [Dan: That’s right. Yeah.]

0:20:36.33

Edward Gallimore

The core tenet of the architecture is WinPart. It… what’s called the SmartPart in the CAB architecture. And it does allow that sort of atomicity of the components. So if you do have three or four WinParts on the screen WinParts can independently go away and get information. They can be independently responsible for saving back the information. If one of the WinParts throws an exception or errors in any way it doesn’t impact on any other WinParts. So as well as giving that parallelism of the data it gives us certain parallelism of development as well. So developers can be independently working on WinParts on a single screen and the WinParts can then potentially be redeployed on other screens.

0:21:28.35

Dan Green

Yeah, that’s an important point that Ed raises there regarding one of the things we discussed upfront was various dimensions of isolation that we wanted to achieve, we knew we gonna be scaling up to a large development team, a 150 or so devs. And there were discussions therefore around do we come up with common scheme? Is a customer a customer everywhere? So the problems with isolation that can bring that someone wants to add an attribute to that object or change or split something and then you have got the coupling that you need to deal with.

0:22:06.16

Dan Green

So what Ed was just talking about in terms of the WinParts achieving that isolation is important from the developer perspective as well in the… the… we retrieve the data from the backend systems through public web services or through a database and aggregate that into a data transfer object we happen to use the derivitive of the dataset and then pass that up to our front end.

0:22:38.43

Dan Green

We… So the scoping of these objects is really at a WinPart level. We haven’t tried to build a one size fits all domain model for our application where you then get into those arguments across the development team and if they want to change something and had he performed impact analysis on what that change might have. So the WinPart model also has affected the way we’ve designed our codebase.

0:23:03.48

Ron Jacobs

So recognizing that a project like this is never really DONE [laughs], but it is in production today…

0:23:28.69

Dan Green

And that’s the key point. On that like the methodology we’ve stuck with stringent 3 month release cycle which has been I think critical part of the success the business has recognized the requirements are never always correct up front, so the sooner you can release something, determine you got that requirement right, do we need to tweak it or not the better.

0:23:37.14

Dan Green

And we knew that up front as well that there will be on that 3 month cycle so that talks to our approach towards instrumentation for example we recognized that this was not going to be zero defect software. But that didn’t make sense. That economically didn’t make sense to the business to be waiting until we have zero defects before we released, I think it made sense to release on the three month mark. Obviously testing is critical and you want to have as fewer defects as possible. We are doing our best on that front.

0:24:05.42

Dan Green

But we knew we are gonna have defects in production so we built an instrumentation piece, baked that in spent fair bit of time on that. Because we knew our developers are going to need roadmaps to fix problems when the occurred in production.

0:24:31.85

Dan Green

And accompanying I guess that instrumentation is also our model about how do you update the application, how do you patch it when you do discover problems and you determine they are critical have you then fixed them and rolled them out. It also talks to a strategy we have I guess about piloting our application. We have this three month cycle and every time we do a release we pick the target of our users and we deploy the new version to those while the existing users continue to use the old versionso we had to bake into the way we do certain things, certain code and techniques to help us that backwards compatibility, the ability to run side by side if you will.

0:24:42.61

Ron Jacobs

So I am just curious though, what has the result been? It’s been working? Are users happy? Is management happy? What’s happening?

0:25:19.62

 Jon Waldron

It’s working very, very well and the general feedback has been very positive and I think as Dan eluded to, it has required a little bit of cultural shift from the whole organization for software that’s as you said never really finished, you keep releasing every quarter, it’s not a big turnkey solution that you got all finished now and with that in mind I think it’s been incredibly successful and users are getting used to that and it’s been doing very, very well.

0:25:28.59

Ron Jacobs

Well guys it sounds terrific that it has been a great success, so thanks so much for joining me today. [All: Thanks Ron. Thanks]

0:25:53.62

 

[Music]

0:25:59.15

The Discussion

  • User profile image
    colinjack

    Without a coherent domain model and a focus on reuse I wonder how the team are going to avoid the system becoming unmaintainable overt the years.

    Might be worth doing a follow up ArcCast at some stage to see how the project has progressed?

    Anyway it was interesting (as always).

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.