Channel 9 Forums - Tech Off - Queue - Processing In Order

Tech Off - Queue - Processing In Order

MetaGunny — Tue, 04 Sep 2007 19:02:37 GMT

Okay. I have a question about the method that I was going to use to process items in a queue in order.

Let's say I have a queue, with a bunch of records, doesn't matt of what. The main point is that each queue, let's say, belongs to a certain account, so let's say we have an AccountID with each queue item.

It may look like this:

Account ID, Record
543, 1
123, 2
543, 3
543, 4

Okay, now, the main criteria here is that I don't want to process record 4 for account id 543 before record 1 is processed for account id 543.

Basically, to keep it simple, let's just say I'm grabbing all the records and then throwing them in a thread pool.

Here is how I was going to do the processing:

1) The first method I was going to do is when I'm looping through the records and throwing them in the thread pool, is on each iteration only throw in unique account id's. So, on the first iterationof the above data, I would only throw records 1 and 2 in. The idea is that if I grab 1000 records at a time, hopefully this will help prevent locking with the second method I use.

2). The second method I use is when I go to process the record, I was going to use SyncLock thread locking method to grab an object for that specific account id. So, when the the next thread comes to process that account id, it would wait until that object is released.

What do you guys think? Or, what is a better method?

Tech Off - Queue - Processing In Order

DoomBringer — Tue, 04 Sep 2007 22:55:07 GMT

I'd make a collection of collections, sorting each record into a sorted list based on the UID. You could use the UID as a hash to lookup the collection for the records.

Tech Off - Queue - Processing In Order

zhuo — Wed, 05 Sep 2007 16:29:42 GMT

I am not sure if I've understood your question.

If you have a queue and presumably that queue is built up chronologically, why would record 1 ever get processed after record 4?

Tech Off - Queue - Processing In Order

DoomBringer — Wed, 05 Sep 2007 23:14:52 GMT

zhuo wrote:

I am not sure if I've understood your question.

If you have a queue and presumably that queue is built up chronologically, why would record 1 ever get processed after record 4?

I think he wants to separate the records by user ID so a single thread can handle a single user ID, and he wants to preserve the ordering.

Tech Off - Queue - Processing In Order

MetaGunny — Thu, 06 Sep 2007 00:23:41 GMT

Thanks Doom. I'll give that a try. The only problem some what is I don't know the user id in the beginning but have to do some processing as well to get that.

They could process out of order because I'm not doing the processing on a single thread, sequentially. I process through each and throw it into a thread pool