Threading Question

I have a multithreaded TCP server that must rely on synchronous communication (blocking I/O). Each connected client to the server runs in its own separate thread. Each thread must be able to communicate with each other by sending “messages” to one another. Additionally, some events must occur at a timed interval (such as, if a client hasn’t authenticated itself in three minutes, it will be shown an error message and be disconnected.)

 

How I solved this in Delphi is to have the blocking read operation on each socket time out every 500 milliseconds and that’s when I’d check to see if any scheduled events were supposed to take place. I’d also check the client thread’s queue to see if any other threads left “messages” for that client and then process them. Then I’d block for 500 more milliseconds until some data arrives, and so on.

 

When porting this application to .NET, I am having some trouble because all of the socket blocking is done at a lower level. I can’t use the blocking operation iterations of the socket to check my queue or check for scheduled events. I’d much rather separate socket operations from my application logic anyway, but I need a strategy to allow me to both block socket operations and check my queue and timed events.

 

How do I go about this one? I could set the socket to have a receive timeout every 500ms, but then an exception is created if the socket times out. But I’m not timing out the socket to disconnect it, simply to interrupt it for a while so I can check my queue and do other stuff. So I’d loop again. That means potentially countless exceptions are created and caught per thread, and that’s not good for performance.

 

I tried peeking the socket, but that causes a noticeable delay to the client and it becomes difficult to tell when the socket has suddenly disconnected. This is due to the fact that I must call a Receive() eventually to tell if the socket’s still there, and that will block indefinitely.

 

What are my options? I don’t even really want to have a queue of messages at all, I’d much rather have them event driven this time so I don’t have to poll to check for messages. But this is complicated in a multithreaded application because events are handled by the same thread that triggered them. That particular problem was easy to solve in a GUI application with Control.Invoke(), but this is not a GUI application.

 

Any ideas?

Here is one idea:

With the socket create a new instance of the NetworkStream class. The NetworkStream object will have a BeginRead method, this method returns an object implementing IAsyncResult.

IAsyncResult is interesting because you can use it to poll if an operation is complete (check the IsCompleted property), or you can use it to block (with a timeout) until an operation completes (call WaitOne on the AsyncWaitHandle property and pass a TimeSpan).


The timeout is not on the operation (the socket read), it is on the wait operation. The underlying read will either complete, or the thread will unblock and you'll have a return value indicating the wait timed out.

All of this follows a common asynchronous programming design pattern in .net, so you should find lots of examples in MSDN and elsewhere. For starter:

MSDN: Asynchronous Programming Design Pattern

To make a complete clone of your Delphi application use Socket.Poll Method instead of Recieve.

if(mySocket.Poll(500,SelectMode.SelectRead)) {
  // Process read
} else {
  // Process events
}

But I may suggest you to search for way to not wait 500 ms to process events. Use Async as proposed - but without timeouts - wait for one of events raised - data received or some wait handle on events raised.

I.e. create 
WaitHadle eventPending = new AutoResetEvent(false);
Use BeginRead() to get IAsyncResult. Obtain AsyncWaitHandle from result and then use WaitHandle.WaitAny() on those two WaitHandle's.

This way your code will get data from socket ASAP as Async callback will notify you and also you will process events ASAP as another thread will raise your eventPending handle.

You can wrap all this logic in one class and reuse it easily )

P.S> I'm unsure - but there can more easy way to obtain WaitHandle from Socket - but this it's unknown to me. 

I like these ideas, and thank you guys so much for participating.

I have some questions though. Each client is already running in its own thread. Since sessions can last a long time and some other intensive operations such as file I/O can happen in a client's thread, it's probably best to keep them this way.

But if I use Async, I'd be using even more threads per client for reading and writing operations. Plus I might have to make many many calls for reading from the socket (unfortunately this is a remote control telnet server, so I have to get one character at a time from each client).

So my question is, is Async (1) costing me too many threads in this case, or (2) too intensive and costly since it'd be used VERY frequently?

Or would it be ok to implement?

I don't mind if Async programming is tough in my situation, I just want to know if it's good practice to pursue it in this scenario. Would it be too CPU/resource costly?

It would be nice to have a more logical way of dealing with events with the async model though.

And you guys are gigantic teases. The worst flirt of all is Avalon. Avalon will be scalable, it will give you beauty, it will give you flexibility, it will micromanage typography, and it will turn water into wine without you asking!

But in the meantime use sizing algorithms, Win32 programming, and manually handle Windows messages to reduce flickering and solve complex control painting problems.

Yeah, thanks! >:o

But I do appreciate your work and I'm glad the calvary is on the way. I've been very impressed with .NET so far and I'm anxious to see WinFX.

You have to compare Async performance vs. waking up threads each 500ms for no reason.

Overall Async must be better for CPU - as you wake up your threads to process real data only and keep them waiting while there is nothing to process.
Consider user does not input anything and there is no any or only a few events waiting for him.
You will wake up thread with no reason or wait too long to send him event he is waiting.

But Async will add minor processing latency to Socket inputs. Also you must take care on synchronization and memory caching issues for SMP.

You have to decide yourself if Async processing overhead will cover 500ms latency for events and waking up threads.

You have options. There is no definite answer.

Sockets: Async or non-blocking I/O?

P.S> It will be good if you remove all your threads and use also async IO for file I/O. In case of completely Async processing you will get maximum CPU performance.

AT wrote:

P.S> It will be good if you remove all your threads and use also async IO for file I/O. In case of completely Async processing you will get maximum CPU performance.

I would love to do that, and I will definitely aim for that in another server project that is more asynchronous in nature.

But I cringe at the logic necessary for writing a telnet server where there are prompts that must wait for user input and user input is processed key by key and telnet option codes must be negotiated in the correct order in a specific time frame.

Plus, of course, there's also the timed events that must occur at specific intervals.

The logic of implementing this in a telnet server is making my head hurt already. But I will definitely try it for other servers I'm writing.

Thanks for all the great information by the way.. All of you! I appreciate it a lot.