I've gotten several requests recently from developers wondering how to add multithreading to AsyncSocket and/or the Cocoa HTTP Server. So I whipped up a quick version of the cocoa http server that demonstrates how by splitting incoming connections between multiple threads.
You can download the MultiThreadedHTTPServer from here.
A quick explanation of the code:
Anytime AsyncSocket accepts a new connection it will fire 2 delegate methods:
onSocket:didAcceptNewSocket:
onSocket:wantsRunLoopForNewSocket:
So all one has to do is implement the these delegates, and return the run loop of a background thread in the second delegate method. I'm sure there are various ways to do this. Here's how I architected it in the sample code:
When the server first starts, it splits off multiple background threads. Each thread stores a reference to its run loop in an array, and then starts the run loop, and runs it indefinitely. This array can then be consulted to obtain a reference to the background thread/runloop.
But that was a bit too easy, so I made it a little smarter.
In addition to keeping an array of run loop references, the code also keeps track of how many connections each thread/runloop currently has attached to it. So when a new connection comes it, the code chooses the run loop with the smallest load. So it does a wee bit of load balancing.
The number of background threads is set in a "#define" statement. It can be adjusted as desired.
9 comments:
any quick tips on how to best achieve the same thing in the c# version? i tried simply spinning up a new thread in the DidAccept handler and doing my Reads/Writes in that thread, but it doesnt seem to work properly (calls to Thread.Sleep still block other socket reads). i tried it with AllowMultithreadedCallbacks set to true and false, but that didnt really help.
got any advice?
Hi Brian,
The C# AsyncSocket and Cocoa AsyncSocket are different beasts. Let me explain:
In Cocoa, socket stuff is handled in the run loop. You can start a run loop in any thread, and one is started for you automatically in the main thread. A run loop just basically has a list of "sources" that it waits on. When something happens on one of the run loop's sources, it wakes up, processes the event, and then continues waiting. So a mouse or keyboard could be a source, a timer, file IO, socket IO, etc. The Cocoa version of AsyncSocket just adds the underlying socket to a run loop, and then waits for the run loop to notify it when socket stuff happens, such as data being available, or an incoming connection waiting to be accepted. This actually makes things very easy because everything is happening in a single thread. No need to worry about all those crazy threading issues such as deadlock and non-atomic operations. And the run loop can handle many sockets at the same time with ease. So it's generally no problem for a server to be single threaded. However, if the server performs lengthy operations for clients, or needs to handle thousands of clients, then breaking off multiple threads will help a lot.
C#/.Net operates very differently. When it comes to sockets, you basically have three options: blocking, non-blocking, and asynchronous sockets. One wouldn't want to use blocking sockets in a server because you'd need to create a thread for every connection. And creating threads is very taxing. As for non-blocking sockets... well as a server you'll have to eventually block waiting for incoming data. There's always the select() method, but it becomes complicated managing hundreds of clients like this. And besides, Microsoft has already done all the hard work for us by creating the asynchronous BeingMethod()/EndMethod() functions. When you call Socket.BeginReceive(), it actually does it's thing on a background thread - and it uses the thread pool to make it super fast. And I'm not talking about the worker threads in the thread pool. It uses the IO threads in the thread pool, and you can get thousands of these. Long before you run out of these threads you'll run into other big problems with memory and cpu. Microsoft worked hard designing these asynchronous methods, they created them for a reason, they work great, and they want us to use them. And this is exactly what AsyncSocket does :)
So to get back to your question, the C# version is already multithreaded. It's massively multithreaded in fact, and we were diligent in writing it this way. If you're using DotnetAsyncSocket to write a server, all you have to do is enable multithreaded callbacks. And of course, you'll need to write your server in a thread safe manner.
By the way, are you writing an HTTP server? If so, I may be able to offer more help as I've also ported the CocoaHTTPServer to C#.
thanks for taking the time to respond in such detail.
when i saw that the AsyncSocket was using the Begin* & End* methods, i assumed it was already 'multi-threaded' as well. however, during some simple tests, i would simulate a long-running operation by doing a Thread.Sleep() downstream from handling the DidRead event. if multiple connections were receieved while the thread was sleeping, the DidAccept method would fire for each as expected, but subsequent reads would wait until the original (sleeping thread) was done.
i was only investigating spinning up new threads in an attempt to fix this issue (although it still didnt make sense to me as to why it was necessary for the reasons you pointed out). i guess i will go back to the drawing board and try to figure out what is going wrong.
(and unfortunately, this is not for an HTTP server. it is for a custom TCP-based protocol, although the mechanics of the server would be the same i suppose, just different data to parse).
i dont know why i had such a brain malfunction, but when i ready your comment of "And of course, you'll need to write your server in a thread safe manner." i realized my error.
the AsyncSocket class provides the asynchronous operations during the actual read and write of data, but i was simulating a long operation *after* the read, but *before* the write. so essentially, my server had a single-threaded bottleneck. i changed a couple of lines of code and it works great now. sorry for wasting your time with stupid programming questions. (although the description of the how the Cocoa stuff was very helpful since i am not very familiar with it).
thanks again for the assistance, and for the great component.
Is there a mailing list or discussion group for AsyncSocket. ? I have searched but can't find one.
We just setup two mailing lists:
One for discussing CocoaAsyncSocket, and another for being notified of subversion commits.
Howdy,
There is a bug in the multithreaded HTTP Server. The server creates all those threads but it never cleans them up. When the threads are created they each retain the server object and unless they are released they won't release the server object. Code like this:
HTTPServer* server = [[HTTPServer alloc] init];
[server release];
results in a bunch of threads still existing and the server itself also not being dealloced.
The code is just an example of how to do it, and it's meant to run the server Indefinitely, until the application is terminted.
I need a server that the user can turn on and off even multiple times. Do you recommend that I use a different version of the project? Will the other versions, Secure, Passwd, Simple work well like that?
Alternatively, how would you go about cleaning up the threads?
Post a Comment