C# Asynchronous sockets revisited

Three years ago I wrote a quick code snippet explaining asynchronous sockets. Recently someone found a rather large bug in that code so I decided to rewrite the tutorial (it’s one of my more popular articles and people have long been asking for the source code, which I lost a few years ago. So this way I kill a bug and give people the source code all at once).

I would like to thank stackoverflow user Polity for notifying me of the problems.

The client

Let’s first start with the client. For this example I created a small console application. It reads lines from the console until it sees an empty line. The text is then converted to a byte[] using the UTF8-Encoding schema.  Meanwhile I create a socket and call the BeginConnect method to start the process of creating an asynchronous connection. Asynchronous in this regards means that the entire program can keep crushing while a separate thread setups up the connection and sends the data. To guide the setup and the sending of the data while the program is running several callbacks are used. One to finish the connection setup and one to send the actual data. Before the actual message is sent I first send 4 bytes to tell how long the message is going to be. (Not doing this caused a subtle bug in the previous version which assumed a message was done once the receive buffer of the server was not entirely filled).

The server

The server is also simple. There is one socket continuously accepting new connections. So in contrary to the previous version this version accepts multiple concurrent connections. Once a connection has been made a separate thread is spawned, implicitly, by using the asynchronous BeginAccept method. In a callback the connecting is accepted using a new socket so the server can keep listing for more connections on the old socket. For each connection a small state object is kept to do the bookkeeping. It stores the active socket, the buffer, and fields  with information about the amount of data we expect to receive and how much data we have indeed received. In the read callback the first 4 bytes are extracted so that we know how much data to accept. After that we keep using BeginReceive until all data has made it’s way to the server. Only then we reconstruct the message.

Note that I use the TCP protocol for the sockets so I know that the data will be complete and in-order.

The source code

I’ve created a Visual Studio 2012 solution with the above two projects in them. Everything is compiled using the .NET 4.5 framework. The example will probably also compile in .NET3.5 but I noticed some methods that I previously used were deprecated  in .NET4.5, so I’ve used alternatives that might no be available in .NET3.5 (should be trivial to fix).

You can find the source code here

09
Sep 2012
CATEGORY

Blog, Programming

COMMENTS 5 Comments

Computermanagement software

Lately I’ve been bussy working at the Rijksuniversiteit Groningen as ‘one day a week’  systemadministrator/programmer.

One of the first jobs that was assigned to me was reorganizing a set of 14 computers. These computers run ‘exhibits’ so people interested in going to the university can see what we do. The old setup was kinda odd, all computers where attached to 4 remote controlled powerinterupters and each morning someone at the reception, or me turned on then powerinterupters and each computer turned on because “wake on power” was turned on in their bioses.

Each afternoon someone would again turn off the powerinterupters and each computer would just stop because the power ran out.

If there was something wrong with a computer, you’d just pick up your keys, keyboard and mouse and sat in front of the damn thing until it worked again.

I quickly started working on laying networkcables between all computers, setting up remote desktop configurations, configuring a sever to become a proxy/dhcp server/webserver and setting up all computers to “wake on lan.” After that the administration began, find each computers username, password and mac adress, and setting a helpful computername.

After these changes I could remote desktop to the server, and from there remote desktop to each connected computer, configure it, install new software, reboot it, etc… The proxy server blocked all websites except for the websites of the university so the computers could no longer be used for ‘bad browsing’.

There was still one thing that bothered me though, and that was not being able to see which computers where still working without logging in to each and every one of them. I also wasnt happy with still having to login in to the server to send the wake on lan packets in the morning, and shutting every computer down by login in again in the afternoon.

To overcome this problem I wrote two programs, creatively called ComputerManager and ComputerMonitor. ComputerMonitor is a small application that runs in the background and sends a “I’m alive” packet to ComputerMonitor every 2minutes, it also has a small listenserver running to listen to shutdown and reboot commands. I installed this on all 14 pc’s.

The ComputerManager program listens for those “I’m alive ” packets and keeps track of which computers have not send an “I’m alive” packet for 5 minutes and adds these to a “red list”. The computers that keep responding keep being updated in the “green list” with their last update time, computername and ip-adress.

The program also allows for saving mac adresses of computers. These mac adresses are used to wake-up all computers on 8:55 am. The program also sends a shutdown message to every computer on 6:05pm. This way my job got allot easier. Ofcourse it was allot of work at first but it was worth my effort.

As a final touch I created one more little program that can be run by an exec command in a php script. This allowed me to make a secure webpage where people other than me can only press two buttons: “Turn all on” and “Turn all off”.

I’m releasing the sourcecode of ComputerManager and Monitor to everyone. Allot is still hardcoded, and there are probably some bad design descissons here and there but it might be useful to someone. The asynchronous listen server is solid and a good learning example. There’s also allot of code dealing with win32 calls like shutdown and reboot.

The most recent version can be downloaded here

I won’t be actively updating or supporting this program since it was just written for my personal use, but if you got any questions or comments, feel free to ask.

Sending objects via high speed asynchronous sockets in C# (Serialization + Socket programming)

DISCLAIMER

Some users at stackoverflow found bugs in this implementation, see the rightful complaints here. As it stands now this code will not handle sending data that is larger than 1 packet correctly. You will have to implement some sort of protocol to handle this to replace the wrong check that there is now that just keeps reading if a full packet was received. It also doesn’t close the client-socket properly when everything is done. Leaving that to the client which can be forgotten. I would like to thank the stackoverflow user Polity for posting here to bring the bugs to my attention!

You can find a new updated version here that fixes all these problem, has been brought up-to-date and includes the source code. I’ll keep this old article here for reference, but please use the new article when you’re actually building something!

END DISCLAIMER

Well a while has past since my last useful post here, but here I am at it again a post filled with useful source code to use in your everyday C# programs.

I was very curious how games work with async sockets to keep everyone communicating smoothly so I set up a simple test app to test async sockets in C#, however this seemed quite a bit harder than I thought and after some help at the Dutch website tweakers.net I finally had everything working. (I finally figured out that I shouldn’t be using the beginsendpacket but the beginsend methods). Anyway let’s get straight to business.

I first made a small object that we want to send over the network, let’s call it Status, below is the source code for that object, let’s make a couple of stuff clear.

[Serializable]
    public class Status
    {
        [NonSerialized]
        public Socket Socket;
        [NonSerialized]
        public List<byte> TransmissionBuffer = new List<byte>();
        [NonSerialized]
        public byte[] buffer = new byte[1024];

        public string msg;     //the only thing we really send.

//Usually you shouldn't but these 2 methods in your class because they don't operate specifically on this object
//and we would get allot of duplicate code if we would put those 2 methods in each class we would like to
//be able to send but to not wind up having to write a couple of utility classes (where these should reside)
// I let them reside here for now.
        public byte[] Serialize()
        {
            BinaryFormatter bin = new BinaryFormatter();
            MemoryStream mem = new MemoryStream();
            bin.Serialize(mem, this);
            return mem.GetBuffer();
        }

        public Status DeSerialize()
        {
            byte[] dataBuffer = TransmissionBuffer.ToArray();
            BinaryFormatter bin = new BinaryFormatter();
            MemoryStream mem = new MemoryStream();
            mem.Write(dataBuffer,0, dataBuffer.Length);
            mem.Seek(0, 0);
            return (Status)bin.Deserialize(mem);
        }
    }

As you can see the class is marked serializable with [Serializable] this signals the compiler that we should be able to serialize this object. (Get it from memory, place it in a byte array and send it anywhere (harddrive/network/etc..). Stuff that shouldn’t be send with it like external classes that we would like to send later should be marked [NonSerialized] (the Java equivalent for transient). This way we can cut of some dependencies and keep the overhead low. (For example no matter what the TransmissionBuffer referenced to, because it’s nonserialized it’s reference will not be send aswell and it will appear on the other side as “null”.

As you can see the only real data this object hold is a small string called msg the other objects are for administrative purposes as we will see soon.

Now there are allot of examples out there that show how to send a string, you can easily get the bytearray from a string and send it anywhere, for an object that is slightly harder, as you can see in the serialize() method we have to create a binarrayformatter, this binarray formatter is then fed a direct stream to the memory where our object resides and a reference to our memorystream, the object is serialized in our memorystream’s buffer as a bytearray and then we can do anything we want with it. This method just returns the buffer so we can set it over a network. The deserialize method does exactly the same but then the other way arround except for the mem.Seek(0,0); we see right before return, this seek sets the pointer of the stream at the start of the stream so the binarrayFormatter can start reading and deserializing from the start of the stream. (Forgetting this would give an error telling that the end of the stream was found before deserialzing was completed, which makes sence if you think about it).

Anyway before we get to the real workhorse of the code let’s take a look at the client.

public class Client
{
ManualResetEvent allDone = new ManualResetEvent(false);

///
/// Starts the client and attempts to send an object to the server
///
public void Start()
{
while (true)
{
Console.Out.WriteLine("Waiting for connection...");
allDone.Reset();
Socket sender = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
sender.BeginConnect(new IPEndPoint(IPAddress.Loopback, 1440), Connect, sender);
allDone.WaitOne(); //halts this thread until the connection is accepted
}
}

///
/// Starts when the connection was accepted by the remote hosts and prepares to send data
///
public void Connect(IAsyncResult result)
{
Status status = new Status();
status.Socket = (Socket)result.AsyncState;
status.Socket.EndConnect(result);
status.msg = "Hello webs";
byte[] buffer = status.Serialize(); //fills the buffer with data
status.Socket.BeginSend(buffer, 0, buffer.Length, SocketFlags.None, Send, status);
}

///
/// Ends sending the data, waits for a readline until the thread quits
///
public void Send(IAsyncResult result)
{
Status status = (Status)result.AsyncState;
int size = status.Socket.EndSend(result);
Console.Out.WriteLine("Send data: " + size + " bytes.");
Console.ReadLine();
allDone.Set(); //signals thread to continue so it sends another message
}
}

Don’t mind the manualreset events to much, they’re there so the application doesn’t go to fast so we can see what happens instead of 2 console windows just printing text like mad :). (Remember that they will send as fast as possible because they are asynchronous and don’t have to wait for the first send to complete so yeah some pause points are quite handy for now, in a real client you wouldn’t use while(true) but something more sophisticated like an update interval or when something changed.

As you can see the start method creates a socket and tries to send some data nothing super special here except for that the beginconnect method references the connect method. When the server is ready for a connection the connect method is executed, we create a new status object and place the socket we get returned from the endAccept method in there for bookkeeping (we need it later to send data else we don’t know which socket we where using, this is also why the socket is [unserialized] we don’t need to send it the other way). We also fill the msg of the object and then serialize it to a byte array, we place that bytearray in the beginsend method.

When the server is ready for receiving data the Send method is called. This method uses the socket in the packet to call endsend and read how many bytes where send.

And now, the server!

public class Server
{
ManualResetEvent allDone = new ManualResetEvent(false);

///
/// Starts a server that listens to connections
///
public void Start()
{
Socket listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
listener.Bind(new IPEndPoint(IPAddress.Loopback, 1440));
while (true)
{
Console.Out.WriteLine("Waiting for connection...");
allDone.Reset();
listener.Listen(100);
listener.BeginAccept(Accept, listener);
allDone.WaitOne(); //halts this thread
}
}

///
/// Starts when an incomming connection was requested
///
public void Accept(IAsyncResult result)
{
Console.Out.WriteLine("Connection received");
Status status = new Status();
status.Socket = ((Socket)result.AsyncState).EndAccept(result);
status.Socket.BeginReceive(status.buffer, 0, status.buffer.Length, SocketFlags.None, Receive, status);
}

///
/// Receives the data, puts it in a buffer and checks if we need to receive again.
///
public void Receive(IAsyncResult result)
{
Status status = (Status)result.AsyncState;
int read = status.Socket.EndReceive(result);
if (read &gt; 0)
{
for (int i = 0; i &lt; read; i++)
{
status.TransmissionBuffer.Add(status.buffer[i]);
}

//we need to read again if this is true
if (read == status.buffer.Length)
{
status.Socket.BeginReceive(status.buffer, 0, status.buffer.Length, SocketFlags.None, Receive, status);
Console.Out.WriteLine("Past niet!");
}
else
{
Done(status);
}
}
else
{
Done(status);
}
}

///
/// Deserializes and outputs the received object
///
public void Done(Status status)
{
Console.Out.WriteLine("Received: " + status.msg);
Status send = status.DeSerialize();
Console.WriteLine(send.msg);
allDone.Set(); //signals thread to continue
//So it jumps back to the first while loop and starts waiting for a connection again.
}
}
}

Well start and accept basically do the same as the client but then the other way around, the only big difference we have is the receive method which endreceives() the data, but it’s not done yet, first it has to check if all the bytes where received if not we have to put the object in a listening state again to get the rest of the bytes from the networkcard. Then when all the bytes are safely inside our Transmissionbuffer we deserialize our object and print the msg we place in it.

Allot of work to just send a string accros, but this code will work any object and make your server nonblocking which could make it much faster, just instead of putting “string msg” in your status object put “TheObjectYouWant obj” in your status object and you are free todo as you please.

Feel free to ask questions and comments, the full sourcecode is available here: AsyncSocketServer+Client.rar