Sending objects via high speed asynchronous sockets in C# (Serialization + Socket programming)

DISCLAIMER

Some users at stackoverflow found bugs in this implementation, see the rightful complaints here. As it stands now this code will not handle sending data that is larger than 1 packet correctly. You will have to implement some sort of protocol to handle this to replace the wrong check that there is now that just keeps reading if a full packet was received. It also doesn’t close the client-socket properly when everything is done. Leaving that to the client which can be forgotten. I would like to thank the stackoverflow user Polity for posting here to bring the bugs to my attention!

You can find a new updated version here that fixes all these problem, has been brought up-to-date and includes the source code. I’ll keep this old article here for reference, but please use the new article when you’re actually building something!

END DISCLAIMER

Well a while has past since my last useful post here, but here I am at it again a post filled with useful source code to use in your everyday C# programs.

I was very curious how games work with async sockets to keep everyone communicating smoothly so I set up a simple test app to test async sockets in C#, however this seemed quite a bit harder than I thought and after some help at the Dutch website tweakers.net I finally had everything working. (I finally figured out that I shouldn’t be using the beginsendpacket but the beginsend methods). Anyway let’s get straight to business.

I first made a small object that we want to send over the network, let’s call it Status, below is the source code for that object, let’s make a couple of stuff clear.

[Serializable]
    public class Status
    {
        [NonSerialized]
        public Socket Socket;
        [NonSerialized]
        public List<byte> TransmissionBuffer = new List<byte>();
        [NonSerialized]
        public byte[] buffer = new byte[1024];

        public string msg;     //the only thing we really send.

//Usually you shouldn't but these 2 methods in your class because they don't operate specifically on this object
//and we would get allot of duplicate code if we would put those 2 methods in each class we would like to
//be able to send but to not wind up having to write a couple of utility classes (where these should reside)
// I let them reside here for now.
        public byte[] Serialize()
        {
            BinaryFormatter bin = new BinaryFormatter();
            MemoryStream mem = new MemoryStream();
            bin.Serialize(mem, this);
            return mem.GetBuffer();
        }

        public Status DeSerialize()
        {
            byte[] dataBuffer = TransmissionBuffer.ToArray();
            BinaryFormatter bin = new BinaryFormatter();
            MemoryStream mem = new MemoryStream();
            mem.Write(dataBuffer,0, dataBuffer.Length);
            mem.Seek(0, 0);
            return (Status)bin.Deserialize(mem);
        }
    }

As you can see the class is marked serializable with [Serializable] this signals the compiler that we should be able to serialize this object. (Get it from memory, place it in a byte array and send it anywhere (harddrive/network/etc..). Stuff that shouldn’t be send with it like external classes that we would like to send later should be marked [NonSerialized] (the Java equivalent for transient). This way we can cut of some dependencies and keep the overhead low. (For example no matter what the TransmissionBuffer referenced to, because it’s nonserialized it’s reference will not be send aswell and it will appear on the other side as “null”.

As you can see the only real data this object hold is a small string called msg the other objects are for administrative purposes as we will see soon.

Now there are allot of examples out there that show how to send a string, you can easily get the bytearray from a string and send it anywhere, for an object that is slightly harder, as you can see in the serialize() method we have to create a binarrayformatter, this binarray formatter is then fed a direct stream to the memory where our object resides and a reference to our memorystream, the object is serialized in our memorystream’s buffer as a bytearray and then we can do anything we want with it. This method just returns the buffer so we can set it over a network. The deserialize method does exactly the same but then the other way arround except for the mem.Seek(0,0); we see right before return, this seek sets the pointer of the stream at the start of the stream so the binarrayFormatter can start reading and deserializing from the start of the stream. (Forgetting this would give an error telling that the end of the stream was found before deserialzing was completed, which makes sence if you think about it).

Anyway before we get to the real workhorse of the code let’s take a look at the client.

public class Client
{
ManualResetEvent allDone = new ManualResetEvent(false);

///
/// Starts the client and attempts to send an object to the server
///
public void Start()
{
while (true)
{
Console.Out.WriteLine("Waiting for connection...");
allDone.Reset();
Socket sender = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
sender.BeginConnect(new IPEndPoint(IPAddress.Loopback, 1440), Connect, sender);
allDone.WaitOne(); //halts this thread until the connection is accepted
}
}

///
/// Starts when the connection was accepted by the remote hosts and prepares to send data
///
public void Connect(IAsyncResult result)
{
Status status = new Status();
status.Socket = (Socket)result.AsyncState;
status.Socket.EndConnect(result);
status.msg = "Hello webs";
byte[] buffer = status.Serialize(); //fills the buffer with data
status.Socket.BeginSend(buffer, 0, buffer.Length, SocketFlags.None, Send, status);
}

///
/// Ends sending the data, waits for a readline until the thread quits
///
public void Send(IAsyncResult result)
{
Status status = (Status)result.AsyncState;
int size = status.Socket.EndSend(result);
Console.Out.WriteLine("Send data: " + size + " bytes.");
Console.ReadLine();
allDone.Set(); //signals thread to continue so it sends another message
}
}

Don’t mind the manualreset events to much, they’re there so the application doesn’t go to fast so we can see what happens instead of 2 console windows just printing text like mad :). (Remember that they will send as fast as possible because they are asynchronous and don’t have to wait for the first send to complete so yeah some pause points are quite handy for now, in a real client you wouldn’t use while(true) but something more sophisticated like an update interval or when something changed.

As you can see the start method creates a socket and tries to send some data nothing super special here except for that the beginconnect method references the connect method. When the server is ready for a connection the connect method is executed, we create a new status object and place the socket we get returned from the endAccept method in there for bookkeeping (we need it later to send data else we don’t know which socket we where using, this is also why the socket is [unserialized] we don’t need to send it the other way). We also fill the msg of the object and then serialize it to a byte array, we place that bytearray in the beginsend method.

When the server is ready for receiving data the Send method is called. This method uses the socket in the packet to call endsend and read how many bytes where send.

And now, the server!

public class Server
{
ManualResetEvent allDone = new ManualResetEvent(false);

///
/// Starts a server that listens to connections
///
public void Start()
{
Socket listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
listener.Bind(new IPEndPoint(IPAddress.Loopback, 1440));
while (true)
{
Console.Out.WriteLine("Waiting for connection...");
allDone.Reset();
listener.Listen(100);
listener.BeginAccept(Accept, listener);
allDone.WaitOne(); //halts this thread
}
}

///
/// Starts when an incomming connection was requested
///
public void Accept(IAsyncResult result)
{
Console.Out.WriteLine("Connection received");
Status status = new Status();
status.Socket = ((Socket)result.AsyncState).EndAccept(result);
status.Socket.BeginReceive(status.buffer, 0, status.buffer.Length, SocketFlags.None, Receive, status);
}

///
/// Receives the data, puts it in a buffer and checks if we need to receive again.
///
public void Receive(IAsyncResult result)
{
Status status = (Status)result.AsyncState;
int read = status.Socket.EndReceive(result);
if (read &gt; 0)
{
for (int i = 0; i &lt; read; i++)
{
status.TransmissionBuffer.Add(status.buffer[i]);
}

//we need to read again if this is true
if (read == status.buffer.Length)
{
status.Socket.BeginReceive(status.buffer, 0, status.buffer.Length, SocketFlags.None, Receive, status);
Console.Out.WriteLine("Past niet!");
}
else
{
Done(status);
}
}
else
{
Done(status);
}
}

///
/// Deserializes and outputs the received object
///
public void Done(Status status)
{
Console.Out.WriteLine("Received: " + status.msg);
Status send = status.DeSerialize();
Console.WriteLine(send.msg);
allDone.Set(); //signals thread to continue
//So it jumps back to the first while loop and starts waiting for a connection again.
}
}
}

Well start and accept basically do the same as the client but then the other way around, the only big difference we have is the receive method which endreceives() the data, but it’s not done yet, first it has to check if all the bytes where received if not we have to put the object in a listening state again to get the rest of the bytes from the networkcard. Then when all the bytes are safely inside our Transmissionbuffer we deserialize our object and print the msg we place in it.

Allot of work to just send a string accros, but this code will work any object and make your server nonblocking which could make it much faster, just instead of putting “string msg” in your status object put “TheObjectYouWant obj” in your status object and you are free todo as you please.

Feel free to ask questions and comments, the full sourcecode is available here: AsyncSocketServer+Client.rar

34 Responses to “Sending objects via high speed asynchronous sockets in C# (Serialization + Socket programming)”

  1. Farsen says:

    That seems very useful, I will try and play around with it.

    Thanks alot!!

  2. Fujimi says:

    hi,

    I’ve tried to copy those classes into 2 different console applications, just to test ‘em.
    however, on the server side I’m getting a SerializationException ‘Unable to find assembly’…

    o_0

    are you familiar with this problem?

    thanks!

  3. Fujimi says:

    it’s me again.. forget about my last post, as I’ve managed to figure out what was the cause for my problem.

    (I just needed to get the status object, and put it in one dll, and reference it from both server and client).

    works fine! great post! thanks a lot!

  4. royalexander says:

    Ah, that’s exactly what I wanted to tell you. :)

  5. jay says:

    Hi,

    very interesting post. Im having the same error, i understand i need to create a dll and reference it from both client and server. However im new to this and have not created a dll before. Ive researched the web but there are still a few points i dont understand.

    How would i create this dll is visual studio and refeence it from both?

    Many thanks

  6. royalexander says:

    Hey Jay,

    I assume that you have 2 projects in one solution. In one of these projects you are missing the class. In that project, find “references” in the project explorer. Right click it, and choose add. Then there choose the tab solution and add the other project as a reference. Now you should be able to resolve any problems.

  7. jay says:

    Ah thanks :), however now i have encountered a new problem. I took your source code and applied it to a media center addin i was building. The server code sits on my media center app and runs fine, however i tried to put the client code onto a smart device running windows mobile 6.

    I have been encontering problems because serialization is not supported in .net compact 3.5, i cant use the BinaryFormatter. So now i am trying to find a way to implement custom serialization on the status object you created in your source code.

    Any ideas how i can serialise the status object without using BinaryFormatter.

  8. jay says:

    and de-serialize it

  9. royalexander says:

    Hey Jay,

    Nasty that the cf doesnt support the binaryformatter. You might be able to use the XMLSerializer though. Or you can ‘serialize’ your objects to a string and then just send the string back and forth. (You need to use the encoding classes then, I usually use UTF8 to send stuff around).

  10. jay says:

    Hi Roy thanks, i’ll look into that :)

  11. bar says:

    hi roy thanks for the code.
    i want to make the server send “status” to cliant but i dont succeed .
    please you can help me?

    sory about my english…

    thanks bar .

  12. Dear Bar,

    Thanks for also posting your question here, instead of at the old place. Much appreciated!

    As regard to your question, I’m sorry but if you don’t provide me more information I can’t help you. What do find hard to understand, and where does it go wrong sending status, maybe provide some sample code of what you got right now and what is going wrong, maybe some error messages?

    If you got more info for more I will try to help you. Please post more details here :).

  13. BigDirty says:

    Have you actually tried sending when msg has a lot of text in it? In particular more than 1024 bytes? Doesn’t work.

    Try it, put a lot of text in your msg and send. The loop where you receive never completes. Bummer :((

  14. Hey BigDirty,

    Thanks for your comment, I actually tried large texts and it seemed fine (there’s a special check in the code to see if we could expect another fragment). Are you sure my code is faulty? Could be that I missed something. Unfortunately I don’t have the sample project anymore, but please play around a bit and if you’ve isolated the problem let me know :).

  15. Iko says:

    Hi
    I dont know why, but this program doest work when you try to do this :
    example ..
    when the buffer in status is 256 bytes and you send message that has exactly 256 bytes.
    or in other words when the size of your message is divisible by the size of the buffer .

    in server after (read == status.buffer.Length) when it tries BeginReceive it just freeze

    can somebody try it and tell me ?

    thx..

  16. Tobias says:

    Hi Iko

    have the same problem. But didn’t have a solution right now. But still trying …

  17. Tobias says:

    Hi Iko

    campe up with 2 possible solutions.

    1) (quick and dirty) try to deserialize each time and catch the exception

    readRequest is function Receive, evaluateRequest is function Done


    private void readRequest(IAsyncResult result)
    {
    Status status = (Status)result.AsyncState;
    int read = status.socket.EndReceive(result);

    if (read > 0)
    {
    for (int i = 0; i < read; i++) status.transmissionBuffer.Add(status.buffer[i]);
    //try to derialize to see if data send completely
    try
    {
    Status receivedStatus = status.deserialize();
    evaluateRequest(status);
    }
    catch (SerializationException)
    {
    Console.WriteLine("read again..");
    status.socket.BeginReceive(status.buffer, 0, status.buffer.Length, SocketFlags.None, readRequest, status);
    }
    }
    else evaluateRequest(status);
    }

    2) Much nicer solution is to send the message length at the beginning of each message. This solution fails if the message length exceeds int.MaxValue Bytes. (There is still the long datatype)


    //Add in Server following variables
    private int incomingMessageLength = 0;
    private Boolean firstPackage = true;
    byte[] tmpLentgh = new byte[256];

    //in the while loop where the server listens set
    firstPackage = false;

    //modify the Receive function like this
    private void Receive(IAsyncResult result)
    {
    Status status = (Status)result.AsyncState;
    int read = status.socket.EndReceive(result);

    if (read > 0)
    {
    if(firstPackage){
    tmpLentgh = new byte[256];
    }

    if (firstPackage)
    { //first 256 bytes are message length
    for (int i = 0; i < read; i++)
    {
    if (i < 256)
    {
    tmpLentgh[i] = status.buffer[i];
    }
    else status.transmissionBuffer.Add(status.buffer[i]);
    }
    BinaryFormatter bin = new BinaryFormatter();
    MemoryStream mem = new MemoryStream();

    mem.Write(tmpLentgh, 0, tmpLentgh.Length);
    mem.Seek(0, 0);
    incomingMessageLength = (int)bin.Deserialize(mem);
    firstPackage = false;
    }
    else if (!firstPackage)
    {
    for (int i = 0; i status.transmissionBuffer.Count)
    { //read again
    Console.WriteLine("read again");
    status.socket.BeginReceive(status.buffer, 0, status.buffer.Length, SocketFlags.None, Receive, status);
    }
    else Done(status);
    }
    else Done(status);
    }

    //in the connect function of the client change
    byte[] buffer = status.Serialize(); //fills the buffer with data
    status.Socket.BeginSend(buffer, 0, buffer.Length, SocketFlags.None, Send, status);

    //to:
    byte[] buffer = status.serialize();

    outgoingMessageLength = buffer.Length;
    BinaryFormatter bin = new BinaryFormatter();
    MemoryStream mem = new MemoryStream();
    bin.Serialize(mem, outgoingMessageLength);
    byte[] messageLength = mem.GetBuffer();
    List messageByteList = messageLength.ToList();
    for (int i = 0; i < buffer.Length; i++) messageByteList.Add(buffer[i]);
    buffer = messageByteList.ToArray();

    status.socket.BeginSend(buffer, 0, buffer.Length, SocketFlags.None, send, status);

    Thank you Roy for this nice tutorial. Pointed me in the right direction :)

  18. Tobias says:

    In solution 2 it has to be

    //in the while loop where the server listens set
    firstPackage = true;

  19. Hey Tobias, thanks for the great comments. And sorry guys that I wasn’t able to create a bigger sample. This article was supposed to be just a quick’n’dirty way to get everybody started :).

  20. Ciwan says:

    Hi Roy

    Thanks for the tutorial. I have tried doing the same thing, but without using serialization. I just want two programs (client and server) that will be able to send and receive a product object. Below is the code for the Product class, and the Client class << I think them two are fine. However my Server class (also below) is messed up, I just don't know how to get it to work.

    Product Class Code

    Client Class Code

    Server Class Code

    As you can see my server class is hardly doing anything, but the problem is I dont’ know what to do. I know that I have to somehow de-serialize the product object, but I don’t know how I’d go about doing that. Looking at your code didn’t help either :(

    I would greatly appreciate it you could show me how this would be done.

    Thank You.

  21. Ciwan says:

    Sorry there is a mistake in my last comment.

    What I mean to say was ” Without using Threading ” … My bad :)

  22. Hey Ciwan, your server is only listening for 100ms, you need to add a while(true) loop so that your server keeps listening. Also you’re not doing anything with the data you might receive. After you’ve accepted a connection (this will give you a client object I think) you need to start receiving data, this will probably have you end up with a byte[]. Then you have to convert this byte[] back to a product item.

  23. Ciwan says:

    Hello Roy

    You were right, you can probably tell I am a n00b to this.

    Here is what my code looks like right now:

    Product Class

    Server Class

    I haven’t touched the client class, because what I understood from you, is that it was fine ! :)

    As you can see in my Product class I have added a Byte Array of size 1024 << would that be enough to hold the byte data from a Product object ?

    Also can you please tell me (in code) how I would grab the byte data, and transform it back to a Product object.

    I greatly appreciate your help.

    Thanks Roy.

  24. parv sondhi says:

    could you please the provide the source code, the existing link is not working

  25. Hey Parv,

    I’m afraid I don’t have all the source code anymore but you shouldn’t need much more than is written here.

  26. parv sondhi says:

    No problem

    Thanks Alot

  27. parv sondhi says:

    hey,

    please excuse me if this turns out to be an stupid problem, i am kind of a noob at this

    I couldn’t understand the following segment in the server side code

    /*if (read > 0)
    {
    for (int i = 0; i < read; i++)
    {*/

    i did not understand the purpose, use of &gt and &lt

    Thanks

  28. parv sondhi says:

    Sorry figured it out myself

    thanks

  29. No problem, glad you figured it out :)

  30. Polity says:

    Hi Roy,

    Please take a look at my reply in http://stackoverflow.com/questions/12335542/why-im-forced-to-close-c-sharp-asynchronous-client-socket-after-every-transa/12336829#12336829

    The code posted in this article is very much broken and confusing for people who simply want to learn about networking.

    I hope you can take appropriate action.

    Thanks, Koen

  31. Hey Koen,

    Thanks for the heads up. I’ve checked and you’re absolutely right. I’ve added the disclaimer on top and will try to replace this post with something better when I have time.

  32. Vinayak says:

    Great post…..very helpful…

Leave a Reply