Easy and perfomant client/server communication with protobuf-net & Griffin.Framework

ProtoBuf is Googles open serialization format which can be used to serialize objects in a standardized way. With it, different platforms can communicate with a format that is much more efficient than XML. Combine the most popular implementation if it, protobuf-net, with Griffin.Framework and you get an easy and fast way of sending information between processes.

(this is a follow up on my previous post)

As I know that the format is quite efficient I felt that it would be a powerful combination. I’ve therefore created a simple client/server where I made a benchmark for how long it would take for a Griffin.Framework client to send 20 000 messages to a Griffin.Framework server. The requirement was that the client have to receive the reply before sending the next message. To spice everything up I changed so that the client application used 20 clients simultaneously, and all clients had to receive all replies before the time was stopped.

As MicroMsg sends the type over the network it works well together with protobuf-net (which requires a specific type to be able to deserialize the message),

Both the server and client was running on the same machine.

The message

The message being transferred and it’s reply looked like this:

[ProtoContract]
public class Authenticate
{
    [ProtoMember(1)]
    public string UserName { get; set; }

    [ProtoMember(2)]
    public string Password { get; set; }
}

[ProtoContract]
public class AuthenticateReply
{
    [ProtoMember(1)]
    public bool Success { get; set; }

    [ProtoMember(2)]
    public string Decision { get; set; }
}

The serializer

To start with, I had to write a serializer for the library which uses protobuf.

class ProtoBufSerializer : IMessageSerializer
{
    static ConcurrentDictionary<string, Type> _types = new ConcurrentDictionary<string, Type>();

    public void Serialize(object source, Stream destination, out string contentType)
    {
        Serializer.NonGeneric.Serialize( destination, source);
        contentType = "application/protobuf;" + source.GetType().FullName;
    }

    public object Deserialize(string contentType, Stream source)
    {
        Type type;
        if (!_types.TryGetValue(contentType, out type))
        {
            int pos = contentType.IndexOf(";");
            if (pos == -1)
                throw new NotSupportedException("Expected protobuf");

            type = Type.GetType(contentType.Substring(pos + 1), true);
            _types[contentType] = type;
        }
            
        return Serializer.NonGeneric.Deserialize(type, source);
    }
}

As you can see, I also use a Type cache as Type.GetType() is terrible slow, even thought it has a built in cache. You could implement a JSON or Xml serializer in the same way.

The server

As a server you typically just create a ChannelTcpListener:

var settings = new ChannelTcpListenerConfiguration(
    () => new MicroMessageDecoder(new ProtoBufSerializer()),
    () => new MicroMessageEncoder(new ProtoBufSerializer())
    );

var server = new MicroMessageTcpListener(settings);
server.MessageReceived = OnServerMessage;

server.Start(IPAddress.Any, 1234); 

That’s it, all you need now is a callback to receive messages from all clients:

private static void OnServerMessage(ITcpChannel channel, object message)
{
    var auth = (Authenticate) message;
    channel.Send(new AuthenticateReply() {Success = true});
}

How much code/configuration would you need in WCF to achieve the same thing?

The client

In the client you do about the same, but you use async/await instead of a callback.

private static async Task RunClient()
{
    var client = new ChannelTcpClient<object>(
        new MicroMessageEncoder(new ProtoBufSerializer()),
        new MicroMessageDecoder(new ProtoBufSerializer())
        );

    await client.ConnectAsync(IPAddress.Parse("192.168.1.3"), 1234);

    for (int i = 0; i < 20000; i++)
    {
        await client.SendAsync(new Authenticate { UserName = "jonas", Password = "king123" });
        var reply = (AuthenticateReply)await client.ReceiveAsync();

        if (reply.Success)
        {
            //Console.WriteLine("Client: Yay, we are logged in.");
        }
        else
        {
            //Console.WriteLine("Client: " + reply.Decision);
        }
    }

    await client.CloseAsync();
}

The result

griffinframework

If you study the screenshot, you will see that Griffin.Framework only takes 28% of the total time for those messages. 37% of the time is used by protobuf-net. The remaining 35% is spent in .NET/Windows.

On my machine that means that 800 000 .NET objects (400 000 requests and 400 000 replies) where transferred in 11 seconds time. That’s 0,01398375 milliseconds per object. Those digits are not as important as the fact that Griffin.Framework only took 28% of the time being used by the application.

  • Dariusz Lenartowicz

    Really nice. I like the idea. Checkout my try to hide infrastructure just behind simply interface. 🙂

    https://github.com/dario-l/MicroServices

    • Cool. Will check it out when I get home. Just made a tweet with it.

  • Ali Esf

    nice library. i am interested in your library but i have some problems in Client side and need your help
    it is my client side code

    public async Task Send(ServerDataPacket dataPacket)
    {
    ChannelTcpClient client = new ChannelTcpClient(new MicroMessageEncoder(new ProtoBufSerializer()), new MicroMessageDecoder(new ProtoBufSerializer()));
    IPAddress ip = IPAddress.Parse(“169.254.116.8″);
    await client.ConnectAsync(ip, 1111);
    await client.SendAsync(dataPacket);
    ClientDataPacket response = (ClientDataPacket) await client.ReceiveAsync();
    await client.CloseAsync();
    client.Dispose();
    return response;
    }

    the code runs without any problem for the first time but i get the Socket got Disconnected for the second time that is click the botton.

    this is my server side code

    public class Server
    {
    public event EventHandler Log;
    private readonly ChannelTcpListener _server;
    public Server()
    {
    var settings = new ChannelTcpListenerConfiguration(
    () => new MicroMessageDecoder(new ProtoBufSerializer()),
    () => new MicroMessageEncoder(new ProtoBufSerializer())
    );
    _server = new ChannelTcpListener(settings);
    _server.MessageReceived += OnMessage;
    _server.ClientConnected += OnClientConnected;
    _server.ClientDisconnected += OnClientDisconnected;
    }
    public void Start()
    {
    _server.Start(IPAddress.Any, 1111);
    }
    private void OnClientConnected(object sender, ClientConnectedEventArgs e)
    {
    OnLog(@”Got connection from client with ip ” + e.Channel.RemoteEndpoint);
    }
    private void OnClientDisconnected(object sender, ClientDisconnectedEventArgs e)
    {
    OnLog(@”Disconnected: ” + e.Channel.RemoteEndpoint.ToString());
    }
    private void OnMessage(ITcpChannel channel, object message)
    {
    ServerDataPacket packet = (ServerDataPacket) message;
    OnLog(@”Server received: ” + packet.StrText);
    channel.Send(new ClientDataPacket() {Msg = packet.StrText + ” received”});
    }
    protected virtual void OnLog(string message)
    {
    logEventArgs args = new logEventArgs() {LogMsg = message};
    var handler = Log;
    if (handler != null)
    handler(this, args);
    }
    }

    is there any bug in griffin.Framework?

    email address = zoofa@ymail.com

    with best regards