Tired of looking for errors in log files? Use OneTrueError - Automated exception management in .NET.

New version of Griffin.Networking – A networking library for .NET

I’ve refactored Griffin.Network quite a lot (I did say that it was still an early release). The new version is called v0.5 (the library is taking shape, but there is still some things to take care of).

Highlights:

1. The pipeline is no longer mandatory
2. I’m using the Async methods instead of Begin/EndXxxx
3. The buffer management is a more effecient
4. It’s easier to work with the buffers
5. Introducing messaging support

Dead easy to get started

If you just want to use the library, here is how you can configure it to send objects over the network using JSON and a small binary header:

public class Program
{
    private static void Main(string[] args)
    {
        // configuring the server
        var serviceFactory = new MyServiceFactory();
        var messageFactory = new BasicMessageFactory();
        var configuration = new MessagingServerConfiguration(messageFactory);
        var server = new MessagingServer(serviceFactory, configuration);
        server.Start(new IPEndPoint(IPAddress.Any, 7652));

        // configuring the client
        var client = new MessagingClient(new BasicMessageFactory());
        client.Connect(new IPEndPoint(IPAddress.Loopback, 7652));

        // Look here! We receive objects!
        client.Received += (sender, eventArgs) => Console.WriteLine("We received: " + eventArgs.Message);

        // And here we are sending one.
        client.Send(new OpenDoor {Id = Guid.NewGuid().ToString()});

        Console.ReadLine();
    }
}

The complete sample

Let’s see how that works…

Messaging

Typically you want to create an application that sends and receives messages over the network. It usually is a flow like this:

Receiving

1. Receive byte[] buffers
2. Receive enough to be able to build a message (by using a length header or a delimiter)
3. Build the message
4. Process the message

Sending

1. Do your magic
2. Send a message
3. Serialize it
4. Send it asynchronously (might require several SendAsync)

Instead of having to do all those steps there is now a Messaging server available. What you have to do now is simply to specify a message builder and a message serializer and you’ll automagically get to work with messages directly.

Sample implementation

The first thing we have to do is to define our factory that creates serializers and message builders. Let’s decide that we should only use a binary header (version + length) and JSON to transport everything.

When we are sending things over the network with Griffin.Networking we’ll refer to messages instead of objects. For that’s what the library to. It serializes the objects, put them in a message (typically with a binary header) and send everything to the remote end point.

Serializing messages

To convert objects to byte arrays we’ll use serializers. They look just like typical .NET serializers. Here is the one for the basic protocol:

public class BasicMessageSerializer : IMessageSerializer
{
	private static readonly byte[] VersionBuffer = new byte[] {1};
	
	public void Serialize(object message, IBufferWriter writer)
	{
		var str = JsonConvert.SerializeObject(message);
		var bodyBytes = Encoding.UTF8.GetBytes(str);

		// version
		writer.Write(VersionBuffer, 0, VersionBuffer.Length);

		//length
		var buffer = BitConverter.GetBytes(bodyBytes.Length);
		writer.Write(buffer, 0, buffer.Length);

		//body
		writer.Write(bodyBytes, 0, bodyBytes.Length);
	}
}

We first serialize the object as JSON and then start by writing the binary header. Finally we also write the string.

Building incoming messages

When receiving bytes from the socket we’ll have to do more, since TCP sends everything in chunks. There is no guarantee that everything is received directly. Hence we need build objects and not just try to deserialize them directly.

public class BasicMessageBuilder : IMessageBuilder
{
	private readonly byte[] _header = new byte[Packet.HeaderLength];
	private readonly Queue<Packet> _messages = new Queue<Packet>();
	private int _bytesLeft = Packet.HeaderLength;
	private Packet _packet;
	private Func<IBufferReader, bool> _parserMethod;

	public BasicMessageBuilder()
	{
		_parserMethod = ReadHeaderBytes;
	}

	// invoked by the server every time something have been received
	public bool Append(IBufferReader reader)
	{
		while (_parserMethod(reader))
		{
		}
		return _messages.Count > 0;
	}

	// server is trying to dequeue a message (if one has been completed)
	public bool TryDequeue(out object message)
	{
		if (_messages.Count == 0)
		{
			message = null;
			return false;
		}

		var packet = _messages.Dequeue();

		using (var reader = new StreamReader(packet.Message))
		{
			var str = reader.ReadToEnd();
			message = JsonConvert.DeserializeObject(str);
		}

		return true;
	}

	// read header bytes from the incoming bytes
	protected virtual bool ReadHeaderBytes(IBufferReader stream)
	{
		var bytesLeftInStream = stream.Count - stream.Position;
		var bytesToCopy = bytesLeftInStream < _bytesLeft
							  ? bytesLeftInStream
							  : _bytesLeft;

		stream.Read(_header, 0, bytesToCopy);

		_bytesLeft -= bytesToCopy;
		if (_bytesLeft > 0)
			return false;

		_packet = CreatePacket(_header);

		_bytesLeft = _packet.ContentLength;
		_parserMethod = ReadBodyBytes;
		return true;
	}

	// build the body
	protected virtual bool ReadBodyBytes(IBufferReader reader)
	{
		var bytesLeftInStream = reader.Count - reader.Position;
		var bytesToCopy = bytesLeftInStream < _bytesLeft
							  ? bytesLeftInStream
							  : _bytesLeft;


		reader.CopyTo(_packet.Message, bytesToCopy);

		_bytesLeft -= bytesToCopy;
		if (_bytesLeft > 0)
			return false;

		_packet.Message.Position = 0;
		_messages.Enqueue(_packet);
		_packet = null;

		_bytesLeft = Packet.HeaderLength;
		_parserMethod = ReadHeaderBytes;
		return true;
	}

	// returns the class which is used to temporarily store the body
	// until we've received the messages entire content
	protected virtual Packet CreatePacket(byte[] header)
	{
		var message = new Packet
			{
				Version = _header[0],
				ContentLength = BitConverter.ToInt32(header, 1)
			};

		if (message.Version <= 0)
			throw new InvalidDataException(string.Format(
				"Received '{0}' as version. Must be larger or equal to 1.", message.Version));
		if (message.ContentLength <= 0)
			throw new InvalidDataException(string.Format(
				"Got invalid content length: '{0}', expected 1 or larger.", message.ContentLength));

		message.Message = new MemoryStream(message.ContentLength);
		return message;
	}
}

The message factory

To be able to customize the message handling we’ll provide the serializer and the message builder through an abstract factory. It got two methods, one for creating serializers and one for creating message builders.

public class BasicMessageFormatterFactory : IMessageFormatterFactory
{
	public IMessageSerializer CreateSerializer()
	{
		return new BasicMessageSerializer();
	}

	public IMessageBuilder CreateBuilder()
	{
		return new BasicMessageBuilder();
	}
}

Working with the messages

Do note that messaging is just one of the options that is included in Griffin.Networking. You can for instance handle the byte arrays directly if you like to. The framework will still take care of all buffer hanlding, network IO and connection management for you.

Before creating the server we have to define the class that will handle all incoming messages. Compare it to a WCF service:

public class MyService : MessagingService
{
	public override void HandleReceive(object message)
	{
		// We can only receive this kind of command
		var msg = (OpenDoor)message;

		Console.WriteLine("Should open door: {0}.", msg.Id);

		// Send a reply
		Context.Write(new DoorOpened(msg.Id));
	}
}

All messages are getting into the same method since we haven’t specified what the server should do. Nothing says that it has to be an RPC server like WCF.

We can of course build something on top of the service to get an approach similar to WCF service classes.

Starting the server

We’ve put all pieces together. Let’s create a server:

class Program
{
    static void Main(string[] args)
    {
        // factory that produces our service classes which
        // will handle all incoming messages
        var serviceFactory = new MyServiceFactory();

        // factory used to create the classes that will
        // serialize and build our messages
        var messageFactory = new BasicMessageFactory();

        // server configuration.
        // you can limit the number of clients etc.
        var configuration = new MessagingServerConfiguration(messageFactory);

        // actual server
        var server = new MessagingServer(serviceFactory, configuration);
        server.Start(new IPEndPoint(IPAddress.Any, 7652));

        //to prevent the server from shutting down
        Console.ReadLine();
	}
}

Yay.

Example client

We also need something which can communicate with the server.

// same factory as server side.
var messageFactory = new BasicMessageFactory();

// tell the client to use the factory
var client = new MessagingClient(messageFactory);

// connect
client.Connect(new IPEndPoint(IPAddress.Loopback, 7652));

// Server sent us something
client.Received += (sender, eventArgs) => Console.WriteLine("We received: " + eventArgs.Message);

// Send the initial command to the server
client.Send(new OpenDoor{Id = Guid.NewGuid().ToString()});

Summary

That was a short introduction to the new messaging alternative of Griffin.Networking. The pipeline that I described in the last Griffin.Networking blog post is still included, but it’s not the only option any more.

As a bonus, here is a basic HTTP server:

// console app
internal class Program
{
    public static void RunDemo()
    {
        var server = new MessagingServer(new MyHttpServiceFactory(),
                                            new MessagingServerConfiguration(new HttpMessageFactory()));
        server.Start(new IPEndPoint(IPAddress.Loopback, 8888));
    }
}

// factory
public class MyHttpServiceFactory : IServiceFactory
{
    public IServerService CreateClient(EndPoint remoteEndPoint)
    {
        return new MyHttpService();
    }
}

// and the handler
public class MyHttpService : HttpService
{
    private static readonly BufferSliceStack Stack = new BufferSliceStack(50, 32000);

    public MyHttpService()
        : base(Stack)
    {
    }

    public override void Dispose()
    {
    }

    public override void HandleReceive(object message)
    {
        var msg = (IRequest) message;

        var response = msg.CreateResponse(HttpStatusCode.OK, "Welcome");

        response.Body = new MemoryStream();
        response.ContentType = "text/plain";
        var buffer = Encoding.UTF8.GetBytes("Hello world");
        response.Body.Write(buffer, 0, buffer.Length);
        response.Body.Position = 0;

        Send(response);
    }
}

I’m currently writing a more complete server which will replace my old C# WebServer.

Documentation is also available, code at github.

Nuget:

install-package griffin.networking
install-package griffin.networking.protocol.basic

This entry was posted in Architecture, CodeProject and tagged , , . Bookmark the permalink.
  • ripryness

    Thanks for your contributions they really are amazing.

    I’ll throw this out there as something I think would be useful. Based on some google searches (and not finding much) there seems to be somewhat of a demand.

    When self hosting the new Web API there can be a need to serve static content. Currently I have this implemented by injecting a custom message handler to the front of the pipeline. But this is just the most basic server. (I did find your ContentTypeHelper to be quite useful for this).

    What would be really awesome is if your C# Webserver could be injected into MessageHandler pipeline for self hosting scenarios. It is on my list to dig into your code and see if I can do this but my lame solution is working for now.

    • JonasGauffin

      My http server will be an alternative to WebApi for self hosting purposes.

  • Boddlnagg

    Great work! Just one question: Will you add a WebSocket implementation that can be combined with the HTTP server? I’m looking for a way to do that without too much effort (and without using the standard .NET HttpListener). If you add support for that it would be awesome!

    • JonasGauffin

      yes. A web socket implementation will be fixed when I’m finished with the HTTP server.

  • Grzegorz Russek

    I’m just curious if some bugs are fixed:
    * Source IP Address in HttpRequest.
    * Cookie parsing
    * Adding cookie to response.
    * Exception handling (try to throw in http processing some common exception). Yes I know… this should not happen, but sometimes if you have 20 developers you might have one who forgot to handle that, and whole service dies :(
    * GZip requests (some exotic browsers always ask with gzip’ed header)

    • JonasGauffin

      First of all: If you find any bugs it would be real nice if you could add them to the issue tracker at github. I’m grateful for all reported bugs since I may not find all myself.

      1) The source ip address is not added to the headers yet, but it’s easy to fix.

      2) Fixed

      3) The IHttpCookieCollection now got an Add method

      4) Exceptions are important. Any messages thrown by the HTTP modules are caught and added to the LastException property in https://github.com/jgauffin/Griffin.WebServer/blob/master/Source/Griffin.WebServer/Griffin.WebServer/IHttpContext.cs. Any lower level exceptions (like parsing the request) are caught by the exception event in ServerBase.

      5) GZip has not been added yet. Feel free to create a pull request.

      Please do not cross post comments. One is enough. There are also a Google Group for both Griffin.WebServer and Griffin.Networking. The address can be find in github.

      • Grzegorz Russek

        Sorry for double posting and thanks for update. I’m really impressed with your work and I hope this amazing server will improve. Good luck.

  • dicker

    thanks for making this usefull library. I need encryption feature, does it support?