We’ve recently been using protobufs as a serialization format for RPC here at Miso. While there are already some existing protobuf RPC solutions, we wanted something that could stream down any number of protobuf objects, as well, we wanted the ability to have void return types.

We built Protoplasm to solve these problems. Protoplasm’s server is built on top of Eventmachine. Object serialization is done using Beefcake.

A common pattern for RPC using a serialization format is to have the request class have an enum for the request type and a series of optional fields to fill out the actual request details. Using Beefcake, our request object might look something like this:

class AddCommand
  include Beefcake::Message
  required :left, :int32, 1
  required :right, :int32, 2
end

class SubtractCommand
  include Beefcake::Message
  required :left, :int32, 1
  required :right, :int32, 2
end

class Command
  include Beefcake::Message
  module Type
    ADD = 1
    SUB = 2
  end
  required :type, Type, 1
  optional :add_command, AddCommand, 2
  optional :subtract_command, SubtractCommand, 3
end

In this example, we support two kinds of requests, ADD and SUB. So, let’s get to implementing this.

First of all, we need to tie our Type enum to the command fields inside the command object. To do this in Protoplasm, we create a Types module, and include Protoplasm::Types into there and define the relationship between the type, field and response type. Here is a complete example of how to do that:

require 'protoplasm'

module Types
  include Protoplasm::Types

  class AddCommand
    include Beefcake::Message
    required :left, :int32, 1
    required :right, :int32, 2
  end

  class SubCommand
    include Beefcake::Message
    required :left, :int32, 1
    required :right, :int32, 2
  end

  class MathAnswer
    include Beefcake::Message
    required :answer, :int32, 1
  end

  class Command
    include Beefcake::Message
    module Type
      ADD = 1
      SUB = 2
    end
    required :type, Type, 1
    optional :add_command, AddCommand, 2
    optional :sub_command, SubCommand, 3
  end

  request_class Command
  request_type_field :type
  rpc_map Command::Type::ADD, :add_command, MathAnswer
  rpc_map Command::Type::SUB, :sub_command, MathAnswer
end

Now we have all our definitions. request_class defines the class to use for our request. request_type_field tells us where to look for the command type in any given request object. The rpc_map method ties together the enum value, the field in the request object and response type.

With all this under our feet, let’s get to building a client and server for this. To write a simple Server for this, we could do the following:

require 'protoplasm'
require './types'

class Server < Protoplasm::EMServer.for_types(Types)
  def process_add_command(cmd)
    send_response(:answer => cmd.left + cmd.right)
  end

  def process_sub_command(cmd)
    send_response(:answer => cmd.left - cmd.right)
  end
end

Then, we can start our server by adding

Server.start(40000)

To create a corresponding client, most of the work is done for you. Here is a sample client that would work with this server:

class Client < Protoplasm::BlockingClient.for_types(Types)
  def add(l, r)
    send_request(:add_command, :left => l, :right => r).answer
  end

  def subtract(l, r)
    send_request(:sub_command, :left => l, :right => r).answer
  end

  def host_port
    ['localhost', 40000]
  end
end

This client will always try to connect via localhost, and on port 40000, but other than that, this is a completely working example. We can then issue requests to a running server by doing the following:

client = Client.new
client.add(2, 3)        # => 5
client.subtract(10, 7)  # => 3
client.subtract(-10, 7) # => -17

The entire source code for this example is at https://github.com/bazaarlabs/protoplasm-example.

When a request is made what is really going on is the following. There are nine bytes sent as a header, then the entire serialized protobuf object is sent. The first byte is a reserved byte, the next eight are a 64-bit unsigned, native endian number. This is the size in bytes of the protobuf object.

The server responds with first the reserved byte. If it’s void, it stops sending data. If it’s streaming it will continue to send the full header plus each serialized object. The reserved byte in this case serves the purpose of indicating when streaming should stop. The client has no way to abort streaming aside from dropping the connection.

The full implementation of Protoplasm is available at https://github.com/bazaarlabs/protoplasm.

Bonus: Fun with Gemspecs!

Another common problem with this sort of RPC client/server arrangement in Ruby is where do you put the types information. Though you could use a third gem to hold onto just the types, a simpler arrangement is to use multiple gemspecs within the same repo. This is the technique employed by protoplasm itself, so, if you’re interesting, take a look at the source. Each gemspec has the same library files in common, but each gemspec has different dependencies. We avoid loading all dependencies by using autoload, but the same thing could be achieved by requiring the individual server and client ruby files.