Tag Archives: serialization

Node Definition

As I mentioned in my previous post I’m designing a domain specific language to define the node properties. So far it looks like this:

import "UAV_Config.def"

namespace silk

struct Multirotor_Config : public UAV_Config
    struct Motor
        vec3f position = {0, 0, 0};
        bool clockwise = false;

    string name;
    float mass = 1.f : [ ui_name = "Mass (Kg)", min = 0.f ];
    float height = 0.5f : [ ui_name = "Height (m)", min = 0.f ];
    float radius = 0.5f : [ ui_name = "Radius (m)", min = 0.f ];
    float motor_thrust = 1.f : [ ui_name = "Motor Thrust (N)", min = 0.f ];
    float motor_z_thrust = 1.f : [ ui_name = "Motor Z Torque (Nm)", min = 0.f ];
    float motor_acceleration = 10.f : [ ui_name = "Motor Acceleration (N/s)", min = 0.f ];
    float motor_deceleration = 10.f : [ ui_name = "Motor Deceleration (N/s)", min = 0.f ];
    vector<Motor> motors : [ ui_name = "Motors" ];


It looks a lot like C++ but allows some extra attributes on types and members, like ui_name to have a nicer name displayed in the editor, a min/max value for numeric types, the number of decimals for floating points etc.

The grammar is done with flex/bison and it will actually be a full typesystem that I will use to generate serialization/deserialization code offline and to create UIs online in the GS.

The language will support:

  • namespaces, needed to be able to generate the code in the correct namespace in C++ and to separate the types properly
  • basic types, like ints, floats but also uint8_t and friends
  • structs and classes together with inheritance.
  • templated types like vectors.
  • math types like vec2/3/4, quat and matrices
  • initializer lists for vec3/4 etc
  • custom attributes
  • imports

The compiler will take a file and output a typesystem that contains all the types defined in the file. These types will have reflection so they can be traversed. From this typesystem I can generate the C++ code for the brain – the serialization/deserialization to binary and json – and I can generate UI widgets in the GS as well.

I’m still struggling with the grammar definition but over this weekend I will get it done.

I have to admit – doing a domain specific language and a typesystem is actually very satisfying and challenging. I wish C++ would help a bit more with common things like reflection and such. I even considered moving everything to rust or go but then I’ll have another problem – QT and other library bindings…



Node properties

One of the nice things about silkopter is that is very easy to add new types of nodes to the system.

A node can be anything from a sensor to a low pass filter to a PWM generator like PIGPIO. All nodes have the following properties:

  • Zero or more input streams. Each stream has a type and a rate
  • Zero or more output streams. Same as the inputs, they have a type and rate
  • An Init_Params struct that decides the initialization params of the node. Once the node is created this cannot be changed. The usual params that go in this struct is the rate (process frequency), number of channels for a PWM sink etc.
  • A Config struct that holds changable configuration. This can be changed at any time and contains PID settings for example

The Init_Params and the Config structs are custom, per node. They have to be serializable to json for loading/saving, serializable to binary for comms (the json serialization can be used here) and they need to be editable in an UI for the Ground Station

Silkopter defines the Init_Params and Comms in a json file – one per node – and then generates the actual C++ structs and serialization code from this json description. The UI for editing is dynamic, based on the serialized json of the structs.

So for example the load flow is this:

  • the settings.json file is loaded and converted to a json structure using the rapidjson library
  • the list of nodes is iterated and nodes are created from it
  • each node has its init method called with the init_params json passed to it
  • the init method deserializes the init_params json using generated code into a Init_Params C++ struct

The save flow is the inverse of the one above.

Sending the node Init_Params and the Config to the GS and editing it:

  • The get_init_params (or get_config) is called for a node. This serializes the internal Init_Params or Config to json using the generated code
  • The json stringified using rapidjson and sent to the GS
  • The GS parses the string using rapidjson back into a json data structure
  • A QT item model is used over this json to populate a tree view with the data
  • When data is changed by QT as a response to user interaction (typing a new value for example) the json changes. As a result of this its serialized back to string, sent to the Brain which then calls set_config with the json.

So in order to add a new node, 3 steps have to be done – all of them in the brain. The GS is unchanged:

  1. Create the node C++ class with all the processing needed
  2. Create a json description of the Init_Params and Config and generate the code for them using autojsoncxx library
        "definition" : true,
        "namespace" : "sz::ADC_Ammeter",
        "name": "Init_Params",
            ["uint32_t", "rate", {"required": true, "json_key" : "Rate (Hz)", "default" : 0}]
        "definition" : true,
        "namespace" : "sz::ADC_Ammeter",
        "name": "Config",
            ["float", "scale", {"required": false, "json_key" : "Scale", "default" : 1}],
            ["float", "bias", {"required": false, "json_key" : "Bias", "default" : 0}]

3. Register the new node in the UAV node factory:

m_node_factory.add<ADC_Ammeter>("ADC Ammeter", *this);


The big advantage of having the serialization code generated is that it’s impossible to make mistakes.

The nice thing about having the UI generated is that the GS doesn’t have to change every time a new node is added or a new property is added/removed from an existing node config or init params.


So in the new GS I intend to keep this but I really want to get rid of the json definition file and replace it with a domain specific language. Main reason is that the json is not as expressive as I want and doesn’t support all the attributes I need – like ranges for the values.

I already started working on a flex/bison grammar for this and a library that will handle the reflection.

More details and a working example soon.