Tag Archives: code generation

Definition Language

aaand it’s done!

The definition language for specifying node configuration is finished – at least in principle.

It ended up being a stand-alone library called def_lang (uninspired, I know)

Basically I can now define things like this:

namespace silk
{
  alias mass_t = ufloat;

  struct UAV_Config_Base
  {
    string name : [ui_name = "Name"];
    mass_t mass : [ui_name = "Mass (Kg)"];
    ufloat moment_of_inertia : [ui_name = "Moment Of Inertia"];
  };

  struct Multirotor_Config : public UAV_Config_Base
  {
    struct Motor
    {
      math::vec3f position;
      direction_t thrust_vector = {0.f, 0.f, 1.f};
      bool clockwise = false;
    };

    ufloat radius = 1.f; //m
    ufloat height = 1.f; //m
    ufloat motor_z_torque = 0.f; //Nm
    ufloat motor_thrust = 1.f; //N
    ufloat motor_acceleration = 10.f; //N/s
    ufloat motor_deceleration = 10.f; //N/s
    vector<Motor> motors;
  };
}

And use this definition in 2 possible ways:

    • Dynamically, through a type system:
ts::ast::Builder builder;

auto parse_result = builder.parse_file("definition_file.def");
if (parse_result != ts::success)
{
  std::cerr << parse_result.error().what();
  return;
}

//walk and display the AST
std::cout << builder.get_ast_root_node().to_string(0, true) << std::endl;

ts::Type_System ts;
ts.populate_builtin_types(); //this adds standard built-in types like float and double

auto compile_result = builder.compile(ts); //populate the typesystem from the AST
if (compile_result != ts::success)
{
  std::cerr << compile_result.error().what();
  return;
}

//now the type system is fully populated and all the types is it are instantiable
std::shared_ptr<ts::IStruct_Type> base_type = ts.find_specialized_symbol_by_path<ts::IStruct_Type>("silk::UAV_Config_Base");
std::shared_ptr<ts::IStruct_Type> type = ts.find_specialized_symbol_by_path<ts::IStruct_Type>("silk::Multirotor_Config");

assert(base_type->is_base_of(*type)); //inheritance is respected

std::shared_ptr<ts::IStruct_Value> multirotor_config_value = type->create_specialized_value(); //create a value of type Multirotor_Config
auto construct_result = multirotor_config_value->construct(); //construct it. This struct is default constructible. An alternative is to initialize it using an initializer list

//let's get (select) some members
//the full dot sintax is supported when selecting so things like motors[0].position works
std::shared_ptr<ts::IString_Value> name_value = multirotor_config_value->select_specialized<ts::IString_Value>("name");

//all setters might fail so I need to check the return value
auto result = name_value->set_value("silkopter");
TS_ASSERT(result == ts::success);

//values are serializable as well to an intermediary tree
auto serialize_result = multirotor_config_value->serialize();
TS_ASSERT(serialize_result == ts::success);

//the intermediary tree can be serialized to json or binary
std::string json = ts::serialization::to_json(serialize_result.payload(), true);

//a more complicated example:
//get the motors vector
std::shared_ptr<ts::IVector_Value> motors_vector_value = multirotor_config_value->select_specialized<ts::IVector_Value>("motors");

//create a new motor
std::shared_ptr<ts::IStruct_Value> motor_value = ts.create_value<ts::IStruct_Type>("silk::Multirotor_Config::Motor");

//change it's position
motor_value->select_specialized<ts::IVec3f_Value>("position")->set_value(ts::vec3f(0, 1, 0));

//add it to the vector
auto result = motors_vector_value->insert_value(0, motor_value);
assert(result == ts::success);

Basically  it’s a full type system implementation similar to the C++ one with type safety, templated types, aliases, values, casting, inheritance etc etc. But on top of the standard C++ features it also allows custom attributes on members and types (like ui_name for example), it supports reflection so everything can be enumerated and it’s serializable. This dynamic one will be used in the GS to generate UI for all the nodes. Like this the GS can work with new nodes without me having to do custom UI for each node I intend to add (or everytime I change the node config to add/remove a parameter, or change a type etc)

 

  • Statically through generated C++ code

From the definition above I can also generate C++ code for all the structs and work with them through getters and setters. Each type in the type system can be mapped to a native C++ type. So in this example, the above structure would be usable like this from generated code:

assert(std::is_base_of<silk::UAV_Config_Base, silk::Multirotor_Config>::value); //inheritance is respected

//create a value of type Multirotor_Config. This also default constructs it
silk::Multirotor_Config multirotor_config_value;

//all members have setters and getters generated for them
std::string name_value = multirotor_config_value.get_name();
multirotor_config.set_value("silkopter");

//values are serializable as well to an intermediary tree
auto serialize_result = silk::serialize(multirotor_config_value); //serialization methods are generated as well
TS_ASSERT(serialize_result == ts::success);

//the intermediary tree can be serialized to json or binary
std::string json = ts::serialization::to_json(serialize_result.payload(), true);

//a more complicated example:
//get the motors vector
std::vector<silk::Multirotor_Config::Motor> motors_vector_value = multirotor_config_value.get_motors();

//create a new motor
silk::Multirotor_Config::Motor motor_value;

//change it's position
motor_value.set_position(ts::vec3f(0, 1, 0));

//add it to the vector
motors_vector_value.insert(motors_vector_value.begin(), motor_value);

//set it back in the config
multirotor_config_value.set_motors(motors_vector_value);

The generated code is simpler to work with partly because it’s a lower level of abstraction (I use the build-in C++ typesystem and work in value space) but also because it’s familiar.

The interested thing is that the generated code keeps inside as a string the definition file contents that it was generated from. This is useful to implement the following flow:

  1. I write the definition file for the bran and all its nodes
  2. I generate the c++ code for the brain. Now all the nodes can use C++ structs for the parameters and config structs. Simple and type-safe as it’s impossible to access non-existing fields or use them with the wrong type. It’s standard, simple C++ structs
  3. When the brain connects to the GS, it sends the def file it was generated from
  4. The GS instantiates a dynamic Typesystem and parses the def file from the brain, so now it will understand and be able to (de)serialize the data it will receive from the brain – like node configs, etc

If I want to add a new node, I add in the def, regenerate the code and use the resulting config struct in the new node. The GS will automatically know how to handle the new node as it will receive the def for it next time it connects to the brain

 

Easy peasy

Advertisements

Node properties

One of the nice things about silkopter is that is very easy to add new types of nodes to the system.

A node can be anything from a sensor to a low pass filter to a PWM generator like PIGPIO. All nodes have the following properties:

  • Zero or more input streams. Each stream has a type and a rate
  • Zero or more output streams. Same as the inputs, they have a type and rate
  • An Init_Params struct that decides the initialization params of the node. Once the node is created this cannot be changed. The usual params that go in this struct is the rate (process frequency), number of channels for a PWM sink etc.
  • A Config struct that holds changable configuration. This can be changed at any time and contains PID settings for example

The Init_Params and the Config structs are custom, per node. They have to be serializable to json for loading/saving, serializable to binary for comms (the json serialization can be used here) and they need to be editable in an UI for the Ground Station

Silkopter defines the Init_Params and Comms in a json file – one per node – and then generates the actual C++ structs and serialization code from this json description. The UI for editing is dynamic, based on the serialized json of the structs.

So for example the load flow is this:

  • the settings.json file is loaded and converted to a json structure using the rapidjson library
  • the list of nodes is iterated and nodes are created from it
  • each node has its init method called with the init_params json passed to it
  • the init method deserializes the init_params json using generated code into a Init_Params C++ struct

The save flow is the inverse of the one above.

Sending the node Init_Params and the Config to the GS and editing it:

  • The get_init_params (or get_config) is called for a node. This serializes the internal Init_Params or Config to json using the generated code
  • The json stringified using rapidjson and sent to the GS
  • The GS parses the string using rapidjson back into a json data structure
  • A QT item model is used over this json to populate a tree view with the data
  • When data is changed by QT as a response to user interaction (typing a new value for example) the json changes. As a result of this its serialized back to string, sent to the Brain which then calls set_config with the json.

So in order to add a new node, 3 steps have to be done – all of them in the brain. The GS is unchanged:

  1. Create the node C++ class with all the processing needed
  2. Create a json description of the Init_Params and Config and generate the code for them using autojsoncxx library
[
    {
        "definition" : true,
        "namespace" : "sz::ADC_Ammeter",
        "name": "Init_Params",
        "members":
        [
            ["uint32_t", "rate", {"required": true, "json_key" : "Rate (Hz)", "default" : 0}]
        ]
    },
    {
        "definition" : true,
        "namespace" : "sz::ADC_Ammeter",
        "name": "Config",
        "members":
        [
            ["float", "scale", {"required": false, "json_key" : "Scale", "default" : 1}],
            ["float", "bias", {"required": false, "json_key" : "Bias", "default" : 0}]
        ]
    }
]

3. Register the new node in the UAV node factory:

m_node_factory.add<ADC_Ammeter>("ADC Ammeter", *this);

 

The big advantage of having the serialization code generated is that it’s impossible to make mistakes.

The nice thing about having the UI generated is that the GS doesn’t have to change every time a new node is added or a new property is added/removed from an existing node config or init params.

 

So in the new GS I intend to keep this but I really want to get rid of the json definition file and replace it with a domain specific language. Main reason is that the json is not as expressive as I want and doesn’t support all the attributes I need – like ranges for the values.

I already started working on a flex/bison grammar for this and a library that will handle the reflection.

More details and a working example soon.