Tag Archives: DSL

Definition Language

aaand it’s done!

The definition language for specifying node configuration is finished – at least in principle.

It ended up being a stand-alone library called def_lang (uninspired, I know)

Basically I can now define things like this:

namespace silk
{
  alias mass_t = ufloat;

  struct UAV_Config_Base
  {
    string name : [ui_name = "Name"];
    mass_t mass : [ui_name = "Mass (Kg)"];
    ufloat moment_of_inertia : [ui_name = "Moment Of Inertia"];
  };

  struct Multirotor_Config : public UAV_Config_Base
  {
    struct Motor
    {
      math::vec3f position;
      direction_t thrust_vector = {0.f, 0.f, 1.f};
      bool clockwise = false;
    };

    ufloat radius = 1.f; //m
    ufloat height = 1.f; //m
    ufloat motor_z_torque = 0.f; //Nm
    ufloat motor_thrust = 1.f; //N
    ufloat motor_acceleration = 10.f; //N/s
    ufloat motor_deceleration = 10.f; //N/s
    vector<Motor> motors;
  };
}

And use this definition in 2 possible ways:

    • Dynamically, through a type system:
ts::ast::Builder builder;

auto parse_result = builder.parse_file("definition_file.def");
if (parse_result != ts::success)
{
  std::cerr << parse_result.error().what();
  return;
}

//walk and display the AST
std::cout << builder.get_ast_root_node().to_string(0, true) << std::endl;

ts::Type_System ts;
ts.populate_builtin_types(); //this adds standard built-in types like float and double

auto compile_result = builder.compile(ts); //populate the typesystem from the AST
if (compile_result != ts::success)
{
  std::cerr << compile_result.error().what();
  return;
}

//now the type system is fully populated and all the types is it are instantiable
std::shared_ptr<ts::IStruct_Type> base_type = ts.find_specialized_symbol_by_path<ts::IStruct_Type>("silk::UAV_Config_Base");
std::shared_ptr<ts::IStruct_Type> type = ts.find_specialized_symbol_by_path<ts::IStruct_Type>("silk::Multirotor_Config");

assert(base_type->is_base_of(*type)); //inheritance is respected

std::shared_ptr<ts::IStruct_Value> multirotor_config_value = type->create_specialized_value(); //create a value of type Multirotor_Config
auto construct_result = multirotor_config_value->construct(); //construct it. This struct is default constructible. An alternative is to initialize it using an initializer list

//let's get (select) some members
//the full dot sintax is supported when selecting so things like motors[0].position works
std::shared_ptr<ts::IString_Value> name_value = multirotor_config_value->select_specialized<ts::IString_Value>("name");

//all setters might fail so I need to check the return value
auto result = name_value->set_value("silkopter");
TS_ASSERT(result == ts::success);

//values are serializable as well to an intermediary tree
auto serialize_result = multirotor_config_value->serialize();
TS_ASSERT(serialize_result == ts::success);

//the intermediary tree can be serialized to json or binary
std::string json = ts::serialization::to_json(serialize_result.payload(), true);

//a more complicated example:
//get the motors vector
std::shared_ptr<ts::IVector_Value> motors_vector_value = multirotor_config_value->select_specialized<ts::IVector_Value>("motors");

//create a new motor
std::shared_ptr<ts::IStruct_Value> motor_value = ts.create_value<ts::IStruct_Type>("silk::Multirotor_Config::Motor");

//change it's position
motor_value->select_specialized<ts::IVec3f_Value>("position")->set_value(ts::vec3f(0, 1, 0));

//add it to the vector
auto result = motors_vector_value->insert_value(0, motor_value);
assert(result == ts::success);

Basically  it’s a full type system implementation similar to the C++ one with type safety, templated types, aliases, values, casting, inheritance etc etc. But on top of the standard C++ features it also allows custom attributes on members and types (like ui_name for example), it supports reflection so everything can be enumerated and it’s serializable. This dynamic one will be used in the GS to generate UI for all the nodes. Like this the GS can work with new nodes without me having to do custom UI for each node I intend to add (or everytime I change the node config to add/remove a parameter, or change a type etc)

 

  • Statically through generated C++ code

From the definition above I can also generate C++ code for all the structs and work with them through getters and setters. Each type in the type system can be mapped to a native C++ type. So in this example, the above structure would be usable like this from generated code:

assert(std::is_base_of<silk::UAV_Config_Base, silk::Multirotor_Config>::value); //inheritance is respected

//create a value of type Multirotor_Config. This also default constructs it
silk::Multirotor_Config multirotor_config_value;

//all members have setters and getters generated for them
std::string name_value = multirotor_config_value.get_name();
multirotor_config.set_value("silkopter");

//values are serializable as well to an intermediary tree
auto serialize_result = silk::serialize(multirotor_config_value); //serialization methods are generated as well
TS_ASSERT(serialize_result == ts::success);

//the intermediary tree can be serialized to json or binary
std::string json = ts::serialization::to_json(serialize_result.payload(), true);

//a more complicated example:
//get the motors vector
std::vector<silk::Multirotor_Config::Motor> motors_vector_value = multirotor_config_value.get_motors();

//create a new motor
silk::Multirotor_Config::Motor motor_value;

//change it's position
motor_value.set_position(ts::vec3f(0, 1, 0));

//add it to the vector
motors_vector_value.insert(motors_vector_value.begin(), motor_value);

//set it back in the config
multirotor_config_value.set_motors(motors_vector_value);

The generated code is simpler to work with partly because it’s a lower level of abstraction (I use the build-in C++ typesystem and work in value space) but also because it’s familiar.

The interested thing is that the generated code keeps inside as a string the definition file contents that it was generated from. This is useful to implement the following flow:

  1. I write the definition file for the bran and all its nodes
  2. I generate the c++ code for the brain. Now all the nodes can use C++ structs for the parameters and config structs. Simple and type-safe as it’s impossible to access non-existing fields or use them with the wrong type. It’s standard, simple C++ structs
  3. When the brain connects to the GS, it sends the def file it was generated from
  4. The GS instantiates a dynamic Typesystem and parses the def file from the brain, so now it will understand and be able to (de)serialize the data it will receive from the brain – like node configs, etc

If I want to add a new node, I add in the def, regenerate the code and use the resulting config struct in the new node. The GS will automatically know how to handle the new node as it will receive the def for it next time it connects to the brain

 

Easy peasy

Advertisements

Node Definition

As I mentioned in my previous post I’m designing a domain specific language to define the node properties. So far it looks like this:

import "UAV_Config.def"

namespace silk
{

struct Multirotor_Config : public UAV_Config
{
    struct Motor
    {
        vec3f position = {0, 0, 0};
        bool clockwise = false;
    };

    string name;
    float mass = 1.f : [ ui_name = "Mass (Kg)", min = 0.f ];
    float height = 0.5f : [ ui_name = "Height (m)", min = 0.f ];
    float radius = 0.5f : [ ui_name = "Radius (m)", min = 0.f ];
    float motor_thrust = 1.f : [ ui_name = "Motor Thrust (N)", min = 0.f ];
    float motor_z_thrust = 1.f : [ ui_name = "Motor Z Torque (Nm)", min = 0.f ];
    float motor_acceleration = 10.f : [ ui_name = "Motor Acceleration (N/s)", min = 0.f ];
    float motor_deceleration = 10.f : [ ui_name = "Motor Deceleration (N/s)", min = 0.f ];
    vector<Motor> motors : [ ui_name = "Motors" ];
}

}

It looks a lot like C++ but allows some extra attributes on types and members, like ui_name to have a nicer name displayed in the editor, a min/max value for numeric types, the number of decimals for floating points etc.

The grammar is done with flex/bison and it will actually be a full typesystem that I will use to generate serialization/deserialization code offline and to create UIs online in the GS.

The language will support:

  • namespaces, needed to be able to generate the code in the correct namespace in C++ and to separate the types properly
  • basic types, like ints, floats but also uint8_t and friends
  • structs and classes together with inheritance.
  • templated types like vectors.
  • math types like vec2/3/4, quat and matrices
  • initializer lists for vec3/4 etc
  • custom attributes
  • imports

The compiler will take a file and output a typesystem that contains all the types defined in the file. These types will have reflection so they can be traversed. From this typesystem I can generate the C++ code for the brain – the serialization/deserialization to binary and json – and I can generate UI widgets in the GS as well.

I’m still struggling with the grammar definition but over this weekend I will get it done.

I have to admit – doing a domain specific language and a typesystem is actually very satisfying and challenging. I wish C++ would help a bit more with common things like reflection and such. I even considered moving everything to rust or go but then I’ll have another problem – QT and other library bindings…