Using GCCXML to automate C++ wrappers creation

(2 votes, average: 5.00 out of 5)

GCCXML uses GCC as a front-end to parse C or C++ files. It then generates XML files for the interface, that is, it generates tags for the types and prototypes it parses. Then, pygccxml is a wrapper over it which parses the XML file to generate a Python object with every information one may need.

So I will quicly show here how it is possible to generate serialization/deserialization and then how to wrap functions with my custom serialization functions.

Using pygccxml

So first, I will parse the file whose name is in a variable named myfile. The config variable is only needed if you need additional include paths or stuff like this. Indeed, GCCXML crashes if one header is not readable (and thus accessible).

  import pygccxml
  config = pygccxml.parser.config.config_t(include_paths = "Additional paths you may need")
 
  d = pygccxml.parser.parse([myfile], config=config)
  global_ns = pygccxml.declarations.get_global_namespace(d[0])

Once the file is parsed, you can get its global namespace: it’s where you will be able to efficiently access every type and method/function.

Serialization/deserialization

As the XML library, I’ve chosen TinyXML. So the serialization/deserialization functions look like:

  Type& get_value(const TiXmlElement* node, Type& value);
  void set_value(TiXmlElement* node, const Type& value);

I’ve written these functions for the simple types (int, float, …) as well as for STL containers. It’s easy to write, so I won’t write them here (some lexical_cast<> are enough for the native types, and a for loop is needed for the container, that’s all). get_value() modifies the argument based on an XML element, and set_value() sets the XML Element according to the

Let’s say I’d like to get all type declarations in a header file that I’ve already parsed and I have know global_ns

def get_members(new_class):
  members = {}
  for variable in new_class.variables():
    if variable.access_type == "public":
      members[variable.name] = variable.type.decl_string
  return members

For each variable in a class, if it is public, I will add it to the member to serialize or deserialize. Its name is given by the property name and its complete type by the property type.decl_string.

Now that I can extract the scope of a class. This is not recursively until I hit the top_parent namespace.

def get_scope(namespace):
  scope = []
  current_namespace = namespace.parent
  while(current_namespace != namespace.top_parent):
    scope.append(current_namespace.name)
    current_namespace = current_namespace.parent
 
  scope = scope[::-1]
  return scope

def populate_classes_in_namespace(namespace, complete_path):
  structures = {}
 
  for new_class in namespace.classes():
    if(new_class.location._file_name == complete_path):
      members = get_members(new_class)
      scope = get_scope(new_class)
 
      structures['::'.join(scope + [new_class.name])] = members
  return structures
 
structures = populate_functions_in_namespace(global_ns, os.path.abspath(myfile))

Here, I browse the different class definitions GCCXML has found, and if the declaration was made in my header file (I have to check this because GCCXML populates the structure with the types in every included file), I extract its member and its namespace scope. This is then saved in a dictionary.

Now, to generate a serialization/deserialization functions couple, I can write their prototype and then the source code like this (for instance):

def write_source(structures, header_name, source_xml):
  """
  Writes the XML source file
  """
  source_xml.write("""/**
 * \\file xml_%s.cpp
 */
 
#include <stdexcept>
 
#include "xml_%s.h"
#include <xml/serilaization_types.h>
 
namespace XML
{
""" % (header_name, header_name))
 
  for name, structure in structures.items():
    source_xml.write("""
  %s& get_value(const TiXmlElement* node, %s& value)
  {
""" % (name, name))
 
    for variable, type in structure.items():
      source_xml.write("""    const TiXmlElement* node_%s = node->FirstChildElement("%s");
    if(node_%s == NULL)
    {
      throw std::invalid_argument("Argument %s invalid");
    }
    get_value(node_%s, value.%s);
""" % (variable, variable, variable, variable, variable, variable))  
 
    source_xml.write("""    return value;
  }
""")
 
    source_xml.write("""
  void set_value(TiXmlElement* node, const %s& value)
  {
""" % (name))
 
    for variable, type in structure.items():
      source_xml.write("""    TiXmlElement* node_%s = new TiXmlElement("%s");
    set_value(node_%s, value.%s);
    node->LinkEndChild(node_%s);
""" % (variable, variable, variable, variable, variable))  
 
    source_xml.write("""  }
""")
 
  source_xml.write("""}
""")

Of course, there are some pitfalls in what I’ve just written: I’ve assumed that the types inside my structure can be serialized, that I have no static member, … This is left as an exercice.

Wrapping functions

Wrapping functions with this can get to RPC, and thus SOAP, as I’m using here XML as channel.
In fact, there are not many differences between wrapping functions and wrapping types. The only difference is that I don’t search for the same things in pygccxml structure. Besides, functions can modify their parameters, there may be pointers, … I myself only dealt with functions modifying their parameters, but I will not add this complexity here.

def get_characteristics(member):
  """
  List of a fucntion parameters
  """
  args = []
  for arg in member.arguments:
    args.append((arg.name, arg.type.decl_string))
 
  return member.return_type.decl_string, args
 
def get_members(new_class):
  """
  Map of class members
  """
  members = {}
  for member in new_class.public_members:
    if (member.__class__.__name__ != "member_function_t") or (member.has_static):
      continue
    members[member.name] = get_characteristics(member)
  return members
 
def populate_functions_in_namespace(namespace, complete_path):
  structures = {}
 
  for new_class in namespace.classes():
    if(new_class.location._file_name == complete_path):
      members = get_members(new_class)
      scope = get_scope(new_class)
 
      structures['::'.join(scope + [new_class.name])] = members
  return structures

Here I save inside a map another map of function members. Now, I can create the RPC functions. Depending on which side you are, you need a class with the same interface that will make the RPC call, thus serializing parameters and deserialize the result after, and a class called by the RPC server that will deserialize the parameters, make the actual method call and then serialize the result.

For instance, generating a method call for a class can be done this way, with inner being a pointer to the actual wrapped class instance:

  TiXmlElement* XML_%s_Server::%s(const TiXmlElement* input)
  {
    TiXmlElement* root = new TiXmlElement("%sResponse");
""" % (structure_name, function_name, function_name))
 
  for argument in arguments[1]:
    source_xml.write("""
    boost::remove_const<boost::remove_reference< %s >::type>::type %s;\n""" % (argument[1], argument[0]))
    source_soap.write("""    const TiXmlElement* node_in_%s = input->FirstChildElement(\"%s\");
    if(node_in_%s == NULL)
    {
      throw std::invalid_argument("Missing argument %s");
    }
    XML::get_value(node_in_%s, %s);
""" % (argument[0], argument[0], argument[0], argument[0],argument[0], argument[0]))
 
  if arguments[0] != "void":
    source_xml.write("\n    boost::remove_const<boost::remove_reference< %s >::type>::type result;\n" % (arguments[0]))
    source_xml.write("    result = inner->%s(%s);\n" % (function_name, ",".join([argument[0] for argument in arguments[1]])))
  else:
    source_xml.write("\n    inner->%s(%s);\n" % (function_name, ",".join([argument[0] for argument in arguments[1]])))
 
  if arguments[0] != "void":
      source_xml.write("""
    TiXmlElement* node_out = new TiXmlElement("result");
    XML::set_value(node_out, result);
    root->LinkEndChild(node_out);
""")
 
  source_xml.write("""
    return root;
  }
 
""")

Final word

I’m using pygccxml for some RPCs in a client/server environment for clusters, to know if a computation is done or how much time is remaining. It is easy to use SCons with pygccxml to create a Builder that will create the files on the fly (and everything stays in Python).

So a big thanks to the GCCXML and pygccxml teams for this work.

Tags: C++, GCCXML, Python, Serialization

No Comments yet »

Matthieu Brucher's blog