Matt’s blog

When moving to Python, the real big problem that arises is the transformation of a Python array into the C++ container the team used for years.
Let’s set some hypothesis :

there is a separation between the class containing the data and the class that uses the data (iterators, …)
the containing class can be changed (policy or strategy pattern)

The first hypothesis is derived from the responsibility principle, the two classes have two distinct responsibilities, the first allocates the data space and allows simple access to it, the second allows usual operations (assignation, comparison tests or iterations for instance).
The second one will be the heart of the wrapper. It allows to change the way data is stored and accessed in a simple way.

So here is a simplification of a 3D container class that will store a 3D numpy array. It must be capable of creating a container from a PyObject* and have a method to get the stored array, here getContainer():

^?View Code CPP

#include &lt;Python.h&gt;
#include &lt;numpy/arrayobject.h&gt;
 
  template&lt;class Im3DValue&gt;
  class Container3DPython
  {
  protected:
    /// The inner data object
    PyArrayObject* imageData;
 
    /// Shortcut for self type reference
    typedef Container3DPython&lt;Im3DValue&gt; Self;
  public:
    /// So that other templates know what Im3DValue is
    typedef Im3DValue value_type;
 
  protected:
    /**
     * Tries to allocate a new 3D array
     * @param width is the width of the new array
     * @param depth is the depth of the array
     * @param height is the height of the array
     */
    void allocate(int width, int height, int depth)
    {
      deallocate();
      PyArray_Dims dims;
      dims.ptr = (npy_intp*)malloc(3 * sizeof(npy_intp));
      dims.len = 3;
      dims.ptr[2] = width;
      dims.ptr[1] = height;
      dims.ptr[0] = depth;
 
      imageData = reinterpret_cast&lt;PyArrayObject*&gt;(PyArray_SimpleNewFromDescr(dims.len, dims.ptr, PyArray_DescrNewFromType(DataTypeTraits&lt;value_type&gt;::type_num)));
      Py_INCREF(imageData);
 
      free(dims.ptr);
    }
 
    /**
     * Tries to allocate an array based on a Python object
     * @param obj is the Python object that will be shared or copied, depending on its content and type
     */
    void allocate(PyObject* obj)
    {
      deallocate();
      PyArrayObject* array = reinterpret_cast&lt;PyArrayObject*&gt;(PyArray_FromAny(obj, NULL, 3, 3, NPY_FARRAY, NULL));
      imageData = reinterpret_cast&lt;PyArrayObject*&gt;(PyArray_CastToType(array, PyArray_DescrNewFromType(DataTypeTraits&lt;value_type&gt;::type_num), 0));
      Py_INCREF(imageData);
      Py_DECREF(array);
    }
 
    /// Deallocates the Python object
    void deallocate()
    {
      Py_XDECREF(imageData);
      imageData = NULL;
    }
 
  public:
    ///@{ \name Image3D Accessors
    /**
     * Returns the width of the image
     */
    unsigned int width() const
    {
      return imageData-&gt;dimensions[2];
    }
    /**
     * Returns the depth of the image
     */
    unsigned int height() const
    {
      return imageData-&gt;dimensions[1];
    }
    /**
     * Returns the depth of the image
     * @return the depth of the image
     */
    unsigned int depth() const
    {
      return imageData-&gt;dimensions[0];
    }
    /**
     * Gets the value at a position in the image
     * @param x is the first coordinate
     * @param y is the second coordinate
     * @param z is the third coordinate
     * @return a reference to the value
     */
    Im3DValue&amp; operator()(int x,int y,int z)
    {
      return reinterpret_cast&lt;value_type*&gt;(imageData-&gt;data)[x+y*width()+z*width()*height()];
    }
    /**
     * Gets the value at a position in the image
     * @param x is the first coordinate
     * @param y is the second coordinate
     * @param z is the third coordinate
     * @return a const reference to the value
     */
    const Im3DValue&amp; operator()(int x,int y,int z) const
    {
      return reinterpret_cast&lt;value_type*&gt;(imageData-&gt;data)[x+y*width()+z*width()*height()];
    }
    /**
     * Returns the inner shared pointer
     * @return the shared pointer
     */
    PyArrayObject* getContainer() const
    {
      Py_XINCREF(imageData);
      return imageData;
    }
    ///@}
 
    ///@{ \name Operators
    /**
     * Assignment operator from a different type of image
     * @param other is the array
     * @return self
     */
    Container3DPython&amp; operator=(PyObject* other)
    {
      allocate(other);
      return *this;
    }
 
    /// Destructor
    virtual ~Container3DPython()
    {
      deallocate();
    }
    /**
     * Simple constructor
     * @param width is the width of the new image
     * @param height is the height of the new image
     * @param depth is the depth of the new image
     */
    Container3DPython(unsigned int width, unsigned int height, unsigned int depth)
    : imageData(NULL)
    {
      allocate(width, height, depth);
    }
 
    /**
     * Constructor from an array
     * @param array is the array to copy
     */
    Container3DPython(PyObject* array)
    :imageData(NULL)
    {
      allocate(array);
    }
    ///@}
  };

Once this skeleton is ready, it is possible to create typedefs to reference new containers.

^?View Code CPP

typedef Container&lt;Container3DPython&lt;float&gt; &gt; Container3DfPython;

Once this is done, the associated swig file will declare some simple typemaps :

^?View Code SWIG

%{
#define SWIG_FILE_WITH_INIT
#define PY_ARRAY_UNIQUE_SYMBOL PyArray_API
%}
%include "numpy.i"
%init %{
import_array();
%}
%{
#include "stdio.h"
#include &lt;numpy/arrayobject.h&gt;
#include &lt;Container3DPython.hpp&gt;
#include &lt;Container.hpp&gt;
%}
 
%typemap(in) Container3DfPython
{
  $1 = new Container3DfPython($input);
}
 
%typemap(freearg) Container3DfPython
{
  delete $1;
}
 
%typemap(out) Container3DfPython
{
  $result = reinterpret_cast&lt;PyObject*&gt;($1-&gt;getContainer());
  delete $1;
}

With this method, the wrapped container can be passed to a class constructor and stored in an instance, deleted at the end. If the numpy SWIG wrappers are used, only pointers are given to the function, and it is freed at the end of the function, thus class instance cannot be created and used later.

Note that the described container class will cast the array into the appropriate type thanks to a trait structure DataTypeTraits with the attribute type_num. This means that the array may not be modified in place. If this is not intended, use typechecks.

Note also that if you have multiple C++ files that uses the container, you will have to define NO_IMPORT_ARRAY for them (but not for the SWIG generated file).

1 Comment »

Wrapping a C++ container in Python

One Response to “Wrapping a C++ container in Python”

Leave a Reply

Search

Categories

Family matters

LinkedIn

Others

Scientific computing

Monthly

Pages

Meta