Pelib: Programming Environment Library

Documentation and Examples

Overview Publications Download License Ongoing Work Contact Acknowledgments

You can access to this page through the link http://www.ida.liu.se/labs/pelab/pelib/index.en.shtml


Pelib stands for Programming Environment Library, as the reference to PELAB (Programming Environment Laboratory). It is a swiss army knife for various research activity, in particular around on-chip pipelining and scheduling research. The pelib includes a flexible C++ programming interface to read and write simple and complex data structure in various text format such as AMPL input data, AMPL output results, GraphML-based streaming applications models and XML-based schedule descriptions. The capability to parse and generate AMPL input and output data simplifies considerably the automation of experimental performance comparison between Linear Programming solvers and C++ algorithm implementations. A C++ program can generate AMPL input problem instances to feed a linear solver with a AMPL frontend and parse its output to compare with another solving method, such as a C++ implementation of a heuristic. We use this approach in our previous and future research papers. The pelib also provides convenient generic C data structures with type checking, based on an object-oriented design, providing fast ways to show the precise state of internal data structures and functionality composition. These data structures are extensively used used in the Drake C stream programming framework.
This pages reviews the main features describes above, how to use and extend them. Section overview gives a brief overview of pelib. Section download provides downloading link to the latest pelib source package. Section licence provides information about our publication licence. Section publication gives the list of articles we published using this library. Section installation provides some explanations to compile and install the library. Section Parsing and generating text representation of complex data structures gives code examples on how to use the parser and output classes provded by pelib and describe the underlying structure to respect when writing new parser and output classes. Section Generic C data structures gives an extended description of the C data structures implemented, how to extend their functionalities as well as adding new generic data structures. Check also the doxygen documentation for more detailed information. Section Ongoing work details the additional features planned for the future. Section Contact gives contact information if you want to contact us. Finally, the page ends with Acknowledgments.


Overview

Main features:



Download

Download source code

Download unit test source code



Software License

The source code available above is released under GPLv3 licence. Please contact Nicolas Melot (nicolas.melot@liu.se) if you want to obtain another licence.



Publications

This library is heavily used internally by Crown scheduling heuristic C++ implementations, C++ schedule analytic evaluators used in our Crown Scheduler paper as well as in the Drake C stream programming framework. Below is a list of papers whose experimental work relied on the pelib library.


Installation

Pelib requires the libraries igraph (>= 0.7.1), libdl, libboost-regex, libboost-graph and libexpat, pthread and libxml++-2.6 to compile and run correctly. Once the dependencies are installed, define the environment variables CC=gcc (or any other C compiler you prefer, provided its switches are the same as gcc) and CXX=g++. Download and extract the pelib source package. cd' into the pelib source directory, make sure you use GNU make and type

make install
to compile and install the pelib. This will install pelib libraries in $HOME/.local/lib, pelib include files in $HOME/.local/include and pkg-config file in $HOME/.local/lib/pkgconfig. If you wish to install it elsewhere, type
make install prefix=</any/path/at/all>
Make will create directories bin, lib, lib/pkgconfig and include in the the installation directory you may choose. The compilation of Pelib is quite long due to inefficient Makefiles, so feel free to do something else while pelib is being compiled.



Parsing and generating text representation of complex data structures

This section gives examples on pelib's functionalities to parse complex data structures. Section General usage gives example on how to use parser and output classes provided by the pelib. Section Contributing gives general indication on how to write your own parser or output class that integrates in pelib.

General usage

This section gives general usage of the pelib parsing and output framework as well as the conversion command line tool pelib-convert.

Pelib parsing and output framework

We suppose in this example that we want to load the value of p written in the AMPL input data file "parse-example-1.dat" as follows:
param p := 3.141592653;
The code snippet below uses the AMPL input data parser to reads the input file "example.dat" and outputs the value of scalar "p" to standard output.
#include <pelib/AmplInput.hpp>

using namespace std;
using namespace pelib;

int
main(int argc, char** argv)
{
	// Variables declaration
        ifstream input; // File input stream
        AmplInput parser(AmplInput::intFloatHandlers()); // AMPL input data parser class with integer and float input parsers and output
        Algebra container; // Collection of Algebraic data structures
        const Scalar<float> *scalar_p; // A scalar
        float p; // Final variable that receives the value parsed

	// Open the file, use AmplInput to parse it into an Algebra container and close the file
        input.open("parse-example-1-data.dat");
        container = parser.parse(input);
        input.close();

	// Find and return the floating-point scalar value named "p" in the algebra container
        scalar_p = container.find<Scalar<float> >("p");

	// Read scalr p's value into float variable p
        p = scalar_p->getValue();

	// Output p
        cout << "p = " &tl;< p << endl;

        return 0;
}
You can build this example with the command line
g++ example-parse-1.cpp `pkg-config --cflags --libs pelib` -o parse-example-1
Make sure the file parse-example-1.dat is available in the active directory and contains the text above, then simply type
./parser-example-1

Suppose now that we want to generate some AMPL output text format from a array of integers. The code snippet below performs this task:

#include <pelib/AmplOutput.hpp>
#include <pelib/Vector.hpp>

using namespace std;
using namespace pelib;

int
main(int argc, char** argv)
{
        // Variables declaration
        float F[4] = {1, 2.71, 3.14, 42};
        map<int, float> map_f;

        // Fills a map of values and their keys
        for(size_t i = 0; i < 4; i++)
        {
                map_f.insert(pair<int, float>((int)i + 1, F[i]));
        }

        // Builds a vector named "F" with values in map_f
        Vector<int, float> vector_f("F", map_f);

        // Output vector_f to standard output
        AmplOutput(AmplOutput::intFloatHandlers()).dump(cout, vector_f);

        return 0;
}

Suppose finally, that we want to convert an AMPL input data file into AMPL output format:

#include <pelib/AmplInput.hpp>
#include <pelib/AmplOutput.hpp>

using namespace std;
using namespace pelib;

int
main(int argc, char** argv)
{       
        ifstream input;
        Algebra container;
        const Scalar<float> *scalar_p;
        float p;
        
        input.open("example.dat");
        container = AmplInput(AmplInput::intFloatHandlers()).parse(input);
        input.close();

        AmplOutput(AmplOutput::intFloatHandlers()).dump(cout, container);

        return 0;
}

Parsers and output are also available for more complex data structures, such as streaming application descriptions (taskgraphs) and schedules. Just as Algebraic data structures have Algebra containers, taskgraphs, platform and schedule have their own container classes. Taskgraph and Schedule containers are instanciated by dedicated parsers classes (GraphML and XMLScehdule are available now, respectively for taskgraph and Schedules) and can be output by some output classes (GraphML and XMLSchedule can also output taskgraphs and schedules, respectively) in the exact same manner as shown above. Platform description have no format of its own yet, and is instead read as an algebraic container that is used to instanciate an instance of Platform. Taskgraph, Schedule and Platform classes can all be instanciated from an algebraic description of their respective data structure. Also, they can all build an Algebra container that holds all information about themselves. See the doxygen documentation to get more information about this feature.

Here is a list of classes able to parse and output data structures:

Formula parsing

Taskgraph description uses formulas to express quantities that can only be computed when more information is available, such as the number of cores available in the execution platform that should execute the streaming application described in a taskgraph. This is the case for the graph property "deadline" as well as the node property "efficiency".

The overal structure of a formula parsed by the pelib consists of an optional prefix and a body. The prefix is a string terminated by a colon ":" and that defines the parsing implementation used to parses the formula. The body is a string that respect the grammer used by the parsiing formula. If no prefix is provided, then the formula body is casted from string to C++ base type double. The prefix "exprtk:" enables parsing with exprtk. The symbols defined depend on the context in which the formula is parsed, that is if this is a deadline formula or an efficiency formula.

When parsing a formula to compute a taskgraph's deadline in millisecond, the following symbols are provided by pelib:

When parsing formula to compute a task's parallel efficiency, the symbols below are define:

Pelib conversion tool

This section provides a manual to the pelib flexible command line file format conversion tool pelib-convert. pelib-convert can read one or several input stream in any format pelib can parse and output it in any format pelib can output. It uses dynamic libraries to handle input and output streams so that any conversion scenario can be performed without having been explicitely programmed. Here is the syntax of pelib-convert

pelib-convert (--input ( --format <format> | --lib <library> ) ( --stdin | --file /path/to/input/file ) [ --args <arguments> [ -- ] ] [ --name <name> ] )* (--output ( --format <format> | --lib <library> ) ( --stdout | --stderr | --file /path/to/output/file ) [ --name <name> ] [ --args <arguments> [ -- ] ] )*
<format> takes shape as <structure>-<syntax> where <structure> is a complex data structure such as taskgraph, schedule or platform and <syntax> is its text format, such as graphml or xml as well as ampl_input or ampl_output. For algebraic data to parse or output, <structure> must be omitted and <syntax> is one of ampl_input or ampl_output.
<library> denotes the filename of a dynamic library (typically ending with .so) that contains the code to parse or output data in some format. If a path (absolute or relative) is not given, then pelib-convert searches for the library in folders defined by the environment variable LD_RUN_PATH as it was set when pelib-convert was linked, or in LD_LIBRARY_PATH, if defined. "--format <format>" is actually a shortcut for "--lib libpelib-<format>.so" is any sequence of strings, separated by one or several spaces or tabs. Arguments can be grouped using quote or double quotes or anything your shell provide to pass arguments with space characters. Arguments passed to the parsing or output library stop at the first occurrence of the string "--" or when the last command line argument is read.
<arguments> is any sequence of strings, separated by one or several spaces or tabs. Arguments can be grouped using quote or double quotes or anything your shell provide to pass arguments with space characters. Arguments passed to the parsing or output library stop at the first occurrence of the string "--" or when the last command line argument is read. "--" is mandatory if more arguments for pelib-convert follow. libtaskgraph-streamit takes one optional argument to name the taskgraph being parsed. None of the other libraries among ampl_*, taskgraph-*, schedule-* or platform-* require any argument.
<name> gives a id to identify an input data source so that several input of the same type (taskgraph, schedule or platform) can be given in a single command line. Some output library may require two instances of the same time. In this case, it needs to differenciate what input serves for a particular purpose. Refer the manual of the output library you use for more information. None of the output libraries ampl_*, taskgraph-*, schedule-* or platform-* require any such name.

As an example, the command line below is equivalent to the last example given in Section pelib parsing and output framework:

pelib-convert --input --format ampl_input --file example.dat --output --format ampl_output --stdout

Contributing

This section gives insights on the general structure of pelib parsers and output classes so that contributions can preserve its highly flexible feature. Section Extending pelib framework discusses the general structure of the pelib C++ framework and Section Extend pelib-convert features discusses the structure of a pelib-convert parsing and output plugin.

Extending pelib framework

Every parser and output classes in pelib store or read the data they parse or output in intermediate, format-agnostic data structure classes. These include Algebra, Taskgraph, Platform and Schedule. They serve as an independant pivot point when conversion between formats need to be performed, or as data to feed C++ algorithm implementation with or that C++ algorithm implementations produce. This is a central concept crucial to the flexibility of pelib framework.
Every data structure pelib can handle is a descendant of class pelib::Data or pelib::Record. Descendant classes of pelib::Data represent simple data structures that can be grouped in a collection that derives from pelib::Record. In contrast, pelib::Record classes represent complex data structures. In particular, any algebraic data structure (scalar, vector, set or matrix) derives from pelib::AlgebraData that is itself a descendant of pelib::Data, whereas classes Taskgraph, Platform and Schedule are complex data structures that derive from pelib::Record. All Algebraic data structures (that descend from pelib::AlgebraData) can be included in a algebraic data container (pelib::Algebra) that is considered as a complex data structure and derives from pelib::Record.

Parser and an output classes for simple data structures derives from pelib::DataParser and pelib::DataOutput, respectively, or both. More specifically, Algebraic data structure parser and output classes derive from pelib::AlgebraDataParser and pelib::AlgebraDataOutput, respectively, or both. In contrast, parsers and output classes descend from pelib::RecordParser or pelib::RecordOutput, respectively, or both. Again, an algebraic data structure collection parser or output derive from pelib::AlgebraParser or pelib::Algebra parser, that respectively derive from pelib::RecordParser and pelib::RecordOutput. Parser and output classes for Taskgraph, Platform and Schedule also derive from pelib::RecordParser and pelib::RecordOutput (or again, from both). Each simple or complex parser or output class derive from pelib::DataParser or pelib::RecordParser, or their output counterpart, to classes that specialize into different text formats, for instance pelib::GraphMLParser or pelib::AmplInputParser. Finally, parser and output classes specialized for a given format are merged into a single parsing and output class, such as pelib::GraphML or pelib::AmplInput.

Parser and output classes of complex data structure composed as several simple data structures (such as pelib::Algebra) are composed of several parsers of simple data structures. For example, pelib::AmplInput parses simple data structure such as pelib::Scalar, pelib::Vector, pelib::Set and pelib::Matrix using pelib::AmplInputScalar (see doxygen documentation for a detailed view of its inheritance diagram), pelib::AmplInputVector, pelib::AmplInputSet and pelib::AmplInputMatrix. These classes can also generate outputs in AmplInput formats because they inherit from pelib::AmplInputDataOutput.

There are several possible scenarii to extend pelib with new parser or output.

Extend pelib-convert features

This section explain how to create plugins to add conversion features to pelib-convert. The plugin takes shape of a dynamic library that can be built using the pelib framework to parse or output data structures. The library needs to implement the C function pelib_parse() (#include <pelib/parser.h>) if must offer a parsing feature and pelib_dump() (#include <pelib/output.h>). Below is the code for a plugin that parses and output Taskgraphs in GraphML text format.

#include <iostream>

#include <pelib/parser.h>
#include <pelib/output.h>

#include <pelib/GraphML.hpp>

using namespace std;
using namespace pelib;

#ifdef __cplusplus
extern "C" {
#endif

pelib::Record*
pelib_parse(std::istream& cin, size_t argc, char **argv)
{
        return GraphML().parse(cin);
}

void
pelib_delete(Record *obj)
{
	delete obj;
}

void
pelib_dump(std::ostream& cout, std::map<const char*, Record*> records, size_t argc, char **argv)
{
        Taskgraph* tg = (Taskgraph*)records.find(typeid(Taskgraph).name())->second;
        Platform* pf = (Platform*)records.find(typeid(Platform).name())->second;

        if(records.find(typeid(Platform).name()) != records.end())
        {
                GraphML().dump(cout, tg, pf);
        }
        else
        {
                GraphML().dump(cout, tg);
        }
}

#ifdef __cplusplus
}
#endif
The code is rather short, because all parsers and output are already implemented in the pelib framework. It is highly recommended to implement parser and outputs as part as the pelib framework first and then use it in a pelib-convert plugin. Note that the whole code is surrounded by a the clause "extern "C" {". This is because libdl can only operate on C functions. Therefore, it is mandatory to surround the implementation of pelib_parse(), pelib_delete() and pelib_dump() with this clause.

pelib_parse() takes an input stream (cin) as well as an array of optional string arguments (argv) and the number of arguments in the array. String arguments can be used to alter the behavior of the parser. They can be used in any manner defined by the plugin programmer, or not used at all. The last element of argv (argv[argc]) is valid and always equals NULL. argv is dynamically allocated and is freed after pelib_parse returns. pelib_parse must return a valid pointer to an instance of a class derived from pelib::Record.
pelib_delete takes a pointer to a class instance derived from pelib::Record. pelib_delete is always called with an object pelib_parse returned previously. pelib_delete must delete the object instance and free its associated memory.

pelib_dump() takes an input stream (output), a collection of pointers pelib::Record in a map as well as an array of optional string arguments (argv) and the number of arguments in the array. As for pelib_parse, String arguments can be used to alter the behavior of the output class. They can be used in any manner defined by the plugin programmer, or not used at all. The last element of argv (argv[argc]) is valid and always equals NULL. argv is dynamically allocated and is freed after pelib_dump returns.
pelib_dump() can find the complex data structure it need to output thanks to a unique class string identifier used as key in the std::map passed as argument (records). For instance, if pelib_dump() requires to output an instance of pelib::taskgraph, it can find it in records with records.find(typeid(pelib::Taskgraph).name()). This can be useful if an output requires more than one instance of pelib::Record to be performed. For instance, pelib::DrakeCSchedule needs and instance of pelib::Schedule, an instance of pelib::Taskgraph and an instance of pelib::Platform to generate some output. Similarly, the example above optionally requires an instance of pelib::Platform to output an instance of pelib::Taskgraph in GraphML format. If your implementation requires more that one instance of the same class derived from pelib::Record, then the class's type name is not enough to find either class instance. In this case, the user can use the pelib-convert switch --name with a predefined string in an input clause (pelib-convert --input ...). pelib_dump() can use this string to find the right object instance in the map collection.

The plugin can be compiled with the line command line below

g++ -shared -Wl,-soname=libpelib-<structure>-<syntax>.so -o /tmp/libpelib-<structure>-<syntax>.so -fPIC `pkg-config --cflags --libs pelib` plugin.cpp
where <structure> is the class name of the data structure parsed and/or output (if both, priority is to the class parsed) and <syntax> is the text format parsed and/or output by the plugin. If the structure is pelib::Algebra, then <structure> should be omitted in the library name. The library must be placed in any directory defined in LD_RUN_PATH at the time pelib-convert was linked ($HOME/.local/lib by default).



Generic C data structures

Programming often invlide the use of more or less elaborate data structures. While there are many existing framework that implement efficient data structures, programming for research projects often requires the possibility to understand deeply the internal design of these data structures and the possibility to modify it to fit the project's need. The pelib C data structures are used in research on On-chip pipelining, high-performance parallel computing research project, as in the experiment C streaming framework Drake. The pelib C data structures are kept as simple as possible, require only a C compile and preprocessor, can be used with any data type while preserving C compilers' type-checking capabilities and follow an object-oriented design that enables code and objects composition. For example, the programmer can define and use basic (integer, float, etc.) and complex (struct { ... }) data types and define pelib object basic operations such as allocation, deallocation, initialization, destruction, or display, to name a few. Also, the programmer can define generic collections of pelib data structures with the same operations as for pelib data types (allocation, etc.) but used in composition of objects it holds. Composability simplifies the development of data structures for new data types. Therefore, the implementation of collections of a new data type only requires the definition of the data type as well as its basic pelib operations and the declaration of collections using this data type. For instance, one can define a display operation of a basic data type and the display operation of a pelib collection can use it to display its the precise content, showing also the content of objects it contains. This simplifies debugging as it makes it easy to write routines to observe the precise state of any complex data structure.

Section Pelib C basics introduces the basic use of pelib data structures, including pelib collections and section Develop new data structures describes how to implement a custom data structure.

Pelib C basics

Using pelib data structures is very similar to using C++ templates. Let us take the example of integer. The code snippet below declares two pelib integers, initialize and compare them, then increments one integer and compare them both again

#include <stdio.h>
#include <stdlib.h>

#include <pelib/integer.h>

int
main(int argc, char **argv)
{
        // Declares a usual integer
        int a, b; 

        // Gives an initial value to this integer, typically 0
        pelib_init(int)(&a);
        pelib_init(int)(&b);

        // Displays both integers
        pelib_printf(int)(stdout, a);
        pelib_printf(int)(stdout, b);

        // Pelib can compare two generic data structures
        int a_lower_than_b = pelib_compare(int)(a, b);
        printf("(C built-in < ) %s\n", a < b ? "A is lower than B" : "A is not lower than B");
        printf("(pelib_compare) %s\n", (a_lower_than_b < 0) ? "A is lower than B" : "A is not lower than B");

        // Now let us modify 3
        b = 1;

        // And let us display both integers again
        pelib_printf(int)(stdout, a);
        pelib_printf(int)(stdout, b);

        // Finally, we can compare both a and b again 
        a_lower_than_b = pelib_compare(int)(a, b);
        printf("(C built-in < ) %s\n", a < b ? "A is lower than B" : "A is not lower than B");
        printf("(pelib_compare) %s\n", (a_lower_than_b < 0) ? "A is lower than B" : "A is not lower than B");

        return EXIT_SUCCESS;
}
Compile with:
gcc test.c -o test `pkg-config --cflags --libs pelib`
Note the peculiar function(type)(arguments) function call syntax. This is to compare to C++ template function calls syntax (function<type>(arguments)). Here, we use the pelib functions for an integer. Consult Doxygen documentation for more information on basic operations for all pelib objects. The code above looks over-complicated. However, it gets more interesting with more complex data structures, such as a complex number. We coded a simple complex number in pelib for the sake of demonstration. In particular, both real and imaginary values are integer and assumed to be greater or equal than 0. Also, we consider that a complex number is higher than another of its real value is higher. If both real parts are equal, then we compare the imaginary part in the same manner.
#include <stdio.h>
#include <stdlib.h>

#include <pelib/complex.h>

int
main(int argc, char **argv)
{
        // Declares a usual complex
        complex_t a, b;

        // Gives an initial value to this complex, typically 0
        pelib_init(complex_t)(&a);
        pelib_init(complex_t)(&b);

        // Displays both complexs
        pelib_printf(complex_t)(stdout, a);
        pelib_printf(complex_t)(stdout, b);

        // Pelib can compare two generic data structures
        int a_lower_than_b = pelib_compare(complex_t)(a, b);
        //printf("(C built-in < ) %s\n", a < b ? "A is lower than B" : "A is not lower than B");
        printf("(pelib_compare) %s\n", (a_lower_than_b < 0) ? "A is lower than B" : "A is not lower than B");

        // Now let us modify 3
        b.r = 1;

        // And let us display both complex again
        pelib_printf(complex_t)(stdout, a);
        pelib_printf(complex_t)(stdout, b);

        // Finally, we can compare both a and b again 
        a_lower_than_b = pelib_compare(complex_t)(a, b);
        //printf("(C built-in < ) %s\n", a < b ? "A is lower than B" : "A is not lower than B");
        printf("(pelib_compare) %s\n", (a_lower_than_b < 0) ? "A is lower than B" : "A is not lower than B");

        return EXIT_SUCCESS;
}
This code is roughly the same as above; only the type of a and b, as well as the type of calls to generic functions, have changed from int to complex_t. Note that on this example, we cannot use the C built-in comparison operator < anymore, as C doesn't know how to use it with complex numbers. Instead, only pelib_compare can use the right comparison function and compute a meaningful result.

Finally, pelib data types and their function can be composed together. The code below illustrates the use if arrays of integers, arrays of complex and arrays of fifos of complex

#include <stdio.h>
#include <stdlib.h>

#include <pelib/integer.h>
#include <pelib/complex.h>

// Create a fifo data structure that handles arrays of complex
// Declare the fifo structure and methods first
#define CFIFO_T array_t(complex_t)
#include <pelib/fifo.h>
// Close the definition of fifo for arrays of complex
#define DONE_cfifo_array_complex_t_t
// Then generate the corresponding code
#define CFIFO_T array_t(complex_t)
#include <pelib/fifo.c>

int
main(int argc, char **argv)
{
        // Iteration variable
        size_t i;

        // Variable declaration and initialization
        int a;
        pelib_init(int)(&a);
        complex_t b;
        pelib_init(complex_t)(&b);

        // Declare and initialize an array of integers, and allocate its buffer
        array_t(int) array_int;
        pelib_init(array_t(int))(&array_int);
        pelib_alloc_buffer(array_t(int))(&array_int, 10);

        // Declare and initialize an array of complex, and allocate its buffer
        array_t(complex_t) array_complex;
        pelib_init(array_t(complex_t))(&array_complex);
        pelib_alloc_buffer(array_t(complex_t))(&array_complex, 10);

        // Fill both integer and complex arrays with increasing number (only in the real value for complex numbers)
        for(i = 0; i < 10; i++)
        {
                pelib_array_append(int)(&array_int, (int)i);
                b.r = (int)i;
                pelib_array_append(complex_t)(&array_complex, b);
        }

        // Display both arrays
        pelib_printf(array_t(int))(stdout, array_int);
        pelib_printf(array_t(complex_t))(stdout, array_complex);

        // Declare and initialize a fifo of arrays of complex, and allocate its buffer
        cfifo_t(array_t(complex_t)) complex_fifo;
        pelib_init(cfifo_t(array_t(complex_t)))(&complex_fifo);
        pelib_alloc_buffer(cfifo_t(array_t(complex_t)))(&complex_fifo, 5);
        // Reinitialize the array of complex (just set its length to 0)
        pelib_init(array_t(complex_t))(&array_complex);

        // Fill the fifo of increasing size arrays sharing the same data buffer
        for(i = 0; i < 3; i++)
        {
                b.r = 10 - (int)i;
                pelib_array_append(complex_t)(&array_complex, b);
		// Below is a bad practice: don't duplicate arrays using a simple copy as it won't copy the array's buffer,
		// but simply copy the pointer to it. It is better to implement pelib_copy() for arrays instead, but
		// we never actually needed it so it's not done yet.
                pelib_cfifo_push(array_t(complex_t))(&complex_fifo, array_complex);
        }

        // Display the composite fifo
        pelib_printf(cfifo_t(array_t(complex_t)))(stdout, complex_fifo);

        return EXIT_SUCCESS;
}
The code snippet above illustrate the use of a pelib collection with integers of complex data structures. The declaration of an array of integer consists of the type itself (array_t) as well as a containing type specifier (int) that defines what elements can be stored in the array. Any array method called on this array instance must by typed as int. Basic operation method calls (that applies to the most basic pelib object, such as initialization, see Doxygen documentation) must be typed as the complete object being manipulated. In line 31 for example, the initialization method is typed as "array_t(int)" as an array of integer is being allocated. More specific methods, such as array's method "append" (line 42) can only be typed with the data type the array handles. This is possible because the method (pelib_array_append) is already specified as an array method in its name.
The code snippet above generates a new type of fifo able to receive arrays of complex. Line 9 defines a fifo content type (CFIFO) as an array of complex (array_t(complex)) and generates the data structure and method declaration in line 10. After definition, the new type must be marked as done (DONE_*) to avoid multiple definitions. This is done in line 12. Finally the actual method implementations are generated in line 15 by redefining the fifo element type and including the fifo generic code. The remaining of this code example resets the array of complex by calling its initialization method (that just reset its number of elements to 0) and insert in the fifo several copies of increasing size arrays. Note that as array_complex is given as a copy at each call of fifo push, each copy uses the same data buffer, but they record different number of elements inside. This is generally a bad practice; a better approach would be to implement the copy method of an array and reuse it here. However in this example, it provides a simple way to show illustrate the use of composite objects.

Develop new data structures

Developing new pelib data structure is somewhat challenging, due to the difficulty to read code that makes heavy use of C preprocessor and interpret compilers' outputs. This section gives code snippets that can help to develop new data structures for pelib. Below is a snippet for its header

// Defines PELIB_CONCAT_2 and PELIB_CONCAT_3 used below
#include <pelib/template.h>
 
// This part is optional, just like any normal header
// Only to declare type-independent stuff related to our structure
#ifndef MYSTRUCT_H
#define MYSTRUCT_H
 
#endif
 
// Here begins the definition of our new data structure
// This line prevents the following block to be imported twice
// for the same type, provided DONE_* is defined after each
// inclusion of this header
#if PELIB_CONCAT_2(DONE_mystruct_, MYSTRUCT) == 0
 
// Declare the name of the new structure and its generic type MYSTRUCT 
// We need to declare two variants: one for the structure itself, another
// for its space-less type alias (see below)
#define mystruct(elem) PELIB_CONCAT_2(mystruct_, elem)
#define mystruct_t(elem) PELIB_CONCAT_2(mystruct(elem), _t)
 
// Declare a frontend name for each function that handle our new structure
#define foo(elem) PELIB_CONCAT_3(tuto_, mystruct_t(elem), _foo)
#define bar(elem) PELIB_CONCAT_3(tuto_, mystruct_t(elem), _bar)
 
// Our actual generic structure
struct mystruct(MYSTRUCT)
{
    int foo;
    MYSTRUCT bar;
};
// A pelib structure cannot have a space in its name
// because of the structure name is used to generate
// function name, therefore we must use a type alias
// using the second type name defined above
typedef struct mystruct(MYSTRUCT) mystruct_t(MYSTRUCT);
 
// This defines the basic object method (allocate, initialize, printf, string, etc.)
// for our new structure
#define STRUCT_T mystruct_t(MYSTRUCT)
#include <pelib/structure.h>
 
// Specific methods for our new structure
// Actual definition of method foo
int foo(MYSTRUCT)(mystruct_t(MYSTRUCT)* obj);
size_t bar(MYSTRUCT)(mystruct_t(MYSTRUCT) obj, size_t n); 
 
// End of definition
#endif
 
// Undefined the generic type name so the portion above cannot be parsed again by error.
#undef MYSTRUCT
Note that any copy error in the code above can lead to quite verbose compiler output, very difficult to understand. The code above generates a header with actual C structure and function definitions for each specialized version of our new generic structure. The snippet below gives an actual implementation. We limit it for the purprose of demonstration to the definition of a base method pelib_init and two specialized methods foo and bar. They both use the generic type MYSTRUCT to perform some operations. Just as our structure mystruct, pelib objects are not required to implement all functions defined in pelib/structure.h. However, if your object tries to call a basic method that is unimplemented, the linker will fail to build the final executable. If you want to use a data structure that requires to implement a method you can't implement, then define the missing method, make it to show an error message and abort the program execution, if needed, then avoid to use any method in the data structure you use that may use the missing function.
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <stddef.h>

#ifndef MYSTRUCT
#error Using generic mystruct without specialization type
#endif

// Basic pelib object mdethods implementations
// Initialization
int
pelib_init(mystruct_t(MYSTRUCT))(mystruct_t(MYSTRUCT)* mystruct)
{
        mystruct->foo = 56; 
        return pelib_init(MYSTRUCT)(&mystruct->bar);
}

FILE*
pelib_printf(mystruct_t(MYSTRUCT))(FILE* stream, mystruct_t(MYSTRUCT) mystruct)
{
        fprintf(stream, "%d, ", mystruct.foo);
        pelib_printf(MYSTRUCT)(stream, mystruct.bar);
        return stream;
}

// Many more basic operations should be implemented, but we won't show them for concision
int
foo(MYSTRUCT)(mystruct_t(MYSTRUCT)* obj)
{
        MYSTRUCT ref;

        // By this line we require that any pelib typed used with our structure requires the implementation
        // of basic function init()
        pelib_init(MYSTRUCT)(&ref);

        // Here, we require that the pelib type using our new pelib data structure implements compare()
        return obj->foo * pelib_compare(MYSTRUCT)(obj->bar, ref);
}

size_t
bar(MYSTRUCT)(mystruct_t(MYSTRUCT) obj, size_t n)
{
        // Trivial operation, just as an example
        return sizeof(MYSTRUCT) * n;
}
The code snippet above should never be compile alone. Instead, it should be included in concrete pelib object implementation. The code below shows the example of an integer making use of our new data structure. As mystruct implements pelib_printf and runs its with the type it is specialized with, we need our integer object to implement some pelib printf method. Let's consider for the sake of example that we can't or don't want to implement it. Instead, we just show an error message.
#include <stdio.h>
#include <stdlib.h>

#include "integer.h"

int
pelib_init(int)(int *val)
{
        *val = 0;
        return 1;
}

int
pelib_compare(int)(int a, int b)
{
        return a - b;
}

FILE*
pelib_printf(int)(FILE* stream, int a)
{
        fprintf(stderr, "[%s:%s:%d][ERROR] Not implemented\n", __FILE__, __FUNCTION__, __LINE__);
        return stream;
}

// Generate mystruct functions for integer
#define MYSTRUCT int
#include "mystruct.c"
As integer must also be a pelib object, we can have the header below
#include <stdio.h>
#include <stdlib.h>

#define STRUCT_T int
#include <pelib/structure.h>
#define DONE_int

#define MYSTRUCT int
#include "mystruct.h"
#define DONE_mystruct
We can fnally use our integer object with mystruct in a program. We declare an instance of mystruct for integers. Because the field bar is generic, it is an int in our example and we can assign it an integer. Then we can use generic forms of methods foo and bar. the pelib basic printf method cannot succeed as our integer doesn't implement it. However as we provided a simple implementation, the program can be compiled and run.
#include <stdio.h>
#include <stdlib.h>

#include "integer.h"

#define show_int(var) printf("[%s:%s:%d] %s = %d\n", __FILE__, __FUNCTION__, __LINE__, #var, var)
#define show_size_t(var) printf("[%s:%s:%d] %s = %zu\n", __FILE__, __FUNCTION__, __LINE__, #var, var)

int
main(int argc, char **argv)
{
	// Declare and initialize mystruct for int
        mystruct_t(int) stuff;
        pelib_init(mystruct_t(int))(&stuff);

	// Declare and initialize an integer
        int mybar = 7;

	// We know stuff is a mystruct for int, then we know that stuff.bar is an int and therefore, the line below is correct.
        stuff.bar = mybar;

	// Run the method foo with our mystruct instance and print it value returned on screen
        show_int(foo(int)(&stuff));

	// Same procedure with bar method
        show_size_t(bar(int)(stuff, 42));

	// Finally, let's try the print method.
        pelib_printf(mystruct_t(int))(stdout, stuff);
        return EXIT_SUCCESS;
}



Ongoing work

Additional features to be developped include:

If you would like to contribute, please let us know.



Contact

For reporting bugs, please email to "<firstname> DOT <lastname> AT liu DOT se".

Acknowledgments

This work was partly funded by the EU FP7 project EXCESS and by SeRC project OpCoReS.