Introduction

There are 3 parts of this lab and you are expected to complete each of them:

  1. MetaProgramming

  2. Profiling in Julia

  3. General-purpose debugging and profiling tools

The lab skeleton is on Gitlab and the instructions for the individual parts are below.

Julia

Some hints for working with the Julia REPL:

  • Typing ? opens the help. cos gives you the documentation for cos. Hitting <backspace> returns to the regular REPL.

  • Typing ] opens the package manager. add DataStructures would install that package.

  • Typing @less cos(1.0) gives you the documentation for that method. Hitting q brings you back.

  • Julia scripting looks similar to Python, but Julia is strongly typed.

  • Julia supports unicode characters as identifiers in the language. You can use assignments such as häst=1. In the REPL, typing \:horse:<TAB>=1 results in 🐴=1 being used, which is convenient for characters that are not on your keyboard (but not needed for the labs).

1. MetaProgramming

The metaprogramming exercise is performed in the Julia programming language since it has access to a reflection API (where a program can query or update its own state), as well as a powerful macro syntax where an abstract syntax tree representation is sent to a macro which modifies the syntax tree and returns a modified version of the tree (which is what the compiler then uses). Is is also capable of building a string or syntax tree on the fly and then evaluate that.

The code for the exercise exists in the julia and reflections folder. Type make run to download julia and run the example (it will not pass until you write some code). Type make shell to start a julia session (or julia-1.2.0/bin/julia).

The meta-programming exercise consists of three small parts:

  • Template programming

  • Macro programming

  • Reflective programming

Template programming

The lab skeleton contains a file (run.jl) that contains a for-loop that takes a list of arguments from which you will build functions. For example, from:

name = :add
op = :+

You will build up a method from those variables:

add(x, y) = x + y

A naïve implementation would create a string corresponding to this and parse + evaluate that:

eval(Meta.parse("$name(x, y) = x $op y"))

Your implementation should not use Meta.parse but build the abstract syntax tree directly using code quoting.

Note

Julia is different from template meta-programming in C++ in that there are almost no rules here.

The Julia source code is basically a script that runs Julia code and adds code to the abstract syntax tree on the fly. It is very flexible, but if you overuse the metaprogramming parts, you are shooting yourself in the foot.

Macro programming

Macro programming is similar to template programming, but more powerful. Instead of only being able to construct an abstract syntax tree and filling in the blanks, you are able to manipulate an abstract syntax tree. In the lab you will create a macro (in dowhile.jl) that takes a block and a condition, and create a do-while block corresponding to this.

You can use code quoting inside the macro, but it is sometimes a bit more complicated since you need to sanitize names in order to know what is a temporary introduced by the macro and what is a name in the code block passed to the macro.

Note

Julia only allows a single source location for a line of code, and only the line that starts the expression which means that Julia will throw exceptions with source code locations pointing to the macro instead of for example the body of the statements that you wrote.

In previous versions of the course, there was a task to remove the source code locations in order to give better error locations.

Feel free to implement that as well if you are curious how it works.

Reflective programming

C# has a powerful version of reflective programming. PizzaInterface.cs contains an interface IPizza, which is used by ThirdPartyPlugin.cs. ThirdPartyPlugin.cs is compiled into ThirdPartyPlugin.dll and contains a few different Pizzas - this file should not be modified and not be compiled into the Program executable. At runtime, the Program will load ThirdPartyPlugin.dll and your task is to create a factory (RandomPizzaMaker) that randomly returns either a Pizza defined in the Program or ThirdPartyPlugin.dll.

Note

See reflection/README.md for some hints on how you can compile the code and make development easier.

Note

You may solve this exercise either by looking through the assembly of ThirdPartyPlugin.dll or (probably simpler) all loaded assemblies. Either solution is acceptable.

The list of IPizza implementations must be created through the reflections API - pretend you have never seen ThirdPartyPlugin.cs.

2. Profiling in Julia

Premature optimization is the root of all evil.

—Donald Knuth

However, sometimes we need to optimise. The question is what to optimise! To help us with this we can use various profiling tools. In some simple cases, valgrind --tool=cachegrind is enough (when the 100x slowdown is tolerable). Otherwise, statistical profilers are usually used (or simply running the program and printing time between statements).

However, sometimes more advanced tools are needed. In this exercise you will experiment with different profiling tools for the Julia language.

Note

If you think the output of the built-in tools is hard to read, you can use Julia packages that provide an easier output like StatProfilerHTML or Coverage.

If you want to try those package, install them by typing ] add StatProfilerHTML Coverage.

Note

You should solve the exercise twice: once using memory analysis and once using the CPU profiler. Simply fixing the performance bottleneck is not enough.

In your lab report, show how you found the performance bottle neck using both CPU profiling and memory analysis.

3. General-purpose debugging and profiling

Sometimes what is known as caveman or print debugging is not enough. A better overview is needed. To help software developer with this numerous tools exists. In this lab there are two tools that might be interesting to use: GDB and Valgrind, among others.

To start gdb, issue:

gdb --args <name-of-executable>

You can set breakpoints by typing for example:

br main.c:85 # Stops on line 85 in main.c
br foo # Stops on call to foo

To start the program:

run

There are many more commands for GDB, for reference see the documentation.

Valgrind is a suite of tools for debugging, etc. It is very useful to detect different issues that relate to memory such as memory leaks. To use valgrind, simply type:

valgrind --tool=<debugging-tool> <program>

  1. The file debugging/dividearrays.cpp crashes. For which values does it crash (use a debugger to find out; no printing to terminal) and how would you solve the problem?

  2. The file debugging/workitems.c contains a race condition. Use a debugging tool to locate it, and then fix it. There are useful functions in pthread.h

  3. The file debugging/fileutil.cpp crashes. Fix all the crashes in a good way such that you can copy any file without modifying the content. Use make run to verify that the file is copied properly.

Note

In your lab report, show how you found the problem. Preferably, attach some logs (in gdb, use set logging on).