Introduction ------------ There are 3 parts of this lab and you are expected to complete each of them: #. MetaProgramming #. Profiling in Julia #. General-purpose debugging and profiling tools The lab skeleton is on `Gitlab `_ and the instructions for the individual parts are below. Julia ^^^^^ Some hints for working with the Julia REPL: * Typing ``?`` opens the help. ``cos`` gives you the documentation for ``cos``. Hitting ```` returns to the regular REPL. * Typing ``]`` opens the package manager. ``add DataStructures`` would install that package. * Typing ``@less cos(1.0)`` gives you the documentation for that method. Hitting ``q`` brings you back. * Julia scripting looks similar to Python, but Julia is strongly typed. * Julia supports unicode characters as identifiers in the language. You can use assignments such as ``häst=1``. In the REPL, typing ``\:horse:=1`` results in ``🐴=1`` being used, which is convenient for characters that are not on your keyboard (but not needed for the labs). 1. MetaProgramming ------------------ The metaprogramming exercise is performed in the Julia programming language since it has access to a reflection API (where a program can query or update its own state), as well as a powerful macro syntax where an abstract syntax tree representation is sent to a macro which modifies the syntax tree and returns a modified version of the tree (which is what the compiler then uses). Is is also capable of building a string or syntax tree on the fly and then evaluate that. The code for the exercise exists in the ``julia`` folder. Type ``make run`` to download julia and run the example (it will not pass until you write some code). Type ``make shell`` to start a julia session (or ``julia-1.0.4/bin/julia``). The meta-programming exercise consists of three small parts: * Template programming * Reflective programming * Macro programming Template programming ^^^^^^^^^^^^^^^^^^^^ The lab skeleton contains a for-loop that takes a list of arguments from which you will build functions. For example, from: .. code-block :: julia name = :add op = :+ You will build up a method from those variables: .. code-block :: julia add(x, y) = x + y A naïve implementation create a string corresponding to this and parse + evaluate that; your implementation should build the abstract syntax tree directly. .. note :: Julia is different from template meta-programming in C++ in that there are almost no rules here. The Julia source code is basically a script that runs Julia code and adds code to the abstract syntax tree on the fly. It is very flexible, but if you overuse the metaprogramming parts, you are shooting yourself in the foot. Reflective programming ^^^^^^^^^^^^^^^^^^^^^^ Reflective programming is often used to implement plugins or similar code that does not exist at compile-time. So the program needs to be able to examine and manipulate which functions or classes exist at runtime. The exercise in Julia is to delete a method from a module. Note that Julia uses multiple dispatch so a single function can have 2 different methods; deleting the method will thus not remove the entire function. Macro programming ^^^^^^^^^^^^^^^^^ Macro programming is similar to template programming, but more powerful. Instead of only being able to construct an abstract syntax tree and filling in the blanks, you are able to manipulate an abstract syntax tree. In the lab you will create a macro that takes a block and a condition, and create a do-while block corresponding to this. The code skeleton contains some print statements to see what is input and output of the macro. Note that code created inside the macro by default will get source locations pointing to the macro, which are not always desirable. You can either construct the output such that these source locations are never added, or replace any such additions with references to the actual source location. Note that Julia only allows a single source location for a line of code, and only the line that starts the expression. 2. Profiling in Julia --------------------- Premature optimization is the root of all evil. -- `Donald Knuth `_ However, sometimes we need to optimise. The question is what to optimise! To help us with this we can use various profiling tools. In some simple cases, ``valgrind --tool=cachegrind`` is enough (when the 100x slowdown is tolerable). Otherwise, statistical profilers are usually used (or simply running the program and printing time between statements). However, sometimes more advanced tools are needed. In this exercise you will experiment with different profiling tools for the Julia language. * Use Julia's `built-in profiling and memory analysis tools `_ to figure out where the performance bottleneck in ``profiling/test.jl`` lies. * Fix the performance bottleneck. .. note :: If you think the output of the built-in tools is hard to read, you can use Julia packages that provide an easier output like `StatProfilerHTML `_ or `Coverage `_. If you want to try those package, install them by typing ``] add StatProfilerHTML Coverage``. .. note :: You should solve the exercise twice: once using memory analysis and once using the CPU profiler. Simply fixing the performance bottleneck is not enough. In your lab report, show how you found the performance bottle neck using both CPU profiling and memory analysis. 3. General-purpose debugging and profiling ------------------------------------------ Sometimes what is known as caveman or print debugging is not enough. A better overview is needed. To help software developer with this numerous tools exists. In this lab there are two tools that might be interesting to use: GDB and Valgrind, among others. To start gdb, issue: .. code-block :: bash gdb --args You can set breakpoints by typing for example: .. code-block :: bash br main.c:85 # Stops on line 85 in main.c br foo # Stops on call to foo To start the program: .. code-block :: bash run There are many more commands for GDB, for reference see the `documentation `_. `Valgrind `_ is a suite of tools for debugging, etc. It is very useful to detect different issues that relate to memory such as memory leaks. To use valgrind, simply type: .. code-block :: bash valgrind --tool= #. The file ``debugging/dividearrays.cpp`` crashes. For which values does it crash (use a debugger to find out; no printing to terminal) and how would you solve the problem? #. The file ``debugging/workitems.c`` contains a race condition. Use a debugging tool to locate it, and then fix it. There are useful functions in `pthread.h `_ #. The file ``debugger/fileutil.cpp`` crashes. Fix all the crashes in a good way such that you can copy any file without modifying the content. .. note :: In your lab report, show how you found the problem. Preferably, attach some logs (in gdb, use ``set logging on``).