Introduction
There are 3 parts of this lab and you are expected to complete each of them:
MetaProgramming
Profiling in Julia
General-purpose debugging and profiling tools
The lab skeleton is on Gitlab and the instructions for the individual parts are below.
Julia
Some hints for working with the Julia REPL:
Typing
?
opens the help.cos
gives you the documentation forcos
. Hitting<backspace>
returns to the regular REPL.Typing
]
opens the package manager.add DataStructures
would install that package.Typing
@less cos(1.0)
gives you the documentation for that method. Hittingq
brings you back.Julia scripting looks similar to Python, but Julia is strongly typed.
Julia supports unicode characters as identifiers in the language. You can use assignments such as
häst=1
. In the REPL, typing\:horse:<TAB>=1
results in🐴=1
being used, which is convenient for characters that are not on your keyboard (but not needed for the labs).
1. MetaProgramming
The metaprogramming exercise is performed in the Julia programming language since it has access to a reflection API (where a program can query or update its own state), as well as a powerful macro syntax where an abstract syntax tree representation is sent to a macro which modifies the syntax tree and returns a modified version of the tree (which is what the compiler then uses). Is is also capable of building a string or syntax tree on the fly and then evaluate that.
The code for the exercise exists in the julia
and reflections
folder.
Type make run
to download julia and run the example (it will not pass until you write some code).
Type make shell
to start a julia session (or julia-1.2.0/bin/julia
).
The meta-programming exercise consists of three small parts:
Template programming
Macro programming
Reflective programming
Template programming
The lab skeleton contains a file (run.jl) that contains a for-loop that takes a list of arguments from which you will build functions. For example, from:
name = :add
op = :+
You will build up a method from those variables:
add(x, y) = x + y
A naïve implementation would create a string corresponding to this and parse + evaluate that:
eval(Meta.parse("$name(x, y) = x $op y"))
Your implementation should not use Meta.parse but build the abstract syntax tree directly using code quoting.
Note
Julia is different from template meta-programming in C++ in that there are almost no rules here.
The Julia source code is basically a script that runs Julia code and adds code to the abstract syntax tree on the fly. It is very flexible, but if you overuse the metaprogramming parts, you are shooting yourself in the foot.
Macro programming
Macro programming is similar to template programming, but more powerful. Instead of only being able to construct an abstract syntax tree and filling in the blanks, you are able to manipulate an abstract syntax tree. In the lab you will create a macro (in dowhile.jl) that takes a block and a condition, and create a do-while block corresponding to this.
You can use code quoting inside the macro, but it is sometimes a bit more complicated since you need to sanitize names in order to know what is a temporary introduced by the macro and what is a name in the code block passed to the macro.
Note
Julia only allows a single source location for a line of code, and only the line that starts the expression which means that Julia will throw exceptions with source code locations pointing to the macro instead of for example the body of the statements that you wrote.
In previous versions of the course, there was a task to remove the source code locations in order to give better error locations.
Feel free to implement that as well if you are curious how it works.
Reflective programming
C# has a powerful version of reflective programming. PizzaInterface.cs contains an interface IPizza, which is used by ThirdPartyPlugin.cs. ThirdPartyPlugin.cs is compiled into ThirdPartyPlugin.dll and contains a few different Pizzas - this file should not be modified and not be compiled into the Program executable. At runtime, the Program will load ThirdPartyPlugin.dll and your task is to create a factory (RandomPizzaMaker) that randomly returns either a Pizza defined in the Program or ThirdPartyPlugin.dll.
Note
See reflection/README.md for some hints on how you can compile the code and make development easier.
Note
You may solve this exercise either by looking through the assembly of ThirdPartyPlugin.dll or (probably simpler) all loaded assemblies. Either solution is acceptable.
The list of IPizza implementations must be created through the reflections API - pretend you have never seen ThirdPartyPlugin.cs.
2. Profiling in Julia
Premature optimization is the root of all evil.
However, sometimes we need to optimise.
The question is what to optimise!
To help us with this we can use various profiling tools.
In some simple cases, valgrind --tool=cachegrind
is enough (when the 100x slowdown is tolerable).
Otherwise, statistical profilers are usually used (or simply running the program and printing time between statements).
However, sometimes more advanced tools are needed. In this exercise you will experiment with different profiling tools for the Julia language.
Use Julia’s built-in profiling and memory analysis tools to figure out where the performance bottleneck in
profiling/test.jl
lies.Fix the performance bottleneck.
Note
If you think the output of the built-in tools is hard to read, you can use Julia packages that provide an easier output like StatProfilerHTML or Coverage.
If you want to try those package, install them by typing ] add StatProfilerHTML Coverage
.
Note
You should solve the exercise twice: once using memory analysis and once using the CPU profiler. Simply fixing the performance bottleneck is not enough.
In your lab report, show how you found the performance bottle neck using both CPU profiling and memory analysis.
3. General-purpose debugging and profiling
Sometimes what is known as caveman or print debugging is not enough. A better overview is needed. To help software developer with this numerous tools exists. In this lab there are two tools that might be interesting to use: GDB and Valgrind, among others.
To start gdb, issue:
gdb --args <name-of-executable>
You can set breakpoints by typing for example:
br main.c:85 # Stops on line 85 in main.c
br foo # Stops on call to foo
To start the program:
run
There are many more commands for GDB, for reference see the documentation.
Valgrind is a suite of tools for debugging, etc. It is very useful to detect different issues that relate to memory such as memory leaks. To use valgrind, simply type:
valgrind --tool=<debugging-tool> <program>
The file
debugging/dividearrays.cpp
crashes. For which values does it crash (use a debugger to find out; no printing to terminal) and how would you solve the problem?The file
debugging/workitems.c
contains a race condition. Use a debugging tool to locate it, and then fix it. There are useful functions in pthread.hThe file
debugging/fileutil.cpp
crashes. Fix all the crashes in a good way such that you can copy any file without modifying the content. Usemake run
to verify that the file is copied properly.
Note
In your lab report, show how you found the problem.
Preferably, attach some logs (in gdb, use set logging on
).