Hide menu

Running Planners

Introduction

A large number of planners, both from the latest International Planning Competition and from other sources, are available for use in the labs. We have tested and verified these planners on typical planning competition domains. The planners however are in general state of the art research prototypes and there is no guarantee that they are free from bugs or that they work perfectly in all domains. In fact, labs seem to be an excellent way of finding strange bugs in planners, since you as students tend to have a greater variety in your domain models than the researchers developing the planners, who tend to follow their own "patterns".

So if you encounter problems with one planner, first check the list of common problems at the end. Then try other planners as well – this helps you see whether the problem lies in the planner or in your domain definition. Also, please ask us about problems at the labs or by e-mail. Never spend hours or days trying to solve a problem!

Satisficing and Optimizing Planners

We distinguish between satisficing and optimizing planners. Both types are available.

An optimizing planner should always return a plan that is optimal in terms of some cost function defined on plans, no matter how much time it takes to generate such a plan. For example, it may return plans with the smallest number of actions, or one may define costs for each action and generate plans with minimal total cost.

We might say that the "opposite" of an optimizing planner is one that returns a plan as quickly as possible, regardless of how bad the plan is. However, this is seldom what we want: We rarely prefer a plan of cost 10000 over a plan of cost 20, even if the first one takes slightly less time to produce.

Therefore, satisficing planners generally try to find a good balance between time and quality. Some planners generate a single plan using a built-in heuristic tuned for both criteria. Other planners provide more flexibility by generating an initial plan but then continues to search for better plans until explicitly interrupted by the user, either because the current plan is "good enough" or because one cannot wait any longer.

In fact, the satisficing track of the 2011/2014 planning competitions reward those planners that yield the highest quality plans within a time limit of 30 minutes. Several of the planners below will therefore continue trying to generate better plans until interrupted, as discussed above. We will show later how to run these planners.

Running Planners

The planners used in lab 1-3 are from different sources. There are planners from the latest planning competition, which took place in 2014. These planners are state of the art. There are also planners from the competition of 2011, which in many cases are also state of the art, as well as older planners. All planners are run by the command:

    planner mydomain.pddl mydomain-prob1.pddl plan-output

The planner will then use the supplied domain and problem to try to generate a plan. The first generated plan will be called plan-output or plan-output.1 and subsequent plans, in case there are any, will get increasing numbers. This is true for all the competition planners. The older planners, FF and IPP, only output the plan in the terminal. Also, be aware that some planners may create a lot of auxiliary files in the current directory which you must remove manually.

The planners from IPC-2014 are installed under /home/TDDD48/planners/ipc2014 and the planners from IPC-2011 are installed in /home/TDDD48/planners/ipc2011. Both these locations contain planners from the different competition planning tracks.

    /home/TDDD48/planners/ipc2014/seq-opt        -- sequential optimizing
    /home/TDDD48/planners/ipc2014/seq-sat        -- sequential satisficing
    /home/TDDD48/planners/ipc2014/tempo-sat      -- temporal satisficing
    /home/TDDD48/planners/ipc2011/seq-opt        -- sequential optimizing
    /home/TDDD48/planners/ipc2011/seq-sat        -- sequential satisficing
    /home/TDDD48/planners/ipc2011/tempo-sat      -- temporal satisficing

The names of the individual planners are shown in the tables below. For example, the winner of the sequential satisficing (seq-sat) track in 2014 was IBaCoP2, while SymBA*-2 won the sequential optimizing (seq-opt) track. These two planners can then be executed as follows:

    /home/TDDD48/planners/ipc2014/seq-sat/seq-sat-ibacop2/plan [args]
    /home/TDDD48/planners/ipc2014/seq-opt/seq-opt-symba-2/plan [args]

Many planners write quite a bit of text to standard output, and it may be difficult to see everything. This is particularly problematic for those planners that continually search for better plans: You may miss the fact that one or several plans have actually been generated already. In this case you may let less buffer the output:

    /home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput |& less

Simply press space to continue to the next screen of output, or use the arrow keys to go up and down. Beware: When the planner has written a full screen of output, it may block, waiting for you to press space until it can go on. You can't simply leave the planner running piped through less and go through the pages of output later, unfortunately.

To append the output to a log file instead:

    /home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput |& tee -a mylogfile

Available Planners

There are seven sections of planners available:

Sequential Satisficing Planners from IPC-2014 (seq-sat)

All of the following planners generate sequential plans and support STRIPS expressivity. Many also support some extensions, but beware that this support may be incomplete in certain ways. Entries in parentheses denote cases where a PDDL feature is supported by the planner, but where inputs not using the feature are preferred (this may mean that the planner is slower if the feature is used, for example). More information can be found in the questionnaire files in the same directory as the planner, and even more at the IPC-2014 competition homepage.

The following planners are among the most interesting ones from IPC-2014. Additional planners may be made available later.

Name Negative
Conditions
ADL
Conditions
Conditional
Effects
Universal
Effects
Comments
BFS(f) yes yes yes yes
BiFD yes yes
Freelunch yes yes
Jasper yes yes yes yes
YAHSP3

In the planning competition, the available planners finished in this order (best first): Jasper, BFS(f), BiFD, Freelunch, YAHSP3. Note that many planners were quite close in performance, and that a planner that is better for one domain is often worse for another!

Sequential Optimizing Planners from IPC-2014 (seq-opt)

All of the following planners generate sequential optimal plans and support at least STRIPS expressivity. More information can be found at the IPC-2014 competition homepage.

Name Negative Conditions ADL Conditions Conditional Effects Universal Effects Comments
AllPACA yes yes This is a portfolio planner which uses machine learning to decide which out of 5 2011 seq-opt planners to use.
Rational Lazy A* yes yes
SPM&S yes yes yes
SymBA*-1 yes yes yes
SymBA*-2 yes yes yes

In the planning competition, the top planners finished in this order (best first): SymBA*-2, SymBA*-1, SPM&S, AllPACA, Rational Lazy A*.

Temporal Satisficing Planners from IPC-2014 (tempo-sat)

These planners are specialized in reasoning about concurrency and time.
Name Negative Conditions ADL Conditions Conditional Effects Universal Effects Comments
ITSAT
Temporal Fast Downward yes yes
YAHSP3 Does not support any additional features.
YAHSP3-MT Multi-threaded version of YAHSP3 but confined to one core.

In the planning competition, the planners finished in this order (best first): YAHSP3-MT, temporal-fast-downward, yahsp3, ITSAT. Note that performance differs across domains and that results relate to the quality of the best plan generated within 30 minutes.

Sequential Satisficing Planners from IPC-2011 (seq-sat)

All of the following planners generate sequential plans and support STRIPS expressivity. Many also support some extensions, but beware that this support may be incomplete in certain ways. Entries in parentheses denote cases where a PDDL feature is supported by the planner, but where inputs not using the feature are preferred (this may mean that the planner is slower if the feature is used, for example). More information can be found in the official list of planners in IPC-2011.

Name Negative
Conditions
ADL
Conditions
Conditional
Effects
Universal
Effects
Comments
acoplan Reports syntax errors in correct files
acoplan2 Reports syntax errors in correct files
arvand (yes) (yes) (yes) (yes)
brt (yes) (yes) (yes) (yes) Seems quite slow for the lab domains
cbp yes yes yes yes
cbp2 yes yes yes yes
cpt4
fd-autotune-1 (yes) (yes) (yes) (yes) See lama-2008
fd-autotune-2 (yes) (yes) (yes) (yes) See lama-2008
forkuniform
lama-2008 (yes) (yes) (yes) (yes)

Cannot handle the case when one action has two conditional effects that assign opposite values to an atom: (and (when foo (is-open door)) (when bar (not (is-open door)))).

The planner can find one solution, try to find a better solution, and then say "No solution found" which actually means "No more solutions found"!

lama-2011 (yes) (yes) (yes) (yes) See lama-2008
lamar (yes) (yes) (yes) (yes)
madagascar yes yes yes yes Only writes plans to output file, not to standard output
madagascar-p yes yes yes yes
probe yes yes Not compiled
randward (yes) (yes) (yes) (yes)
sharaabi
yahsp2 One of the planners that generates multiple plans in separate output files

In the planning competition, the planners finished in this order (best first): lama-2011, fd-autotune-1,fd-autotune-2, forkuniform, probe, arvand, lama-2008, lamar, randward, brt, cbp2, yahsp2, cbp, madagascar-p, madagascar, cpt4, sharaabi, acoplan, acoplan2. Note that many planners were quite close in performance, and that a planner that is better for one domain is often worse for another!

Sequential Optimizing Planners from IPC-2011 (seq-opt)

All of the following planners generate sequential optimal plans and support STRIPS expressivity. One also supports some extensions, but beware that this support may be incomplete in certain ways. More information can be found in the official list of planners in IPC-2011.

Name Negative Conditions ADL Conditions Conditional Effects Universal Effects Comments
bjolp Big Joint Optimal Landmarks Planner
cpt4 Seems very slow for the lab domains
fd-autotune
lmcut
selmax

In the planning competition, the planners finished in this order (best first): selmax, lmcut, fd-autotune, bjolp, cpt4.

Temporal Satisficing Planners from IPC-2011 (tempo-sat)

These planners are specialized in reasoning about concurrency and time.
Name Negative Conditions ADL Conditions Conditional Effects Universal Effects Comments
cpt4 Segfaults on the new hardware.
dae_yahsp DAE = Divide and Evolve, combining an "ordinary" planner with evolutionary algorithms. Reports "Argument list too long" on the new hardware.
lmtd yes yes yes Gives sound plans, even though it reports errors.
popf2 yes
sharaabi
tlp-gp Disabled
yahsp2
yahsp2-mt

In the planning competition, the planners finished in this order (best first): dae_yahsp, yahsp2-mt, popf2, yahsp2, lmtd, cpt4, charaabi, tlp-gp. Note that performance differs across domains. For one particular domain cpt4 was the best, for a couple of domains, popf2 and lmtd (third and fifth place) did considerably better than the other planners, and so on. Note also that results relate to the quality of the best plan generated within 30 minutes.

Additional Planners

Fast Forward (FF)

Name Negative Conditions ADL Conditions Conditional Effects Universal Effects
ff yes yes yes yes

FF is a very well-known forward-chaining heuristic search planner producing sequential plans. Although it does not guarantee optimality, it tends to find good solutions very quickly most of the time and parts of the planner have been used as the basis of many other planners.

FF searches the state space forward from the initial state, using a variant of hill-climbing search and a non-admissible heuristic. The heuristic is computed using a simplified form of the same graph construction used by Graphplan and IPP, but FF does not actually search for plans in this graph.

In ordinary hill-climbing search, the best (according to the heuristic) neighboring node is chosen. The hill-climbing variant used by FF works such that if all neighboring nodes are worse than the current node, a breadth-first search is started outwards from the current node and continued until a better node is found. FF also uses some cut-off techniques so that it does not have to consider all neighboring nodes when searching for a better node.

Before starting search, FF performs some analysis to try to simplify the goal. If the goal simplifies to the constant FALSE, the problem is provably unsolvable and the planner exits.

Please note the following:

  • FF requires all the arguments to an operator to be different, i.e. an action such as (move A B A) is considered impossible regardless of the domain definition.
  • Nested when:s and and:s confuse FF. Try to avoid this.
  • Avoid the use of exists in action effects (not even in when-conditions). Sometimes forall provides a better alternative.

Running FF

Running FF is similar to running any of the other planners that are installed:

    /home/TDDD48/planners/ff/plan [args]

All planners take the same three arguments: A domain file, a problem file, and a solution file (which may be generated by the planner or may be ignored, depending on the planner). For example (in a single line):

    /home/TDDD48/planners/ff/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput

The Output of FF

Typically, FF's output will look like the following. After some preamble (files read, etc.), hill-climbing search starts ("goal distance" is the heuristic value of the current node):

	Cueing down from goal distance:   31 into depth [1]
        30            [1]
        29            [1][2]
        27            [1][2]
        26            [1]
        28            [1][2]
	.
	.
	.    

The number to the right is the depth of the breadth-first search needed to find a better node. This continues until a plan is found:

	ff: found legal plan as follows

	step    0: LOAD PACKET3 TRUCK3 OFFICE3
        1: LOAD PACKET4 TRUCK4 OFFICE4
	.
	.
	.    

Since hill-climbing can get stuck in "dead ends" of the search space, the breadth-first search for a better node may fail. In this case, FF reverts to plain best-first search from the initial state:

	Enforced Hill-climbing failed !
	switching to Best-first Search now.

	advancing to distance :    5
        4
	.
	.
	.
    

Switching to best-first search is always "safe" (i.e. it makes the planner complete), but it is less efficient.

IPP: Interference Progression Planner

About IPP

Note that IPP, like FF, requires all the arguments to an operator to be different, i.e. an action such as (move A B A) is considered impossible regardless of the domain definition.

The IPP planner does handle subtypes. However, it requires every type to be defined before it is used. This means you can only specify ":types A B - C" if type C has already been defined. Simply defining ":types C A B - C" does not work, of course: This says that C is a subtype of itself. IPP has a built-in type "object" that can be used as the topmost type: ":types C - object A B - C".

Running IPP

Running IPP is similar to running any of the other planners that are installed:

    /home/TDDD48/planners/ipp/plan [args]

All planners take the same three arguments: A domain file, a problem file, and a solution file (which may be generated by the planner or may be ignored, depending on the planner). For example (in a single line):

    /home/TDDD48/planners/ipp/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput

Common Problems

  • No such file or directory: 'output' – often caused by calling an output file "output" with a planner that also wants to use "output" as a temporary file name. For example:

    args: ['/home/TDDD48/planners/ipc2011/seq-opt/seq-opt-fdss-2/src/search/downward-1', '--search', 'astar(blind())', '--plan-file', 'output']
    Traceback (most recent call last):
      File "/home/TDDD48/planners/ipc2011/seq-opt/seq-opt-fdss-2/src/search/downward-seq-opt-fdss-2.py", line 19, in <module>
        seq_opt_portfolio.run(CONFIGS)
      File "/home/TDDD48/planners/ipc2011/seq-opt/seq-opt-fdss-2/src/search/seq_opt_portfolio.py", line 60, in run
        os.open("output", os.O_RDONLY)
        OSError: [Errno 2] No such file or directory: 'output'
  • Do you get the following or similar output when you try to run a planner?

    1. Running translator
    'import site' failed; use -v for traceback
    Traceback (most recent call last):
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/
            translate/translate.py", line 4, in <module>
        from __future__ import with_statement
    ImportError: No module named __future__
    

    In this case you must do "unset PYTHONHOME" in the shell before you run the planner. (The problem is caused by an environment variable that may be transferred from a remote system when using Thinlinc or ssh. It causes Python to look for libraries in the wrong location.)

  • Do you get the following or similar output when you try to run a planner?

    Completing instantiation...
    Traceback (most recent call last):
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/translate.py", line 592, in <module>
        sas_task = pddl_to_sas(task)
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/translate.py", line 424, in pddl_to_sas
        reachable_action_params) = instantiate.explore(task)
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/instantiate.py", line 79, in explore
        return instantiate(task, model)
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/instantiate.py", line 55, in instantiate
        fluent_facts, type_to_objects)
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/pddl/actions.py", line 132, in instantiate
        cost = int(self.cost.instantiate(var_mapping, init_facts).expression.value)
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/pddl/f_expression.py", line 105, in instantiate
        expression = self.expression.instantiate(var_mapping, init_facts)
      File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/pddl/f_expression.py", line 82, in instantiate
        assert False, "Could not find instantiation for PNE!"
    AssertionError: Could not find instantiation for PNE!
    

    A PNE is a PrimitiveNumericExpression. This most likely means that you are writing a cost-based domain with costs defined by a function, and that the function does not specify the costs for all combinations of arguments. Maybe you missed the case where a UAV flies from a location to the same location?


Page responsible: Jonas Kvarnström
Last updated: 2015-04-14