Running Planners

Introduction

A large number of planners, both from the latest International Planning Competition and from other sources, are available for use in the labs. We have tested and verified these planners on typical planning competition domains. The planners however are in general state of the art research prototypes and there is no guarantee that they are free from bugs or that they work perfectly in all domains. In fact, labs seem to be an excellent way of finding strange bugs in planners, since you as students tend to have a greater variety in your domain models than the researchers developing the planners, who tend to follow their own "patterns".

So if you encounter problems with one planner, first check the list of common problems at the end. Then try other planners as well – this helps you see whether the problem lies in the planner or in your domain definition. Also, please ask us about problems at the labs or by e-mail. Never spend hours or days trying to solve a problem!

Satisficing and Optimizing Planners

We distinguish between satisficing and optimizing planners. Both types are available.

An optimizing planner should always return a plan that is optimal in terms of some cost function defined on plans, no matter how much time it takes to generate such a plan. For example, it may return plans with the smallest number of actions, or one may define costs for each action and generate plans with minimal total cost.

We might say that the "opposite" of an optimizing planner is one that returns a plan as quickly as possible, regardless of how bad the plan is. However, this is seldom what we want: We rarely prefer a plan of cost 10000 over a plan of cost 20, even if the first one takes slightly less time to produce.

Therefore, satisficing planners generally try to find a good balance between time and quality. Some planners generate a single plan using a built-in heuristic tuned for both criteria. Other planners provide more flexibility by generating an initial plan but then continues to search for better plans until explicitly interrupted by the user, either because the current plan is "good enough" or because one cannot wait any longer.

In fact, the satisficing track of the 2011/2014 planning competitions reward those planners that yield the highest quality plans within a time limit of 30 minutes. Several of the planners below will therefore continue trying to generate better plans until interrupted, as discussed above. We will show later how to run these planners.

Running Planners

The planners used in lab 1-3 are from different sources. There are planners from the latest planning competition, which took place in 2014. These planners are state of the art. There are also planners from the competition of 2011, which in many cases are also state of the art, as well as older planners. All planners are run by the command:

    planner mydomain.pddl mydomain-prob1.pddl plan-output

The planner will then use the supplied domain and problem to try to generate a plan. The first generated plan will be called plan-output or plan-output.1 and subsequent plans, in case there are any, will get increasing numbers. This is true for all the competition planners. The older planners, FF and IPP, only output the plan in the terminal. Also, be aware that some planners may create a lot of auxiliary files in the current directory which you must remove manually.

The planners from IPC-2014 are installed under /home/TDDD48/planners/ipc2014 and the planners from IPC-2011 are installed in /home/TDDD48/planners/ipc2011. Both these locations contain planners from the different competition planning tracks.

    /home/TDDD48/planners/ipc2014/seq-opt        -- sequential optimizing
    /home/TDDD48/planners/ipc2014/seq-sat        -- sequential satisficing
    /home/TDDD48/planners/ipc2014/tempo-sat      -- temporal satisficing
    /home/TDDD48/planners/ipc2011/seq-opt        -- sequential optimizing
    /home/TDDD48/planners/ipc2011/seq-sat        -- sequential satisficing
    /home/TDDD48/planners/ipc2011/tempo-sat      -- temporal satisficing

The names of the individual planners are shown in the tables below. For example, the winner of the sequential satisficing (seq-sat) track in 2014 was IBaCoP2, while SymBA*-2 won the sequential optimizing (seq-opt) track. These two planners can then be executed as follows:

    /home/TDDD48/planners/ipc2014/seq-sat/seq-sat-ibacop2/plan [args]
    /home/TDDD48/planners/ipc2014/seq-opt/seq-opt-symba-2/plan [args]

Many planners write quite a bit of text to standard output, and it may be difficult to see everything. This is particularly problematic for those planners that continually search for better plans: You may miss the fact that one or several plans have actually been generated already. In this case you may let less buffer the output:

    /home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput |& less

Simply press space to continue to the next screen of output, or use the arrow keys to go up and down. Beware: When the planner has written a full screen of output, it may block, waiting for you to press space until it can go on. You can't simply leave the planner running piped through less and go through the pages of output later, unfortunately.

To append the output to a log file instead:

    /home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput |& tee -a mylogfile

Available Planners

There are seven sections of planners available:

• Sequential Satisficing from IPC-2014
• Sequential Optimizing from IPC-2014
• Temporal Satisficing from IPC-2014
• Sequential Satisficing from IPC-2011
• Sequential Optimizing from IPC-2011
• Temporal Satisficing from IPC-2011
• Planners from other sources
For each planner we report on their capabilities as well as provide some comments about the general behavior expected.

In the planning competition, the available planners finished in this order (best first): Jasper, BFS(f), BiFD, Freelunch, YAHSP3. Note that many planners were quite close in performance, and that a planner that is better for one domain is often worse for another!

Sequential Optimizing Planners from IPC-2014 (seq-opt)

All of the following planners generate sequential optimal plans and support at least STRIPS expressivity. More information can be found at the IPC-2014 competition homepage.

In the planning competition, the top planners finished in this order (best first): SymBA*-2, SymBA*-1, SPM&S, AllPACA, Rational Lazy A*.

Temporal Satisficing Planners from IPC-2014 (tempo-sat)

These planners are specialized in reasoning about concurrency and time.

In the planning competition, the planners finished in this order (best first): lama-2011, fd-autotune-1,fd-autotune-2, forkuniform, probe, arvand, lama-2008, lamar, randward, brt, cbp2, yahsp2, cbp, madagascar-p, madagascar, cpt4, sharaabi, acoplan, acoplan2. Note that many planners were quite close in performance, and that a planner that is better for one domain is often worse for another!

Sequential Optimizing Planners from IPC-2011 (seq-opt)

All of the following planners generate sequential optimal plans and support STRIPS expressivity. One also supports some extensions, but beware that this support may be incomplete in certain ways. More information can be found in the official list of planners in IPC-2011.

In the planning competition, the planners finished in this order (best first): selmax, lmcut, fd-autotune, bjolp, cpt4.

Temporal Satisficing Planners from IPC-2011 (tempo-sat)

These planners are specialized in reasoning about concurrency and time.

Additional Planners

Fast Forward (FF)

Name	Negative Conditions	ADL Conditions	Conditional Effects	Universal Effects
ff	yes	yes	yes	yes

FF is a very well-known forward-chaining heuristic search planner producing sequential plans. Although it does not guarantee optimality, it tends to find good solutions very quickly most of the time and parts of the planner have been used as the basis of many other planners.

FF searches the state space forward from the initial state, using a variant of hill-climbing search and a non-admissible heuristic. The heuristic is computed using a simplified form of the same graph construction used by Graphplan and IPP, but FF does not actually search for plans in this graph.

In ordinary hill-climbing search, the best (according to the heuristic) neighboring node is chosen. The hill-climbing variant used by FF works such that if all neighboring nodes are worse than the current node, a breadth-first search is started outwards from the current node and continued until a better node is found. FF also uses some cut-off techniques so that it does not have to consider all neighboring nodes when searching for a better node.

Before starting search, FF performs some analysis to try to simplify the goal. If the goal simplifies to the constant FALSE, the problem is provably unsolvable and the planner exits.

Please note the following:

• FF requires all the arguments to an operator to be different, i.e. an action such as (move A B A) is considered impossible regardless of the domain definition.
• Nested when:s and and:s confuse FF. Try to avoid this.
• Avoid the use of exists in action effects (not even in when-conditions). Sometimes forall provides a better alternative.

Running FF

Running FF is similar to running any of the other planners that are installed:

    /home/TDDD48/planners/ff/plan [args]

All planners take the same three arguments: A domain file, a problem file, and a solution file (which may be generated by the planner or may be ignored, depending on the planner). For example (in a single line):

    /home/TDDD48/planners/ff/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput

The Output of FF

Typically, FF's output will look like the following. After some preamble (files read, etc.), hill-climbing search starts ("goal distance" is the heuristic value of the current node):

	Cueing down from goal distance:   31 into depth [1]
        30            [1]
        29            [1][2]
        27            [1][2]
        26            [1]
        28            [1][2]
	.
	.
	.    

The number to the right is the depth of the breadth-first search needed to find a better node. This continues until a plan is found:

	ff: found legal plan as follows

	step    0: LOAD PACKET3 TRUCK3 OFFICE3
        1: LOAD PACKET4 TRUCK4 OFFICE4
	.
	.
	.    

Since hill-climbing can get stuck in "dead ends" of the search space, the breadth-first search for a better node may fail. In this case, FF reverts to plain best-first search from the initial state:

	Enforced Hill-climbing failed !
	switching to Best-first Search now.

	advancing to distance :    5
        4
	.
	.
	.
    

Switching to best-first search is always "safe" (i.e. it makes the planner complete), but it is less efficient.

IPP: Interference Progression Planner

About IPP

Note that IPP, like FF, requires all the arguments to an operator to be different, i.e. an action such as (move A B A) is considered impossible regardless of the domain definition.

The IPP planner does handle subtypes. However, it requires every type to be defined before it is used. This means you can only specify ":types A B - C" if type C has already been defined. Simply defining ":types C A B - C" does not work, of course: This says that C is a subtype of itself. IPP has a built-in type "object" that can be used as the topmost type: ":types C - object A B - C".

Running IPP

Running IPP is similar to running any of the other planners that are installed:

    /home/TDDD48/planners/ipp/plan [args]

All planners take the same three arguments: A domain file, a problem file, and a solution file (which may be generated by the planner or may be ignored, depending on the planner). For example (in a single line):

    /home/TDDD48/planners/ipp/plan
        mydomain.pddl mydomain-prob1.pddl tempoutput

Common Problems

• No such file or directory: 'output' – often caused by calling an output file "output" with a planner that also wants to use "output" as a temporary file name. For example:

  args: ['/home/TDDD48/planners/ipc2011/seq-opt/seq-opt-fdss-2/src/search/downward-1', '--search', 'astar(blind())', '--plan-file', 'output']
  Traceback (most recent call last):
    File "/home/TDDD48/planners/ipc2011/seq-opt/seq-opt-fdss-2/src/search/downward-seq-opt-fdss-2.py", line 19, in <module>
      seq_opt_portfolio.run(CONFIGS)
    File "/home/TDDD48/planners/ipc2011/seq-opt/seq-opt-fdss-2/src/search/seq_opt_portfolio.py", line 60, in run
      os.open("output", os.O_RDONLY)
      OSError: [Errno 2] No such file or directory: 'output'

• Do you get the following or similar output when you try to run a planner?

  1. Running translator
  'import site' failed; use -v for traceback
  Traceback (most recent call last):
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/
          translate/translate.py", line 4, in <module>
      from __future__ import with_statement
  ImportError: No module named __future__
  

  In this case you must do "unset PYTHONHOME" in the shell before you run the planner. (The problem is caused by an environment variable that may be transferred from a remote system when using Thinlinc or ssh. It causes Python to look for libraries in the wrong location.)

• Do you get the following or similar output when you try to run a planner?

  Completing instantiation...
  Traceback (most recent call last):
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/translate.py", line 592, in <module>
      sas_task = pddl_to_sas(task)
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/translate.py", line 424, in pddl_to_sas
      reachable_action_params) = instantiate.explore(task)
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/instantiate.py", line 79, in explore
      return instantiate(task, model)
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/instantiate.py", line 55, in instantiate
      fluent_facts, type_to_objects)
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/pddl/actions.py", line 132, in instantiate
      cost = int(self.cost.instantiate(var_mapping, init_facts).expression.value)
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/pddl/f_expression.py", line 105, in instantiate
      expression = self.expression.instantiate(var_mapping, init_facts)
    File "/home/TDDD48/planners/ipc2011/seq-sat/seq-sat-lama-2011/src/translate/pddl/f_expression.py", line 82, in instantiate
      assert False, "Could not find instantiation for PNE!"
  AssertionError: Could not find instantiation for PNE!
  

  A PNE is a PrimitiveNumericExpression. This most likely means that you are writing a cost-based domain with costs defined by a function, and that the function does not specify the costs for all combinations of arguments. Maybe you missed the case where a UAV flies from a location to the same location?