Skip to content

MDP and RL interface for PDDL domains via PDDL.jl + POMDPs.jl.

License

Notifications You must be signed in to change notification settings

JuliaPlanners/SymbolicMDPs.jl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SymbolicMDPs.jl

SymbolicMDPs.jl wraps the PDDL.jl interface for PDDL domains and problems within the POMDPs.jl interface for Markov decision processes (MDPs).

Since POMDPs.jl supports the reinforcement learning interface defined by CommonRLInterface.jl, this package also allows PDDL domains to be treated as RL environments compatible with libraries such as ReinforcementLearning.jl.

Installation

Install via the Pkg REPL by running:

add https://github.com/JuliaPlanners/SymbolicMDPs.jl.git

Usage

Once a PDDL domain and problem are loaded (e.g. using PDDL.load_domain and PDDL.load_problem), a corresponding MDP can be constructed as follows:

mdp = SymbolicMDP(domain, problem)

Alternatively, we can manually specify an initial state, goal formula and cost metric formula:

mdp = SymbolicMDP(domain, state, goal, metric)

By default, every action leads to -1 reward. If metric is specified, then the reward will be equal to the difference in the value of the metric between consecutive states. More customization of e.g. reward functions will be supported in the future. States that satisfy the goal condition are considered terminal.

We can also construct RL environments as follows:

env = SymbolicRLEnv(domain, problem)

For many reinforcement learning methods, the underlying PDDL state has to be converted to an integer or vector representation (eg. for tabular or deep reinforcement learning respectively). We can do this by specifying a conversion type:

tabular_env = SymbolicRLEnv(Int, domain, problem)
vector_env = SymbolicRLEnv(Vector, domain, problem)

To use these environments with ReinforcementLearning.jl, we have to perform one further step of conversion:

using ReinforcementLearning
env = convert(RLBase.AbstractEnv, env)

For an example of tabular Q-learning on a small Blocksworld problem with two blocks, see examples/tabular_q.jl.

About

MDP and RL interface for PDDL domains via PDDL.jl + POMDPs.jl.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages