Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Python package with new ScenarI/O APIs #176

Merged
merged 35 commits into from
Apr 20, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
7b0b014
Revert "Disable gym_ignition package"
diegoferigo Apr 8, 2020
aaf3395
Update Python files with new GazeboSimulator class
diegoferigo Mar 12, 2020
ceccd7f
Remove robot interfaces from Python
diegoferigo Mar 31, 2020
a113c01
Remove gazebo robot in favor of ScenarI/O
diegoferigo Mar 31, 2020
d111c9a
Move tasks to the new gym_ignition_environments package
diegoferigo Mar 31, 2020
7658ea3
Remove deprecated pybullet runtime
diegoferigo Mar 31, 2020
5f802f0
Remove pybullet envs
diegoferigo Mar 31, 2020
146e397
Move environments to gym_ignition_environments
diegoferigo Mar 31, 2020
511a4c0
Move gympp_env to experimental
diegoferigo Apr 9, 2020
e49366d
Update base.task.Task
diegoferigo Apr 15, 2020
afe72d5
Update base.runtime.Runtime
diegoferigo Apr 15, 2020
70404c1
Update runtimes.gazebo_runtime
diegoferigo Apr 15, 2020
3981215
Update runtimes.realtime_runtime
diegoferigo Apr 15, 2020
acba20e
Update environment configuration
diegoferigo Apr 15, 2020
2663f19
Update utils.logger
diegoferigo Apr 15, 2020
6fbb653
Remove utils.misc
diegoferigo Apr 15, 2020
9e08947
Update utils.typing
diegoferigo Apr 15, 2020
03e8a54
Expose ECMSingleton to Python
diegoferigo Apr 15, 2020
7260696
New utils.scenario helper module
diegoferigo Apr 15, 2020
4d6db91
Update main init file
diegoferigo Apr 15, 2020
97241fc
Configure the ScenarI/O verbosity from Python
diegoferigo Apr 15, 2020
889d455
Configure the environment in the package init file
diegoferigo Apr 15, 2020
57338d1
Find by default resources from IGN_GAZEBO_RESOURCE_PATH
diegoferigo Apr 15, 2020
e684c60
Update demo tasks
diegoferigo Apr 15, 2020
db84c8e
Add new CartPoleContinuousSwingup task
diegoferigo Apr 15, 2020
5ed481e
Fix import loop
diegoferigo Apr 15, 2020
6062f12
Setup the environment variables for gym-ignition-models
diegoferigo Apr 15, 2020
80f0db7
Store the seed in task.Task as public member
diegoferigo Apr 19, 2020
cf1743a
Add a contextmanager to change locally the verbosity
diegoferigo Apr 19, 2020
e632660
Helper to write a string to a tempfile
diegoferigo Apr 19, 2020
5ec561a
Update demo tasks
diegoferigo Apr 19, 2020
2325327
New interfaces for ScenarI/O models
diegoferigo Apr 19, 2020
981285e
New cartpole and pendulum Python models
diegoferigo Apr 19, 2020
4ad6ed9
Remove pybullet dependency
diegoferigo Apr 20, 2020
9e8f58e
Document utils.h
diegoferigo Apr 20, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions bindings/scenario/scenario_bindings.i
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#include "scenario/gazebo/Model.h"
#include "scenario/gazebo/utils.h"
#include "scenario/gazebo/World.h"
#include "scenario/plugins/gazebo/ECMSingleton.h"
#include <cstdint>
%}

Expand Down Expand Up @@ -65,3 +66,10 @@
// GazeboSimulator
%shared_ptr(scenario::gazebo::GazeboSimulator)
%include "scenario/gazebo/GazeboSimulator.h"

// ECMSingleton
%ignore scenario::plugins::gazebo::ECMSingleton::clean;
%ignore scenario::plugins::gazebo::ECMSingleton::getECM;
%ignore scenario::plugins::gazebo::ECMSingleton::getEventManager;
%ignore scenario::plugins::gazebo::ECMSingleton::storePtrs;
%include "scenario/plugins/gazebo/ECMSingleton.h"
5 changes: 5 additions & 0 deletions cpp/scenario/gazebo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,11 @@ target_link_libraries(ScenarioGazebo
set_target_properties(ScenarioGazebo PROPERTIES
PUBLIC_HEADER "${SCENARIO_PUBLIC_HDRS}")

if(NOT CMAKE_BUILD_TYPE STREQUAL "PyPI")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make life complicated for anyone packaging the project in any form, being it as part of a superbuild, as Debian packages or a conda recipe. Instead of conflating build logic and packaging logic, could we have an explicit CMake variable to select if the gym-ignition project get installed in User mode or Developer mode, and then just specify the default based on the CMAKE_BUILD_TYPE , something like:

if(NOT CMAKE_BUILD_TYPE STREQUAL "PyPI")
  set(GYM_IGNITION_DEVELOPER_MODE_DEFAULT ON)
else()
   set(GYM_IGNITION_DEVELOPER_MODE_DEFAULT OFF)
endif() 
option(GYM_IGNITION_DEVELOPER_MODE "If ON, gym-ignition is installed in Developer mode, otherwise in user mode" ${GYM_IGNITION_DEVELOPER_MODE_DEFAULT})
if(GYM_IGNITION_DEVELOPER_MODE)
    target_compile_options(ScenarioGazebo PRIVATE
        -DGYMIGNITION_CMAKE_INSTALL_PREFIX="${CMAKE_INSTALL_PREFIX}")
endif()

Copy link
Member Author

@diegoferigo diegoferigo Apr 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion of propagating the build type as normal option. I will have a look at it. I don't remember precisely why we had to define a new build type for PyPI in the first place, for sure I tried to use a normal CMake variable as first attempt and then I faced problems that led me to this build type workaround, that is much more convoluted. What I'm sure is that I didn't do it if there were more simple solutions :)

I am planning to give a refreshing also to the CMake project, I will keep you comment in mind.

This will make life complicated for anyone packaging the project in any form, being it as part of a superbuild, as Debian packages or a conda recipe.

This is something I don't understand. The PyPI type is not meant to be used by any downstream user or packager. It is meant to be used to create packages for PyPI, as the name (and we are the maintainers of the packages). Any other usage is not supported.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make life complicated for anyone packaging the project in any form, being it as part of a superbuild, as Debian packages or a conda recipe.

This is something I don't understand. The PyPI type is not meant to be used by any downstream user or packager. It is meant to be used to create packages for PyPI, as the name (and we are the maintainers of the packages). Any other usage is not supported.

Exactly. With this modification, now the only way to properly install gym-ignition in a relocatable way is to set the CMAKE_BUILD_TYPE to PyPI, and I guess that this is not supported if you are not installing for PyPI. I guess we want to support "proper" installation also if the project is not installed with PyPI .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ow ok now I understood your initial point: supporting a relocatable installation which could be implemented in the same way we package the project for PyPI.

Note the using PyPI for this purpose is wrong. In fact, due to how the shared libraries must be placed in the site_package folder, you need to have the following installation tree:

install_tree/
├── gym_ignition
│   ├── [...]
│   ├── __init__.py
│   └── plugins
│       ├── libControllerRunner.so
│       ├── libECMProvider.so
│       ├── libJointController.so
│       └── libPhysicsSystem.so
├── gym_ignition_environments
│   ├── __init__.py
│   └── [...]
├── libECMSingleton.so
├── scenario_bindings.py
└── _scenario_bindings.so

As you can see, shared libraries are installed in the root of the CMAKE_INSTALL_PREFIX, and plugins inside gym_ignition/plugins. This is not what you want to obtain in a regular installation! In light of this, we should start thinking how to make coexist a (static) relocatable Developer installation with the PyPI build type without messing too much with the CMake project.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not what you want to obtain in a regular installation! In light of this, we should start thinking how to make coexist a (static) relocatable Developer installation with the PyPI build type without messing too much with the CMake project.

Yes, we can also have a separate issue for that.

target_compile_options(ScenarioGazebo PRIVATE
-DGYMIGNITION_CMAKE_INSTALL_PREFIX="${CMAKE_INSTALL_PREFIX}")
endif()

# ===============
# GazeboSimulator
# ===============
Expand Down
112 changes: 112 additions & 0 deletions cpp/scenario/gazebo/include/scenario/gazebo/utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,130 @@
namespace scenario {
namespace gazebo {
namespace utils {
/**
* Set the verbosity process-wise.
*
* Accepted levels are the following:
*
* - ``<= 0``: No messages.
* - ``1``: Error messages.
* - ``2``: Error and warning messages.
* - ``3``: Error, warning, and info messages.
* - ``>= 4``: Error, warning, info, and debug messages.
*
* If called without specifying the level, it will use
* level 2 or level 4 depending if the project was compiled
* respectively with Release or Debug flags.
*
* @param level The verbosity level.
*/
void setVerbosity(const int level = DEFAULT_VERBOSITY);

/**
* Find a SDF file in the filesystem.
*
* The search path is defined with the ``IGN_GAZEBO_RESOURCE_PATH``
* environment variable.
*
* @param fileName The SDF file name.
* @return The absolute path to the file if found, an empty string
* otherwise.
*/
std::string findSdfFile(const std::string& fileName);

/**
* Check if a SDF string is valid.
*
* An SDF string could contain for instance an SDF model or
* an SDF world, and it is valid if it can be parsed successfully
* by the SDFormat library.
*
* @param sdfString The SDF string to check.
* @return True if the SDF string is valid, false otherwise.
*/
bool sdfStringValid(const std::string& sdfString);

/**
* Get an SDF string from a SDF file.
*
* @param fileName An SDF file. It could be either an absolute path
* to the file or the file name if the parent folder is part
* of the ``IGN_GAZEBO_RESOURCE_PATH`` environment variable.
* @return The SDF string if the file was found and is valid, an
* empty string otherwise.
*/
std::string getSdfString(const std::string& fileName);

/**
* Get the name of a model from a SDF file.
*
* @param fileName An SDF file. It could be either an absolute path
* to the file or the file name if the parent folder is part
* of the ``IGN_GAZEBO_RESOURCE_PATH`` environment variable.
* @param modelIndex The index of the model in the SDF file. By
* default it finds the first model.
* @return The name of the model.
*/
std::string getModelNameFromSdf(const std::string& fileName,
const size_t modelIndex = 0);

/**
* Get the name of a world from a SDF file.
*
* @param fileName An SDF file. It could be either an absolute path
* to the file or the file name if the parent folder is part
* of the ``IGN_GAZEBO_RESOURCE_PATH`` environment variable.
* @param worldIndex The index of the world in the SDF file. By
* default it finds the first world.
* @return The name of the world.
*/
std::string getWorldNameFromSdf(const std::string& fileName,
const size_t worldIndex = 0);

/**
* Return a SDF string with an empty world.
*
* An empty world only has a sun and the default DART
* physics profile enabled.
*
* @note The empty world does not have any ground plane.
*
* @return A SDF string with the empty world.
*/
std::string getEmptyWorld();

/**
* Get a SDF model file from a Fuel URI.
*
* A valid URI has the following form:
*
* ``https://fuel.ignitionrobotics.org/openrobotics/models/model_name``
*
* @param URI A valid Fuel URI.
* @param useCache Load the model from the local cache.
* @return The absolute path to the SDF model.
*/
std::string getModelFileFromFuel(const std::string& URI,
const bool useCache = false);

/**
* Generate a random alpha numeric string.
*
* @param length The length of the string.
* @return The random string.
*/
std::string getRandomString(const size_t length);

/**
* Get the install prefix used by the CMake project.
*
* @note It is defined only if the project is installed in
* Developer mode.
*
* @return A string with the install prefix if the project is
* installed in Developer mode, an empty string otherwise.
*/
std::string getInstallPrefix();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to document that this is only valid if gym-ignition is installed in Developer mode.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I take advantage and I will document all the utils functions together in a new commit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9e8f58e

} // namespace utils
} // namespace gazebo
} // namespace scenario
Expand Down
11 changes: 11 additions & 0 deletions cpp/scenario/gazebo/src/utils.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -231,3 +231,14 @@ std::string utils::getRandomString(const size_t length)
std::generate_n(str.begin(), length, randchar);
return str;
}

std::string utils::getInstallPrefix()
{
#ifdef GYMIGNITION_CMAKE_INSTALL_PREFIX
return GYMIGNITION_CMAKE_INSTALL_PREFIX;
#else
gymppDebug << "User installation detected. The install prefix "
<< "could be detected from the Python module path." << std::endl;
return "";
#endif
}
164 changes: 41 additions & 123 deletions gym_ignition/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,133 +2,51 @@
# This software may be modified and distributed under the terms of the
# GNU Lesser General Public License v2.1 or any later version.

import scenario_bindings

# Workaround for https:/osrf/sdformat/issues/227.
# It has to be done before loading the bindings.
import gym_ignition_models
gym_ignition_models.setup_environment()

# Import SWIG bindings
# See https:/robotology/gym-ignition/issues/7
# https://stackoverflow.com/a/45473441/12150968
# import sys
# if sys.platform.startswith('linux') or sys.platform.startswith('darwin'):
# import os
# dlopen_flags = sys.getdlopenflags()
# if "gympp_bindings" not in sys.modules:
# sys.setdlopenflags(dlopen_flags | os.RTLD_GLOBAL)
# else:
# sys.setdlopenflags(dlopen_flags | os.RTLD_LAZY | os.RTLD_NOLOAD | os.RTLD_GLOBAL)
#
# import gympp_bindings
#
# # Restore the flags
# sys.setdlopenflags(dlopen_flags)
# else:
# import gympp_bindings

import sys
if sys.platform.startswith('linux') or sys.platform.startswith('darwin'):
import os
dlopen_flags = sys.getdlopenflags()
if "scenario_bindings" not in sys.modules:
sys.setdlopenflags(dlopen_flags | os.RTLD_GLOBAL)
else:
sys.setdlopenflags(dlopen_flags | os.RTLD_LAZY | os.RTLD_NOLOAD | os.RTLD_GLOBAL)

import scenario_bindings
scenario_bindings.setVerbosity()

try:
import gympp_bindings
except ImportError:
pass

# Restore the flags
sys.setdlopenflags(dlopen_flags)
else:
import scenario_bindings
scenario_bindings.setVerbosity()

try:
import gympp_bindings
except ImportError:
pass


# Configure the verbosity depending on the selected CMAKE_BUILD_TYPE
scenario_bindings.setVerbosity()

# Configure OS environment variables
# from gym_ignition.utils import gazebo_env_vars, resource_finder
# gazebo_env_vars.setup_gazebo_env_vars()
# resource_finder.add_path_from_env_var("IGN_GAZEBO_RESOURCE_PATH")

# =========================
# REGISTER THE ENVIRONMENTS
# =========================

# from gym.envs.registration import register
# from gym_ignition.utils import resource_finder

# Import the robots
# from gym_ignition.robots import rt
# from gym_ignition.robots import sim

# Import the tasks
# from gym_ignition.tasks import pendulum_swingup
# from gym_ignition.tasks import cartpole_discrete
# from gym_ignition.tasks import cartpole_continuous

# ======================
# GYMPP C++ ENVIRONMENTS
# ======================

# import numpy
# max_float = float(numpy.finfo(numpy.float32).max)
#
# register(
# id='CartPoleDiscrete-Gympp-v0',
# max_episode_steps=5000,
# entry_point='gym_ignition.gympp.cartpole:CartPoleDiscrete')

# ============================
# IGNITION GAZEBO ENVIRONMENTS
# ============================

# register(
# id='Pendulum-Gazebo-v0',
# entry_point='gym_ignition.runtimes.gazebo_runtime:GazeboRuntime',
# max_episode_steps=5000,
# kwargs={'task_cls': pendulum_swingup.PendulumSwingUp,
# 'robot_cls': sim.gazebo.pendulum.PendulumGazeboRobot,
# 'model': "Pendulum/Pendulum.urdf",
# 'world': "DefaultEmptyWorld.world",
# 'rtf': max_float,
# 'agent_rate': 1000,
# 'physics_rate': 1000,
# })
#
# register(
# id='CartPoleDiscrete-Gazebo-v0',
# entry_point='gym_ignition.runtimes.gazebo_runtime:GazeboRuntime',
# max_episode_steps=5000,
# kwargs={'task_cls': cartpole_discrete.CartPoleDiscrete,
# 'robot_cls': sim.gazebo.cartpole.CartPoleGazeboRobot,
# 'model': "CartPole/CartPole.urdf",
# 'world': "DefaultEmptyWorld.world",
# 'rtf': max_float,
# 'agent_rate': 1000,
# 'physics_rate': 1000,
# })
#
# register(
# id='CartPoleContinuous-Gazebo-v0',
# entry_point='gym_ignition.runtimes.gazebo_runtime:GazeboRuntime',
# max_episode_steps=5000,
# kwargs={'task_cls': cartpole_continuous.CartPoleContinuous,
# 'robot_cls': sim.gazebo.cartpole.CartPoleGazeboRobot,
# 'model': "CartPole/CartPole.urdf",
# 'world': "DefaultEmptyWorld.world",
# 'rtf': max_float,
# 'agent_rate': 1000,
# 'physics_rate': 1000,
# })

# =====================
# PYBULLET ENVIRONMENTS
# =====================
from gym_ignition.utils import gazebo_env_vars, resource_finder
gazebo_env_vars.setup_gazebo_env_vars()

# register(
# id='Pendulum-PyBullet-v0',
# entry_point='gym_ignition.runtimes.pybullet_runtime:PyBulletRuntime',
# max_episode_steps=5000,
# kwargs={'task_cls': pendulum_swingup.PendulumSwingUp,
# 'robot_cls': sim.pybullet.pendulum.PendulumPyBulletRobot,
# 'model': "Pendulum/Pendulum.urdf",
# 'world': "plane_implicit.urdf",
# 'rtf': max_float,
# 'agent_rate': 1000,
# 'physics_rate': 1000,
# })
#
# register(
# id='CartPoleDiscrete-PyBullet-v0',
# entry_point='gym_ignition.runtimes.pybullet_runtime:PyBulletRuntime',
# max_episode_steps=5000,
# kwargs={
# # PyBulletRuntime
# 'task_cls': cartpole_discrete.CartPoleDiscrete,
# 'robot_cls': sim.pybullet.cartpole.CartPolePyBulletRobot,
# 'model': "CartPole/CartPole.urdf",
# 'world': "plane_implicit.urdf",
# 'rtf': max_float,
# 'agent_rate': 1000,
# 'physics_rate': 1000,
# })
# Add IGN_GAZEBO_RESOURCE_PATH to the default search path
import os
if "IGN_GAZEBO_RESOURCE_PATH" in os.environ:
resource_finder.add_path_from_env_var("IGN_GAZEBO_RESOURCE_PATH")
3 changes: 0 additions & 3 deletions gym_ignition/base/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,5 @@

# Abstract classes
from . import task
from . import robot
from . import runtime

# Base C++ environment
from . import gympp_env
13 changes: 0 additions & 13 deletions gym_ignition/base/robot/__init__.py

This file was deleted.

21 changes: 0 additions & 21 deletions gym_ignition/base/robot/robot_abc.py

This file was deleted.

Loading