Feature request: update from Gym to Gymnasium, PettingZoo version

[elliottower]

Hi, I was wondering if it would be possible for Kaggle environments to use gymnasium rather than gym? We would like to list this of third party environments for Gymnasium and PettingZoo, but are only adding environments which are compatible with current versions.

Gymnasium a maintained fork of openai gym and is designed as a drop-in replacement (import gym -> import gymnasium as gym). Beyond just bugfixes, many RL training libraries have also switched (rllib, tianshou, CleanRL), or are planning to switch (stable-baselines3). It would be great if users could train agents on Habitat using the latest models and features from these libraries (e.g., scalable distributed training/model serving using Ray/RLlib).

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues (for example, many popular tutorials/implementations of Q learning using gym were actually incorrect because of done, there will be an upcoming blog post explaining more details about this on the Farama site (https://farama.org/blog).

The update to gymnasium should not be too difficult, after that is done it should be relatively simple to update to the most recent Pettingzoo version as well, as it involves the same changes from done to truncated and terminated. We don't have the bandwith to contribute PRs updating every third party environment but I am happy to help with questions/review code, and can direct new Gymnasium/PettingZoo contributors to this if it's not able to be fixed.

[astrojuanlu]

+1. Notice that gym does not install anymore with modern versions of setuptools: openai/gym#3176