Release notes for v0.26.0
This release is aimed to be the last of the major API changes to the core API. All of the previously "turned off" changes of the base API (step termination / truncation, reset info, no seed function, render mode determined by initialization) are now expected by default. We still plan to make breaking changes to Gym itself, but to things that are very easy to upgrade (environments and wrappers), and things that aren't super commonly used (the vector API). Once those aspects are stabilized, we'll do a proper 1.0 release and follow semantic versioning. Additionally, unless something goes terribly wrong with this release and we have to release a patched version, this will be the last release of Gym for a while.
If you've been waiting for a "stable" release of Gym to upgrade your project given all the changes that have been going on, this is the one.
We also just wanted to say that we tremendously appreciate the communities patience with us as we've gone on this journey taking over the maintenance of Gym and making all of these huge changes to the core API. We appreciate your patience and support, but hopefully, all the changes from here on out will be much more minor.
Breaking backward compatibility
These changes are true of all gym's internal wrappers and environments but for environments not updated, we provide the EnvCompatibility wrapper for users to convert old gym v21 / 22 environments to the new core API. This wrapper can be easily applied in gym.make and gym.register through the apply_api_compatibility parameters.
Step Termination / truncation - The Env.step function returns 5 values instead of 4 previously (observations, reward, termination, truncation, info). A blog with more details will be released soon to explain this decision. @arjun-kg- Reset info - The
Env.reset function returns two values (obs and info) with no return_info parameter for gym wrappers and environments. This is important for some environments that provided action masking information for each actions which was not possible for resets. @balisujohn
- No
Seed function - While Env.seed was a helpful function, this was almost solely used for the beginning of the episode and is added to gym.reset(seed=...). In addition, for several environments like Atari that utilise external random number generators, it was not possible to set the seed at any time other than reset. Therefore, seed is no longer expected to function within gym environments and is removed from all gym environments @balisujohn
- Rendering - It is normal to only use a single render mode and to help open and close the rendering window, we have changed
Env.render to not take any arguments and so all render arguments can be part of the environment's constructor i.e., gym.make("CartPole-v1", render_mode="human"). For more detail on the new API, see blog post @younik
Major changes
- Render modes - In
v25, there was a change in the meaning of render modes, i.e. "rgb_array" returned a list of rendered frames with "single_rgb_array" returned a single frame. This has been reverted in this release with "rgb_array" having the same meaning as previously to return a single frame with a new mode "rgb_array_list" returning a list of RGB arrays. The capability to return a list of rendering observations achieved through a wrapper applied during gym.make. #3040 @pseudo-rnd-thoughts @younik
- Added
save_video that uses moviepy to render a list of RGB frames and updated RecordVideo to use this function. This removes support for recording ansi outputs. #3016 @younik
RandomNumberGenerator functions: rand, randn, randint, get_state, set_state, hash_seed, create_seed, _bigint_from_bytes and _int_list_from_bigint have been removed. @balisujohn- Bump
ale-py to 0.8.0 which is compatibility with the new core API
- Added
EnvAPICompatibility wrapper @RedTachyon
Minor changes
详细ChangeLog
Release notes
This release finally introduces all new API changes that have been planned for the past year or more, all of which will be turned on by default in a subsequent release. After this point, Gym development should get massively smoother. This release also fixes large bugs present in 0.24.0 and 0.24.1, and we highly discourage using those releases.
API Changes
Step - A majority of deep reinforcement learning algorithm implementations are incorrect due to an important difference in theory and practice as done is not equivalent to termination. As a result, we have modified the step function to return five values, obs, reward, termination, truncation, info. The full theoretical and practical reason (along with example code changes) for these changes will be explained in a soon-to-be-released blog post. The aim for the change to be backward compatible (for now), for issues, please put report the issue on github or the discord. @arjun-kgRender - The render API is changed such that the mode has to be specified during gym.make with the keyword render_mode, after which, the render mode is fixed. For further details see https://younis.dev/blog/2022/render-api/ and #2671. This has the additional changes
- with
render_mode="human" you don't need to call .render(), rendering will happen automatically on env.step()
- with
render_mode="rgb_array", .render() pops the list of frames rendered since the last .reset()
- with
render_mode="single_rgb_array", .render() returns a single frame, like before.
Space.sample(mask=...) allows a mask when sampling actions to enable/disable certain actions from being randomly sampled. We recommend developers add this to the info parameter returned by reset(return_info=True) and step. See #2906 for example implementations of the masks or the individual spaces. We have added an example version of this in the taxi environment. @pseudo-rnd-thoughts- Add
Graph for environments that use graph style observation or action spaces. Currently, the node and edge spaces can only be Box or Discrete spaces. @jjshoots
- Add
Text space for Reinforcement Learning that involves communication between agents and have dynamic length messages (otherwise MultiDiscrete can be used). @ryanrudes @pseudo-rnd-thoughts
Bug fixes
- Fixed car racing termination where if the agent finishes the final lap, then the environment ends through truncation not termination. This added a version bump to Car racing to v2 and removed Car racing discrete in favour of
gym.make("CarRacing-v2", continuous=False) @araffin
- In
v0.24.0, opencv-python was an accidental requirement for the project. This has been reverted. @KexianShen @pseudo-rnd-thoughts
- Updated
utils.play such that if the environment specifies keys_to_action, the function will automatically use that data. @Markus28
- When rendering the blackjack environment, fixed bug where rendering would change the dealers top car. @balisujohn
- Updated mujoco docstring to reflect changes that were accidently overwritten. @Markus28
Misc
- The whole project is partially type hinted using pyright (none of the project files is ignored by the type hinter). @RedTachyon @pseudo-rnd-thoughts (Future work will add strict type hinting to the core API)
- Action masking added to the taxi environment (no version bump due to being backwards compatible) @pseudo-rnd-thoughts
- The
Box space shape inference is allows high and low scalars to be automatically set to (1,) shape. Minor changes to identifying scalars. @pseudo-rnd-thoughts
- Added option support in classic control environment to modify the bounds on the initial random state of the environment @psc-g
- The
RecordVideo wrapper is becoming deprecated with no support for TextEncoder with the new render API. The plan is to replace RecordVideo with a single function that will receive a list of frames from an environment and automatically render them as a video using MoviePy. @johnMinelli
- The gym
py.Dockerfile is optimised from 2Gb to 1.5Gb through a number of optimisations @TheDen
详细ChangeLog
