Environment Wrapping Utils

Note

GOPS provides a number of additional functions through environment wrappers, such as observation scaling, observation noising, action clipping, and reward shaping. You can wrap an environment with certain functions by defining the corresponding parameters in the training configuration file as follows:

parser.add_argument("--obs_shift", type=float, default=0.0)
parser.add_argument("--obs_scale", type=float, default=1.0)
parser.add_argument("--obs_noise_type", type=str, default="normal")
parser.add_argument("--obs_noise_data", type=list, default=[0.0, 1.0])
parser.add_argument("--clip_action", type=bool, default=True)
parser.add_argument("--reward_shift", type=float, default=0.0)
parser.add_argument("--reward_scale", type=float, default=1.0)
parser.add_argument("--max_episode_steps", type=int, default=200)
gops.env.wrapper.wrapping_utils.wrapping_env(env, max_episode_steps: int | None = None, reward_shift: float | None = None, reward_scale: float | None = None, obs_shift: ndarray | float | list | None = None, obs_scale: ndarray | float | list | None = None, obs_noise_type: str | None = None, obs_noise_data: list | None = None, repeat_num: int | None = None, sum_reward: bool = True, action_scale: bool = True, min_action: float | int | ndarray | list = -1.0, max_action: float | int | ndarray | list = 1.0)
Automatically wrap data type environment according to input arguments. Wrapper will not be used

if all corresponding parameters are set to None.

Parameters:

  • env – original data type environment.

  • max_episode_steps (Optional[int]) – parameter for gym.wrappers.time_limit.TimeLimit wrapper. if it is set to None but environment has ‘max_episode_steps’ attribute, it will be filled in TimeLimit wrapper alternatively.

  • reward_shift (Optional[float]) – parameter for reward shaping wrapper.

  • reward_scale (Optional[float]) – parameter for reward shaping wrapper.

  • obs_shift (Union[np.ndarray, float, list, None]) – parameter for observation scale wrapper.

  • obs_scale (Union[np.ndarray, float, list, None]) – parameter for observation scale wrapper.

  • obs_noise_type (Optional[str]) – parameter for observation noise wrapper.

  • obs_noise_data (Optional[list]) – parameter for observation noise wrapper.

  • repeat_num (Optional[int]) – parameter for action repeat wrapper.

  • sum_reward (bool) – parameter for action repeat wrapper.

  • action_scale (bool) – parameter for scale action wrapper, default to True.

  • min_action (Union[float, int, np.ndarray, list]) – minimum action after scaling.

  • max_action (Union[float, int, np.ndarray, list]) – maximum action after scaling.

Returns:

wrapped data type environment.

gops.env.wrapper.wrapping_utils.wrapping_model(model, reward_shift: float | None = None, reward_scale: float | None = None, obs_shift: ndarray | float | list | None = None, obs_scale: ndarray | float | list | None = None, clip_obs: bool = True, clip_action: bool = True, mask_at_done: bool = True, repeat_num: int | None = None, sum_reward: bool = True, action_scale: bool = True, min_action: float | int | ndarray | list = -1.0, max_action: float | int | ndarray | list = 1.0)
Automatically wrap model type environment according to input arguments. Wrapper will not be used

if all corresponding parameters are set to None.

Parameters:

  • model – original data type environment.

  • reward_shift (Optional[float]) – parameter for reward shaping wrapper.

  • reward_scale (Optional[float]) – parameter for reward shaping wrapper.

  • obs_shift (Union[np.ndarray, float, list, None]) – parameter for observation scale wrapper.

  • obs_scale (Union[np.ndarray, float, list, None]) – parameter for observation scale wrapper.

  • clip_obs (bool) – parameter for clip observation wrapper, default to True.

  • clip_action (bool) – parameter for clip action wrapper, default to True.

  • mask_at_done (bool) – parameter for mask at done wrapper, default to True.

  • repeat_num (Optional[int]) – parameter for action repeat wrapper.

  • sum_reward (bool) – parameter for action repeat wrapper.

  • action_scale (bool) – parameter for scale action wrapper, default to True.

  • min_action (Union[float, int, np.ndarray, list]) – minimum action after scaling.

  • max_action (Union[float, int, np.ndarray, list]) – maximum action after scaling.

Returns:

wrapped model type environment.