Suggested Software

• Starter repo
  LUH-AI template for code repositories. It contains crucial features such as linting, formatting, automated tests, and checking docstrings. This should be used for all projects.
• The art of debugging
  Comprehensive overview of debugging strategies.
• How to debug your ML algorithm
  Blogpost about debugging ML code
• How to debug your RL algorithm
  Blogpost about debugging RL code

• Torch
  Our standard ML framework. Torch should be the basis for most projects and should only be deviated from in very special cases.
• Jax
  Alternative to Torch, but only recommended for project with high hardware demands. The startup cost is high and thus this should be discussed with the supervisor before using.
• Scikit-Learn
  Framework with many options for classic ML & Data Science methods.
• Tensorflow
  Alternative to Torch. Tensorflow should not be used in projects with us.
• Keras
  Interface for multiple ML frameworks. There should be no reason to use Keras over Torch.

Tools

• SMAC 
  Our HPO Tool, the default choice for optimization.
• Hypersweeper
  Parallel execution of different HPO tools. Should be combined with SMAC for expensive projects.
• AutoPyTorch
  Pipeline search for Torch.
• AutoSKlearn
  Pipeline search for SKlearn. Useful for tabular data.

Benchmarks

• HPOBench
  General HPO benchmark
• DACBench 
  Benchmark for dynamic HPO
• ARLBench 
  HPO specifically for RL

• DeepCave 
  HPO analysis, e.g. hyperparameter importance.
• Penzai
  Analysis of JAX networks.

RL Algorithms

• Mighty 
  Flexible implementations of (Meta-)RL algorithms in Torch.
• CleanRL
  Implementations of various algorithms and environments.
• PureJax
  Very fast, but not very flexible or simple to change
• Stoix
  Alternative zu PureJax with simpler interface

Environments

• Classic Control
  Very simple, but good for testing & debugging.
• MiniGrid
  Testing exploration with a large variety of difficulties.
• CARL
  Extensions of classic environments to test generalization.
• Mujoco
  Robotic simulations.
• Brax
  Alternative to Mujoco, only recommend for use cases with high parallelization.
• MiniHack
  NetHack variations, that have a lot of variance and can become very difficult, but also have complicated observation spaces. Ideal to test skills or for very hard tasks.
• Crafter/Craftax
  Minecraft-like environment with simpler observations. Can become quite complex and contains temporal dependencies.
• Robotics Gym
  Complex robotics tasks on realistic models.
• Atari
  Atari game. Somewhat difficult, but most of all expensive to evaluate. Only recommended for HPO, offline RL or in the 100k version.
• ProcGen
  Similar to Atari with added complexity.
• Pufferlib
  Faster execution of many different environments.