• CartPole - Exploring the Parameter Space

    43 views 2 weeks ago
    In this video, we use random guessing to explore the linear parameter space of the CartPole problem. This gives us an idea of what the parameter space of machine learning problems look like, and helps us build an intuition for how moving through a parameter space will affect the performance of that model.

    With this simple problem, we can use random guessing to find a very good solution, but in more complex problems, we'll want to use more complex models that have larger parameters spaces which can't be explored effectively by random guessing. So instead we'll use methods which estimate the gradient of the parameter space with respect to the expected reward, and move our parameters in the direction which increases that expected reward. We'll cover these techniques in future videos.

    Note: In this video, we only used a linear layer with weights for the environment variables without using a bias variable. It turned out that adding a random bias variable consistently made the model perform worse. Can you guess why? If you think about what the bias would do, it makes sense. It would make the model favor moving the cart one way over the other. And since we want the cart to stay upright and in the middle of the screen, it makes sense that adding a bias would lower performance. So in this video, I didn't include exploring using a bias variable. Also, the fewer parameters there were to plot, the easier it was to visualize them.

    OpenAI CartPole environment:

    Socials (@elliotwaite):

    My Gear: https://kit.com/elliotwaite

    Outro Music: ksolis - Nobody Else Show less
    Read more
  • Uploads Play all

    This item has been hidden
  • Follow Along Workouts Play all

    This item has been hidden
  • REMIXES!!! Play all

    This item has been hidden
to add this to Watch Later

Add to

Loading playlists...