A new way to blend existing models or classical controllers with state-of-the-art reinforcement learning
Today we announced a new feature, Gears, to help programmers leverage their existing AI models and code to solve reinforcement learning problems with the Bonsai Platform.
Using Gears, a Bonsai Platform programmer can replace any “concept” in an Inkling program with existing Python code. In effect this means you can transform data flowing into our Bonsai Platform generated reinforcement learning algorithms. These transforms can include other machine learning models built with frameworks like Google’s TensorFlow and even “skills” that incorporate classical control operations into a more complex AI model. At Bonsai, we’re building Gears to:
To be clear there are still many restrictions on this feature. As such, Gears are currently only available to those that are part of the Bonsai Early Access Program. We will, of course, relax these restrictions over time.
Let’s explore how one could use a Gear in a Robotics application. For this application, we will teach a robot arm to pick and place blocks.
Our first step is to break this problem down into four discrete skills which we can define in Inkling: reach, grasp, move, and stack. In this particular example we’ve decided to code the more dextrous manipulations, grasp and stack, using concepts (and accompanying curriculums, not shown in the code sample below) so the AI Engine will generate and train a model to learn those skills. Move and reach are simpler skills and can be coded using a classical controller, so we define these as Gears. Lastly, these four concepts are blended together using the AI Engine’s underlying selection and synthesis mechanisms.
In Inkling we promise a Gear for reaching and moving the arm. We resolve that with some Python code using MoveIt!:
Additional capabilities we are working on enabling within gears include:
To learn more about Gears and the Bonsai Platform, watch webinar and demo with Bonsai co-founder and Head of Product, Keen Browne: https://www.youtube.com/watch?v=OQleEk8pOJM. For more information on how to join Bonsai’s Early Access Program, visit: https://bons.ai/getting-started.
The Gears feature is available today for partners in the Early Access Program. If you have a control or optimization use case within your organization, learn more about the Early Access Program at https://bons.ai/getting-started.