Research: Backpropagation through camera

We optimized a computer vision neural network by backpropagating through a camera.

Posted by 317070 on January 16, 2017

I finally implemented the backpropagation through a camera.

We have optimized a controller for the pendulum-cart system, by backpropagating through the physics, vision and controller. By treating the whole system as a differentiable function, we could update the controller parameters, and have it solve this problem. In only 2420 update steps (or trials in the simulator), it managed to completely learn the vision and the control task, and solve the pendulum-cart system. I hope the paper for ICLR will be accepted soon, such that I can publish the code and see what other people will do with it.

Next step: evaluating this method on the real setup!