The acrobatic dance videos are flashy, but what are the actual technical breakthroughs? What is happening to the Korean robotics industry? — The robotics company that has a knack for viral technology videos showcasing little things robots can do, parkour, bullying robots, and more. A central tenet of Boston Dynamics is the idea of athletic intelligence — movement patterns that are robust, flexible, and maybe even human. These videos and technologies have…

Multi-agent scenarios make reward maximization a risk. Discussing when, rather than if, we should believe in the Reward Hypothesis. — Dopamine (below) is central to human experience. It is known to be involved in feeling a current pleasure — that is something you are experiencing and enjoying. Dopamine also plays a crucial role in predicting a future please — that is done in planning by releasing the same feeling of…

I make the case why people iteratively training any model should learn some core concerns of reinforcement learning. — Thought exercise Consider these examples of an ML problem that has across-time effects on its models. This time-dependent function of reinforcement learning is where most of the “core pieces”(below) will emerge. Consider a system that is trying to reduce the churn (ended subscriptions) of a paid online membership. Such a company could…

Lessons learned from a PhD in a young field. — I saw an example of this debugging lessons for model-free RL and felt fairly obliged to repeat it for MBRL. Ultimately MBRL is so much younger and less pervasive, so if I want it to keep growing I need to invest that time in all of you. For an illustrative…

How behaving as a digital platform rather than a manufacturer will be the sweet spot for the next generation of robotics companies — Data and deep learning have been changing the world for the last decade. Most robotics companies have not tapped into the datavolution and the mainstream vision for what robotics can be is still far from realized. …

Where a hot new research sub-field is going in RL. — As many of you know, I am doing my PhD centered around model-based reinforcement learning (MBRL). This post is not talking about the technical details and recent work, but rather why I am bullish on it for the future. Beyond the prospects of how well it can perform (it’s much…

Lessons about AI from lessons about our mind. Focused on the nature of free will. — I have been trying to train my mind through meditation for about two years now. It is remarkable to me the frequency and ease by which you can notice the actual modus operandi of the brain differing from the perceived modus operandi. …

I tried my best to absorb a lot of content at NeurIPs 2020, and it was just as overwhelming as ever. Everyone makes decisions of what content they want to focus on, and it is always an exploration (learn new things) versus exploitation (further mastering material in your area of…

A small step into the research community’s most opaque framework. — Reinforcement learning (RL) — the framework of interacting with an environment to learn a policy to act and achieve some objective — is on the up in many domains. Some domains make sense, some will never work, and most are bound to fall somewhere in the middle. RL is so…