State of the art in LLMs + Robotics 2023

tldr I write about some of the more interesting works that shaped my understanding of applying LLMs for AI agents and robotic applications. Introduction What is this LLMs as a fad - a caveat Are LLMs actually going to be useful for robotics? Instruct based Benchmarking LLM basics n-shot and reasoning via prompting Chain of Thoughts Self consistency ReAct Tree of Thoughts Automating our automatons Lies, safeguards, and Waluigi Building LLM Agents Finetuning With Tool APIs MRKL Toolformer TALM MM-React Generative Agents: Interactive Simulacra of Human Behavior Socratic Models LLMs and Robotics Code generation and multi agents SayCan Inner Monologues Code As Policies ProgPrompt Statler LM-Nav RT2 Final Thoughts Introduction What is this I’m kicking off a project that is centered around the idea of applying large language models (LLMs) as a context engine; understanding contextual information about the environment and deriving additional steps from human intent to shape its actions.