@hlfshell
@programming.devhttps://hlfshell.ai/posts/llms-and-robotics-papers-2023/
tldr I write about some of the more interesting works that shaped my understanding of applying LLMs for AI agents and robotic applications. Introduction What is this LLMs as a fad - a caveat Are LLMs actually going to be useful for robotics? Instruct based Benchmarking LLM basics n-shot and reasoning via prompting Chain of Thoughts Self consistency ReAct Tree of Thoughts Automating our automatons Lies, safeguards, and Waluigi Building LLM Agents Finetuning With Tool APIs MRKL Toolformer TALM MM-React Generative Agents: Interactive Simulacra of Human Behavior Socratic Models LLMs and Robotics Code generation and multi agents SayCan Inner Monologues Code As Policies ProgPrompt Statler LM-Nav RT2 Final Thoughts Introduction What is this I’m kicking off a project that is centered around the idea of applying large language models (LLMs) as a context engine; understanding contextual information about the environment and deriving additional steps from human intent to shape its actions.
https://hlfshell.ai/posts/llms-and-robotics-papers-2023/
tldr I write about some of the more interesting works that shaped my understanding of applying LLMs for AI agents and robotic applications. Introduction What is this LLMs as a fad - a caveat Are LLMs actually going to be useful for robotics? Instruct based Benchmarking LLM basics n-shot and reasoning via prompting Chain of Thoughts Self consistency ReAct Tree of Thoughts Automating our automatons Lies, safeguards, and Waluigi Building LLM Agents Finetuning With Tool APIs MRKL Toolformer TALM MM-React Generative Agents: Interactive Simulacra of Human Behavior Socratic Models LLMs and Robotics Code generation and multi agents SayCan Inner Monologues Code As Policies ProgPrompt Statler LM-Nav RT2 Final Thoughts Introduction What is this I’m kicking off a project that is centered around the idea of applying large language models (LLMs) as a context engine; understanding contextual information about the environment and deriving additional steps from human intent to shape its actions.
https://www.nature.com/articles/s41586-023-06419-4
An autonomous system is described that combines deep reinforcement learning with onboard sensors collecting data from the physical world, enabling it to fly faster than human world champion drone pilots around a race track.
https://robopen.github.io/
Towards Sample Efficient Robot Manipulation with Semantic Augmentations and Action Chunking
https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/
We use ChatGPT to allow a user to intuitively control multiple robots such as drones, robot arms, home assistant robots using only language.
https://arxiv.org/abs/2307.15236
This is the routine thread where we discuss what you'll be working on this week! A cool robot? A computer vision project? Something cool in reinforcement learning? 3d printing a drive train? Let us know!
Maybe instead you're studying something, or reading a paper that just came out? Post about it!
It’s also okay to say “nothing” too - it’s great for your mental health to take a break!
Looking for help? Ask a question! See someone working on something cool? Ask them about it! No project is too small or too "newbish"!
Playing multiple co-op campaigns, and every druid across multiple campaigns experience, randomly, unexpected deaths when being knocked out of wild shape. Not death saving throws either - straight into full blown d-e-a-d.
Per the rules, it should go from wild shape -> spill over damage being removed from the remaining HP of the druid. Reading the combat logs we see that while enough damage is done to knock the druid out of their wild shape, the remainder is not nearly enough to put them into death saving throws, let alone outright kill them.
Is anyone else encountering this issue?