Highlights of the Week
Welcome! This week we delve into the use of tools in applications of Large Language Models (LLMs). But that happens only after looking at a keynote by Yann LeCun, in which he critically analyzes the current state of LLMs and suggests additional model types necessary for real progress.
While ‘tools’ and ‘agents’ are often referenced together, it’s crucial to distinguish between them. When discussing LLMs, ‘tools’ often refer to software, applications, and resources that aid in creating, managing, deploying, or enhancing these models. This includes programming languages, libraries, platforms, APIs, among others. ‘Agents’, on the other hand, are the entities that interact with the models.
Next, we explore “Flows”, a groundbreaking AI framework from researchers at the École Polytechnique Fédérale de Lausanne (EPFL) and Paris Sciences et Lettres University (PSL University). This approach promises advances in AI system functionality by simplifying the composition of various models and tool linkages.
We then investigate how LLMs can utilize self-supervised learning to enhance their functionality, introducing the Toolformer project and its potential synergy with tools like the previously reviewed Gorilla project.
Finally, we discuss democratizing Reinforcement Learning with Human Feedback and the role of DeepSpeed-Chat in making it more accessible and affordable. Tools await us!
Towards Machines that can Learn, Reason , and Plan
To start us off, I want to draw your attention to a Keynote given by Yann LeCun at the “Impact of chatGPT” talks on July 21, 2023. The address, entitled “Towards Machines that can Learn, Reason, and Plan”, is one one of the best I’ve listened to. He notes the shortcomings of the spate of LLMs now getting big attention, and points out the likely sources of data and AI technologies which, when used in conjunction with the LLMs, could possibly address the problems. The address is available in this video recording . The slides are available as a .pdf here: Objective-Driven AI
According to LeCun, Auto-Regressive Generative Models suck. (His words. Not mine). He goes on to state that what we need are the technologies and systems that address three challenges:
- Learning about and learning to use representations and predictive models of the world.
- Learning to reason.
- Learning to plan complex actions which satisfy objectives.
In addition to the critique of LLMs, he touches on the importance of open source AI, hybrid systems for reasoning and planning. It’s well worth your time.
Reasoning and Collaborating AI
Right after listening to Yann LeCun’s address, I came upon Flows: Building Blocks of Reasoning and Collaborating AI. It was almost as if, in response to one of the key messages of the address, this project from École Polytechnique Fédérale de Lausanne and Paris Sciences et Lettres University appeared magically.
Imagine an AI system that’s like a Lego set, with parts that can be assembled, disassembled, and reassembled in various ways to create different structures. This is the concept behind “Flows,” a new AI framework presented in this paper. Flows are like individual building blocks of computation that can communicate with each other. These blocks can be combined in numerous ways to model complex interactions among multiple AI systems and humans. The beauty of Flows is that they reduce complexity by breaking down big tasks into smaller, manageable parts.
As a proof of concept, the researchers used Flows to improve the performance of AI in competitive coding, a task that many advanced AI models find challenging. The result was a significant improvement in the AI’s proficiency. To make this new framework accessible for further research, the authors have introduced the aiFlows library, a collection of Flows that researchers can use and build upon.
How an LLM Might Use Self-supervised Learning About How to Use Tools
As you might recall, one of last week’s recommended readings was Gorilla: Large Language Model Connected with Massive APIs, the open source project which identified the best APIs to be used by LLMs for specific purposes, and guidance about how they might address them. But in this configuration, it’s not clear that an LLM will already have the skills to follow this guidance. That’s where projects and offerings like Toolformer, a language model that can teach itself to use tools, provide the potential solution.
The authors note that large language models (LLMs) have become incredibly popular mainly because of their outstanding performance on a range of natural language processing tasks. One of their most significant differentiating factors is their impressive ability to solve new tasks from just a few examples or text prompts. This makes it all the more puzzling that these ostensibly all-knowing LLMs frequently have difficulties with fundamental functions like executing arithmetic operations or with being able to access up-to-date information. At the same time, much simpler and smaller models perform remarkably well in this space. The work of researchers from Meta AI Research and Universitat Pompeu Fabra reports that Toolformer not only decides which APIs to call, when to call them and what arguments to pass, but it comes by this knowledge and skill by ‘self-supervised learning, requiring nothing more than a handful of demonstrations for each API.’ It would seem that the combination of Gorilla and Toolformer might well be a way forward.
ALL Model Learning May Not Be Self-supervised.
Reinforcement Learning with Human Feedback (RLHF) is a method where an artificial intelligence system learns to improve its actions or decisions based on feedback it receives from humans. DeepSpeed-Chat is a novel system designed to make RLHF training for powerful AI models more readily and economically available. With easy-to-use training, a scalable pipeline replicating InstructGPT, and a robust system that optimizes training and inference, DeepSpeed-Chat claims to offer efficient, cost-effective training for models with billions of parameters. Gain broader access to advanced RLHF training with DeepSpeed-Chat, fostering innovation in AI, even for data scientists with limited resources.
Thanks for reading. FYI … I do at times use GPT-3.5 to summarize articles. I do so less to have someone/something else do the writing. It’s more to check myself and determine whether I’ve identified the important points. I hope that it improves the quality of these posts. – Rich
