• BlenderBot 2.0: FAIR chatbot with long-term memory

    FAIR has introduced the second version of BlenderBot , an open-source chatbot that combines several conversational skills, including personification, empathy and factual knowledge. BlenderBot 2.0 has a long-term memory and the ability to communicate on topics that were not used during training.

    The chatbot stores in long-term memory all the data received during previous conversations with the user, and can use them in subsequent conversations in a few days, weeks, and even months. For example, if you discussed Tom Brady’s chatbot a few weeks ago, he could potentially touch on the topic of the NFL in future conversations, since he knows that this is a relevant topic for you.

    Also, during the dialog, the model generates search queries on the Internet, analyzes the results and includes them in the answers to users ‘ questions. Thus, the model is always up-to-date, since it does not rely on data used in training, which could be outdated.

    To solve these problems, FAIR has developed the Retrieval Augmented Generation algorithm. During the dialog, the model, which combines the information search component with the seq2seq generator, searches for relevant information both in its long-term memory and in documents that it finds using an Internet search. To do this, the traditional encoder-decoder architecture has been supplemented with a module that generates search queries taking into account the conversational context.

    The architecture of BlenderBot 2.0.

    To train neural networks, FAIR has developed two datasets:

    • chats supplemented with new information from Internet search queries (Wizard of the Internet dataset);
    • multi-session, long chats with users with references to data from previous correspondence sessions (Multi-Session Chat dataset).

    The new chatbot surpasses BlenderBot 1.0: the assessment of engagement (the ability to continue communicating from the place where the previous communication session ended) increased by 17%, and the assessment of taking into account the context of previous communication sessions increased by 55%. At the same time, the reliability of the actual data provided by BlenderBot 2.0 increased by 12%. To date, BlenderBot 2.0 surpasses all existing chatbot models.

    Notify of
    Inline Feedbacks
    View all comments