Psy-Copilot: Visual Chain of Thought for Counseling

Psy-Copilot:
Visual Chain Of Thought for Counseling

Anonymous Team ^*^♠ ,

^♠Anonymous University

^*Equal Contribution ^†Corresponding Author

Paper

Psy-COT Graph Code

Dataset

Demo

"True collaboration is not about dividing work between machines and people but about bringing the strengths of both together to solve problems and achieve more than either could alone."

—— Garry Kasparov

Abstract

The datasets of mental support are particularly scarce, especially for Chinese multi-turn dialogue. The Psy-Insight dataset alleviates the scarcity of multi-turn Chinese and English counseling data. The Psy-Insight includes bilingual labels for five types of tasks in emotional support. However, we only conducted experiments in Chinese and English for dialogue fine-tuning (Task 3) and RAG(Task 4) tasks.

Overview

Figure 2: Overview of indexes in Psy-COT and retrieval progress in Psy-Copilot. Psy-COT has two vector indexes for dialog and COT reasoning content respectively. There is a two-stage retrieval argument generation in Psy-Copilot.

This figure presents a screenshot of Psy-Copilot.

Psy-Copilot is a helper tool that uses artificial intelligence to work alongside therapists in giving psychological support. It's made to make sure that people can trust and understand how AI works in this setting. Psy-Copilot acts like a chatbot (left side), offering clear and dependable information (right side) that therapists can follow, such as possible replies, past conversations, and strategies for counseling.

Psy-Copilot has a user-friendly page that shows how the AI comes up with its answers, making it easy to see the steps it takes. Since Psy-Copilot is open-source, it lets people work together and share ideas. It's made to create a good working relationship between AI, therapists, and those seeking help, with the goal of improving mental health.

Datasets

The dataset for Psy-Copilot consists of 941 psycho-counseling sessions collected from psychological blogs. Each session contains dialogue and corresponding explanatory text. These sessions are used to construct the Psy-COT graph, which maps out event connections and strategic transitions in counseling dialogues.

Evaluation Metric

The evaluation metrics for Psy-Copilot focus on four aspects of multi-turn dialogues: Fluency (Flu.), Helpfulness (Hel.), Naturalness (Nat.), and Comforting Effectiveness (Com.). These metrics are used to assess the performance of Psy-Copilot against baseline models. The human evaluation scores are presented in a table, showing the performance of different models across the four metrics. Psy-Copilot outperforms the baseline models in all four metrics, indicating its effectiveness in generating psychological counseling responses.

This figure illustrates the objective evaluation results of Psy-Copilot generated result. Psy-Copilot-Dialog scoring 8.5 for Fluency, 7.4 for Helpfulness, 7.2 for Naturalness, and 8.2 for Comforting Effectiveness. The Psy-Copilot-COT variant showed even higher scores, achieving 8.6 for Fluency and 7.5 for Helpfulness, with a slight drop to 7.0 for Naturalness but a higher 7.9 for Comforting Effectiveness. These results indicate that Psy-Copilot not only meets but exceeds the performance of existing models, showcasing its potential to enhance the synergy between AI and human therapists in providing psychological support.

The above displays the overview and detailed setting of the Psy-copilot, as well as the dataset and evaluation metrics. You can click the buttons above to view the respective contents.

Psy-COT Graph

A small sub-graph of Psy-COT graph. The Psy-COT maps events and strategies to dialogue units, preserving causal and temporal relationships.

A small sub-graph of Psy-COT graph in Neo4j. Our graph and dataset are open-sourced on Github. You can try and interact with Psy-COT by buttons on homepage.

Previous Works on Knowledge Graph (KG).

Psy-COT has similarities to commonsense graphs and knowledge graphs. Figure 1 and2 shows how Psy-COT differs from previous studies. Unlike knowledge graphs, which focus on entities, Psy-COT emphasizes counseling descriptions and logic relationship ("Disappointed" vs Expand Client’s Client’s relationship).

Multi-Level Indexing for Retrieval

Psy-COT features the construction of two specialized indexing structures, one for COT (Chain of Thought) nodes and another for dialogue nodes. This design allows for more precise information retrieval, enhancing the efficiency and accuracy of finding relevant strategies and dialogues in psychological counseling conversations. By distinguishing between the content of the chain of thought and dialogue content, Psy-COT enables more accurate vector retrieval, thereby boosting the performance of LLMs.

Visualizing the Thought Process

Psy-COT presents a graphical representation of the thought process in psychological counseling dialogues, allowing therapists to intuitively understand the reasoning behind AI models. It displays semi-structured counseling conversations alongside step-by-step annotations that capture the reasoning and insights of therapists, enhancing the transparency of AI decision-making. Unlike traditional knowledge graphs, Psy-COT emphasizes the logical chain of causality in events and the temporal evolution of strategies in counseling, rather than just the inclusion relationships between entities.

Take away

Open open-sourced Interactive Platform

Combined with Psy-Copilot, Psy-COT has built an interactive platform that offers a user-friendly interface for therapists to engage with the AI system. During each dialogue round, the platform dynamically retrieves and displays relevant sub-graphs, making it easier for therapists to understand the logic chain behind AI-generated responses. This interactive experience not only strengthens therapists' trust in the AI system but also fosters collaboration between AI and human therapists, advancing mental health development.

1. The effectiveness of Psy-Copilot's two-stage retrieval argument generation process is evident in the high scores, suggesting that the combination of dialogue and reasoning content retrieval is a robust approach for generating counseling responses.
2. The Psy-Copilot's can allow for broader AI-Therapist collaboration in the field of emotional support conversation.

BibTeX

@article{Psy-Insight, author = {Anomyous until 2023-11-15}, title = {Psy-Copilot: Visual Chain Of Thought for Counseling}, journal = {AAAI}, volume = {}, year = {2024}, url = {}, doi = {} }