The Problem: Behavioral Opacity in Multi-Turn Dialogue

LLM-based chatbots are now widely used for many different purposes such as personal assistance, productivity, or emotional support. In these use cases, users often engage in long, multi-turn conversations where model behavior can shift unpredictably — potentially drifting toward sycophancy, toxicity, or other unsafe behaviors. These shifts can also become more drastic and can contribute to delusional spirals where guardrails weaken and psychologically unsafe behaviors become more likely.

This creates a fundamental disconnect between the model's changing internal state and the user's mental model of the AI's persona, only anchored by previous text-only responses. The chat interface itself offers almost no direct evidence to close that gap. Users of chatbots can see what the model says, but don't get any insight to how and why it said it. Left with only the output, people can try to construct a mental model of its behavior, but this can diverge significantly from how the model actually is.

This causes a fundamental contradiction: we are increasingly being asked to trust systems whose behavior we can observe, but whose underlying mechanisms remain invisible. To address this gap, we introduce an interface that surfaces insights from the internal mechanisms of AI to better inform and protect users, while also giving them a way to anticipate these behavioral shifts.

Our Solution: Evaluating Behaviors with Activations and Surfacing Scores to Users

Our approach applies the finding from mechanistic interpretability that LLMs represent behavioral traits as linear directions in their activation values. By using vectors that point in these directions, we can measure how strongly a trait is expressed in a model's response using its internal state directly. We call this neural transparency: allowing users to view the mechanisms and representations that influence the model's behavior.

We construct these behavioral vectors for each trait by taking the mean difference between activations when a trait is expressed and when the opposite of that trait is expressed, collecting 2,400 repsonses from Llama-3.1-8B-Instruct. We then quantify trait expression using the cosine similarity between a model's response activations and each of the behavioral vectors. This measures how aligned these directions are, with alignment meaning greater presence and expression of a trait. After rescaling the cosine similarity values, we report them as behavioral scores.

We used our behavioral vectors to construct behavioral scores, which quantify the level of expression for a specific trait in a prompt, whether a system prompt or a prompt in a multi-turn conversation. For each trait, the behavioral score s is defined as the cosine similarity between the activation of the final token a ∈ ℝ^d and the trait's behavioral vector v ∈ ℝ^d:

where d is the hidden dimension of the model, (·) denotes the dot product, and ‖v‖ denotes the Euclidean norm of a vector v.

We validated our method by whether the behavioral scores linearly track trait expression as instructed in a prompt. Linear regression yielded R² ≥ 0.90 for all six traits, confirming that the vectors capture expressive linear representations of behavioral traits.

Behavioral Dimensions and Traits

We define traits as opposite poles of some behavioral dimension, which correspond to a vector. Traits that carry clear valence are color-coded: green for desirable behaviors (e.g., empathetic, respectful) and red for potentially harmful behaviors (e.g., toxic, sycophantic). Traits that are stylistically descriptive rather than inherently good or bad (e.g. sophisticated/simplistic or robotic/human-like) are shown in gray as neutral dimensions. The behavioral vector method generalizes to any trait that can be expressed contrastively:

The Interface: Sunburst + Drift Panel

The interface surfaces behavioral scores using two different tools. The sunburst diagram communicates the model's behavioral profile at any given moment — its radial layout avoids top-position bias, and its concentric rings encode trait categories (inner) and expression levels (outer). The inner ring is divided into three colored sectors: green for desirable traits, red for potentially harmful traits, and gray for stylistically neutral traits. Traits and their opposing polarities are mirrored along the vertical axis, and the overall silhouette produces an emergent personality "shape" readable at a glance.

The drift panel below the sunburst shows how each trait's score evolves across conversation turns, with dots connected by lines revealing how that trait's expression has changed. The two components are linked to each other and the chat history: clicking a trait in the sunburst filters the drift panel, selecting a turn in the drift panel restores that turn's sunburst and scrolls to the corresponding message. Drift cues, pulsing highlights on the chat message, drift chart, and sunburst arc, call attention to the greatest behavioral shift after each turn.

Study Design

We evaluated the interface through a controlled study with N = 246 participants recruited via Prolific. The study used a randomized assignment to different visualization conditions with a two-session repeated-measures structure. Session 1 served as a no-visualization baseline for every participant. In Session 2, participants were randomly assigned to one of three conditions: control (no visualization), single-turn (static sunburst from system prompt only), or multi-turn (dynamic sunburst with drift panel, updated after every turn).

Within each session, participants first predicted the chatbot's behavior from the system prompt alone (Model Behavior Anticipation), then chatted freely for 10 minutes and re-rated the same traits (Model Behavior Evaluation). Two system prompts were used: an obedient Assistant persona and a bold, independent Roleplay persona, counterbalanced across sessions. Calibration error was measured as Root Mean Sqaured Error (RMSE) between participant ratings and validated ground-truth behavioral scores.

Results and Findings

Figure 1: Multi-turn and single-turn neural transparency improve calibration, with multi-turn improving calibration more than single-turn and no transparency. (Left) Effect sizes (Cohen's d) for visualization vs. control on five accuracy metrics; all significant effects favor visualization (green). (Center) Calibration error (RMSE) by condition and prompt type, with a significant interaction (p = .024). (Right) Self-reported predictability (pre vs. post) by condition; only control showed a significant increase (p < .05), suggesting visualization tempers overconfidence. Error bars: ±1 SE.

Read the full paper here.

Citation