Profile-Driven User Simulation via Reinforcement Learning with Hierarchical Rewards for Task-Oriented Dialogues

What it claims

ProfileSim is a user simulator for Chinese-market-style financial marketing outbound calls, built by Bairong Inc. (arXiv, October 2026). It is trained on 72,296 real voice dialogues. Simulated callers are conditioned on an explicit user profile and a business profile. A Markov transition over 9 intent groups, estimated from real call logs, moves the business profile from turn to turn.

How it works

The pipeline has three stages: data cleaning and profile extraction, SFT, then GRPO.

• Data. 242,438 raw outbound-call sessions were desensitized, transcription-corrected and quality-filtered down to 72,296 dialogues, averaging 7 turns each. User utterances were manually labelled with intents.
• User profile. An LLM first summarized objective features (gender, age range, occupation, income status, language style) and Big Five traits. These summaries formed 10,000 dialogue-profile pairs. Qwen3.5-9B was then fine-tuned with LoRA on those pairs, giving a dedicated profile-generation model.
• Business profile. Utterances were mapped to 73 fine-grained intents (e.g. "Loan process consultation"), grouped into 9 intent groups. Each intent was rewritten as a natural-language prompt ("Next, you may ask questions about ...").
• Intent transitions. A 9×9 group transition matrix is counted from consecutive user turns in real dialogues. A fixed stay probability of 0.1 keeps the current group. Otherwise the next group is drawn from the empirical transitions, and a specific intent is then picked uniformly within that group.
• SFT. Each training sample is a tuple of product info, dialogue history, user profile, business profile and target utterance, trained with cross-entropy.
• GRPO. G full dialogues are rolled out per context, with intent transitions resampled each turn. The customer-service agent is frozen and acts as the environment. Rewards apply at both dialogue level and utterance level.

flowchart LR
  P[User profile + product info] --> S[User simulator]
  T{Stay p=0.1?} -->|yes| G1[Same intent group]
  T -->|no| G2[Sample from 9x9 matrix]
  G1 --> B[Pick intent uniformly]
  G2 --> B
  B --> S
  S --> A[Frozen agent: SOP stage + RAG]
  A --> H[Dialogue history]
  H --> S
  H --> T

The agent picks responses from the current SOP stage and retrieved knowledge, and it predicts the next stage.

Evidence

The fetched text ends partway through the GRPO section. It contains no results tables, no reward definitions, no named baselines and no metric values. The comparative claims (highest colloquial naturalness, strongest profile adherence, lowest agent score on long-tail intents, shorter utterances at comparable compliance) appear only in the abstract. The introduction adds that the baselines are "strong zero-shot LLM" generators and that an ablation shows each component contributes.

Limits

• The training corpus is proprietary outbound-call data from one company. The page mentions no release of data, weights or code, and gives no licence.
• The base model for the simulator itself is not named in the fetched text. Only the profile extractor (Qwen3.5-9B with LoRA) is named.
• Choosing an intent uniformly within a group ignores how often each fine-grained intent occurs in real calls. Only group-level transitions are learned.
• The stay probability of 0.1 is a fixed hyperparameter, and no justification for the value appears in the fetched text.
• The results, reward formulas and evaluation protocol fall in the part of the paper cut off from the fetched text. They need checking in the full paper.