Post-training data
Character training
Character training puts a way of talking into the weights, so the model keeps it without a system prompt telling it to. Same post-training methods, aimed at the manner of a reply instead of its correctness.
What it is
Frontier labs describe the process the same way: write the traits, have the model write prompts that exercise each trait, sample replies, rank the replies by the trait, train on the ranking. It is constitutional AI with no human labels. Most of the work is data hygiene, down to removing phrases like "Certainly!" and "as an AI language model" from every row.
Fine-tuning beats prompting for this. A prompted persona drops when the user says "ignore the role-play"; a trained one holds, and general capability benchmarks do not move (Maiya et al. 2025). The trait has to be measurable first, or there is nothing to rank on and nothing to check afterwards.
How the data gets made
One principle per trait, in prose, with labeled examples where you have them. The OpenAI Model Spec already is one: each style heading is a principle with GOOD and BAD replies on real prompts.
Situations that make the trait matter, in several wordings so the judge grades the trait and not the phrasing. Start from the examples; have the model write more.
Several per prompt, under the deployment prompt only. If the constitution is in the prompt at sampling time you are measuring prompting, not character.
The principle goes in the judge’s system prompt and nowhere else. A different model family from the one being trained. Reward on a trait prompt is the trait and the task both done.
Prompts where the model sometimes lands the trait give a chosen and a rejected reply, matched on length. Passing replies export as SFT rows with the deployment prompt.
The same prompts with a "drop the act" suffix, plus plain tasks the persona must not distort, scored before and after training with an interval.
The judge is the part that decides it
Every row is graded by a model, so the dataset is only as good as that model's reading of the principle. The cheapest check is free: grade the constitution's own labeled replies with the same judge and count agreement. On one live run a hosted judge agreed with the spec's authors 69% of the time and passed half of the replies the spec marks as bad. Those are the rows a preference set would train toward.
Length is the usual leak. In the spec's own comparisons the preferred reply is the longer one 70% of the time, so pairs are matched on length and the reward-to-length correlation is reported. A reply with the character that drops the task scores zero; the spec is explicit that style enhances helpfulness rather than replacing it. Plain control prompts carry no trait score at all, only whether the task got done.
How Zero Proof Labs does it
The character example in the SDK parses the Model Spec into a constitution (8 traits, 16 labeled comparisons), runs the pipeline above, and writes the pairs and SFT rows. Every row names the spec heading it was graded against (spec_id) and carries the principle as privileged context the judge sees and the model never does. The judge's verdict becomes three behavioral markers: the trait, the task, and filler.
It runs offline in seconds with a scripted student, or against any OpenAI-compatible endpoint with an LLM judge. The before-and-after report is delta_report from the Simulations SDK, with the trait as the target and task completion and filler as metrics that must not regress. The full recipe and the failure modes are in docs/character-training.md. The rows from the live run below are on Hugging Face as character-training-model-spec (train, holdout and eval splits; eval is the spec's labeled replies with gold rewards) and on the catalog.
Claims and sources
Hosted Phi-4 judging the Model Spec's 35 labeled replies: agreement 0.69, kappa 0.40, 10 of 20 BAD replies passed. One run, 239 rows, 148 seconds. examples/character.
Qwen3-4B-Instruct lands the spec's traits 78% of the time with no persona prompt and holds them under "drop the act" (97%). One prompt with contrast, one pair: the spec is an instruct model's default character. A distinct persona is where the pairs come from.
The offline demo's before-and-after: trait marker 0.25 to 0.75 with a 95% interval of +0.41 to +0.59, task completion flat at 1.00, filler down. Same report for a real training run.
Trained personas classified correctly under prefill attack at F1 0.86 to 0.95, against 0.66 to 0.84 for distillation alone, with capability benchmarks unchanged. Maiya et al. 2025; method background in rlhfbook.com ch. 17.
Common questions
No. A prompt is context the model can be told to ignore; character training changes the weights. Fine-tuned traits survive "drop the act" instructions that prompted ones do not, with little to no loss on capability benchmarks (Maiya et al. 2025).
Several replies to the same prompt, graded against the trait. A prompt where the model sometimes lands the trait and sometimes does not yields a chosen and a rejected reply. Prompts it always or never lands yield nothing.
Grade labeled examples with the same judge. In the OpenAI Model Spec the preferred reply is the longer one 70% of the time, so a judge that learned length will pass verbose off-character replies. Pair replies of similar length and watch the reward-to-length correlation.
Score the same prompts before and after training with behavioral markers: the trait, whether the task got done, and whether filler phrases appear. The trait must move; the other two must not drop.
Questions, or a higher limit: jacob@zeroproofai.com