Post-training data
Post-training data for AI agents
Post-training data is graded examples of the work a model will actually do, used after pretraining to set its behavior rather than its knowledge.
Four stages, four shapes of data
Post-training is not one step. Each stage reads a different shape of data, and a set built for one stage usually does not train the next.
A large corpus shaped like the target domain, so the model meets the tools and formats before it is taught how to behave in them. Defined on the glossary.
One good trajectory per situation, which the model copies. What a usable SFT row is made of.
Several graded runs per prompt, failures included, so the update has contrast to read. RL datasets for agents.
Replies graded against a written principle, then paired and matched on length. How the data gets made.
The grade is what makes it post-training data
A transcript is not training data until something scores it. An ungraded run teaches a model its own mistakes, because nothing in the row says which turns were right.
The score comes from a judge or a verifier, and it has to be checked before it is trusted. A judge that prefers long replies trains a verbose model. LLM judge and post-training are defined on the glossary; the trust checks measure the judge against labels a person wrote.
How Zero Proof Labs does it
The engine reads an agent's tools and system prompt and builds a world from them. It writes the users, plays the agent against scheduled tool faults, and grades every trajectory. You can watch one conversation become four objects, or generate rows from a spec with the Simulations SDK.
Rows come out as JSONL with four objects: Task, Rollout, Judgment and Marker. One row serves SFT, RL, evals and a mid-training corpus without reshaping. Datasets covers the shape, and zps.train runs SFT, GRPO or DPO on it.
Behavioral markers score production traces sent over OpenTelemetry, so one metric reads before training and after it. Published sets are on Hugging Face.
Claims and sources
A 15 dollar synthetic SFT set with no human-written row, on tau2-bench telecom. The same count of expert-written rows reached 17.5%. The fine-tuning post.
Gradient support that trains under a group relative method. Under 0.05 the set is flat. Read the profile.
What a two-minute simulation run produces, graded, depending on mode. Measured.
Macro intent accuracy for zeroproof-ecommerce-1b, at 18 cents per million output tokens. Intent model.
Common questions
Graded examples of the work a model will do, used after pretraining to set its behavior. It covers mid-training, supervised fine-tuning, reinforcement learning and character training.
Pretraining data is raw text at enormous scale, and it is not graded. Post-training data is small, scored, and shaped like the task the model will be asked to do.
Fewer rows than most people expect, when every row is graded. Contrast matters more than count: a prompt group where every run passes moves no weights.
Yes. A 15 dollar synthetic set with no human-written row took Llama-3.1-8B from 5.0% to 30.0% on tau2-bench telecom, against 17.5% for the same count of expert-written rows.
Questions, or a higher limit: jacob@zeroproofai.com