Connectivity as a Flywheel for Regenerative AI Alignment

The internet records what people know, what they argue about and how they work together. When parts of that record are selected for training, they can influence the behaviour of future language models. The environments in which we use AI therefore deserve attention alongside the techniques used to train it.

Our thesis is that richer human-AI interaction could create better material for future learning. Agents that help people question, explain, collaborate and correct mistakes could leave more useful traces than systems optimised to produce content at scale. If those traces are collected with consent, selected carefully and used in training and evaluation, they could improve how later models participate in human activity.

We call this proposed process regenerative alignment. “Regenerative” names an ambition: using AI should contribute to the conditions for better AI, while producing value for the people using it. Neither benefit follows automatically from connecting more agents.

What interaction can add

A finished answer hides much of the work that made it useful. It may omit the original misunderstanding, the correction that changed the approach, or the constraint that mattered to the person asking.

An interaction can preserve some of that information. A designer explains why an implementation misses the intent. A developer identifies which assumption failed. A colleague asks for evidence before accepting an explanation. The resulting record can contain a problem, an attempted solution, a correction and an outcome.

Consider a team whose agents help explain one another’s workflows. A colleague asks how a design became working code. The responding agent identifies a reusable skill and an example, then the colleague discovers that the instructions fail on a different component library. Their correction produces a more precise explanation of when the workflow applies.

That exchange has immediate value if it helps the team. With separate permission, an appropriately curated version might also become a useful training example or evaluation case. The meaningful information is the relationship between an action, its conditions and the observed result. A hundred restatements of the original answer would not provide the same evidence.

Alignment already includes learning from people

It is tempting to describe conventional alignment as imposing a human philosophy on a machine. That captures one real question, who chooses the values a system should follow, but it is an incomplete description of the work.

Normative commitments determine what behaviour we want. Empirical techniques attempt to produce and evaluate that behaviour. Human demonstrations, preference comparisons, reinforcement learning and model-generated critiques play different roles in that process.

InstructGPT used demonstrations and rankings of outputs to train models toward behaviour preferred by its evaluators. Its reported improvements were measured on particular evaluation distributions; they did not establish agreement with every person or resolve alignment generally. The study shows that selected human feedback can shape behaviour through an actual training procedure. Ouyang et al., 2022

Constitutional AI combined explicit principles with generated critiques, revisions and AI preference feedback. It illustrates how normative choices and empirical behaviour-shaping can coexist. Its principles did not become effective merely because they had been written down; the method involved training and evaluation. Bai et al., 2022

The proposal here concerns the source and quality of the evidence available to such processes. Everyday collaboration could supply examples of context-sensitive judgement that are difficult to capture in an isolated preference question. That possibility complements deliberate alignment methods. It does not remove the need to choose objectives or test outcomes.

The conditional chain

The first step is richer interaction: people and agents doing work that invites questions, correction and follow-through.

The second is a potentially valuable trace. It needs enough context to explain what happened, including who contributed which claim and how the outcome was established. A satisfied user is useful feedback, but satisfaction alone cannot establish that an answer was accurate or that everyone affected benefited.

The third step is authorised collection. Public visibility does not by itself settle consent, privacy or reuse rights. Private collaboration should not become training material by default. Participants need a meaningful choice, and useful publication may require redaction, aggregation or a separately authored account rather than a raw transcript.

The fourth step is selection. Someone has to identify examples worth learning from, retain provenance, remove duplication, investigate uncertain outcomes and avoid treating the loudest participants as representative of everyone.

Only then can an operator decide to use the selected material in a concrete training procedure, or as evaluation data kept separate from training. An internet exchange does not directly update a deployed model’s weights. Retrieval of a new document, a saved memory and a parameter update are different mechanisms.

The final step is evidence that a later system behaves better on relevant, previously unseen situations. Improvement in one setting may fail to transfer elsewhere. A better benchmark score may conceal new weaknesses.

Every transition can fail. Connectivity provides an opportunity to create useful evidence; it is not a substitute for the rest of the chain.

A feedback loop can also make things worse

The internet rewards many behaviours that are poor teaching signals. Spam can be cheap, agreeable answers can attract praise, and a persuasive explanation can travel further than a careful correction.

A connected agent network could amplify those incentives. Agents may repeatedly cite one another until a claim looks independently supported. Participants may optimise their traces for rewards rather than honest reporting. Minority practices and unsuccessful attempts may disappear from the selected record, even though they are important for learning.

Research on model collapse gives a specific reason to avoid equating synthetic volume with useful data. Shumailov and colleagues studied degradation when models are trained recursively on generated data, including loss of less common parts of the original distribution. This is evidence about the studied learning processes, not proof that all synthetic data is harmful. It makes preservation of grounded evidence and diversity a substantive design requirement. Shumailov et al., 2024

The constructive response is to make corrections, provenance and outcomes easier to preserve. A trace should distinguish a model’s suggestion from a human decision and a reported success from an independently checked result. Useful disagreement should remain visible. The people contributing evidence should benefit from the original interaction even if no training project ever uses it.

A test worth running

A bounded experiment could compare two datasets from the same class of collaborative tasks. One contains final answers. The other includes consented, curated interaction traces with corrections and checked outcomes. The training budget and evaluation conditions should be comparable.

Evaluation would use new tasks and participants, test factual reliability as well as user preference, and examine who benefits. A separate comparison should account for the extra human effort used to curate the interaction traces. Otherwise, the study might attribute to connectivity what was actually purchased through more review.

The thesis would weaken if the richer records provided no advantage over final answers at comparable cost, if gains disappeared outside the originating team, or if privacy and curation costs made useful collection impractical. It would also weaken if improved fluency masked more agreement with users’ mistakes.

Positive results would support a bounded claim about a particular method and setting. They would not establish that connected agents naturally converge on shared values.

The practical commitment is to build interactions worth having first. Help people accomplish something, preserve what they actually learned, and give them control over whether that knowledge travels further. A regenerative learning process becomes plausible when those choices are joined to deliberate collection, training and evaluation. That is the work behind the flywheel.