Public consultation synthesizer

A public consultation synthesizer earns its keep when the consultation lead can open a theme brief, see the response IDs that sit in that cluster, and know that every quoted line is in the file that was loaded. The brief ranks themes by how the clustering run grouped the text. It does not close the consultation, and it does not write the ministerial note.

Quality here is narrow. A theme that has supporting quotes lists those quotes with IDs. A theme that has no usable quotes keeps the quote field empty. A segment split that the file cannot support stays blank. The model does not invent a support percent to make the page look finished.

Collection and hosting sit elsewhere. Engagement suites from Granicus, CitizenLab, Delib, and Microsoft already hold submissions, identity rules, and closing dates. You export. Then you synthesise. Treating those vendors as interchangeable "consultation products" misses the cut: they are the system of record for what was said; the synthesizer is a reading aid on an extract.

If you later need a public explainer of the same dataset, that is a different job. A public-facing data summarizer is for what you publish; this page is for what you can still defend in clearance.

Load the live response file, cluster so every quote cites an ID in that file, leave blanks blank, and write the ministerial note yourself. Use the extract you actually have, not a sample pack from last year. Confirm the export includes a stable ID per row, the free-text fields you intend to cluster, and whatever stakeholder flags exist in the form (resident, statutory consultee, parish, business, campaign template, and so on). If the file has no IDs, add them before clustering. A brief that cites "several residents" cannot be audited.

Run clustering on those text fields only. For each cluster, require the tool to emit: a theme label, the member response IDs, and quotes drawn exclusively from those members. If a cluster has members but no quote that survives your redaction rules (personal data, threats, identifiability), leave the quote slot empty. Do not backfill from another cluster. Do not paraphrase a missing quote into existence.

Quantify support and opposition only where the file gives you a basis: an explicit agree/disagree field, a closed question mapped to the same ID, or a coding rule you wrote down before the run. If the extract is free text alone, the brief can say how many IDs landed in a theme. It cannot mint a majority. Empty stays empty.

Then you write the note. The synthesizer's last honest line is the ranked list of themes with cites. Tone, weight, "what this means for the Secretary," and what you recommend are human work. If a cluster is thin, say so in the note. Do not ask the model to thicken it.

Stakeholder segment is how you read a cluster, not a substitute for the closing report. A statutory consultee and a campaign copy-paste can sit in the same theme and mean different things for risk. The brief should keep IDs grouped by the segment flags that exist in the file. If a row has no segment, it does not get one invented from the prose.

Opposition in a cluster is not "the public." It is the IDs in that cluster that argue against the proposal, optionally split by segment when the export supports it. Support is the same discipline in the other direction. Ranking themes by cluster size is a reading order, not a finding that the largest theme has won.

Where the live issue is whether the proposal even sits inside current duties, keep the synthesizer in its lane. A regulatory gap scanner looks at the instrument and the duty set. This tool looks at what people wrote. Mixing the two in one generated paragraph is how a briefing sentence becomes untrue.

Suppose you have exported comments on a proposed density change along a corridor. You load that file, not a cleaned "illustrative" spreadsheet. Clustering puts a set of IDs under a label such as overshadowing and garden loss, another under bus capacity and school places, another under support for more homes near the station. One cluster is a handful of near-duplicate campaign lines. Another has only a few IDs and no sentence you are willing to quote after redaction. That quote field stays empty.

You do not write "most respondents oppose the change." You write, in the brief, which IDs sit in which theme, and which quotes (if any) you are using. In the ministerial note you still have to say what you make of statutory consultees versus residents versus the campaign block, whether the consultation is still open, and what you recommend next. The Thursday-evening run is there so you are not hunting IDs by hand at midnight. It is not there so you can skip the note.

If a sentence appears in the draft brief that you cannot find by searching the file for that ID, you delete the sentence. That is the whole quality bar.

Inventing majority support is the first failure. A ranked cluster is not a poll. Theme size can reflect a campaign, a badly designed question, or a loud segment. If you need a share, you compute it from coded fields or from a count you can show. If you cannot, you leave the percent off the page. Putting a round number in the note because the brief "looks empty" is a clearance problem, not a formatting problem.

Treating the cluster run as consultation closed is the second. Clustering an extract does not freeze the inbox, does not satisfy the duty to consider late statutory responses if your process still accepts them, and does not replace the audit trail of what was published and when. Engagement platforms (again, the Granicus, CitizenLab, Delib, and Microsoft class) remain the place where open/closed is a fact. Your brief should date the extract. If responses arrived after that timestamp, they are not in the clusters.

Quoting a response that is not in the file is the third. Models complete. They will offer a vivid resident story, a tidy councillor line, or a "typical" objection that matches the theme label. If that text is not attached to an ID in the load, it is not a consultation response. It is invention. The same rule applies to merging quotes across IDs into a composite voice. Cite or cut.

A secondary slip sits next to these three: letting the theme label drift into policy. "Residents reject the approach" is a conclusion. "Theme: garden loss and overshadowing; IDs listed" is a brief. The note can argue. The synthesizer should not.

Once the briefs exist, the lead still sequences the work. You read against the original questions. You check whether a small cluster is a legal landmine even if it is not the largest. You decide what, if anything, changes in the proposal. Modelling what a change would do is not clustering. That is a policy impact simulation when you need effects, or a scenario comparison matrix when you need options side by side. Those pages assume you already know what was said. This page assumes you must not fake what was said.

Keep the ministerial note in a document you own. Paste IDs and quotes from the brief. Write the judgement in sentences you can stand behind in a meeting. If a cluster is empty of quotes, the note can still mention the theme and the IDs; it should not dress the gap with generated colour.

That is the practitioner loop: load the response file, cluster with cites, leave blanks blank, write the note yourself. The synthesizer is doing its job when a sceptical colleague can pick an ID off the brief and find the same words in the extract.