Continuous source monitor

Research leads turn on a source monitor when a question stays open for months and the record underneath it keeps moving. The output is a small set of new candidate rows in the intake queue. Each row carries the source it came from, the query that caught it, the date it was retrieved, and a stable identifier. A digest email is not the output. A folder of alerts nobody has opened is not the output either.

Expected yield is low and the maintenance is real. Most caught items get screened out. A watch running across a long review may push nothing that changes a conclusion, and that is still a correct result. The reason to run one is narrow: a late arrival on this question would change the recommendation, and finding it three months after publication would be too late.

Set the bar there, or do not build it. Feeds break quietly, sites change their markup, and search operators drift. Someone has to own that maintenance by name. A monitor left running without an owner produces confident silence, which is worse than no monitor, because the team starts believing nothing new has appeared.

The first search pass is a different job. Semantic search across a chosen corpus builds the initial candidate pool. The monitor only watches what happens after that pool closes.

Write the watch definition as a document, not as settings buried inside a tool. It names every source with a URL or feed address, the query applied to each, the person who owns it, and the date the watch stops.

Sources behave differently by class. Publisher feeds and preprint servers deliver structured records with identifiers. Regulatory dockets, agency pages, and standards bodies publish on their own schedule and often without any feed. Study registries update existing entries rather than adding rows, so a crawl looking only for new records will miss a status change. Institutional sites post PDFs with no metadata at all.

Set a stop date on every watch. Open-ended monitors rot. Tie the stop to the decision the review feeds: the panel date, the publication date, the point where the recommendation locks.

State the empty behavior before the first crawl. A source that returns nothing records a successful empty crawl with a timestamp. A source that returns an error, a redirect, or a login wall is a fault, and it goes to the owner rather than into the queue as silence. Those two states must never look the same on a dashboard. A dead feed reporting zero results looks exactly like a quiet field.

If the source list and the queries do not exist in writing, do not start crawling.

Check every caught item against the round already in progress before it becomes an intake row. That includes candidates already screened out. A paper rejected at triage in March that reappears in June under a journal name instead of a preprint identifier gets screened twice, and the second reviewer will not know someone already looked.

Match on more than the title. A preprint and its published version share authors, an abstract, and often a registration number, and they share almost no persistent identifier. Version chains, corrected records, and reposted agency documents need the same treatment. Vector similarity across abstracts catches most of it, and near-duplicate merging on embeddings is the same mechanism triage already runs.

Then apply the hard filters the round agreed on, before anything reaches a person. If the review excludes work published before a cutoff year, non-human studies, or jurisdictions outside scope, the monitor applies those rules itself. Use the same rule set as the rules engine that rejects candidates on hard criteria, and use the same file, not a second copy that drifts.

What survives becomes an intake row marked unscreened. It does not get a score, a summary, or a relevance rank from the monitor. Ranking new arrivals against a question nobody has read them against invents a judgment that belongs to triage.

Treat the following unit and its timeline as a constructed example. An environmental policy unit is preparing a recommendation on a treatment standard for a class of drinking-water contaminants. The initial search closed in February. The advisory panel meets the following January, so the question stays open for eleven months.

The watch definition names fourteen sources: two preprint servers, six journal feeds filtered to the compound class, a federal rulemaking docket, two state agency pages, a standards body, a health registry, and one practitioner association's publication list. Two queries run against them: compound names with synonyms and registry numbers, and a treatment-method query covering the removal technologies under review.

Across the eleven months the monitor catches 194 items. Deduplication against the round removes 61, most of them preprints already sitting in the candidate pool under a different identifier. The knockout rules drop another 112 for wrong compound class, wrong matrix, or jurisdiction outside scope. Twenty one items reach intake as unscreened rows.

Triage keeps nine. Seven are confirmatory occurrence data that moves nothing. One is a state agency decision that changes what the panel compares against. One is a retraction of a paper already sitting in the deep-assess set, which forces a re-read of a section written around it.

Two of the fourteen sources produce every item that mattered. Six produce nothing all year. Three break at some point: a feed address changes, an agency moves its docket, and a publisher changes its filter syntax. Each break gets caught because empty and error are separate states.

The retraction is the whole return on the year. That single row is why the watch existed.

Volume kills first. A query broad enough to feel safe pushes hundreds of rows a month, triage stops reading them, and the queue becomes the place things go to be ignored. Tighten the query and accept that the monitor will miss things. A narrow watch someone actually reads beats a wide one nobody opens.

Silent breakage is the dangerous failure, because absence gets read as evidence. Track each source's last successful non-empty return. If a feed has produced nothing for longer than its normal publishing rhythm, treat that as a fault until someone confirms the field is genuinely quiet.

Query drift is slower and harder to see. Terminology moves. A compound acquires a common name, a method acquires an acronym, a field renames its outcome measure. The query written in February keeps running and keeps missing the new vocabulary. Re-read it against the terms appearing in recent items at least once a quarter.

Scope creep fills the queue with sources nobody can defend. Every source in the definition needs a written reason and a removal date. Sources with no reason come out.

The last confusion is between a monitor and a refresh. Watching for new evidence while a question is open is this job. Re-testing conclusions after a report has shipped is periodic re-running of key claims against new evidence, on a different cadence and a different owner.

A late arrival does not re-open anything by itself. A named person, usually the evidence lead who owns the review, decides what the queue changes. That decision has three outcomes worth recording: the item enters the round as an ordinary candidate, it is logged as seen and not material, or it forces a re-read of a section already drafted.

Log the second outcome. Items seen and set aside are the only proof that the question was watched. Without that log, the evidence a monitor ran is that nothing happened, which is exactly what a broken crawler also produces.

Set a hard cutoff and publish it. After a stated date, new items do not enter the current round. They go to the next one, or into unresolved uncertainties packaged as research briefs for whoever picks the question up later. A review that accepts evidence right up to the meeting never finishes, and the last item read ends up carrying more weight than the fifty read in March.

Turn the watch off when the decision closes. A monitor that outlives its question keeps producing rows with no owner and no destination, and someone eventually mistakes it for a live process. The closing note should say what it caught, what it missed, and which sources earned their place.