Bias Audit on Admit Decisions

A bias audit of admit and deny decisions produces a cited table of patterns that exist in one locked applicant cohort. Each cell shows a count, a rate only when the count is large enough to report, and a blank when the cell is too thin. The table is a quality artifact for IR, admissions leadership, and equity staff. It is not a legal determination, not a finding of discrimination, and not a trigger that rewrites files.

The model is adversarial only in that it looks for uneven admit and deny patterns the committee did not set out to measure. It does not rank applicants, propose a new class, or auto-change decisions. If a cell is empty because four people is too small to publish, it stays empty. Inventing a disparate-impact ratio to fill the gap would be the audit failing its job.

Counsel or the board may later use the same files under a legal standard. This page is about the operational report IR can stand behind: every number traces to a locked extract, thin cells are suppressed, and humans own the interpretation.

Admissions systems of record, including Slate, Technolutions, Ellucian, and Salesforce implementations, already hold application, decision, and (where collected) demographic fields. The audit reads a freeze of those fields. It does not replace the CRM, the SIS, or the reader workflow that produced reviewer decision assistance notes during the season.

Lock the population before you compare anything. Agree in writing who is in: term, student type (first-year, transfer), residency, complete files only or all files that received a decision, and which decision timestamp counts as final. Waitlist pulls, late athletic admits, and dual-enrollment exceptions move cells if you recut the file every Friday.

Export once from the CRM or SIS, version the extract, and refuse silent refreshes. If leadership wants last week's adds included, that is a new cohort and a new table, not an edit to the old one. Mixing extracts is how a rate appears to move when the only change was files posting after the freeze.

Admit and deny mean what the institution recorded. Defer, waitlist, and withdraw-before-decision need explicit in-or-out rules, including whether a later waitlist admit counts in this freeze. Put the rule on the table footer. An equity lead who cannot explain the denominator cannot defend the numerator.

The freeze also blocks using the audit to launder a quota. A quota is a target class composition. An audit describes what already happened. Recutting a live pool until group ratios look acceptable is not quality control; it is a steering wheel. Run the official audit after the class you intend to report is closed. A mid-cycle cut, if you run one, stays labeled provisional and never becomes the finding.

Compare admit and deny only on attributes IR and legal policy already permit. Where collected and allowed, that often includes reported race and ethnicity, reported sex or gender, first-generation status, residency, and sometimes Pell eligibility. Do not invent SIS fields. Do not scrape essay or zip-code proxies unless IR has an approved method that survives scrutiny.

The comparison is descriptive. For each allowed attribute, and for combinations IR pre-approved, show admit and deny counts, and a rate only when cell size supports it. Intersection cells go blank faster than margins. Do not collapse categories after seeing the numbers unless that collapse was in the analysis plan.

This audit sits downstream of how files were scored. Holistic application scoring and academic success prediction can shape who gets a serious read. Uneven outcomes do not by themselves prove those models caused the pattern. Causal claims need a different design, with IR and often counsel in the room.

Essay authenticity screening is separate. An authenticity flag is not a demographic finding. Mixing flags into the bias table without a pre-specified rule contaminates both reports.

Vendors do not set the attribute list. Whether records live in Slate, a Technolutions stack, Ellucian student information, or a Salesforce recruiting org, policy decides the pull. Keep a data dictionary with the extract.

The deliverable is a table a colleague can cite: group, decision, count, rate if reported, suppression note if not. Empty stays empty if a cell is too thin. Follow IR's existing suppression rule, including complementary cells that would let someone back-calculate. If you have no rule, write one with IR before you generate rates. This page does not pick a magic n for you.

Illustrative path, not a measured case: an IR analyst freezes completed first-year applications for one fall term after the last published decision date. The table lists admit and deny counts by each allowed attribute. One combination, a small reported group whose deny cell has only four people, falls under suppression. The cell is blank. A dean asks for the ratio for the board deck. The analyst refuses. Publishing a ratio from n=4 turns noise into a campus story and can make a family identifiable. Telling the board the cell was suppressed for size is the honest line. A three-decimal impact ratio the file cannot support is not.

Do not invent a disparate-impact ratio to summarize the whole table. One ratio erases cell size, mixes attributes, and sounds like a legal test. Rate comparisons counsel may later run belong in a legal review with a defined comparator and statute. Emitting a ratio because the model can compute one is a failure mode.

When you report a rate, show the two counts behind it. Footnote freeze date, extract version, and suppression rule. If a later audit of the same term disagrees, check the cohort lock first.

Humans interpret the table. The model does not change decisions, re-rank the waitlist, or email applicants. If a reportable pattern looks material, IR and admissions own the finding, with equity and counsel as the institution requires. Remedy might be process review, reader calibration, a look at reviewer decision assistance use, or a planned study of scoring inputs. Remedy is never the audit flipping files overnight.

Treating the report as a legal finding is the second failure mode. A quality table can inform a legal review. It does not complete one, establish intent, pick the legal comparator, or replace counsel. Do not title the PDF as a civil-rights determination or as proof of disparate impact. Title it as an internal cohort audit with a date, a freeze, and a suppression rule.

The third failure mode is using the audit to launder a quota. If leadership already has a numeric target by group, they do not need this table to discover it. Quietly adding admits until published cells look even turns a descriptive control into a covert selection rule and poisons the next freeze. If policy changes lawfully, change it in the open, recode reader instructions, and audit the next closed cohort. Do not ask monitoring to hide the change.

Done means a versioned extract, a table with counts and blanks, an IR memo on what is reportable, and a log of who reviewed it with a note that no automated remedy ran. Empty cells stay empty. Unjustified ratios stay unwritten. The admitted class stays the admitted class unless humans, on the record, change a file for a reason the institution already allows.