oregon-stories · oregon-audits × oregon-kpm, joined by meaning
By meaning alone, 43% of audits land on their own agency's scorecard
Two corpora describe the same agencies: the auditors' reports, and the agencies' own Annual Performance Progress Reports. Read purely by meaning — no metadata, no names — each audit's nearest scorecard belongs to the same agency 43% of the time, against a 1.2% chance. That recovery rate is the method check. What it buys: every audit can be laid beside the scorecard it most resembles, and that scorecard's own at-target share for the year — the table below shows the twelve pairings where the agency's self-reported performance was weakest.
The method check: can meaning alone find the right agency?
Audits beside the weakest scorecards — 116 pairs with ≥5 judged measures, twelve lowest shown
| audit report | nearest same-agency scorecard (by meaning) | measures at target that year |
|---|---|---|
| Cybersecurity Controls Audit information technology · 2026 | apprproposed-ojd-2019-02-14 cos 0.70 · reporting year 2018 | 8% 1 of 13 judged 0 state no direction |
| Clean Water State Revolving Fund Loan Program Enterprise Fund of the S financial · 2025 | appr-deq-2025-09-30 cos 0.76 · reporting year 2025 | 22% 5 of 23 judged 92 state no direction |
| Clean Water State Revolving Fund Loan Program Enterprise Fund of the S financial · 2024 | appr-deq-2025-09-30 cos 0.74 · reporting year 2025 | 22% 5 of 23 judged 92 state no direction |
| Clean Water State Revolving Fund Loan Program Enterprise Fund of the S financial · 2026 | appr-deq-2025-09-30 cos 0.73 · reporting year 2025 | 22% 5 of 23 judged 92 state no direction |
| Clean Water State Revolving Fund Loan Program For the Fiscal Year Ende financial · 2023 | appr-deq-2025-09-30 cos 0.70 · reporting year 2025 | 22% 5 of 23 judged 92 state no direction |
| Ongoing Strategic Planning Can Help DEQ Address Obstacles to Achieving performance · 2024 | appr-deq-2022-10-03-final cos 0.84 · reporting year 2022 | 25% 6 of 24 judged 91 state no direction |
| Clean Water State Revolving Fund Loan Program For the Fiscal Year Ende financial · 2022 | appr-deq-2022-10-03-final cos 0.70 · reporting year 2022 | 25% 6 of 24 judged 91 state no direction |
| Oregon Health Insurance Marketplace Fiscal Year 2019 Compliance financial · 2020 | appr-dcbs-2019-12-19 cos 0.74 · reporting year 2019 | 26% 14 of 54 judged 30 state no direction |
| Oregon Health Insurance Marketplace Financial Statements For the Fisca financial · 2020 | appr-dcbs-2019-12-19 cos 0.74 · reporting year 2019 | 26% 14 of 54 judged 30 state no direction |
| Clean Water State Revolving Fund Loan Program For the Fiscal Year Ende · 2021 | appr-deq-2021-10-04 cos 0.74 · reporting year 2021 | 27% 6 of 22 judged 79 state no direction |
| Uncertainty and Risk Remain After Gambling Industry Has Evolved Faster performance · 2023 | appr-orc-2025-09-18 cos 0.81 · reporting year 2025 | 29% 5 of 17 judged 30 state no direction |
| Oregon Health Insurance Marketplace Fiscal Year 2020 Compliance financial · 2021 | appr-dcbs-2020 cos 0.74 · reporting year 2020 | 30% 17 of 57 judged 30 state no direction |
Read this before quoting the chart
Proximity surfaces candidates, never findings. An audit reading like a scorecard does not mean the audit was prompted by those measures or should have addressed them — both are one agency writing about its own programs, so agency-level overlap is the null hypothesis, not the result; the 43% is a recovery rate, not accuracy. At-target shares use each measure's own stated trend direction (measures stating none are counted, not judged) and targets are agency-set — the same caveats as the KPM story. Counted out: 19 audits carry no registry slug; 7 audited agencies have no KPM reports at all. Similarity is cosine over per-document mean embeddings (BAAI/bge-m3); the cache records both artifact fingerprints and is rebuilt deliberately, not per-deploy.