| Evaluation | Evaluator | Timing | Authority to change rules | What it measures |
|---|---|---|---|---|
| Data+Soul cycle evaluation | Internal team | Every cycle | Advisory with required response | Reach and participation, Process quality, Downstream and institutional effects |
| Board annual review and Rulebook vote | Partner-led | Every cycle | Binding | Process quality, Money and distribution, Projects and delivery |
| Public comment at Board meetings and online | Participants themselves | Every cycle | Advisory with required response | Process quality |
| Monitoring of awarded projects | Internal team | Every cycle | Informational only | Projects and delivery, Money and distribution |
Data+Soul Research (formerly MXM Research Group) was hired by OPB in January 2024, before the Rulebook existed, so it is a contractor and counts here as Internal team even though it is an outside firm. Its cycle evaluation uses surveys embedded in the idea portal and voting platform, event surveys, observation, partner and City staff debriefs, resident reflection conversations and quarterly OPB retrospectives, reported to the Board in phase reports and a final report; the evaluation questions were co-developed with OPB, the Finance Cabinet, departments, Board members and the Better Budget Alliance. The External Oversight Board sends an annual review to the Mayor and City Council under ordinance section 5.1.11.7(c) and votes each spring on Rulebook revisions. Public comment takes 15 minutes at every Board meeting, with a written response at the next meeting, and there are online comment periods on the draft Rulebook and the draft ballot proposals. Awarded projects are monitored through OPB tracking with departments and status reports to the Board; a Data+Soul protocol covering costs, who is served and outcomes is planned. The loop visibly closes: FY26 changes (two-part forums, paper ballots, partners contracted for the full cycle, 19 partners instead of 9, demographic questions embedded in the platforms, zip code made optional) trace to pilot findings, and the FY27 changes adopted on May 7, 2026 (partners to run workshops only since tabling and door-knocking proved less effective, the term 'priority populations' dropped, proposal development clarified, paper ballots through partners formalised) trace to FY26 findings. Response rates are the known weakness: about 6% for the pilot portal survey and 3% for the pilot vote survey, rising to 83% for the vote in FY26 once embedded. The Board's four-member subcommittee edited the pilot evaluation, asking that recommendations be split into those possible through process change and those needing more resources. Some public proposals have been heard and declined with reasons (Decidim, a neighbourhood set-aside, budget delegates, more project money, April 2025). Project outcomes are not yet evaluated: a Cycle 2 question on how funded projects meet the ordinance's objectives was planned, and the ordinance requires a public hearing on funding use and demographics served.
Guest book
Notes left on Feedback Loops.
No notes yet. Be the first to leave one on this node.