Written by The dominance of Randomized Controlled Trials (RCTs) as the “gold standard” for evaluating social programs presents significant challenges for nonprofit organizations. While a well-implemented RCT can demonstrate that a program causes a specific, measurable outcome, this singular focus often comes at the cost of continuous learning and organizational strength.
In the Fall 2025 issue of the Stanford Social Innovation Review, the authors argue that elevating the RCT as the primary arbiter of impact can have negative effects, including directing management attention away from community connections and reducing a nonprofit’s ability to respond nimbly to changing social problems.
Lengthy RCTs force organizations to hold program components constant for three to five years, ironically limiting the agility and responsiveness that Lean methodologies seek to maximize. Instead of pursuing high-stakes, time-consuming RCTs, the social sector should adopt an “improvement orientation” focused on continuous learning, adaptation, and ongoing evaluation strategies.
This proposed improvement orientation closely matches the concepts of Lean and Six Sigma methodologies, which prioritize process optimization and continuous flow. They recommend iterative, evidence-based methods rather than waiting for a single “final verdict” from a multi-year evaluation. Specific techniques suggested include using rapid cycle mini-experiments, A/B testing, and plan-do-study-act cycles (PDSA).
The articles also states that RCTs treat nonprofits merely as “platforms for the delivery of those programs,” overlooking the need to cultivate strong, multifaceted organizations. Lean Six Sigma principles teach us that process issues (like the managerial problems and capacity drain noted during RCT participation) must be addressed to ensure reliable outcomes.
In essence, the shift from RCT evaluation to an improvement orientation is a shift from auditing a final result (a “thumbs-up” or “thumbs-down” answer) to optimizing the underlying processes that create impact. Nonprofits must transition from being “locked down by their evaluation methodology” to becoming nimble, learning organizations.
Organizations should use data as “constant conversation starters” rather than inert final verdicts. This dynamic process ensures that effectiveness is blended with adaptability and genuine community engagement, leading to lasting progress.
Thanks to Steve Bell for sharing this article with the community.
You can download the entire article at https://ssir.org/articles/entry/the-problem-with-randomized-controlled-trials
Image provided by https://unimelb.libguides.com/whichstudytype/RCT