Pilots and trials

Pilot programs and trials are another valuable way to test innovations. In some situations agencies may be able to identify a small trial community or environment, take measurements before and after the implementation of a new policy or program, and compare the data. Pilots and trials can reduce the risks associated with implementing a policy by establishing its effectiveness at a lower cost, with a more or less explicit admission that it might or might not be continued.

A caveat is that a successful pilot does not mean that the associated implementation tasks of integrating it into a mainstream service are reduced [1. See for instance Osborne and Brown 2005, Managing change and innovation in public service organizations, Routledge, London]. Care should also be taken not to use the language of pilot if it has already been determined that the innovation will be introduced regardless.

Randomised policy trials are an evolution of pilot programs, adapting the randomised trial methodology of modern medical trials to government policy development. Trial groups are run alongside ‘control’ groups, and the two are alike in all respects except for the trial itself. Randomised policy trials can overcome some of the major problems of pilot programs by establishing a greater level of robustness in the evaluation method.