Wrapping Up the Australian Public Sector Innovation Indicators (APSII) Project

In September 2012 I reported that an innovation-specific pilot survey had been launched by the Australian Public Service Commission (APSC). The survey was part of the APSII Project and aimed to measure innovation in the Australian Public Service (APS).  I’m pleased to report that the pilot survey was very successful and provided insightful information on public sector innovation.  A more detailed summary on key findings is available but here are some of the highlights:

  • An innovation-specific on-line survey directed at SES Band 1 officers in the APS provides a useful snapshot of innovation activity within the Australian Government.  Information derived is both quantitative and qualitative and allows us to develop data cards on the innovation activity of agencies.

  • Survey participants generally understood the concept of public sector innovation and its two essential features: novelty and improvement over existing practices.  However, respondents often had difficulties in differentiating between the five types of innovations identified in the survey (service, process, communication method, policy and goods).

  • The most commonly reported innovation type was process innovation (85%), followed by product and service innovation (47%), policy innovation (44%) and communication innovation (42%).  A total of 91% of all survey respondents reported that some type of innovation had been undertaken by their branch or work unit over a two year period.

  • Respondents often lacked a clear perception of the difference between the development and implementation of an innovation. Many reported innovations had a long implementation period with no clearly defined date of introduction.  Some branches or agencies were also responsible for part of, but not all, of the innovation process.

  • Innovations can fail and information on why this has occurred is highly important.  However, there was reluctance amongst respondents to provide information on unsuccessful innovations, with around 80% survey participants stating that they “did not have a least successful innovation”.

  • There is a high turnover of SES Band 1 officers within agencies.  Nearly 57% of survey respondents had been responsible for their branch or work unit for less than two years and 86% for less than 4 years.  This was in part the result of the Machinery of Government (MOG) changes over the past few years.

  • Notably fewer managers with less than six months of branch / work unit responsibility reported having implemented a process innovation (67% compared to the average 82%). Taking account of innovations underway (i.e. “innovation active” branches or work units) the difference became less (80% compared to 87%).

  • During cognitive testing Interviewees reported a “top-down” and “bottom-up” approach to innovation within their respective agencies.  The former was generally associated with broad policy directives and high-level ideas (“ideas innovation”).  Ideas originating from junior staff tended to be more operationally focused, leading to incremental improvements.

  • The pilot survey provided some support for a qualitative difference in the types of innovation by the source of the idea, with 41% of innovations which originated with the Minister or senior executive taking 12 months or more to develop and implement. This compared to 37% of ideas coming from the leadership group and 31% of ideas coming from junior staff.  Conversely, 58% of ideas coming from junior staff took less than 6 months to implement.

  • Interviewees saw their leadership group (which included themselves and Executive Level officers) as being the “powerhouse” for the implementation of Ministerial ideas, many of which had innovative elements as they dealt with new government initiatives.  Survey results showed that the leadership group accounted for 50% of the sources of ideas for the most successful innovations.  Higher level management and the Minister accounted for a total of 10% (or 6.2% and 3.4% respectively) of ideas.

  • The time spent on innovations proved to be a useful indicator of innovation activity.  Sixty per cent of managers from innovative branches reported that their most successful innovation required six months or longer to develop, while only 10% reported spending less than one month on this innovation.

  • Many of the innovations cited by interviewees and survey respondents were minor or incremental improvements rather than major or transformative changes.  The length of time taken to develop an innovation appears to be a good indicator for distinguishing between these two types. Innovation rate drops to around 60% if the data are screened for major innovations only based implementation time.

  • A large majority of innovations were developed through collaborations with other organisations, replicating results from Europe where collaboration rates in the public sector are almost four times higher than rates in the private sector (80% versus 20%).

  • The pilot survey found that nearly 14% of branches were diffusers of ‘first to Australia’ innovations, with the rate ranging from 4% for branches whose main function was legal or regulatory services to 33% for branches responsible for public relations or marketing. Preliminary analysis suggests that diffusion rate is a good indicator of innovation novelty.

In conclusion:

  • The innovation rate reported for the APS at the branch level (91.3%) is very similar to the public sector innovation rate (91.5%) calculated from the European surveys in 2010.  It is significantly higher than the innovation rates reported for Australian firms in 2010/11 or European firms between 2006 and 2008.

  • The Pilot Survey findings did not support the popular belief that the public sector is non-innovative due to risk aversion, has an anti-innovation culture and lacks innovation incentives. The Pilot Survey demonstrated that innovation exists within the public service.  However, innovation appears to take longer than in the private sector, possibly because managers develop strategies to accommodate the ‘risk averse’ requirements of the public sector.

In terms of survey participation, the response rate to the pilot survey was 33%. This is consistent with voluntary surveys.  Of the 97 APS agencies selected, 61 participated in the Pilot Survey. It included small (13), medium (26) and large (22) sized agencies.  Even allowing for size differences, response rates varied considerably between agencies.  By the time the survey closed, a total of 367 responses had been received. This captured more than 15% of all functional units at a branch level across the APS and was considered representative enough to undertake a preliminary analysis and prepare reports on the survey outcomes.

We also learned some valuable lessons in terms of survey methodology.  Foremost, the pilot survey had been too long and repetitive.  Following the analysis of the survey data, we reduced the number of questions by 25% and the overall length by 30% in the final version of the questionnaire.

The next (and final) stage of the APSII Project involves preparing several Project Reports and submitting these to the Secretaries Board for consideration. The Background Document summarises progress to date with the APSII Project. Once a clear direction for the future of the project is determined we will provide a final blog.

Visit the Department of Industry's Public Sector Innovation page to access the APSII documents in more formats.