Quality Assurance is an integral part of the process of developing Grattan Institute reports. It is vital to our reputation and effectiveness in influencing policy. Grattan’s reputation rests on the credibility of the analysis incorporated in its reports.
Grattan Institute’s Board, CEO and Program Directors have put in place arrangements for internal and external review of every Grattan report, to ensure that Grattan Institute’s work provides a balanced consideration of the issues based on rigorous analysis.
The Institute devotes substantial resources to the development of each research report. Grattan’s Program Directors are highly qualified and experienced experts in their fields, each has a long track record in undertaking and supervising analytical projects.
The direction and structure of each report are regularly discussed by the CEO with the Program Director and the team working on the report. In practice this often leads to clarification of the analysis required and the core logic of the publication. Each report also has a Fellow or Associate not working on the report assigned to be a “critical friend” to the direction of the analysis.
Towards the end of preparing a report, the CEO assigns one or more Associates not substantially involved in the report to validate quantitative analyses. This cross-check may be conducted by working through key analyses and by independently replicating key data points to verify the analysis. This process may also involve checking citations to ensure they support the claims made in the text.
All of Grattan’s programs have a Board-appointed Reference Group of external experts in the subject area. A draft of each report is sent to the relevant Reference Group for comment. This invariably leads to corrections of matters of fact, clarifications, and occasionally substantial shifts of emphasis. Reference Groups have a charter that articulates their role and responsibility for providing frank and fearless feedback on any issues they see in draft reports.
A draft of each report is also circulated for review to a number of people with expertise on the particular topic of the report. It is common for reports to be circulated to around 10 to 20 experts. At least two of these are nominated in advance by the Program Director in consultation with the CEO and asked to provide detailed feedback on the report. All feedback is documented. This feedback is advisory only. When a substantial issue is raised by external feedback but the Program Director does not believe it should be incorporated in the final report, it is raised with the CEO. Where the CEO agrees with the judgment of the Program Director, then the report stands, but these issues, and the reasons for not following the advice of the reviewer’s are formally reported to the Public Policy Committee of the Board who review each of Grattan’s reports and oversee quality assurance processes.