Development Effectiveness

The Australian Government is committed to strengthening the performance orientation of Australia's aid program. Aid must be monitored and evaluated against its objectives, to ensure it is effective in reducing poverty and achieving sustainable development. Australia will continue to undertake and strengthen its evidence-based approach to aid programs.

Performance Assessment and Evaluation Policy

The Australian aid program is committed to strengthening its performance orientation to help managers improve development effectiveness and account for results. A new Performance Assessment and Evaluation Policy [PDF 115KB] sets out expectations for measuring the performance of Australian aid, including a number of principles that apply to all types of reporting.

The policy sets out the minimum expectations for measuring performance at strategy or program level and at activity level. There are three types of reporting processes: annual performance reporting, the quality reporting system and evaluations & reviews.

A major part of performance assessment is the annual performance reporting. It provides an analysis of whether programs are meeting their objectives. The information is then used in making decisions about future program planning and budgeting.

Annual performance reporting include reporting against country and regional strategies. Most program areas are required to complete an Annual Program Performance Report by May 2008.

Annual performance reporting by thematic area will be trialled in 2008. Thematic areas required to complete an Annual Thematic Performance Report are: Governance & Anti-Corruption, Environment, Infrastructure, Health, Education, Humanitarian & emergencies. These areas were selected because they are required to report on new policy proposals. ODE will also work with global programs on reporting for 2008.

Instructions and Guidelines will be available in February following user testing. Existing and new instructions and guidelines on quality reporting, independent completion reports and evaluations will be linked to the Performance Assessment and Evaluation Policy.

This policy replaces the Performance Assessment Framework [PDF 48KB], previously used to measure the effectiveness of Australian aid.

Country and regional strategies

The country strategy provides the overall policy implementation framework for Australia's aid program in any given country. It translates Australia's aid themes and strategies into programs tailored to individual country circumstances and priorities. Country strategies are developed and agreed jointly with partner governments. Country strategies are being upgraded by:

Incorporating the broader ODA effort, including that delivered by Australian Government agencies other than AusAID
Strengthening selectivity - Australia cannot do all things in all countries, and the country strategy will be the main document which determines the focus of effort
Providing a more rigorous performance framework which will articulate expectations at the country level more clearly and provide a better basis for assessing the impact of aid efforts
Agreeing with partners performance frameworks that link additional allocations to mutually agreed performance criteria.

A three-year program has been established for upgrading all country and regional strategies. New country strategies will be informed by reviews of past strategies with ODE undertaking the reviews for the major partner countries.

Aid Quality

AusAID’s new Quality Reporting System addresses the need for ongoing measurement of the quality of activities, as required by the new Performance Assessment and Evaluation Policy. The new system helps to ensure reliable, valid and robust information is available to assist ongoing improvements of program management, and to meet external accountability through AusAID’s Annual Report and Annual Review of Development Effectiveness.

The Quality Reporting System reports on the quality of aid activities at the stages of design (or entry), implementation and completion. The new system uses a 6-point rating scale.

The Quality at Entry report rates the quality of an activity design. Ratings and commentary are initially prepared by the AusAID Activity Manager. A Peer Review then discusses and agrees to ratings, and identifies any actions needed to improve the design before proceeding to implementation.

The Quality at Implementation report provides information and ratings on the quality of activity implementation, and enables contestability of findings and proposed actions. Links to an Overview of the Quality Reporting System and the Guideline and Template for Quality at Implementation Reports are provided below.

The Quality at Completion report follows a similar format, and provides ratings and commentary against the following criteria: 1) Analysis and learning, 2) Effectiveness, 3) Efficiency, 4) Monitoring and Evaluation, 5) Sustainability.

Evaluations

Evaluations are an important part of learning and accountability about aid effectiveness. Good evaluations consider the following elements of aid activity:

relevance
effectiveness
efficiency
impact
sustainability of benefits and systems

A core element of AusAID's system is the Independent Completion Report, mandatory for all significant AusAID initiatives. Program areas within AusAID also commission further independent reviews and evaluations to supplement the information from Independent Completion Reports; for example, by returning two years after activity has finished to better assess impact.

Recent years have seen international debate about evaluation of aid programs, and advances in a number of areas. With growing community interest in aid effectiveness, aid agencies are increasingly seeking credible evaluations of the long term impact of aid initiatives, using rigorous social sciences methods such as randomisation, quasi-experiments, or rigorous qualitative techniques. Methods such as contribution analysis are being used to address the difficult questions of causality and attribution, particularly when considering the effectiveness of country programs. ODE ensures that Australia uses the latest in evaluation methodology and development thinking.

The Office of Development Effectiveness conducts evaluations or reviews of major country programs when a new strategy is to be developed. The Office of Development Effectiveness also manages a program of flagship thematic evaluations, usually led by senior, internationally recognised figures. ODE's evaluation program is based on the following criteria:

Relevance to policy or budget priorities
Whole of government interest
Cross program relevance
Trialling of new evaluation methodologies

Priority forthcoming ODE evaluations include gender and domestic violence, an impact evaluation of our HIV/AIDS support in PNG, and anti-corruption. As a general rule, ODE evaluations will be published.

Office of Development Effectiveness

The Australian Government established the Office of Development Effectiveness (ODE) in 2006 to monitor the quality and evaluate the impact of the Australian aid program.

The ODE answers directly to the Director General of AusAID. It is guided by a Development Effectiveness Steering Committee, including deputy secretaries of the Australian Government departments Prime Minister and Cabinet, Foreign Affairs and Trade, Treasury and Finance and Administration.

Annual Review of Development Effectiveness

The ODE will publish an Annual Review of Development Effectiveness drawing on the breadth of its work, including quality process reviews, evaluations and on the experiences of all Australian agencies delivering Official Development Assistance. The Annual Review will be integrated into the Government's budget cycle and provide a practical link between expanding allocations and increased aid effectiveness.

The first Annual Review of Development Effectiveness was released on 20 March 2008. The Annual Review will complement the Annual Reports of AusAID and other aid-delivering Australian government agencies.

See:

See also: