Tuesday, October 26, 2010

Better metrics about relief and development interventions are needed

Dear Colleagues

There is a need for paradigm shift in the way data are acquired and the way data are used. CordAid is one of the largest international NGOs with an excellent reputation and a lot of experience ... but they use independently prepared "studies" about their work to help them with management as for example as follows:
"This study has been carried out on request of Cordaid. Cordaid wished to better understand the effect of their Pay for Performance Programme (P4P) and the introduction of a flat rate fee system introduced in 2007 in some of the health facilities which Cordaid supports. The effect of the intervention has been measured in terms of client satisfaction and out-of-pocket spending on maternal and malaria health services of households in the Catholic Diocese Bukoba in Muleba, Missenyi and Bukoba Districts in North West Tanzania. A population-based cluster-sampled survey was carried out July-Aug 2007 and included 1946 households with members using all types of health service providers. Two main research questions were addressed: (1) to what extend are health services from the various providers in the area being utilized and how much has been paid for these services by households out-of-pocket, and (2) to what extend are people satisfied with the health services that are being provided in terms of perceptions on quality, access and affordability? The results of the survey are presented in this report and should be regarded as mid-2007 baseline for a follow-up survey in 2-3 years time. Therefore, the report is limited in analysing effects of the P4P and introduction of flat rates, though efforts have been put into comparing faith-based and government health facilities."

Client Satisfaction and Out-of-Pocket Expenditures on Maternal Health and Malaria Health Services. A population-based survey in North-West Tanzania 2007. By Ronald Horstman et al., August 2009. Paper commissioned by Cordaid. Download as PDF from cordaidpartners.com

The report referred to is located at: http://www.phc-amsterdam.nl/artikelen/CL-OOPE_Study_Tanzania_-_final_26Aug2009.pdf

The study works for the commissioning organization ... they have a document that reports that they have done something more or less according to what they planned on doing ... and the work looks good and sounds convincing.

It is, however, almost impossible to verify at moderate cost ... and really does not help with the big question of whether scarce relief and development resources are being deployed in an effective way. The issue of whether or not services are better free or paid is certainly interesting ... but a bigger question might be the question of the value of the service relative to the cost of service independent of the issue of pay or free. In poor communities, it is clear that affordability is a challenge ... but that is no issue when there is no service. At the same time ... the value destruction associated with inadequate service is a very big issue.

A new paradigm for assessing the performance of international relief and development assistance interventions needs to be deployed ... the sooner the better.

Peter Burgess

No comments:

Post a Comment