Pays for itself

These days I am reading a lot of papers from economists that talk about interventions paying for themselves.  These are educational or development interventions where the costs of the intervention are less than the supposed (and pretty shaky) projected additional public money raised in the long term as a result.  The intervention is often recommended on the basis of these calculations.  It strikes me as odd that this is even considered an interesting ratio, since public money is gathered from a range of sources, and is necessarily about redistributing money from one part of the economy to another.  An optimal way of directing funds might be social welfare per pound spent, with some adjustment for social justice, so that opportunity costs are avoided, with some deduction for utility lost in taxation.  It is irrelevant to talk about public returns, and misdirects attention to something that is (a) very hard to project, and (b) a weird thing to do anyway as it is possible to raise taxes in other ways if the net of social utility is improved.  It’s just so strange that “improve maths results with intervention Xit pays for itself” means anything to anyone when that’s not how things are paid for and isn’t necessarily the use of funds that maximizes utility.

Cynically, I wonder if the obsession with the bottom line in government makes these ‘expenditure/returns’ ratios interesting to policy makers.  I also wonder whether the lack of proper engagement in measured utility (e.g. something analogous to the QUALY used in the health sector) allows people to miss the point entirely.

Leave a comment

Filed under Uncategorized

Facebook Ethics

There’s been a lot of gasping about Facebook deliberately making a random half of a random sample of users’ posts a less positive than they would have been for one week.  This was part of a study to understand how emotions are affected, and the results of the study are, apparently, important.  The fuss is that the people involved – many thousands – were not asked if they wanted to take part.

There have been some funny things said about this.  Not least, from the Guardian:

But the study has come in for severe criticism because unlike the advertising that Facebook shows – which arguably aims to alter peoples’ behaviour by making them buy products or services from those advertisers – the changes to the news feeds were made without users’ knowledge or explicit consent

My view, and the view of many in advertising and psychology, is that persuasion does indeed change behaviour without people’s explicit knowledge (whatever that means exactly) and certainly no-one consents to having advertising in their Facebook feed any more than they would to take part in this experiment.

For me, as an epidemiologist who helps with cluster randomized trials, I am surprised that we’re still talking about consent in such a simplistic way.  In many cluster randomized trials it is not deemed necessary by ethics boards for every member of every cluster to be asked for their consent to be randomized (although it probably will be for any data collected, at least at the individual level).  In other contexts, especially in psychology, it is unfeasible to explain the study to the participants since they cannot be blinded to the random change.  In both cases, important research is only made possible with a nuanced view of consent.

From the PNAS paper, it doesn’t appear that Facebook received approval from an ethics board, and balancing the issues touched on above is what ethics boards do.  They may continue to try to claim that agreeing to the data use policy when we sign up constitutes consent (it doesn’t).   These are the criticisms that should be levied here – that they were in violation of procedure – not some naive and knee-jerk reaction to issues of consent in the context of tailored ads and diminishing privacy.

Leave a comment

Filed under Uncategorized

Freedom and feedback

Owen Barder, of the Centre for Global Development, delivered a fascinating talk to the Children’s Investment Fund Foundation, that you can read here.  He was talking about how to think about scaling up development initiatives, and the importance of local adaptability, the ‘process’ of institution building, measuring outcomes, and autonomy for local implementors.  He cites evidence for a link between the last two – measurement and autonomy.  He then draws parallels with business when he says:

I know more about development than I do about managing a hedge fund, but I would hazard a guess that most of this is common sense to a business investor.  As I understand it, a large part of being an excellent venture capitalist is identifying great teams with great strategies, and giving them a sufficient degree of latitude to get on to deliver.

It was this comment that inspired this post, since it strikes at the heart of the ‘aid as business’ perspective that is in vogue.  Barder makes the link between autonomy and feedback, but doesn’t make it clear that in business consumer/client purchasing behaviour is the feedback, and is an in-built assessment of value of the business.  The market administers the evaluation: the value is expressed in such a way that the recipients (clients, customers) dictate it directly, and it can be packaged up and consolidated for the investors (as returns).

Delivery of public services to the poorest of the poor cannot function in such a way, since the recipients are not customers with either:

  • a) choice to express a personal evaluation in a market, or
  • b) the means to express it in a way that can be consolidated directly by donors.

The ‘common sense’ for business investors doesn’t translate. As Barder suggests, the aid as business model doesn’t work without rigorous assessment and evaluation taking on the role of market mechanism, that is, unless recipients are given real choice to allocate things that really matter to the programme administrators (i.e. money or credits).  While I agree with very much of what Barder says in this talk, I am concerned that the parallels drawn with businesses overlook the challenges of best addressing the needs of the powerless and the poor.

1 Comment

Filed under Uncategorized

Job satisfaction after income

I was intrigued by an article on a report from the Cabinet Office that ranked Clergy and Chief Executive as numbers 1 and 2 on a list of the most satisfying jobs in the UK.  Isn’t it likely that they are driven by different things?

I downloaded the data and fitted a quick-and-dirty regression model between income and life satisfaction, and then ranked the jobs according to the residuals from the model.  This is roughly the average satisfaction that is not explained by income.  The top 20 most satisfying jobs, after accounting for income, are below, with the original ranking in brackets:

(1) Clergy
(4) Company secretaries
(15) Fitness instructors
(3) Managers and proprietors in agriculture and horticulture
(17) School secretaries
(8) Farmers
(21) Dental nurses
(6) Health care practice managers
(23) Farm workers
(12) Physiotherapists
(13) Primary and nursery education teaching professionals
(22) Musicians
(50) Teaching assistants
(51) Childminders and related occupations
(9) Hotel and accommodation managers and proprietors
(33) Travel agents
(5) Quality assurance and regulatory professionals
(10) Skilled metal, electrical and electronic trades supervisors
(78) Playworkers
(41) Records clerks and assistants


And the bottom 20 are:


(266) Bar staff
(258) Parking and civil enforcement occupations
(253) Roofers, roof tilers and slaters
(252) Bus and coach drivers
(108) Legal professionals
(248) Mobile machine drivers and operatives
(262) Fishing and other elementary agriculture occupations
(257) Fork-lift truck drivers
(234) Quantity surveyors
(267) Care escorts
(261) Construction operatives
(263) Security guards and related occupations
(265) Plastics process operatives
(268) Sports and leisure assistants
(264) Ambulance staff (excluding paramedics)
(269) Telephone salespersons
(271) Industrial cleaning process occupations
(272) Debt, rent and other cash collectors
(270) Floorers and wall tilers
(273) Elementary construction occupations
(274) Publicans and managers of licensed premises


With Chief Executive languishing somewhere in the end at number 242.

In fact, the variation at the top of the unadjusted ranking (the ranking in the article) is quite large, while at the bottom end of the scale the variation is much smaller.  This can be seen in this graph below, with the original rank on the X axis and the adjusted rank on the Y axis (click to enlarge):



The red dot is Chief Executive -the least satisfied at the upper end of the overall scale.  This suggests that income is more strongly linked at the bottom end of the scale than at the top, which makes sense.

Whatever way you look at it, clergy (say they) are pretty happy!


Leave a comment

Filed under Uncategorized

Health ranks are rank

The global burden of diseases report tells us which diseases are contributing the most to human suffering.  Which is interesting, but is that what we really need to know?  Following from Ian Roberts’ talk at the LSHTM in April, I am more and more convinced that it misses the mark.

Diseases are manifestations of suffering, they are the effects that result from the causes of ill health.  The DALY (disability adjusted life year) goes some way to make this interchangeability a quantitative science: all health matters, all diseases are just categorizations of suffering.  So what does that global ranking tell us then?  Sometimes the manifestations of suffering tell us about the underlying causes, and this is epidemiologically interesting.  For example, the transitions from communicable disease manifestations of suffering to non-communicable diseases tells us something about development, food security, and society.  But what really matters is what we can do about suffering in all its many manifestations.  For that, we need to shift our focus from disease burden to the cost-effectiveness of averting a DALY of suffering, and not particularly worry about the proportions.

By focusing on cost effectiveness for DALY reduction we will change how we look at disease control.  The ‘control arm’ in a trial for a DALY reducing intervention should be the ‘next cheapest method of DALY reduction’, rather than the usual care for the particular disease that is being targeted.  Healthcare spending would be more rational, and save lives that are currently being lost in the ‘opportunity cost’ (rather mild way of putting it) of inefficient healthcare spending.  And by looking outside of the disease silos we might find that multi-modal interventions such as girls education are more cost effective than they had appeared when only looking at single outcomes.

That said, a focus on cost-effectiveness should not be a shift to another narrow view.  A few issues are outstanding:

  1. Disease burden is important for research agenda setting: the total positive effect of a new cost effective intervention (i.e. one that warrants funding in this super-efficient future) for a disease will be the individual effect multiplied by the current burden.  If we developed a very cost effective method to treat a rare disease then it should be used, but the total effect of the intervention will be lower than had we developed something for higher burden diseases/risks.
  2. The cost effectiveness may be a function of the burden.  If a disease is common then national control programmes can find efficiency in scale, and often harvest low hanging fruit.
  3. Evidence for effectiveness will favour single-disease targeted interventions.  The statistical methods that we use, combined with the protocol and reporting standards that have been set to prevent fishing in the data for results, have made it difficult to show evidence of multi-modal effects.

While this is not a new idea at all, I am still hearing and seeing ranking of disease burden used a lot to motivate healthcare spending.

Leave a comment

Filed under Uncategorized

Two point development index

A few weekends ago my friend Sam, who works with Save the Children, was talking about his experience of working in Yemen.  Sounded like a bit of a nightmare (‘kat’ or ‘chat’ a particular problem) and he said that it really didn’t help that many Yemenis have a perception that they are in a developed country (brought on, he thinks, by the proliferation of smart-phones and other superficial baubles of modernity).  Because of this, they’re not too fussed about changing how they’re living.  In response, he suggested the following two-point development index:

1. What do you eat?

2. Where do you sh*t?

If the answer to the former is ‘nothing but [insert low-nutrient carb]’ and the latter is ‘in a field’, then you’re not living in a developed country.  In fact, when I mentioned this to some Nigerian colleagues they thought that question 2 might be sufficient.

My tongue is only lightly in my cheek in saying this.  In India more than 50% of people defecate in fields, and yet India parades itself as an emerging nation – not developed, but taken seriously on the world stage.  Of course, India also fails on a number of other metrics, but open-defecation is unequivocally bad for people and pretty easily solved.  All nations, and maybe all concerned westerners, should strive to meet my friend Sam’s two point index.  It’s a good place to start.

Leave a comment

Filed under Uncategorized

Data biases in economics

Michael Clemens and Gabriel Demombynes recently published this summary paper of what happened and what they thought about the Millennium Villages Project debacle.  It makes interesting reading, and expands to cover their approach to impact evaluation and its limitations.  Something that is not covered, however, is a discussion about data.

Simon Brooker – an epidemiologist working in Kenya with experience of working with economists on trials – once said to me that ‘epidemiologists care about data, economists care about analysis’.  The more that I read, the more that I think that this is broadly true.  If this dichotomy is the case (and I have not taken a systematic approach to this, so, ironically, my data is very poor) then I am sure that it has emerged from the different human genres that the disciplines have grown up in: epidemiologists dealing with messy things like health, and economists with more precise and less subjective things like money.  Nowadays the disciplines are overlapping.  Economists who are sharp as tacks when it comes to analysis, such as Clemens and Demonbynes, seem conspicuously obtuse on discussions of sampling, measurement bias, recall biases, and missing data.  Such biases, especially in the context of an impact evaluation, can be sufficiently large as to make subsequent fiddling with statistical models all but irrelevant.


Leave a comment

Filed under Uncategorized