First study .282004.29 Lancet surveys of Iraq War casualties

1 first study (2004)

1.1 criticisms , countercriticisms
1.2 lancet publications related criticisms
1.3 other responses criticism

first study (2004)

the survey sponsored center international emergency disaster , refugee studies, johns hopkins bloomberg school of public health, baltimore, md, united states (authors l roberts phd, g burnham md) , department of community medicine, college of medicine, al-mustansiriya university, baghdad, iraq. roberts team chosen experience in estimating total mortality in war zones, example estimate of 1.7 million deaths due war in congo not met widespread acceptance , no challenge when published in 2000, resulted , cited in u.n. security council resolution foreign armies must leave congo, united nations request $140 million in aid, , state department pledging additional $10 million in aid. similar studies have been accepted uncritically estimates of wartime mortality in darfur , bosnia.

roberts regular technique estimate total mortality personal surveys of sample of households in area under study; method being chosen in order avoid under-counting inherent in using reported deaths in areas chaotic many deaths unreported, , include deaths not directly attributable violence nevertheless result of conflict through indirect means, such contamination of water supply or unavailability of medical care. baseline mortality rate calculated interviewees reports period prior conflict subtracted reported during conflict, estimate excess mortality may attributed presence of conflict, directly or indirectly. technique has been accepted uncritically in previous mortality surveys discussed above.

because of impracticality of carrying out evenly distributed survey, particularly during war, roberts surveys use cluster sampling , dividing area number of randomly selected, approximately equally populated regions; random point chosen within each region, , fixed number of households closest point surveyed cluster . while not accurate evenly distributed survey of same number of households, technique more accurate merely surveying 1 household each selected point.

in study of iraq, roberts divided country 33 regions, attempting sample 30 households each cluster, , selecting 988 households, 7868 residents. in september 2004, each surveyed household interviewed household composition, births, , deaths since january, 2002. of 78 households members asked show documentation confirm claims after interview finished, 63 able present death certificates. according authors, 5 (0.5%) of 988 households randomly chosen surveyed refused interviewed.

the relative risk of death due 2003 invasion , occupation estimated comparing mortality in 17.8 months after invasion 14.6 months preceding it. authors stated, making conservative assumptions, think 100,000 excess deaths, or more have happened since 2003 invasion of iraq. among such conservative assumptions exclusion of data fallujah in many of findings. since interpreting results of study complicated inclusion of outlier cluster in fallujah, heavy fighting caused far more casualties elsewhere in iraq, study focused on results excluded fallujah cluster. while authors argued fallujah cluster s inclusion justified normal part of sampling strategy (the authors noted other hotspots najaf had not ended being surveyed), , authors presented 2 sets of results in cases (one set including fallujah data , 1 not), article, , press coverage of article, stresses data excluded fallujah cluster.

the main debate in media in u.s. , uk focused on whether 98,000 (95% ci 8000–194,000) more iraqis died result of coalition intervention, calculated estimate of increased mortality of 1.5 times (95% ci 1.1-2.3) prewar rate (excluding fallujah data). had fallujah sample been included, survey s estimate mortality rates had increased 2.5 times since invasion (with 95% ci 1.6-4.2) including fallujah data have resulted in excess of 298,000 deaths (95% ci ?-?), 200,000 concentrated in 3% of iraq around fallujah (roberts et al. p. 5).

according article, violence responsible of deaths whether or not fallujah data excluded. coalition airstrikes main cause of these violent deaths if fallujah data included. study makes controversial conclusion that: violent deaths widespread, reported in 15 of 33 clusters, , attributed coalition forces. , violence accounted of excess deaths , air strikes coalition forces accounted violent deaths. study estimates risk of death violence in iraq during period after invasion approximately 58 times higher in period before war, ci95 being 8.1-419, meaning there 97.5% chance risk of death violence after invasion @ least 8.1 times higher before. newsday reported:

common causes of death before invasion of iraq heart attacks, strokes , other chronic diseases. however, after invasion, violence recorded primary cause of death , attributed coalition forces—with 95 percent of deaths caused bombs or fire helicopter gunships .

criticisms , countercriticisms

some criticisms have focused on relatively broad 95% confidence intervals (ci95), resulting difficulty , scarcity of reliable sources.

lila guterman, after writing long article in january 2005 in chronicle of higher education, wrote short article in columbia journalism review stated: called ten biostatisticians , mortality experts. not 1 of them took issue study’s methods or conclusions. if anything, scientists told me, authors had been cautious in estimates. quick call statistician, reporters have found probability forms bell curve — likelihood small number of deaths fell @ either extreme of range. fall near middle.

a ministerial statement written 17 november 2004, uk government stated government not accept [the study s] central conclusion , because apparently inconsistent figures published iraq ministry of health, based on figures collected hospitals, said between 5 april 2004 , 5 october 2004, 3,853 civilians killed , 15,517 injured .

some critics have said lancet study authors unable visit randomly selected sample areas. in interview on radio program american life however, authors of study never substituted different, more accessible, areas, , every place randomly selected @ beginning of study surveyed in full, despite risk of death surveyors.

critics of lancet study have pointed out other difficulties in obtaining accurate statistics in war zone. authors of study readily acknowledge point , note problems in paper; example state there can dramatic clustering of deaths in wars many die bombings . said data projections based on of limited precision because quality of information depended on accuracy of household interviews used study.

the results of study politically sensitive, since heavy death toll raise questions regarding humanitarian justifications on eve of contested presidential election. critics objected timing of report, claiming hastily prepared , published despite perceived poor quality in order sway u.s. electorate. on topic, les roberts stated emailed in on sept. 30 under condition came out before election. motive in doing not skew election. motive if came out during campaign, both candidates forced pledge protect civilian lives in iraq. opposed war , still think war bad idea, think our science has transcended our perspectives.

lancet publications related criticisms

november 20, 2004. criticism , suggestions peer reviewer professor sheila bird, mrc biostatistics unit, cambridge cb2 2sr, uk, chair of royal statistical society s working party on performance monitoring in public services. calls scientific method described , readily repeatable , says [p]articular attention needed methodology randomly selecting location(s) of cluster(s) within governorates. roberts , colleagues describe rather succinctly . suggests additional information included more precise multipliers (to obtain final estimate) can applied. discusses example hypothetical circumstance incorporating said information, regarding airstrike deaths , collateral damage, under over-counting occur due population density variances among cluster representations.
march 26, 2005. criticism stephen apfelroth, department of pathology, albert einstein college of medicine. criticizes several questionable sampling techniques should have been more thoroughly examined before publication , lists several flaws, including fatal one, in such situation, multiple random sample points required within each geographic region, not 1 per 739000 individuals.
march 26, 2005. response les roberts et al. apfelroth. acknowledges flaws, says key public-health findings of study robust despite imprecision. these findings include: higher death rate after invasion; 58-fold increase in death violence, making main cause of death; , violent deaths being caused air-strikes coalition forces. whether true death toll 90000 or 150000, these 3 findings give ample guidance towards understanding must happen reduce civilian deaths. ... before publication, article critically reviewed many leading authorities in statistics , public health , suggestions incorporated paper. death toll estimated our study indeed imprecise, , interested in international law , historical records should not content our study. encourage apfelroth , others improve on our efforts. in interim, feel study, other published sample survey know of on subject, point violence coalition forces main cause of death , remind number of iraqi deaths many times higher reported passive surveillance methods or in press accounts.

other responses criticism

the chronicle of higher education wrote article discussing differences in survey s reception in popular press on how received in scientific community.

epidemiologist klim mcpherson writes in march 12, 2005 british medical journal: government rejected survey , estimates unreliable; in part absurdly because statistical extrapolation samples thought invalid. imprecise are, known extent. these unique estimates dispassionate survey conducted in dangerous of epidemiological conditions. hence estimates, far can go, unlikely biased, allowing reinstatement of falluja. confuse imprecision bias unjustified.