On swearing versus thuggery: a statistical analysis

A few bloggers commented on the (referenced) use of a bad word (F***) in my previous post (1, 2), and a few more commented below the post itself. I received a large number of comments by E-mail. The post was about suppression of data from Zetia or ezetimibe clinical trials, and the harm to both science and patients that results from such actions. I hypothesized that I would get some very negative responses to the linguistics, but that was not the case (n=13, all either positive or neutral). Sadly, for purposes of this discussion, it was therefore not possible to disprove the null hypothesis of no-offence.

I thought I would use this to discuss two rather different topics of relevance to medicine in general.

1. The statistical relevance of non-occurrence of an event
2. The problem of decorum versus thuggery

The statistical relevance of non-occurrence of an event

In this case there were no negative responses amongst 13 events. This problem has general relevance in medicine. Commonly, in clinical studies there are no observed events in a large number of patients. Forgetting for the moment that study patients might not reflect real-world patients, and that results of studies are often manipulated, it is interesting to examine the statistical meaning of non-occurrence. For example, if there are no deaths following liver biopsy in 200 consecutive patients in a particular hospital, how confident can we be about the mortality rate? If no patients (of 13) die in a Phase I study of an experimental drug, what does this mean? If the public relations officer of a bisphosphonate manufacturer claims that there were no cases of Osteonecrosis of the Jaw (ONJ) in formal clinical trials (n=3000) what are the confidence limits for the rate of ONJ within the population (assuming no-cheating, proper followup, and real-world similarity of trial patients)?

If the true rate of occurrence of an event in the population is R, then the probability P of non-occurrence in n consecutive patients is:

P = (1 - R)ⁿ

We can solve this for n as follows:

n = (log P)/log(1 - R)

This enables calculation of the number of consecutive non-occurrences needed to infer a low rate of occurrence R at a chosen P value. To the naive, the resulting numbers may seem surprising. If there are no deaths in 13 patients in a Phase I trial, then all we can say is that the likely death rate is less than 20% (at P=0.05). To infer a death rate in our Phase I study of not more than 1% we need to observe 299 cases with no death. I wonder whether patients in Phase II trials are provided with such information, either numerically or in spirit?

On the problem of decorum and swearing

I was interested in some of the responses I received. In the words of David Kern:

When you're in an argument with a thug, there are things much more important than civility. I do not like incivility. Yet, I like thugs even less.

My feeling is that our profession cares a great deal more about decorum and a sort of "pseudo-politeness" than we do about actions that are truly immoral, anti-science, and damaging to patients. Patients seem to care rather more about scientific honesty than we imagine. I have written previously of the quotation by the Irish priest Steve Gilhooley about the relation between decorum and thuggery. The quotation was delivered as part of a sermon on El Salvador. Gilhooley spoke passionately from the pulpit:

"I said to them, '70,000 people have been butchered and none of you gave a shit.'"

There was silence. A priest had sworn in the pulpit.

"And the reason I know none of you gave a shit," he continued, "was because none of you fell off your seat when I said '70,000 had been butchered', but nearly all of you fell off your seats when I said 'shit'."

Much of Gilhooley's feelings about the church would apply to the current sad state of medicine. Says Gilhooley [of the church]

"There are those who would rather hide the truth. Those whose priorities have become so skewed that they would protect [sexually] abusive priests before they would protect those who fight for justice and transparency. These are the people who are really in control in the Church. Well, let them get on with it. Let them bury it in the ground, and then we'll all start something else."

The offense caused by words depends on the priority we ascribe to various things. According to the "Hitchhiker's Guide to the Galaxy," the most offensive word in the universe is "Belgium." A journalist in the Melbourne Sunday Age (29.6.2003) provided an eloquent example of the significance of words following Greg Rusedski's Wimbledon soliloquy during his tennis match with Andy Roddick:

"As this is a family paper, both major and minor obscenities have been replaced by the names of birds. The result should be informative and educational."

"I can't do anything if the crowd albatross calls it. Absolutely vulture ridiculous. At least replay the point. Robin ridiculous, falcon ridiculous, budgie ridiculous. Some lesser-crested grebe in the crowd changes the whole match and you allow it to happen. Well done, well done, well done. Absolutely muttonbird..."

Here are some varied educational links:

1. Stupendous BBC/Advertising Standards Authority report on swearing - an impact analysis listing all the bad words (complete with graphs and statistical analysis)
2. Great piece on the evolution of swearing by a Geoffrey Nunberg (a linguist at UC Berkeley)
3. Why we curse. What the F***? by Steven Pinker, Harvard (fantastic, a must read).
4. Clin Psych blog on the Pinker article
5. How stuff works: Swearing
6. Swearing in other languages
7. West Side woman faces jail time for swearing at toilet: Pennsylvania Times-Tribune. Dawn Herb was facing jail for swearing at her overflowing toilet in her own home. She was overheard by an off duty police officer who heard her yell: "Are you f***ing retarded? Get me the f***ing mop." Patrolman Gilman said he then yelled, "Watch your mouth", to which the person replied: "F*** off.". She was later acquitted.
8. College Teacher fired for saying F___ in class: Inside Higher Education
9. Devil's Kitchen: On swearblogging (an excellent, intelligent UK political blog which includes a great deal of swearing).
10. Ben Goldacre: Sweary Mary

Earlier|Later|Main Page

More problems with Ezetimibe (Zetia, Vytorin): Let there be light

Yet more problems with Zetia (ezetimibe) clinical trials are now being reported in the New York Times (see background to the Enhance trial of ezetimibe Ezetimibe (Zetia, Vytorin): the smell of bad science). This has to do with hidden data in other studies of the drug.

Part of the problem here is the redefinition of what we mean by scientific fraud, and a redefinition of science itself. If I (as an individual scientist) were to study 20 rats, and then plot a graph showing only the results from those rats "I liked", or those rats I "considered important enough" then I would be guilty of scientific fraud.

Somehow when the stakes are higher we confuse power with honesty, and the deception is tolerated. It's an oddly familiar pattern of deception.

From 2000 to 2003, Merck and Schering-Plough conducted eight long-term safety studies of Zetia in combination with statins. Only three of those studies were published. Now some questions are being asked about what those studies showed, and why prescribers and patients were not given a full view of the evidence.

A Schering executive, Dr. Robert J. Spiegel, said "the companies had not considered the studies scientifically important enough to publish their findings. Some may eventually be published".

Well fuck you Dr. Spiegel, it's not your decision. When patients might die it's always "scientifically important". You might personally think that other scientists and even patients don't need to know what the evidence is, but it really isn't your call. If you want to sell your product under the banner of science you have to behave like a scientist. Science which cannot be scrutinized is not science at all. If you disable doctors and patients from making rational (and scientific) decisions about the use of your product, then you cannot claim to be a science based-enterprise. There are many other serious problems with suppressing clinical data. Patients in these trials exposed themselves to unknown risks in the interests of all of us. Suppression or distortion of that data is an affront to their dignity as human beings.

Science is, as Carl Sagan put it, a candle in the dark. It shines a light on the world around us and allows us to see beyond our superstitions and fears, beyond our ignorance and delusions, and beyond the magical thinking of our ancestors, who rightfully fought for their survival by fearing and trying to master occult and supernatural powers.

Let there be light.

From other blogs:

1. Brody, Howard (2007-12-21). New Cholesterol Controversies: Regaining Perspective. Hooked: Ethics, Medicine, and Pharma.
2. Aberegg, Scott (2007-12-22). Patients and Physicians should BOYCOTT Zetia and Vytorin: Forcing MRK and SGP to come clean with the data. Medical Evidence Blog.
3. Aberegg, Scott (2007-12-20). Are Merck and Schering-Plough "enhancing" the ENHANCE data?. Medical Evidence Blog.

From the NY Times, By ALEX BERENSON, Published: December 21, 2007:

The discovery of the unpublished research comes as Merck and Schering are already under criticism for not yet releasing data from an important Zetia study, called Enhance, that they completed early last year.

"The unpublished Zetia studies, devised as safety tests, would not prove the drug’s effectiveness. But they would give the public more information about Zetia’s potential risks. All the unpublished studies covered periods at least one year in length and were intended to show whether long-term use of Zetia might pose dangers that short-term use did not.

Most of the studies about Zetia in which Merck and Schering have published the results covered periods of only 12 weeks — not enough time for liver problems to develop in most patients.

The unpublished studies, conducted from 2000 to 2003 according to the F.D.A. documents, were not listed on the industry Web sites where companies are supposed to register the results of all drug trials that were ongoing after October 2002. The New York Times discovered references to the studies in briefing papers on the F.D.A. Web site.

“We keep telling people we want to practice evidence-based medicine, and what we keep finding out is that much of the evidence is obscured,” said Dr. Harlan Krumholz, a cardiologist at Yale, when told about the previously undisclosed studies. “There is important evidence, but it’s not in public view. It’s hidden from investigators.”"

“You don’t want to have data missing,” said Dr. Bruce Psaty, a professor of medicine and epidemiology at the University of Washington. “When there have been adverse effects, when the benefits don’t look impressive, those are the trials that historically don’t make it to press.”

A Schering executive, when asked by a reporter about the unpublished studies, confirmed their existence. But the executive, Dr. Robert J. Spiegel, said the companies had not considered the studies scientifically important enough to publish their findings. Some may eventually be published, he said.

“We’re pretty comfortable that people don’t have trouble tolerating Zetia,” said Dr. Spiegel.

Dr. Eric J. Topol, a cardiologist and director of the Scripps Translational Science Institute in La Jolla, Calif., said that he had asked Merck and Schering more than four years ago to conduct a large, long-term trial to prove that Zetia could reduce heart attacks and strokes. But the companies had little interest, he said.

“They looked at me like I was an alien,” Dr. Topol said.

Dr. Mark Stolk, a gastroenterologist in the Netherlands, last year reported two cases of patients who had developed hepatitis, a liver disease, after taking Zetia alongside Lipitor. One of the patients has since died.

But Dr. Beatrice A. Golomb, an associate professor at the University of California, San Diego, said doctors have lost sight of the purpose of prescribing drugs like Zetia.

The goal of prescribing cholesterol-lowering drugs is not reducing cholesterol, Dr. Golomb said. It is reducing the number of deaths and heart attacks in patients, he said. And without data to prove that Zetia actually reduces heart attacks, doctors cannot be sure they are helping patients when they prescribe the drug, she said.

See post followup: On swearing versus thuggery: a statistical analysis
Earlier|Later|Main Page

A ray of light - well done Dr John P. Kastelein

A note of congratulations to Dr John Kastelein for saying the right thing out loud. I would imagine this is only the first step in the tortuous path that might lead to honest reporting of the ENHANCE trial of ezetimibe. For background see Ezetimibe (Zetia, Vytorin): the smell of bad science.



From the Wall Street Journal

The lead researcher of a long-delayed drug study says he regrets not standing up to Merck & Co. and Schering-Plough Corp. when they first told him last month that they planned to alter the statistical analysis of their jointly sponsored trial.

Under mounting criticism, the companies last week reversed the earlier decision to change the primary measure to evaluate the drug.

John P. Kastelein, a cardiologist at Academic Medical Center, Amsterdam, and principal investigator of the study, said he breathed a 'sigh of relief' when the companies told him last week they were reversing course.

'It's never, ever right to change the primary endpoint of a study,' especially after all the data are in, he says. 'It is statistically not good and it gives the wrong impression to the outside world.' He says he initially went along with the plan but now regrets not firmly resisting it from the outset.

He says the episode was the culmination of a long-running battle over the conduct of the trial and the companies' worries that some deficiencies in the data would jeopardize a good result.

From Health care Renewal: Manipulation of ENHANCE Reversed:

"So this case is unusual, in that an early public outcry resulted in at least some of the clinical research manipulation being reversed. There may be hope yet."

Indeed. The problem is that with so much deception in clinical trials, and with so much at stake it is hard to know when data can be believed. Nevertheless this is a first step. However it still appears that the companies involved had control over the study database in this trial - a crucial problem.

Earlier|Later|Main Page

Christmas gifts for the MHRA and GMC

Quite liked this cover. A new book by John O'Farrell. Gifts for my totally non-corrupted friends at the MHRA (the UK drug regulator) and my totally non-corrupted and non-racist and honest friends at the General Medical Council. All as pure as the driven slush. Merry Christmas to all.

Synopsis
Here at last is a chance to have a good laugh and learn all that stuff you feel you really ought to know by now...you can read how Anglo-Saxon liberals struggled to be positive about immigration; 'Look I think we have to try and respect the religious customs of our new Viking friends - oi, he's nicked my bloody ox!' Discover how England's peculiar class system was established by some snobby French nobles whose posh descendants still have wine cellars and second homes in the Dordogne today. And explore the complex socio-economic reasons why Britain's kings were the first in Europe to be brought to heel; (because the Stuarts were such a useless bunch of untalented, incompetent, arrogant, upper-class thickoes that Parliament didn't have much choice.) A book about then that is also incisive and illuminating about now, "2000 Years of Upper Class Idiots in Charge" is an informative and cantankerous journey through Britain' fascinating and bizarre history. It is as entertaining as a witch burning, and a lot more laughs. Even more laughs than GlaxoSmithKline's study 329 and all those children who died.

Perhaps a copy too for the UK Panel for Research "integrity". I recall an immensely productive and useful meeting with them exactly a year ago.

Earlier|Later|Main Page

Contemptible response of the week - from Health Canada

The prize for the most ludicrous letter of the week goes to Health Canada (the Canadian equivalent of the American FDA or the UK MHRA).

This letter could serve as a model template for all regulators. It is as timely as it is informative. I am sure that the writer is not personally responsible but perhaps she needs to consider the part she plays in the farce. The letter no doubt reflects with some honesty the spirit of the actual procedures used to deal with a serious complaint. Below the letter are details of the complaint (of 2005) that precipitated the response received last week. It is truly amazing how hilarious these "regulatory" bodies are - and how little they care that actual human beings die as a result of their actions.

This letter was the sole response to a formal complaint made by four Canadian health policy researchers about illegal and misleading direct to consumer advertising of the drug Diane-35 (cyproterone and estradiol) during 2005. The complaint mentioned that the advertisements were in contravention of the Food & Drugs Act. Diane-35 increases the risk of venous thromboembolism compared to other similar hormone combinations, and there were at that stage at least six reported deaths in Canada. Diane-35 was the subject of two safety advisories in Canada which warned doctors not to use Diane-35 for two unapproved (off-label) uses: for birth control or mild acne. Diane-35 was only approved for a restricted indication in Canada: treatment of a subgroup of patients with endocrine disorders and very severe acne who have failed to respond to all treatments. Nevertheless the DTC advertisements promoted Diane-35 for unapproved uses, and targeted a vulnerable population: adolescent girls. The ads conveyed scientifically false information. Risks were also not mentioned.

Mild acne was promoted as an indication through the use of images of young girls with flawless skin preening in the mirror and the message to 'Ask your doctor or your dermatologist' at the end of the ad. The addition of dermatologist was a clear hint of the indication. Anyone who had severe acne that had failed to respond to previous treatment would not have the flawless skin of these models, either before or following treatment. There is also no indication for the use of Diane as a contraceptive. Three to 10 extra cases of venous thromboembolism occur per 10,000 women taking Diane-35 per year, as compared to the rate of venous thromboembolism in users of other oral contraceptives. Several other illegal aspects of the advertisements were noted.

Of course it is possible that the "satisfactory" actions taken by Health Canada might have included prison sentences for those implicated (the usual sanction in a sane society when individuals recklessly break laws and endanger real people). Readers might wish to contact Saveen Kamarn at (604)666-8516 or Saveen_Kamarn@hc-sc.gc.ca to find out what happened. As usual my profession remains silent.

Onwards and upwards for honest science.

Earlier|Later|Main Page

Memory Hole (7 December): A case study in ethical medical leadership

Scientific Misconduct Blog Memory Hole: Events of December 7th

9 years ago today: A critical commentary on the role of leadership in medicine

On 7 December 1998 Professor Manuel Buchwald wrote a letter which has become a case study in medical ethics.

In that year Professor Brenda Gallie held the position as leader of the Research Division of Cancer and Blood at the Hospital for Sick Children at the University of Toronto. Her line manager was Professor Manuel Buchwald (OC, PhD FRSC, and Lombard Insurance Chair of Pediatric Research, Hospital for Sick Kids Chief of Research). Buchwald is one of the co-discoverers of the Cystic Fibrosis gene. He is listed in the Canadian Who's Who.

In May 1998 Gallie learned that Dr. Nancy Olivieri, a member of her Division had failed to persuade Buchwald or the hospital administration to assist honestly with a critical ethical crisis involving clinical research funded by the Canadian Pharmaceutical Company Apotex.

Gallie wrote to Buchwald and the hospital President suggesting that the matter required urgent action. They responded by telling her that she "did not know the whole story", and should not become involved. It later became known that Apotex were involved in granting tens of millions of dollars to the University, and that the University were about to attempt to whitewash the embarrassing affair. The funding from Apotex was promised to increase to $55million, and university leaders were encouraged to act according to the wishes of Apotex. As part of a widespread campaign of bullying, the leadership of University of Toronto began to attack a series of academics who had attempted to stand up for honesty in science. On the 7th of December 1998 Buchwald wrote the below letter to Gallie. It is self explanatory. It includes a wonderful series of comments on integrity in medical leadership. Moral duty, our duty to patients and our duty to colleagues are, according to Buchwald, inconsistent with being a leader. Moral integrity is apparently incompatible with the overriding duty of a leader to the institution.


For the complete letter see here - Manuel Buchwald to Brenda Gallie, 7 December 1998

This letter is derived from the morality of Auschwitz. It would seem to me to disqualify someone from being a respected scientist involved in human research. The University of Toronto did nothing to discipline Buchwald. Gallie was appalled, and resigned her leadership position. She moved her research program to the Ontario Cancer Institute.

Great intellect implies neither humanity nor ethical insight. It's all in the ventromedial prefrontal cortex several inches behind the brow. Buchwald, as a younger scientist had held himself as an advocate for human rights.

Earlier|Later|Main Page