Making stuff up

Earlier|Later|Main Page

Micro Statistics Tutorial 01: Lessons from Dave, the forecasting pig

Most (not all) statistics is about prediction. Forecasting is about prediction of future events (usually, but not always, in advance of those events). For example, Carrie Smith Cox, President of Schering-Plough, dumped $28 million worth of stock before anyone knew about negative studies, altered endpoints or anything at all about the drug Vytorin (ezetimibe/simvastatin-combination). That is an example of forecasting. We might however want to predict events in one group of people, knowing what happened in another group.

Here is where we introduce Dave, the Forecasting Pig (Reuters 31 Jan 2008). Dave lives in Ohio. He opines (or oswines) on US economic status, and is a key statistical tool used by the Ohio treasurer's office. Dave decides between a trough of sugar or one of sawdust to gauge the economy's future. Sadly, better methodologies are available (ask Carrie Smith Cox).

Tutorial take home message:
The ingredients of good statistics:

1. A decent, honest, well described set of data
2. Proper definition of terms, specified in advance
3. A clear, well framed and unambiguous question (or problem), specified in advance
4. A plan for examining those data, specified in advance
5. Honest intent

Example of a well framed unambiguous question:
Have you had the measles?
If so, how many?
(Armstrong Ward)

See here for Collated Micro-Statistics Tutorials

Earlier|Later|Main Page

Pharmaceutical Dilbert of the day (4) - Numbers in Medicine

Medicine is about numbers, or at least an evidence-based gut-feel for numbers.

The art of good medicine is based on understanding what is likely to happen to a patient if we do something - or if we do nothing.

Sometimes we gain that understanding from experience. Sometimes understanding is based on statistical findings which go beyond the experience of an individual clinician.

It is infuriating when those who purport to regulate the safety of medicines collude to give the impression that no-information is information. For example, the MHRA (the increasingly beleaguered UK drug regulator) proudly proclaim that

"The MHRA have highlighted that the product information for rosiglitazone (Avandia) has carried warnings about heart failure since 2000 and myocardial infarction since 2006"
and
"Avandia's warning label already warned of the risk of cardiovascular adverse events".

Well that's alright then.

Actually it isn't.

Pharmaceutical companies and doctors must warn ultimate purchasers (patients) of dangers inherent in drugs sold and prescribed. Legitimate lawsuits should be about whether a company took reasonable and timely action to make a drug’s risks known. That means far more than telling consumers that a bad event may occur. Manufacturers do not fulfill their duty to warn by providing meaningless information.

The legal system and regulators regularly misunderstand (or pretend to misunderstand) the numeric basis of medicine. If information is provided without attempting to assign honest estimates of probability (with confidence intervals), that information is essentially without meaning.

Let us suppose for a moment that I am the manufacturer of a motor vehicle. I sell that vehicle with a disclaimer that : "the tires on this vehicle may explode resulting in the death of passengers and bystanders". In a scientific or medical context such information is equivalent to no-information. It is meaningless (without content). It does not inform decision-making in any way.

When the numbers are knowingly obscured, ignored, denied, delayed, disguised or falsified, then medicine stops.

Knowing now what GSK must have known, an appropriate warning for Avandia might have looked something like:

"Reducing blood sugar is not the primary goal of treatment in diabetes. The main goal of treatment in diabetes is to prevent the chance of cardiovascular death. If a drug increases that risk, it can have profound health consequences. There is as yet no evidence to suggest that Avandia reduces the rate of death in diabetes. There is some evidence to suggest that the risk of adverse cardiovascular events death might be increased. The confidence limits for such an increase are from X1 to X2."

The MHRA also helpfully advise that

“patients should not stop treatment with rosiglitazone but should discuss their medication with their doctor at their next routine appointment“.

What exactly should doctors discuss with patients? Given that the MHRA (and FDA) have a history of helping companies hide the information that doctors should be discussing, they might wish to tell us what we should discuss - instead of passing the buck. Alternatively we could turn the whole art and enterprise of medicine over to GSK, the Department of Health, Ian Oulsnam, and an MHRA computer. Why do we as doctors allow ourselves to be placed in the position of having to lie to patients?

See also Pharmagossip.

Earlier|Later|Main Page