Independent Reviewer: ENHANCE Data "Not Acceptable"

This afternoon's WSJ Health Blog provides access to drafts of after-the-fact "minutes" from the November 2007 meeting of independent reviewers of the ENHANCE study data. The draft documents, which include inserted comments from imaging expert James H. Stein, MD, were originally obtained by a Congressional subcommittee, according to the WSJ blog. Many of Stein's comments relate to the integrity and quality of ENHANCE's image database, which he concluded was "not acceptable." The image database resided at Amsterdam's Core Echo Laboratories (CEL), which is directed by ENHANCE's primary investigator John Kastelein, MD, PhD.

A few excerpted comments from Stein:

The imaging and reading protocols in ENHANCE were suboptimal even relative to methodologies employed at the time the study commenced and relative to procedures in clinical and epidemiological studies conducted in the 1990s.

We did not state the data and analysis "are reportable."

When we looked at images and considered the CIMT [carotid intima-media thickness] values of individual subjects over time, almost all the examples we saw showed measurement errors, biologically implausible measurements, biologically implausible changes in CIMT measurements, and/or failure to adhere to the protocol.

Great concern was expressed that the aggregate values may look reasonable, but that they may not reflect reality.

The concerns about unreported and later updated file segments were serious. I recall that I and other panel member[s] specifically stated that we could not determine if the database was "clean and credible" based on the information presented.

It is my recollection that there also were concerns about data management, such as the existence of data files with measurements that either were not entered into the official measurement database or that reflected more "recent" measurements than those in the measurement database. A statement should be inserted here that reflects concerns about the integrity of the data [which] was part of the reason for convening the expert panel. 

It is also revealed in the documents that data for 485 of 640 (~76%) subjects in ENHANCE were missing, and that data for 75 (~12%) subjects were biologically implausible (defined as >0.1-mm CIMT change between baseline and endpoint).* Because the missing data for the common carotid artery were considerably less, either 3% or 8% (depending on which source you take), it was suggested that the primary endpoint might be changed to the CCA measurement.

*The Pathophilia blog has repeatedly expressed its frustration that the volume of missing or implausible data, which would certainly undermine the integrity of and conclusions from the ENHANCE study, had not been revealed.