ulation health management, in particular,
has not gone unnoticed. 4
Verification, the fifth ‘V’ in that list,
requires some additional scrutiny. As millions of health-related records are generated every day from diverse sources – EHR,
claims, billing, appointment and customer
relationship management systems, just to
name a few – the trustworthiness of the
data held within these systems has been
called into question, 5, 6 and a standard
for the verification of the quality of these
sources has yet to emerge. Despite no lack
of attention to the challenge of identifying
data quality gaps, data quality analysis
remains an ad hoc process, subject to local
pressures and tendencies.
Even defining a data quality gap properly
presents an analytic challenge. 7 All of the
following qualify as data quality issues one
might encounter in an EHR:
Logical Observation Identifiers Names
and Codes (LOINC) codes, derailing bulk
analysis.
All of these cases involve very different
causes and data sets, and result in different types of gaps. Some arise as a result of
standard reporting configurations that fail
to transmit crucial information. Others are
the result of clinical practices, which may
stem from EHR configuration, organizational workflow or even user personalities. 8
Regardless of the cause, concerns about
the quality of healthcare data generated in
the clinical environment threaten to derail
efforts to derive organizational and public
value from healthcare data sets. 6
The diversity of quality gaps has made
the challenge of addressing them ever more
complex, splintering these issues across
different areas of the clinical workflow (i.e.
who enters the data) and different components of the healthcare IT environment (i.e.
which system captures the data). There
have been a substantial number of efforts
to quantify data quality in EHR systems, 7
often in the context of a particular need, 9, 10
and typically with a focus on correctness
and completeness of data entry. 11
While errors and gaps in data entry
are demonstrably harmful to the mission
of population health management, 12, 13 the
attention paid to the capture of information may have come at a cost to consideration of data quality degradation following
entry. The effects of clinical workflows and
system configuration can have an equally
deleterious effect on the fitness for use of an
EHR system, to paraphrase a classic definition of quality. 14 This will likely become
especially critical for large organizations
with multiple, diverse systems and spread
across numerous organizational entities
with different habits and cultures.
To address the challenge of quantifying
and qualifying EHR data quality gaps, we
developed a model capable of identifying a
wide range of data quality gaps and assigning specific impacts based on independent
factors. Here, we describe that case study
exploring data quality gaps, as well as the
model that emerged from that process.
We also describe how these results can be
applied in a practical setting to create targeted and effective plans for data quality
improvement and discuss the impacts of
such improvements on clinical and operational effectiveness.
CASE STUDY
Beth Israel Deaconess Care Organization
(BIDCO), a physician and hospital network
with over 2,000 providers in the Northeast United States, sought assistance from
Arcadia Healthcare Solutions in identifying sources of flaws in an existing clinical
measure reporting infrastructure.
To assist BIDCO with its reporting
requirements, each practice EHR system
transmitted a data feed nightly to a third-
party analytic tool, which would calculate
a set of quality measures for reporting.
These data feeds were formatted accord-
ing to vendor-defined Coordination of Care
Document (CCD) Specifications v.C3216,
which were the only source of clinical data
for the analytic tool. BIDCO analysts had
identified inconsistencies in the measures
reported and, due to the number of systems
involved in data storage, transmission and
analysis, were unable to identify the source
of the data quality issues.
For the purposes of this study, Arcadia
consultants and BIDCO subject matter
experts (SME) examined four practices
with a total of 5,800 patients served by 50
providers. Of those 50 providers, all use the
eClinical Works (eCW) Electronic Medical
Record platform. Although model parameters were chosen to fit the configuration of
the eCW platform used at these practices,
our consultants report similar types of
quality issues with other platforms, as well.
DATA QUALITY ANALYSIS
Data quality can mean many things, ranging from predictable data encoding errors
to complete corruption or even absence of
data. In their review of EHR Data Quality
Assessments, Weiskopf and Weng7 proposed a mapping between dimensions of
data quality – How do we define the function of quality? – and methods of assessing
data quality – What criteria do we test data
against? This case study focused on processes that would cause reported measures
to fail to correspond with the captured data.
Therefore, we were primarily focused on
dimensions of Concordance, Plausibility
and Currency: Does the EHR offer a valid,
plausible and relevant representation of
patient state at the time of reporting?
QUALITY MEASURES
For the purposes of studying potential data
quality gaps, we tested existing EHR data
against a subset of ACO quality measures
issued by CMS15 (Table 1). These standards
are nationally recognized clinical indicators, with corresponding indicators in
other standards sets, such as the Health
Effectiveness Data and Information Set
