Kathleen H Pine, PhD1
Lee Anne Landon, RN, CCMC, CCDS2
Claus Bossen, PhD3
ME VanGelder, RN, MEd, CCDS, RHIT2
Abstract
Background: Numbers of clinical documentation integrity specialists (CDIS) and CDI programs have increased rapidly. CDIS review patient records concurrently with patient admissions and visits to ensure that information is accurate, complete and non-ambiguous, and query clinicians when they see opportunities for improving data. The occupation was initially focused on improving data for reimbursement, but rapid changes to clinical coding requirements, technologies and payment systems led to a quickly evolving role for CDI programs and changes in CDIS practice. Objective: This case study seeks to uncover the ongoing innovation and adaptation occurring in a CDI program by tracing the evolution of a single CDI program over time. Method: We present a case study of the CDI program at the HonorHealth hospital system in Arizona. Results: The
HonorHealth CDI program holds a unique hybrid expertise and role within the healthcare organisation that allows it to rapidly adapt to support emergent demands both internal and external to the organisation, such as supporting accurate data collection for the COVID-19 pandemic. Conclusion: CDIS are a vital component in present data-intensive resourcing
efforts. The hybrid expertise of CDIS and capacity for adaption and relationship building has enabled the HonorHealth CDI program to adapt rapidly to meet a growing array of clinical documentation integrity needs, including emergent needs during the COVID-19 pandemic. Implications: The HonorHealth case study can guide other CDI programs in adaptation of the
CDI role and practices in response to changing organisational needs.
Healthcare data: a crucial resource
Multiple related forces, including technological advances, digitisation and demand for data-driven accountability, have given rise to massive “attempts at getting more data, of better quality, on more people” (Hogle, 2016). Scholars have termed this intensified focus on data production and use “data intensive resourcing,” described by Hoeyer as “a dynamic process of creating, collecting, curating and storing data while simultaneously making them available for multiple purposes, including research, governance and economic growth” (Hoeyer, 2016). Scholars studying this phenomenon have argued that data-intensive resourcing will become a dominant force, driving the design, function and governance of healthcare organisations and systems into the future (Hogle,
2016; Hoeyer, 2016). This development places new and increasing demands on healthcare workers. For example, since medical records are the major source of data for extraction,
clinicians face ever-increasing demand for documentation in patient records to accurately reflect the patient’s journey and activity that occurred during their episode of care (Kuhn et al.,
2015), so that other data workers, such as clinical coders and researchers, can code and extract high-quality data. Much research in medical informatics has focused on creating technologies and system improvements to better produce, extract and store healthcare data; examples include natural language processing (Soysal et al., 2018), speech recognition technology (Hodgson et al., 2017), taxonomies of data defects (Zhang and Koru, 2020) and data quality feedback tools (Van der Bij et al., 2017). However, medical informatics has often overlooked the human workforce or
“human infrastructure,” as described by Lee et al. (2006), which must accompany technical tools and infrastructure for data to steward data quality into the data-intensive future. A recent body of work has begun to examine the on-the-ground work required to produce, manage, analyse and deploy data in healthcare (Bjørnstad and Ellingsen, 2019; Bonde et al., 2019; Grisot et al., 2019; Islind et al., 2019; Møller et al., 2020).
One key occupation that has recently expanded rapidly in healthcare and is becoming increasingly important in the endeavour to produce more data of better quality without over-burdening clinicians is that of clinical documentation integrity specialists (CDIS). Clinical documentation integrity (CDI) programs, staffed by CDIS, have increasingly come to play a key role in the United States of America (USA) healthcare organisations and are found in healthcare organisations in other countries as well, such as Australia (Shepheard, 2018). CDI programs themselves have constituted a practice innovation over the past decade that supports data-intensive resourcing as the demand for high-quality EHR and administrative data has intensified (Brazelton et al., 2017). However, the context of clinical documentation is ever-changing as medical evidence, regulatory requirements, financial systems and technological tools change. Thus, the needs of healthcare organisations vis-a-vis clinical documentation integrity are continuously shifting. In this article, we have provided an overview of CDIS professionals and CDI programs as human infrastructure in the
data-intensive resourcing landscape. Further, using the case of an established CDI program in the HonorHealth system in Arizona, USA, we argue that CDI programs and CDIS professionals are in a unique position to continually adapt their clinical documentation integrity activities to support
healthcare organisations in producing healthcare data crucial for data-intensive resourcing.
The emergence and work of clinical documentation integrity specialists
CDIS are a growing part of the healthcare workforce whose work is increasingly essential to producing data namely, claims data (otherwise known as administrative data) that go on to be used for a variety of crucial purposes including billing, quality measurement, quality improvement, and research. CDIS review clinical documentation in near “real time” (e.g. soon after a patient’s outpatient visit or concurrently with a patient’s inpatient stay), looking for ways that the clinical documentation could be improved and querying clinicians when they see opportunity for such improvement. CDIS have credentials and training as clinicians (e.g. registered nurses (RNs), or foreign-trained physicians whose credentials are not recognised in the USA and thus cannot
practice) who worked in clinical practice. The clinical background provides CDIS both an increased capacity to review documentation and (crucially) to develop good relationships with clinicians. Once hired, CDIS receive extensive training in clinical documentation review and clinical
coding (although they do not receive a clinical coding credential). For example, in HonorHealth’s CDIS orientation process, new CDIS spend their first 2 weeks with the CDIS Educator where they are introduced to the CDI workflow and tools including the electronic health record (EHR), the encoder, clinical coding books and coding guidelines. The next 9 weeks are spent with two preceptors, one highly experienced and one less experienced. Uniform education nurtures
and develops a strong foundation that keeps staff engaged in their professional growth and promotes the development of core CDI competencies.
CDI programs have existed since the 1990s (Richard and Graham, 1992). The Association of CDIS (ACDIS) was formed in 2007, and ACDIS introduced certified credentials in 2009. However, clinical documentation integrity programs did not take off in the USA on a larger scale until the national
Department of Health and Human Services authorised the use of International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) for all inpatient medical and clinical coding purposes after 1 October 2015. The transition to the ICD-10-CM in 2015 increased the number of codes from approximately 14,000 to over 70,000, spurring interest in CDI programs (Brazelton et al., 2017). In Australia, interest in CDI programs has increased with the introduction
of activity-based funding in 2011 and a pricing framework focused on quality and patient safety in 2017 (Hay et al., 2020).
CDIS play many roles, including educator, facilitator and relationship builder (Brazelton et al., 2017). As professionals with both clinical and clinical coding expertise, they have both the knowledge and relationships to play a bridging role between clinicians and clinical coders (Shepheard, 2018). The core practice of CDIS is one of “translation” (Pine and Bossen, 2020). CDIS bridge a literal language gap between clinicians and clinical coders, and they also engage in a more abstract translation between the concerns and interests of clinicians on the one hand and the concerns and interests of various stakeholders of standardised administrative datasets that are produced through the process of clinical coding on the other. These translation practices play out through querying, where CDIS review medical charts in near real-time and send various requests for clarification and refinement of the patient’s chart. CDIS know that seemingly small details often seen as inconsequential by clinicians make a large difference in clinical documentation, which is a key contributing factor to improving clinical coding accuracy.
However, requiring clinicians to document in “clinical coding language” would hinder clinical practice and thus patient care. Through querying, CDIS work to get key details into medical documentation without over-burdening clinicians. CDIS also provide education to clinicians, conveying information about clinical coding language and proper documentation in an attempt to shift the clinician’s documentation practices over time and to help clinicians connect the dots between the documentation they produce and its importance as standardised data used for a variety of financial, research and accountability purposes.
The accuracy of coded data is very important. Codes translate into data used for reimbursement of both hospital and physician practices, public health statistics, disease tracking, trending and benchmarking statistics, physician report cards, and other quality reports. The Center for Medicare and Medicaid Service (CMS) gathers quality data from hospitals paid under the Inpatient Prospective Payment System (IPPS). The goal is to drive quality improvement through measurement and transparency (Center for Medicare and Medicaid Service, 2017). Hospital quality measurement sets continue to evolve to meet consumer needs and provide information about clinical effectiveness, health outcomes, patient safety and patient experience related to care (Agency for Healthcare Research and Quality, 2016). Results are tallied and posted on a public scorecard, which reports how well hospitals provide recommended care to their patients in the form of star ratings. The ratings range from one to five stars; the more stars, the better a hospital performed on quality measures. The data collected through the IPPS program are available to consumers and providers on the Hospital Compare website. Data from selected measures are also used to either pay a bonus or impute a penalty based on the quality and efficiency of care, adjusting payments made to hospitals based on performance and quality of care delivered to patients; such programs include the Hospital Value-Based Purchasing Program, Hospital-Acquired
Condition Reduction Program and Hospital Readmissions Reduction Program (Center for Medicare and Medicaid Service, 2017). It is intended to encourage hospitals and clinicians to improve the quality and cost of inpatient care provided to all patients. Other quality reports include
Healthgrades (Healthgrades, 2021) and Care Compare (Medicare, 2021), which are publicly displayed to help consumers make more informed decisions concerning their healthcare.
The value of the CDI program is that it benefits both hospitals and providers by improving their coded data for comparisons, benchmarking and quality reports; it also improves coded data for crucial local and large-scale research conducted using administrative datasets. Clinical coding directives change constantly. Thus, CDI programs have to stay abreast of constant changes to clinical coding conventions and guidelines, trends in denial of payment by healthcare payers and so forth. Below, we provide one example of how a CDI program may successfully meet the challenges of building up expertise for an expanding scope of work within a rapidly changing context.
The clinical documentation integrity program at HonorHealth
Early beginnings
The present HonorHealth network in Arizona, USA, comprises six hospitals, approximately 10,500 employees and 3700 affiliated doctors; this is the result of a merger in 2014 of two existing healthcare networks. Both pre-existing networks employed CDIS and had their own CDI programs. The earliest iteration of the current CDI program began at Scottsdale Health’s campuses in 2003. This innovative program was born of the initiative of an employee who was in a master’s degree program in health information and learned about clinical documentation improvement as part of their program. In 2010, another hospital system John C. Lincoln Medical System developed their own CDI program in collaboration with a software company who helped develop tools for chart review. A few years later, the two healthcare systems merged, forming the present day HonorHealth. In 2016, the CDI programs at the different campuses merged, creating a unified program with integrated management, structure and processes.
A period of rapid change: Implementation of All Patient Refined Diagnosis Related Groups and
international classification of diseases, tenth revision, clinical modification
Prior to the merger in 2015, both programs gradually changed their focus from Medicare patients and identifying comorbidities (CCs) and major comorbidities (MCCs) to accurately capture a patient’s condition for all insurance payers, leading to improved clinical coding and accurate reimbursement. This focus has remained relatively constant since the inception of the two programs. However, starting in 2013, a variety of rapid changes in healthcare accountability,
payment models and information technology led to a stream of rapid shifts in CDI practice at HonorHealth as the program adapted to support the changing needs of the organisation. In
2013, the state of Arizona adopted All Patient Refined Diagnosis Related Groups (APR-DRG) for Medicaid reimbursement, and the CDI program began reviewing Medicaid patient charts and working to improve accuracy of APRDRGs, which entailed addressing severity of illness (SOI)
and risk of mortality (ROM) scores for patients. Assessing SOI and ROM entails more detailed levels of review and education of physicians because accurately capturing SOI and ROM is more complex than documenting the singular conditions that are CCs and MCCs, which had been the major focus until then.With the introduction of SOI and ROM came increased awareness of and focus on quality measurement as a key priority for healthcare documentation. Where the CDI
program had been solely concerned with improving documentation in service of financial reimbursement, as quality measurements started to take on increased importance in the USA, a new concern entered CDIS practice that of improving the integrity of documentation in service of producing accurate quality measurements to preserve institutional and public legitimacy.
The implementation of ICD-10-CM in 2015 provided a watershed moment for the HonorHealth CDI program. The language and expansion of the code set was so large that HonorHealth found themselves increasingly reliant on CDIS to help them make sense of the massive changes to clinical coding and bridge the widening gap between clinical documentation and clinical coding language. Implementation of ICD-10-CM at HonorHealth led to rapid growth in the CDI program, from 16 staff in 2015 to 26 in 2020. This period also led to increased organisational support for the CDI program from the healthcare network. Prior to implementation of ICD-10-CM, clinicians often overlooked the CDI program and reacted negatively to scrutiny of their documentation. With
ICD-10-CM and the move to value-based care (VBC) reimbursement schemes, the increasing complexity of clinical coding led to increased buy-in and support for the CDI program. Under ICD-9-CM, clinicians could document in very general terms and still get reimbursed; not so under ICD-10-CM and new DRG-based VBC payment models. Clinicians began to see CDI as a necessary step that protected their time and helped them manage rapidly changing documentation requirements (Pine and Bossen, 2020), rather than an annoyance that took time away from other more important tasks. Through rapidly adapting to the ICD-10-CM and the shift to value-based payment models, the program was able to expand and deepen relationships with clinicians
and develop a culture of shared responsibility for documentation quality in which CDIS, clinical coders and clinicians are all partners in documentation quality (see Figure 1).
Another key change that occurred in 2016 was adoption of single EHR and CDI data software systems across the network. The CDI data software system is a suite of applications that support a number of clinical documentation integrity functions. It includes artificial intelligence (AI) capabilities in the form of a computer-assisted clinical coding (CAC) tool (also referred to as an encoder) that searches clinical documentation and suggests codes. This software also provides
clinical coding guidelines, a platform for sending queries, and tools for tracking the impact of CDI and queries, among other capabilities.
Ongoing adaptation: Innovations in the clinical documentation integrity program
Today, the CDI program covers six hospital facilities and has a staff of 27 CDIS, including a manager, a CDI analytic lead and CDI educator, all of whom have backgrounds as RNs. The program’s focus is to identify and clarify ambiguous, conflicting or incomplete documentation. The CDI staff facilitates the overall quality and completeness of clinical documentation along several dimensions: acuity, severity of illness, risk of mortality, quality profiles and resource utilisation of patients. Focused communication with the clinical provider (via the query process) and ongoing education are utilised to improve the integrity of clinical documentation. The CDI program has developed a number of unique innovations in response to emergent needs within the HonorHealth system. Thus, although the initial mission of the CDI programs developed in the different hospitals remains, the current CDI program is much larger, broadly networked within the organisation and provides a much wider variety of services than the program that first developed in the Scottsdale hospital in 2003, or even the combined program that emerged in the new HonorHealth system in 2016.
In 2017, as the CDI program was rapidly growing and working to support rapid changes to clinical coding and value-based reimbursement, the network manager of CDI (second author) had more management tasks falling to her than she could reasonably meet. During a leadership exchange, she learned of the idea to develop special working groups that CDIS would join, thus spreading out responsibility and expertise among the entire CDI team. Thus, the program developed a mission statement and five working groups: Preceptor & Orientation (overseeing training and mentorship of new CDIS), Quality & Patient Safety Indicators (collaborating with quality personnel to optimise
documentation for quality measurement and patient safety), Mortality (perform second-level review to assure accurate capture SOI and), Denial management (review DRG downgrade denials from payers and provide the clinical indicators to support the diagnosis and overturn the denial) and Query (reviewing and improving the standard query processes). Thus, in addition to their daily work with optimising the integrity of data in the individual patient records, all CDIS in the program now take part in at least one work group. Each work group has the responsibility to develop a level of expertise related to their specific realm. With this additional knowledge, they review and improve procedures related to group’s specific activities, along with acting as a resource for their peers. Further, the work groups provide a flexible model through which the CDI program can address ongoing changing needs of the organisation and provide an avenue for leadership development.
Another innovation is the development of a new role for a data analyst who has a background in data analytics and management along with CDIS experience that allows her to work with data in unique ways. For example, she detects patterns in the data such as a variation in DRG and is able to skillfully audit to confirm variances in the data. It is this ongoing development of hybrid skill sets that allows the CDI program to build new bridges and perform crucial functions at the intersection of clinical practice, documentation, billing, quality and more.
The CDI program has steadily expanded its reach within the hospital system by working with new stakeholder groups. At the genesis of the program, CDI largely interfaced with clinical coding, financial and medical records departments. Presently, the CDI program interfaces regularly with the readmissions program, service lines, quality and multiple specific departments (e.g. dietary, wound care, and outpatient) (see Figure 2). For example, both the Quality and CDI Department review the record for appropriate documentation regarding clinical conditions that are exclusions for specific programs, code assignment, and “present on admission” status. The Utilisation Management Department uses the “working concurrent DRG” (the DRG that is assigned during
each clinical documentation integrity review), which helps to guide their length of stay criteria. CDI also provides education to various departments regarding the impact of their documentation on data collection and projected healthcare outcomes. One of the key roles played by the CDI program is to monitor changes to clinical coding guidelines and help translate these changes to providers throughout the hospital system. Such changes are so rapid that it would be impossible for many clinicians to keep up; the CDI program eases the burden on clinicians by monitoring changes on behalf of clinicians, acting as a trusted arbiter of new and upcoming documentation requirements. The ongoing expansion and broadening of relationships and roles require ongoing relationship building by the CDI program.
The ongoing COVID-19 pandemic provided just such a rapidly changing context. Clinical coding guidelines and high-quality clinical documentation for COVID-19 are crucial to produce data about COVID-19 used for epidemiology, public health guidance and retrospective research on diagnoses and treatment. Since COVID-19 was a novel disease, clinical coding guidelines changed at a rapid-fire pace, much faster than highly impacted clinicians could keep up with. Thus, the CDI program at HonorHealth has been a crucial part of the COVID-19 response effort, responsible for monitoring clinical coding guidelines and providing rapid turnaround education on the rules for COVID-19 documentation. For example, CDIS are educating clinicians and departments about how to capture comorbid conditions and what are the criteria for a “confirmed” COVID-19 case, both of
which are moving targets and require constant updating; for example, initially only specific COVID-19 tests could be used to document a positive case, and now certain local tests can be used. Following this rapid progression and translating this information to clinicians without over- burdening them is the delicate balance that the CDI program has learned to strike.
Lessons learned
Successfully growing from 16 to 27 staff within 5 years, while at the same time expanding the scope of tasks has of course involved a number of challenges. To onboard and train a sufficient number of CDIS to a high level of clinical coding competence, at a time where there was a deficit of experienced CDIS, it was necessary to develop a learning program to teach RNs to become CDIS by teaching them the required clinical coding guidelines, how to query and build rapport with physicians, understanding the concerns of clinical coders, etc. The orientation process begins with the new hire spending 2 weeks with the CDI educator to learn the basics of CDI. The new hire then transitions to working side by side with a CDIS preceptor. During this time, they delve further into the CDI processes and are shadowed while performing their reviews to allow for immediate feedback and continued learning. At the same time, experienced CDIS had to learn to become trainers for their new colleagues, which later was organisationally stabilised by the formation of the “Preceptor and education group.”
As the demand for clinical coding specificity increased and more queries were made to physicians, it became important to build rapport with, and buy-in from, physicians. Physicians often see CDI programs and queries as taking time from patients, and it is crucial to get physicians not to “build a
wall” between them and CDIS, but see themselves as within the same walls (Lo, 2014). This was achieved by regularly joining physician meetings and showing how specific wording in documentation made a difference for clinical coding and hence quality indicators and billing. By the same logic, it was important not to have a wall between CDIS and clinical coders. CDIS have clinical knowledge and insight that clinical coders may not, whereas clinical coders have insights into the clinical coding rules and regulation, along with legal requirements of clinical coding that CDIS may lack, and hence the two groups might not always understand why their clinical coding differs. To avoid misunderstandings and facilitate cooperation, CDIS have been paired with a “clinical coding buddy,” whom they can ask for clarification and get insight into the clinical coders’
perspective, and joint monthly meetings are held with both CDIS and clinical coders. The work group structure has provided an important means through which to manage the increasing scope of tasks, improve CDIS processes, and handle the CDI program’s work portfolio in a structured way. At the same time, the work group structure builds up individual CDIS’ expertise and knowledge, as participation in the groups facilitates learning between CDIS. For example, the
denials work group builds knowledge about how insurance billing works that CDIS then use to inform CDI practice.
As the program expanded and required more resources, it had to prove its worth to management. This was facilitated by the data analyst and 3M 360 software that helps to track how queries improve clinical coding vis-a-vis billing and quality indicators. However, CDI program staff have learned to be careful about the metrics they use to assess their work and track success. Over time, they have shifted away from looking at the speed of chart review as a metric of success and instead focused on quality of documentation integrity. Relatedly, the CDI program has developed procedures for building skill with working with the CAC system within 3M 360. The CAC can speed up chart review for experienced CDIS by suggesting codes, but the code suggestions require careful scrutiny from CDIS. CDIS are trained without the CAC at first to build a knowledge base, and only add the CAC into their chart review work practices later. Thus, skillful CDI work involves careful balance between efficiency and quality, with quality of chart review often taking precedence over speed.
Conclusion
Overall, the success of the CDI program at HonorHealth can be attributed to its ability to continuously adapt to external demands, monitor the external environment for major shifts that impact documentation, leverage a unique hybrid skill set (clinical documentation integrity and clinical coding), build new relationships across the organisation as the range of documentation stakeholders expands and build an organisational structure that facilitates the management of a broad scope of tasks while at the same time supporting learning.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship,
and/or publication of this article.
References
Agency for Healthcare Research and Quality (2016) Measuring the quality of hospital care. Available at: https://www.ahrq.gov/talkingquality/measures/setting/hospitals/index.html (accessed
17 November 2021).
Bjørnstad C and Ellingsen G (2019) Data work: A condition for integrations in health care. Health Informatics Journal 25: 526–535.
Bonde M, Bossen C and Danholt P (2019) Data-work and friction: investigating the practices of repurposing healthcare data. Health Informatics Journal 25: 558–566.
Brazelton NC, Knuckles MC and Lyons AM (2017) Clinical documentation improvement and nursing informatics. CIN: Computers, Informatics, Nursing 35(6): 271–277.
Center for Medicare and Medicaid Service (2017) Hospital inpatient quality reporting program. Available at: https://www.cms.gov/ Medicare/Quality-Initiatives-Patient-Assessment-Instruments/
HospitalQualityInits/HospitalRHQDAPU (accessed 17 November 2021).
Grisot M, Moltubakk Kempton A, Hagen L, et al. (2019) Data-work for personalized care: examining nurses’ practices in remote monitoring of chronic patients. Health Informatics Journal 25: 608–616.
Hay P, Wilton K, Barker J, et al. (2020) The importance of clinical documentation improvement for Australian hospitals. Health Information Management Journal 49: 69–73.
Healthgrades (2021) Front Page. Available at: www.healthgrades. com (accessed 17 November 2021).
Hodgson T, Magrabi F and Coiera E (2017) Efficiency and safety of speech recognition for documentation in the electronic health record. Journal of the American Medical Informatics Association 24: 1127–1133.
Hoeyer K (2016) Denmark at a Crossroad? Intensified Data Sourcing in a Research Radical Country. Berlin/Heidelberg, Germany: The Ethics of Biomedical Big Data Springer, pp. 73–93.
Hogle LF (2016) Data-intensive resourcing in healthcare. Bio-Societies 11: 372–393.
Islind AS, Lindroth T, Lundin J, et al. (2019) Shift in translations: Data work with patient-generated health data in clinical practice health informatics, The Journal of the South Carolina Medical Association 25: 577–586.
Kuhn T, Basch P, Barr M, et al. (2015) Clinical documentation in the 21st century: executive summary of a policy position paper from the American college of physicians. Annals of Internal
Medicine 162: 301–303.
Lee CP, Dourish P and Mark G (2006) The human infrastructure of cyberinfrastructure. In: Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work,
Banff Alberta Canada, November 4–8, 2006, pp. 483–492.
Lo W (2014) Document like this, not that: CDI insights from the physician and CDI specialist perspective. Journal of Ahima 85: 36–40.
Medicare (2021) Medicare care compare. Available at: https://www.medicare.gov/care-compare/ (accessed 17 November 2021).
Møller NH, Bossen C, Pine KH, et al. (2020) Who does the work of data? Interactions 27: 52–55.
Pine Kathleen H and Bossen Claus (2020) Good organizational reasons for bettermedical records: The datawork of clinical documentation integrity specialists. Big Data & Society 7(2): 2053951720965616doi: https://doi.org/10.1177/2053951720965616.
Richard E and Graham D (1992) The emerging role of the clinical reimbursement coordinator. Journal of Ahima 63(62–64):66–70.
Shepheard J (2018) What do we really want from clinical documentation improvement programs? Health Information Management Journal 47(1): 3–5.
Soysal E, Wang J, Jiang M, et al. (2018) CLAMP–a toolkit for efficiently building customized clinical natural language processing pipelines. Journal of the American Medical Informatics Association 25: 331–336.
van der Bij S, Khan N, Ten Veen P, et al. (2017) Improving the quality of EHR recording in primary care: a data quality feedback tool. Journal of the American Medical Informatics Association 24: 81–87.
Zhang Yand Koru G (2020) Understanding and detecting defects in healthcare administration data: Toward higher data quality to better support healthcare operations and decisions. Journal of the American Medical Informatics Association 27: 386–395.