The move towards activity-based funding as part of national health reform in Australia will put a spotlight on data quality, as hospitals receive funding based on the number and type of cases they treat. A similar regime is also on the horizon in Canada, where the Canadian Institute for Health Information (CIHI) is charged with ensuring the quality of health care data. We asked CIHI Consultant Heather Richards, to highlight the data quality issues to be faced.

IDM: Can you describe what you do and the objectives of the CIHI?

HR: CIHI coordinates, develops and maintains health information in Canada. Our mandate is to remain neutral and objective and to deliver quality, unbiased information. We are not policy-makers, yet we play an integral role in providing relevant and reliable data and analyses to those who manage health care and formulate health policy. Our data helps others with the effective management of Canada’s health care system and with raising public awareness about factors affecting good health. Most of my time at CIHI has been with the Data Quality department; however, last August I accepted a two-year secondment outside the department to work on an Activity-Based Funding (ABF) project. I was drawn to this work by the recent surge of concerns surrounding increases in health care spending. As Canada moves along the path to ABF, my role is to ensure that 1) our data is of sufficient quality for this use, 2) we develop a case mix system that is appropriate for this use, and 3) the changes we observe in the health care system are ones that are favourable rather than unfavourable. For example, my colleagues and I are currently working on a framework to monitor changes in hospital behaviours - including changes in coding quality. Outside CIHI, I am currently serving a two-year term as Director of Publicity with the International Association for Information and Data Quality. This work, and the exposure it provides me to data quality initiatives in other sectors and in other countries, keeps me firmly grounded and aware that there is still much for me to learn in the information and data quality field.

IDM: There is a wide range of electronic data connected with a patient, whether it is the electronic patient record, medical images or associated admin files. What is the scope of your initiative?

HR: CIHI maintains mostly administrative data files, and most of my experience in understanding data quality with health information is in this context. CIHI currently houses 27 databases that collect very different types of data - including data on health services, health spending, prescription drug use, and health human resources. However, CIHI’s scope also includes work on the electronic medical record (EMR). CIHI led the development of Primary Health Care EMR content standards in collaboration with jurisdictions across Canada, as well as Canada Health Infoway. This standard includes specifications of key concepts and value sets that describe a subset of important data routinely captured in EMRs, albeit much of the current collection is free text. The draft standard includes 106 data elements for concepts such as age, diagnosis and procedure. This project excites me. Increased adoption and uptake of EMR content standards by primary health care providers and jurisdictions will support better patient care and improved primary health care in Canada by making available more complete and comparable data.

IDM: What are some of the challenges CIHI has ensuring data quality within the Canadian health care system?

HR: CIHI is a secondary data collector and this position creates our main challenge. The provincial ministries of health, regional health authorities, and hospital administrators — not CIHI — determine the work environment and resources available to the primary data collectors. As a result, factors which influence the quality of the data, such as its data capture, are outside CIHI’s jurisdiction. This poses a data quality challenge as we cannot directly affect how data is captured and collected. To counteract this limitation, we establish data quality activities that strengthen our data providers’ awareness and understanding of the importance of data quality. We assist them with implementing practices that promote the most accurate data, such as data submission manuals, vendor specifications for data entry, and educational workshops on how to code and abstract data to meet the national standards.

IDM: You are working to understand how reliable and complete the data are in Canada's health information databases, which leads to understanding how much this information can be trusted to make informed decisions in health policy and patient care. Can you give us an indication of the scale of this task and how far you are along the way?

HR: To start, I should point out that we have many partners in Canada who work with us in the health information arena, and each partner has a specific use for our data in terms of making informed decisions in health policy. We have national organizations such as Statistics Canada, provincial organizations such as our ministries of health, professional associations that could be either national or provincial. Also, on a regional or municipal level, there are health authorities, health facilities and private sector organisations. And then there are other groups such as researchers and advocates. Our partners sometimes provide the data and sometimes they use the data - sometimes they do both. The data challenges are many. We must accommodate data providers with different coding standards at provincial and territorial levels versus the CIHI national level. At the same time we need to be aware how differing coding standards can affect the needs of our various stakeholders. It’s really a balancing act in terms of allowing some flexibility to address local interests and limitations, while keeping other aspects of data collection standardized across the country to address provincial and national interests. It is important for decision makers to understand this distinction so that they are not misled when developing health policy.
Many health policy decisions are based on information collected in more than one CIHI database. Hence, we need to ensure that our data is usable across a variety of health sectors. As such, we need to continuously structure our date to allow tracking of patient care across health sectors. Given that we have many databases it can be difficult to integrate the information. We have overcome many challenges and have made significant strides in making our data more usable. For example, we have a corporate data dictionary that standardizes how similar data is captured across databases. We also have a database that enables analysts to identify where data is available for each patient across databases and over time. Right now analysing data across health sectors is done by CIHI staff and is not yet integrated into applications available to our external clients, such as the CIHI Portal.
To answer your question about “how far we are along the way,” all I can say is that we are somewhere past the start line. The health care sector is a changing environment, and CIHI will always need to put effort into making our data more usable and understandable by the decision makers who have the ability to effect positive changes in the health care system and the health of Canadians.

IDM: The Australian government has announced an intention to create a personally controlled electronic health record (PCEHR) for every individual. Is any thing similar underway in Canada? How important is data quality to such initiatives?

HR: Canada Health Infoway (not CIHI) is initiating some work on consumer portals and several jurisdictions have announced plans to begin work in this area. Sunnybrook Hospital in Toronto recently announced their patient portal, giving patients access to their test results and health record. So this movement to create a personally-controlled EMR is happening in Canada also and is relatively new. When collecting any data, its quality is paramount in terms of laying the foundation for good decisions. So, data quality remains important in a personally-controlled EMR and the same principles of data quality should be applied to it. That is, the quality of the information it contains should suit the needs of its stakeholders – whether these be the patient, his or her doctor, or a governing body.

IDM: How advanced is data sharing between medical facilities in Canada, which presumably use a range of different applications, databases, etc?

HR: Your question correctly presumes that there is a range of different applications and databases used by the medical facilities across Canada. It is not CIHI’s role to stipulate which vendor package a hospital uses to collect data, but for most of our data holdings we do specify: the minimum data set to submit, the format in which the data is to be submitted, the edit and consistency checks that this data must pass to be accepted by CIHI, and the deadlines by which the data must be received. Having said that, we work closely with data providers and vendor software companies to create collection processes that are efficient and support quality data collection.
To enable data sharing between facilities, CIHI has developed a tool called the CIHI Portal. The Portal reflects a new way of using and accessing health information. It is a dynamic, web-based environment which allows registered users from Canada’s data-submitting health care organizations - such as hospitals, regional health authorities and ministries of health - to access interactive reports on the delivery of health services at the facility, regional, provincial and national level. A dynamic bundle of content, functionality and features, offer users improved evaluation, stronger decision support and broader knowledge transfer.

IDM: What are some of the initiatives aimed at prevention, early detection and resolution of data quality issues?

HR: Each data holding has a process for preventing, detecting, and resolving data quality issues. While the details may differ, the main components that drive these processes are the same. With respect to prevention, I mentioned earlier that CIHI has submission guidelines that must be adhered to by our data providers. Guidelines include data submission deadlines, data formats and edit and consistency checks. All CIHI data holdings have submission guidelines. When data is received by CIHI, further checks are applied to detect possible data quality issues. This can include record-level edit checks for more sophisticated coding rules, such as those that would ensure adherence to the Canadian Coding Standards for ICD-10-CA (International Classification for Diseases and Health Related Problems, 10th Revision, Canada) and CCI (Canadian Classification for Health Interventions). This can also include aggregate-level checks, such as identifying unusual changes in record volumes submitted over time or checking for unexplained changes in the mix of patients treated by a hospital. Each data holding has its own system for detecting data quality issues. Whenever possible each holding lets the data providers know about any issues so that errors can be remedied and the ‘cleaned’ data can be resubmitted to CIHI. Resolution of data quality issues involves a feedback loop between CIHI and the data providers. This feedback loop can be a formal meeting, such as the National Clinical Administrative Database bi-annual meetings in which provincial and territorial representatives and CIHI staff meet to collaborate on changes needed to improve data usability and data quality. The feedback loop can also be less formal. For example, each program area has Client Services Representatives (CSR). Data providers can contact the CSR if there are data quality issues. CIHI also offers services such as the eQuery tool. Using eQuery, hospital coders can solicit personalized instruction on how to capture data for a specific patient episode. Some data quality issues are managed outside data provider feedback loops. Earlier, I briefly touched on the fact that the provinces often have individual collection needs. As a result, data collected at a provincial level may deviate from CIHI’s national standards. When this happens, we process the data in a way that allows comparisons between provinces. As an example, the Canada-wide roll out of ICD-10-CA and CCI was implemented province by province rather than all at once. During this time CIHI offered tools that allowed analysts and researchers to map the new ICD-10-CA and CCI codes to the older classification systems – enabling analysts to monitor disease prevalence over time or among provinces. However, while we do create tools like this, CIHI’s primary data quality focus remains on working with our data providers to promote consistent data submission.

IDM: What is the role that can be played by standards for data collection, recording and measurement? How prevalent are they in hospitals and general practise?

HR: Data standards are fundamental for any data collection environment with multiple data providers and data collectors. I have mentioned the ICD-10-CA and CCI classifications maintained by CIHI. These standards play an important role for hospitals submitting clinical data. The classification rules maintained in the Canadian Coding Standards provide the framework by which coders understand how to report the clinical data. These standards allow the data abstracted by thousands of coders across the country to be comparableand for measurements on health system performance and the health of Canadians to have the foundation it needs for comparable statistics across the country. While I’ve just highlighted coding standards and their roles, it should be noted that CIHI’s Standards Framework extends beyond coding to include data transmission standards, data standards, information standards, and outside of the data quality realm. It also includes privacy and security standards. The prevalence of these standards in hospitals varies depending on the standard in question. Compliance is high for data standards, such as data providers submitting the minimum data set. Compliance to transmission standards is also high, as most providers adhere to CIHI’s data submission standards. We have seen inconsistencies within the national coding standards. With respect to ICD-10-CA coding in the inpatient setting, reabstraction studies have found differences in the specificity of some health conditions reported to CIHI (e.g.. unspecified stroke vs. ischemic stroke).
We have also found inconsistencies in diagnosis typing for certain conditions; diagnosis typing, indicating if a health condition affected the patient’s stay (e.g. did the diabetes make the treatment of the patient more resource intensive?). The detailed findings from these studies are available in public reports on our web site. The coding quality we are observing in Canada’s inpatient data is consistent with what is seen in other developed countries that have done similar types of studies. ‘The root causes for our coding inconsistencies range from the quality of the physician chart documentation at the hospital, to the education and training of the coders, to the resources and educational offerings provided from CIHI. Unsurprisingly, these are areas in which we continue to focus on developing and promoting.

Heather Richards is a keynote speaker at the Data Quality Asia Pacific Congress 2011 being held on March 28-29 at the Citigate Central Sydney. www.dqasiapacific.com