Development of Evidence Based Diagnostic Algorithm for Primary Ciliary Dyskinesia
Date Issued
June 2017
Author(s)
Advisor
Abstract
Introduction
Primary Ciliary Dyskinesia (PCD) is a rare, genetically heterogeneous disorder which results from the dysfunction of small hair-like organelles, called motile cilia. Motile cilia project from the apical side of epithelial cells that line up the upper and lower airways (respiratory cilia) and can be found in a variety of other tissues. PCD patients usually suffer from recurrent respiratory infections which lead to chronic destructive airway disease characterized by progressive loss of lung function and structural damage of the airways (bronchiectasis).
Despite the fact that many of the manifestations of PCD present early in life, diagnosis is often delayed or missed completely, primarily due to the low specificity of some symptoms (e.g. cough, rhinorrhea), lack of awareness for PCD among clinicians and difficulties in the availability and interpretation of specialised diagnostic testing. Diagnostic testing for PCD usually involves at least three laboratory procedures: (a) nasal Nitric Oxide measurement, (b) assessment of ciliary motility and (c) examination of ciliary ultrastructure. Diagnostic testing for PCD is laborious and time consuming and many centers may lack access to necessary equipment or expertise to perform all required tests. As a result, different diagnostic algorithms for PCD diagnosis may be followed by different centers and this phenomenon is further influenced by the lack of knowledge regarding the diagnostic effectiveness and average cost of each test.
Aims
Towards further illuminating the decision making process for the establishment of the most efficacious diagnostic algorithm for PCD, we aimed first to characterize the diagnostic properties of the three main tests for PCD (nNO, TEM and HSVM) and second to evaluate different diagnostic algorithms in terms of overall health benefits for PCD patients and overall costs to the healthcare systems.
Methods
In separate systematic reviews and meta-analyses, all major electronic databases were searched from inception until 2016 using appropriate terms towards identifying eligible studies that reported estimates of diagnostic accuracy for TEM, nNO and HSVM as well as estimates of the prevalence of PCD in consecutive referrals of suspect cases. Eligible studies included diagnostic information on PCD patients or PCD referrals that underwent a combination of diagnostic tests which included nNO, TEM, HSVM and genetic testing.
For the meta-analysis of nNO diagnostic accuracy, estimates of sensitivity and specificity of nNO measurement was calculated for each included study and a two-level mixed logistic regression model conditional on the sensitivity and the specificity of each study and a bivariate normal model for the sensitivity and specificity between studies were fitted. Summary receiver operating characteristic (HSROC) curves were drawn using the parameters of the fitted models separately depending on the breathing technique, Vellum Closure (VC) or non-Vellum Closure (non-VC), used for nNO measurement. For the meta-analysis of PCD prevalence in consecutive referrals of suspect cases and for the meta-analysis of TEM detection rate, a meta-analysis of proportions using a random effects model was performed. Meta-analysis of proportions allows the calculation of the pooled proportion across studies containing binomial data while random effects allow for each study to be assigned a weight which includes the within study variance and the between studies variance. Heterogeneity was assessed with the I2 which describes the proportion of total variation in the effect estimate that results from the between-studies heterogeneity and ranges from 0 to 100%.
The evidence regarding the diagnostic properties of nNO and TEM as well as the evidence regarding the prevalence of PCD among suspect patients were combined along with diagnostic accuracy estimates for HSVM from individual studies to develop a probabilistic decision model that allowed the calculation of net sensitivity and specificity as well as the cost-effectiveness (CE) and incremental cost effectiveness for three diagnostic algorithms that were characterized by different combinations of nNO, TEM and HSVM. The evaluated combinations were (a) nNO+TEM in sequence, (b) nNO+HSVM in sequence and (c) nNO/HSVM in parallel followed, in cases with conflicting results, by confirmatory TEM (nNO/HSVM+TEM) and the model followed a hypothetical initial population of 1000 referrals (expected 320 PCD patients). Number of PCD patients identified, CE and ICE ratios were calculated using Monte Carlo analysis in ANALYTICA.
Results
PCD prevalence among referrals was 32% (95% CI: 25–39%, I2 = 92%). TEM detection rate among PCD patients was 83% (95% CI: 75–90%, I2 = 90%). Exclusion of studies reporting isolated inner dynein arm defects as PCD, reduced TEM detection rate and explained an important fraction of observed heterogeneity (74%, 95% CI: 66–83%, I2 = 66%).
The overall sensitivity of nNO measured by VC techniques was 0.95 (95 % CI 0.91–0.97), while specificity was 0.94 (95 % CI 0.88–0.97). The positive likelihood ratio (LR+) of the test was 15.8 (95 % CI 8.1–30.6), whereas the negative likelihood ratio (LR-) was 0.06 (95 % CI 0.04–0.09). For non-VC techniques, the overall sensitivity of nNO measurement was 0.93 (95 % CI 0.89–0.96) whereas specificity was 0.95 (95 % CI 0.82–0.99). The LR+ of the test was 18.5 (95 % CI 4.6–73.8) whereas the LR- was 0.07 (95 % CI 0.04–0.12).
Regarding the probabilistic decision analysis model, out of 320 PCD patients, 311 were identified by nNO/HSVM+TEM, 274 with nNO+HSVM and 198 with nNO+TEM. The nNO/HSVM+TEM had the higher mean cost (€97K) followed by nNO+TEM (€56K) and nNO+HSVM (€39K). The nNO+HSVM algorithm dominated the nNO+TEM algorithm (less costly and more effective). The ICE ratio for nNO/HSVM+EM was €1600 per additional PCD patient identified.
Conclusions
Many centers for the diagnosis and treatment of PCD in the developed world follow different tests and a variety of algorithms for diagnosing PCD. In some low income countries, most likely, there is a complete lack of specialized diagnostic testing. The results of this PhD thesis suggest that diagnostic accuracy of nNO measurement both with VC and non-VC maneuvers is high and can be effectively employed in the clinical setting to detect PCD even in young children, thus potentiating early diagnosis. On the contrary, a significant percentage, at least as high as 26%, is missed by TEM and this limitation that should be accounted toward the development of an efficacious PCD diagnostic algorithm. The results of decision analysis approach employed in this study also suggest that a diagnostic algorithm which includes nNO during VC as a screening test followed by confirmatory HSVM identifies approximately 86% of PCD patients with a mean CER of 140€ per PCD case identified. The algorithm which maximizes the number of PCD patients identified involves parallel performance of nNO and HSVM as the first step, followed by TEM as a confirmatory test for the few cases where nNO and HSVM yield conflicting results, with a corresponding ICER of 1620€ per additional PCD patient identified. These findings can inform the dialogue about the development of evidence-based guidelines for PCD diagnostic testing and can illuminate discussions about how these guidelines can best be implemented across various healthcare systems.
Primary Ciliary Dyskinesia (PCD) is a rare, genetically heterogeneous disorder which results from the dysfunction of small hair-like organelles, called motile cilia. Motile cilia project from the apical side of epithelial cells that line up the upper and lower airways (respiratory cilia) and can be found in a variety of other tissues. PCD patients usually suffer from recurrent respiratory infections which lead to chronic destructive airway disease characterized by progressive loss of lung function and structural damage of the airways (bronchiectasis).
Despite the fact that many of the manifestations of PCD present early in life, diagnosis is often delayed or missed completely, primarily due to the low specificity of some symptoms (e.g. cough, rhinorrhea), lack of awareness for PCD among clinicians and difficulties in the availability and interpretation of specialised diagnostic testing. Diagnostic testing for PCD usually involves at least three laboratory procedures: (a) nasal Nitric Oxide measurement, (b) assessment of ciliary motility and (c) examination of ciliary ultrastructure. Diagnostic testing for PCD is laborious and time consuming and many centers may lack access to necessary equipment or expertise to perform all required tests. As a result, different diagnostic algorithms for PCD diagnosis may be followed by different centers and this phenomenon is further influenced by the lack of knowledge regarding the diagnostic effectiveness and average cost of each test.
Aims
Towards further illuminating the decision making process for the establishment of the most efficacious diagnostic algorithm for PCD, we aimed first to characterize the diagnostic properties of the three main tests for PCD (nNO, TEM and HSVM) and second to evaluate different diagnostic algorithms in terms of overall health benefits for PCD patients and overall costs to the healthcare systems.
Methods
In separate systematic reviews and meta-analyses, all major electronic databases were searched from inception until 2016 using appropriate terms towards identifying eligible studies that reported estimates of diagnostic accuracy for TEM, nNO and HSVM as well as estimates of the prevalence of PCD in consecutive referrals of suspect cases. Eligible studies included diagnostic information on PCD patients or PCD referrals that underwent a combination of diagnostic tests which included nNO, TEM, HSVM and genetic testing.
For the meta-analysis of nNO diagnostic accuracy, estimates of sensitivity and specificity of nNO measurement was calculated for each included study and a two-level mixed logistic regression model conditional on the sensitivity and the specificity of each study and a bivariate normal model for the sensitivity and specificity between studies were fitted. Summary receiver operating characteristic (HSROC) curves were drawn using the parameters of the fitted models separately depending on the breathing technique, Vellum Closure (VC) or non-Vellum Closure (non-VC), used for nNO measurement. For the meta-analysis of PCD prevalence in consecutive referrals of suspect cases and for the meta-analysis of TEM detection rate, a meta-analysis of proportions using a random effects model was performed. Meta-analysis of proportions allows the calculation of the pooled proportion across studies containing binomial data while random effects allow for each study to be assigned a weight which includes the within study variance and the between studies variance. Heterogeneity was assessed with the I2 which describes the proportion of total variation in the effect estimate that results from the between-studies heterogeneity and ranges from 0 to 100%.
The evidence regarding the diagnostic properties of nNO and TEM as well as the evidence regarding the prevalence of PCD among suspect patients were combined along with diagnostic accuracy estimates for HSVM from individual studies to develop a probabilistic decision model that allowed the calculation of net sensitivity and specificity as well as the cost-effectiveness (CE) and incremental cost effectiveness for three diagnostic algorithms that were characterized by different combinations of nNO, TEM and HSVM. The evaluated combinations were (a) nNO+TEM in sequence, (b) nNO+HSVM in sequence and (c) nNO/HSVM in parallel followed, in cases with conflicting results, by confirmatory TEM (nNO/HSVM+TEM) and the model followed a hypothetical initial population of 1000 referrals (expected 320 PCD patients). Number of PCD patients identified, CE and ICE ratios were calculated using Monte Carlo analysis in ANALYTICA.
Results
PCD prevalence among referrals was 32% (95% CI: 25–39%, I2 = 92%). TEM detection rate among PCD patients was 83% (95% CI: 75–90%, I2 = 90%). Exclusion of studies reporting isolated inner dynein arm defects as PCD, reduced TEM detection rate and explained an important fraction of observed heterogeneity (74%, 95% CI: 66–83%, I2 = 66%).
The overall sensitivity of nNO measured by VC techniques was 0.95 (95 % CI 0.91–0.97), while specificity was 0.94 (95 % CI 0.88–0.97). The positive likelihood ratio (LR+) of the test was 15.8 (95 % CI 8.1–30.6), whereas the negative likelihood ratio (LR-) was 0.06 (95 % CI 0.04–0.09). For non-VC techniques, the overall sensitivity of nNO measurement was 0.93 (95 % CI 0.89–0.96) whereas specificity was 0.95 (95 % CI 0.82–0.99). The LR+ of the test was 18.5 (95 % CI 4.6–73.8) whereas the LR- was 0.07 (95 % CI 0.04–0.12).
Regarding the probabilistic decision analysis model, out of 320 PCD patients, 311 were identified by nNO/HSVM+TEM, 274 with nNO+HSVM and 198 with nNO+TEM. The nNO/HSVM+TEM had the higher mean cost (€97K) followed by nNO+TEM (€56K) and nNO+HSVM (€39K). The nNO+HSVM algorithm dominated the nNO+TEM algorithm (less costly and more effective). The ICE ratio for nNO/HSVM+EM was €1600 per additional PCD patient identified.
Conclusions
Many centers for the diagnosis and treatment of PCD in the developed world follow different tests and a variety of algorithms for diagnosing PCD. In some low income countries, most likely, there is a complete lack of specialized diagnostic testing. The results of this PhD thesis suggest that diagnostic accuracy of nNO measurement both with VC and non-VC maneuvers is high and can be effectively employed in the clinical setting to detect PCD even in young children, thus potentiating early diagnosis. On the contrary, a significant percentage, at least as high as 26%, is missed by TEM and this limitation that should be accounted toward the development of an efficacious PCD diagnostic algorithm. The results of decision analysis approach employed in this study also suggest that a diagnostic algorithm which includes nNO during VC as a screening test followed by confirmatory HSVM identifies approximately 86% of PCD patients with a mean CER of 140€ per PCD case identified. The algorithm which maximizes the number of PCD patients identified involves parallel performance of nNO and HSVM as the first step, followed by TEM as a confirmatory test for the few cases where nNO and HSVM yield conflicting results, with a corresponding ICER of 1620€ per additional PCD patient identified. These findings can inform the dialogue about the development of evidence-based guidelines for PCD diagnostic testing and can illuminate discussions about how these guidelines can best be implemented across various healthcare systems.
File(s)![Thumbnail Image]()
![Thumbnail Image]()
Name
PhD Thesis Panayiotis Kouis Abstract English and Greek.pdf
Size
369.7 KB
Format
Adobe PDF
Checksum (MD5)
9bbb64d505476e29d6bb8447b35c9f46
Name
PhD Thesis Panayiotis Kouis.pdf
Size
4.79 MB
Format
Adobe PDF
Checksum (MD5)
b5bc277d9171fb676c943fed6376d264