What are the 4 types of decision-making analytical models?

Author information Copyright and License information Disclaimer

Copyright © 2008 Health Research and Educational Trust

To illustrate the use of decision-analytic modeling to assist decision making in organizational innovations.

Regarding an organizational innovation (shared care in hearing aid provision) available evidence from different sources was synthesized.

A probabilistic Markov model was constructed.

We modeled the long-term cost-effectiveness of different organizational formats of shared care as opposed to the current organization. We assessed the expected value of perfect information (EVPI) for several groups of parameters in the model.

The current organization had the highest probability of being cost-effective. Additional research is worthwhile, especially on access to care and safety (sensitivity to detect pathology).

Decision-analytic modeling in an early stage of organizational innovation is a valuable tool to facilitate evidence-based decision making.

Keywords: Decision-analytic modeling, cost-effectiveness analysis, uncertainty, organizational innovation, hearing aid provision, expected value of perfect information

Health technology assessment focuses increasingly more on organizational innovations than on specific technologies. With regard to decision making in organizational innovation there are two major issues: (1) there is not enough data to answer questions with enough precision for use in decision making and (2) collecting data is expensive and there is not enough budgeted to provide helpful answers (Hadley 2000). This paper illustrates how decision-analytic modeling can assist decision making in organizational innovations that are system changes rather than specific technologies. Especially in an early stage, organizational innovations can be designed in various formats, which makes it difficult to perform trials to assess safety and efficiency. However, this lack of evidence may not mean that decisions are postponed. It is better to inform a decision with the available evidence than without any evidence at all (Weinstein et al. 2003). Decision-analytic modeling is a useful tool to synthesize available evidence, and is increasingly used to estimate the safety and efficiency of health care interventions. It is not intended to reveal scientific truth (Weinstein 2006), but to guide decision making. Data from different sources can be combined to inform decision making under conditions of uncertainty (Briggs, Claxton, and Sculpher 2006). Additionally, one can assess the expected value of perfect information (EVPI), and specifically what type of additional evidence would be most valuable. It can therefore assist in making a two-fold decision: (1) whether or not an intervention should be implemented, given the current evidence, and (2) whether it is worthwhile to perform additional research to reduce the uncertainty surrounding the first decision (Claxton, Sculpher, and Drummond 2002).

The current paper aims to illustrate how decision-analytic modeling can assist decision making in organizational innovations. As an illustration, we examined shared care in hearing aid provision.

A key step in decision-analytic modeling is defining the model structure. When a Markov model is used, mutually exclusive health states need to be defined. In organizational innovation these health states should not only be based on health outcomes and costs, but also on organizational aspects. Especially in an early stage of an innovation, many alternative formats may be possible. Therefore, it is also important that the model structure allows all relevant organizational formats to be compared.

The main advantage of decision-analytic modeling is that data from different sources can be synthesized. Especially in an early stage of organizational innovation little evidence is available, and the limited evidence is often found in different sources. This uncertainty can be incorporated in the model by assigning distributions to the model parameters. Incorporating uncertainty also makes results easier to interpret by decision makers, as one can calculate which format has the highest probability of being cost-effective. One can also calculate the worth of acquiring additional information through further research, by calculating the EVPI (Claxton and Posnett 1996). Additionally one can examine for which parameters further research is the most valuable, by calculating the EVPI for (groups of) parameters (Felli and Hazen 1998).

Shared Care in Hearing Aid Provision

As only a small proportion of hearing-impaired adults require management other than hearing aids, it is advocated that not all persons with hearing complaints need to be examined by an ear nose throat (ENT) specialist or audiological center (AC) (Swan and Browning 1994). Increasing attention is paid to shared care initiatives in hearing aid provision, including direct hearing aid provision by private dispensers. However, it is uncertain whether private dispensers are capable of identifying the subgroup of persons who require medical care (safety). This subgroup will be referred to as “patients” throughout this paper, while persons not in need of medical care are referred to as “clients.” Also, it is unclear whether the transfer of tasks will indeed reduce costs while maintaining the quality of care (efficiency). However, this lack of evidence did not mean that decisions were postponed. In fact, some health insurance companies in the Netherlands already allow private dispensers to take over tasks from ENT specialists and ACs. This illustration examines the long-term cost-effectiveness of different shared care formats as opposed to the current organization of hearing aid provision. Additionally, we assessed the EVPI, and for which topics further research is most valuable.

Model Structure

A Markov model was constructed with mutually exclusive health states. The model simulated the lifetime course of events in a hypothetical cohort of persons with hearing complaints, aged 50 years or older. The cycle length of the model was set to 1 year.

Health states were based on hearing aid use and pathology. Regarding hearing aid use, health states were “no hearing aid,”“hearing aid user,” and “ex-user.” Hearing aid users were subdivided into two health states: good and poor quality hearing aid fitting. Pathology states were divided into acoustic neuroma, chronic otitis media, otosclerosis, and psychosocial problems, either detected or undetected. Pathology states were further divided into hearing aid users and nonusers. The final absorbing state was “death.”Figure 1 presents a schematic diagram of the model.

Simplified Schematic Diagram of the Markov Model

1. Process of hearing aid fitting. 2. Incidence and detection of pathology. 3. Successful treatment of pathology. 4. Becoming ex-user. 5. Detecting undetected pathology. *Hearing aid users are subdivided into users with a poor and a good quality hearing aid fitting. **The pathology state is subdivided into “acoustic neuroma,”“chronic otitis media,”“otosclerosis,” and “psychosocial problems.” These pathologies can be either detected or undetected. For detected acoustic neuroma, before and after treatment states are distinguished. Pathology states are further subdivided into hearing aid users and nonusers, as is illustrated for the no pathology states

In the “current format” persons seek help with the general practitioner (GP) and ENT specialist or AC, and all persons fitted with a hearing aid have an evaluation of their hearing aid fitting at the ENT specialist or AC. We defined three shared care formats. In the “total direct format” persons seek help with a private dispenser, and only patients have a follow-up at the ENT specialist to evaluate the hearing aid fitting. In the “triage format” persons seek help with the private dispenser, but all persons fitted with hearing aids have an evaluation at the ENT specialist or AC. In the “follow-up format” persons seek help with their GP, but for clients there is no need for an evaluation of the hearing aid fitting by the ENT specialist or AC. The organizational formats are presented in Table S1.

Data Sources

Input parameters for the model were derived from a multicenter prospective study, a patient cohort study, published literature, and, if no other source was available, expert opinion. Unless stated otherwise, expert opinion was provided by the authors through an informal process. In the prospective study shared care versus the current organization was examined. Details of this study (N=269) are presented elsewhere (Grutters et al. 2007). The cohort study consisted of 1,000 randomly selected patients consulting for hearing problems in a community hospital in the Netherlands in 2002 (J. A. Duijvestijn et al. unpublished data). All input parameters are listed in Table S2.

Analysis

The analysis was performed from the societal perspective, and all costs were reported in Euros (€1 is US $1.25, average 2006 conversion rate). Future costs and effects were discounted to their present value by a rate of 3 percent (Siegel et al. 1997).

We compared the cost-effectiveness of the four organizational formats. Incremental cost-effectiveness ratios (ICERs) were calculated, dividing the incremental costs by the incremental quality-adjusted life years (QALYs). ICERs were calculated by comparing each format with the next most effective format. Whether a format is deemed efficient depends on how much society is willing to pay for a gain in effect, which is referred to as the ceiling ratio. The National Institute for Health and Clinical Excellence in the United Kingdom uses a ceiling ratio between £20,000 and £30,000 per QALY (Buxton 2006), which is roughly €40,000.

We assigned distributions to the model parameters, to reflect the second-order uncertainty in the estimation of that parameter (Weinstein 2006). See Table S2 for the assigned distributions. Parameter values were drawn at random from the assigned distributions, using Monte Carlo simulation with 10,000 iterations. To illustrate the results of the simulation, cost-effectiveness acceptability curves (CEACs) were calculated (van Hout et al. 1994; Fenwick, Claxton, and Sculpher 2001). For different ceiling ratios, the net monetary benefit was calculated for each format by subtracting the costs from the effects, multiplied by the ceiling ratio. CEACs show the probability that a format has the highest net monetary benefit, and thus is deemed cost-effective, given different ceiling ratios.

In the base case analysis age was set at 70 years, as this was the median age of participants in the prospective study and was estimated to be the median age for persons seeking help for their hearing complaints. In a subgroup analysis, we varied age to 50 and 85 years.

As uncertainty exists, there is always a chance that the “wrong” decision will be made (Briggs, Claxton, and Sculpher 2006). The EVPI is the expected value of obtaining perfect knowledge of the “true” values of all parameters. We calculated the total EVPI by subtracting the net monetary benefit of the organizational format we would choose under conditions of uncertainty, from the net monetary benefit of the optimal decision we would make if we knew the “true” parameter values. Finally, we calculated the partial EVPI for (groups of) parameters (Felli and Hazen 1998), for a ceiling ratio of €40,000 per QALY, to examine for which parameters further research is the most valuable. We calculated this partial EVPI by subtracting the net monetary benefit of the format we would choose under conditions of uncertainty for a given parameter (group), from the net monetary benefit of the optimal decision we would make if we knew the “true” value of that parameter (group).

For a 70-year-old population, the current format yielded the most QALYs (6.765; Table 1), followed by the follow-up format. The follow-up format was associated with a QALY loss of 0.004 and cost savings of €43, resulting in a mean ICER of €10,972 saved per QALY lost. Because the incremental results are negative (less effective and less expensive), in this case higher ICERs are more acceptable. That is, with higher ICERs more money is saved per QALY lost. As both the triage format and the total direct format were somewhat more expensive than the follow-up format and associated with a loss in QALYs, these formats were judged as inferior.

Base Case Results and Subgroup Analyses

In Figure 2 the CEACs show that considerable uncertainty surrounds the decision which format is deemed cost-effective. Given a ceiling ratio of €40,000, the current format had the highest probability of being cost-effective (51 percent), followed by the triage (40 percent), the follow-up (6 percent), and the total direct format (3 percent).

The subgroup analysis for age showed that younger age (50 years) was associated with a higher probability (58 percent) that the current format is cost-effective than older age (85 years; 42 percent). Otherwise, age did not affect the results.

For the base case analysis, the uncertainty surrounding the decision whether or not to implement shared care resulted in an EVPI of €87 per person, given a ceiling ratio of €40,000. Implementing shared care affects all persons with hearing complaints of 50 years and older without a hearing aid, being a total of 1,155,719 persons in the Netherlands in the next 10 years (Central Bureau for Statistics 2007). This makes the population EVPI €100 million, meaning that perfect information on this topic is worth €100 million. The population EVPI for different ceiling ratios is presented in Table S3.

Additionally, the EVPI for parameters was calculated. The parameter most valuable for further research was about access to care: whether persons with hearing complaints would seek help sooner when they can do so directly with the private dispenser. Other valuable parameters for further research were about the safety of care: the probability that the private dispenser refers patients to the ENT specialist (sensitivity), followed by the probability that undetected pathology will nevertheless be detected before any harm is done. Additional research with regard to the utility scores was also worthwhile. For the remaining (groups of) parameters the EVPI was relatively low (Table S4).

Some years ago Hadley presented two major issues in health services research: (1) there is not enough data to answer questions with enough precision for people to use in decision making, and (2) collecting data is expensive, and health services research does not have large enough budgets to provide helpful answers (Hadley 2000). The current paper shows that decision-analytic modeling can assist in both these issues. First, all available evidence can be synthesized in a model to guide decision making. Next, EVPI analyses can help to allocate research budgets in the most efficient way. This will obviously not answer all the questions that exist in health services research, but will at least help to guide the research agenda.

There are a number of limitations of the present study that need to be addressed. First, due to the lack of evidence in this early stage of the innovation, many uncertain parameters were included in the model, based on expert opinion or small study populations. However, a model should not be faulted because existing data fall short of ideal standards of scientific rigor (Weinstein et al. 2003). That is, decisions will be made; hence, it is better to inform the decision with the available evidence than without any evidence at all. Using decision-analytic modeling we were able to incorporate the uncertainty in the model, and results were shown with surrounding uncertainty. We did assume “perfect implementation,” although. Less than perfect implementation reduces efficiency, and it is unclear whether patients and professionals will change their behavior if shared care would be implemented. If shared care in hearing aid provision would be deemed efficient, it would therefore be interesting to examine the value of perfect implementation. This would allow for a tradeoff between the costs and benefits of implementation strategies (Fenwick, Claxton, and Sculpher 2008; Hoomans et al. 2007). Also beyond the scope of this paper, but worth considering if in a later stage shared care is found to be cost-effective and may be implemented, is the possible disadvantage that the prescriber of the hearing aid financially benefits from selling hearing aids.

Another limitation is that decision-analytic modeling is rather computer-intensive. Especially for the partial EVPI, many iterations need to be drawn. In the present study, we ran 200 Monte Carlo simulations by 200 draws for the parameter (group) of interest. When repeatedly running our analysis the ordering of which parameters were most valuable for further research remained stable. Thus, we could accurately estimate which parameters are more and less valuable for further research. One should, however, be aware that, to answer the question how valuable obtaining perfect information for each parameter exactly is, many more iterations would be required.

Finally, modeling is always a simplification of a complex reality. In our case, we incorporated four types of pathology, while other types also may affect hearing and may be missed by dispensers. However, we feel that we included the most important types of pathology in this study.

Overall, this study shows that decision-analytic modeling in an early stage of organizational innovation is a valuable tool to facilitate evidence-based decision making. Despite the limited evidence, the illustrative decision-analytic model provided valuable information for decision makers on whether or not to implement shared care in hearing aid provision. Moreover, it informed decision makers and researchers on whether or not to perform additional research and for which topics additional research is most valuable.

Joint Acknowledgment/Disclosure Statement: Financial support from the Heinsius Houbolt Foundation and the Dutch Health Care Insurance Board (CvZ) is gratefully acknowledged. We would like to thank Wouter Dreschler, Hans Verschuure, Jan Duijvestijn, Michelene Chenault, and Harry Streukens for providing us with data. Preliminary results were presented at the 8th congress of the European Federation of Audiological Societies, June 6–9, 2007, in Heidelberg, Germany.

Disclosures: None.

The following material is available for this article online:

Appendix SA1 Author Matrix.

Table S1. Alternative Organizational Formats of Hearing Aid Provision.

Table S2. List of Input Parameters.

Table S3. Population Expected Value of Perfect Information Curve.

Table S4. Expected Value of Perfect Information for (Groups of) Parameters.

This material is a vailable as part of the online article form: This material is available as part of the online article from: http://www.blackwellsynergy.com/doi/abs/10.1111/j.1475-6773.2008.00872.x (this link will take you to the article abstract).

Click here to view.(31K, doc)

Click here to view.(189K, doc)

Click here to view.(26K, doc)

Click here to view.(24K, doc)

Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford: Oxford University Press; 2006. [Google Scholar]
• Buxton MJ. Economic Evaluation and Decision Making in the UK. Pharmacoeconomics. 2006;24(11):1133–42. [PubMed] [Google Scholar]
• Central Bureau for Statistics. 2007. [accessed on May 9, 2008]. “StatLine database”. Available at http://www.cbs.nl/nl-NL/menu/cijfers/statline/toegang/default.htm.
• Claxton K, Posnett J. An Economic Approach to Clinical Trial Design and Research Priority-Setting. Health Economics. 1996;5(6):513–24. [PubMed] [Google Scholar]
• Claxton K, Sculpher M, Drummond M. A Rational Framework for Decision Making by the National Institute for Clinical Excellence (NICE) Lancet. 2002;360(9334):711–5. [PubMed] [Google Scholar]
• Felli JC, Hazen GB. Sensitivity Analysis and the Expected Value of Perfect Information. Medical Decision Making. 1998;18(1):95–109. [PubMed] [Google Scholar]
• Fenwick E, Claxton K, Sculpher M. Representing Uncertainty: The Role of Cost-Effectiveness Acceptability Curves. Health Economics. 2001;10(8):779–87. [PubMed] [Google Scholar]
• Fenwick E, Claxton K, Sculpher M. The Value of Implementation and the Value of Information: Combined and Uneven Development. Medical Decision Making. 2008;28(1):21–32. [PubMed] [Google Scholar]
• Grutters JPC, Van der Horst F, Joore MA, Verschuure H, Dreschler WA, Anteunis LJC. Potential Barriers and Facilitators for Implementation of an Integrated Care Pathway for Hearing-Impaired Persons: An Exploratory Survey among Patients and Professionals. BMC Health Services Research. 2007;7(57) [PMC free article] [PubMed] [Google Scholar]
• Hadley J. Better Health Care Decisions: Fulfilling the Promise of Health Services Research. Health Services Research. 2000;35:175–86. 1, part 2. [PMC free article] [PubMed] [Google Scholar]
• Hoomans T, Ament A, Evers S, Severens JL. 2007. “Worthwhile Implementation of Evidence-Based Guidelines into Clinical Practice: How to Determine the Investment Potential for Guideline Implementation and the Value for Money of Implementation Strategies?” 6th World Congress of the International Health Economics Association (iHEA), Copenhagen, Denmark. [PubMed] [Google Scholar]
• Siegel JE, Torrance GW, Russell LB, Luce BR, Weinstein MC, Gold MR. Guidelines for Pharmacoeconomic Studies. Recommendations from the Panel on Cost Effectiveness in Health and Medicine. Panel on Cost Effectiveness in Health and Medicine. Pharmacoeconomics. 1997;11(2):159–68. [PubMed] [Google Scholar]
• Swan IR, Browning GG. A Prospective Evaluation of Direct Referral to Audiology Departments for Hearing Aids. Journal of Laryngology and Otology. 1994;108(2):120–4. [PubMed] [Google Scholar]
• van Hout BA, Al MJ, Gordon GS, Rutten FF. Costs, Effects and C/E-Ratios Alongside a Clinical Trial. Health Economics. 1994;3(5):309–19. [PubMed] [Google Scholar]
• Weinstein MC. Recent Developments in Decision-Analytic Modelling for Economic Evaluation. Pharmacoeconomics. 2006;24(11):1043–53. [PubMed] [Google Scholar]
• Weinstein MC, O'Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, Luce BR. Principles of Good Practice for Decision Analytic Modeling in Health-Care Evaluation: Report of the ISPOR Task Force on Good Research Practices—Modeling Studies. Value in Health. 2003;6(1):9–17. [PubMed] [Google Scholar]