Decision theory applied to image quality control in radiology
© Lessa et al; licensee BioMed Central Ltd. 2008
Received: 26 May 2008
Accepted: 13 November 2008
Published: 13 November 2008
The present work aims at the application of the decision theory to radiological image quality control (QC) in diagnostic routine. The main problem addressed in the framework of decision theory is to accept or reject a film lot of a radiology service. The probability of each decision of a determined set of variables was obtained from the selected films.
Based on a radiology service routine a decision probability function was determined for each considered group of combination characteristics. These characteristics were related to the film quality control. These parameters were also framed in a set of 8 possibilities, resulting in 256 possible decision rules. In order to determine a general utility application function to access the decision risk, we have used a simple unique parameter called r. The payoffs chosen were: diagnostic's result (correct/incorrect), cost (high/low), and patient satisfaction (yes/no) resulting in eight possible combinations.
Depending on the value of r, more or less risk will occur related to the decision-making. The utility function was evaluated in order to determine the probability of a decision. The decision was made with patients or administrators' opinions from a radiology service center.
The model is a formal quantitative approach to make a decision related to the medical imaging quality, providing an instrument to discriminate what is really necessary to accept or reject a film or a film lot. The method presented herein can help to access the risk level of an incorrect radiological diagnosis decision.
The X-Ray is the main investigation method for the human body in various clinical conditions. Therefore, quality control procedures were implemented to standardize and define the minimum requirements for a gold standard radiology service. These requirements are worldwide reviewed by health institutions, including Brazilian National Institute of Health (Secretaria de Estado da Saúde, Centro de Vigilância Sanitária and Ministério da Saúde), as well as international institutions (International Commission on Radiological Protection – ICRP, World Health Organization – WHO, and International Commission of the European Communities – ICRU) monitoring the radiology practice .
The main idea behind the regulations in X-Ray application is to protect patients from radiation injury. The basic principles can be summarized as those described by ICRP 73 : ionization radiation should be applied in cases where the benefits overcome the potential hazards. The objective is to maximize the gains over the potential harms using procedures to protect against radiation. The applications of these two principles are the main goal of a Quality Control (QC) program: apply the minimum required radiation to maintain the diagnostic value of the images. Radiation must be controlled and the radiological images should have always enough quality to provide the required information.
Excellence in imaging quality standards is achieved through published guidelines of radiological societies (e.g. Radiological Society of North America guidelines). There are standard procedures for each exam that the technologist knows and follows when acquiring an image. There are also general rules related to examination steps and acquisition device settings. Every step is dependent of the patient's characteristics, such as height. In addition, some issues are difficult to standardize, although the major of technical procedures were monitored for a proper quality assurance . These non-measurable factors usually have a negative impact in the standardization of radiological practice with undesirable effects. A preliminary study made between 1992 and 2003 in Brazil showed that 5.15% of the films were rejected films; resulting in patient's re-expositions, equipment lifetime reduction and an overall increased operational cost. The main cause of film loss identified in this study was lack of standardization .
A QC program is typically a result of multidisciplinary interaction, involving several steps: habitual verification of technical parameters in X-ray equipments, systems registry (films, digital cassettes) and other procedures used to deliver the images for medical interpretation. The definition of QC also includes a series of standardized tests developed to detect changes in X-ray equipments [5, 6]. The final result of a successful QC should primarily be a reduction in the number of re-exposed or over-exposed patients and a high quality imaging diagnostic. Also, it should provide minimal financial costs and increase the equipment lifetime. This usually requires a complex rule of decisions, usually made by the physicists, engineers and radiologists in a joint effort to maximize two principles: improve the image quality and reduce radiation exposure.
Decision Making Theory (DMT) offers an integrated framework to mathematically define a preference and/or probability model for each step of the QC program. Priority of preference in sets of possible outcomes and states of information (uncertainty or imprecision) are represented as distribution possibilities  and therefore statistically analyzed. This formulation allows a more direct measurement of the outcomes from a certain decision and has been implemented successfully in many different contexts. One of the most used implementations of DMT was proposed by Von Neumann and Morgenstern . These authors modeled the quantitative utility function based on a set of axioms that described decision-maker's behavior. Various implementations made in medicine were based on their approach with practical examples [9, 10]. Their work has demonstrated the importance of this kind of framework according to a decision policy regarding a case of a preoperative patient management care before opting for a major elective surgery. All available information to decision-making, through informed priors, allow the appropriate decisions to be made .
In essence, what the decision theory formalizes is the common sense idea that an individual should take the best action based upon what he or she wants, knows, and can do [11, 12]. In a radiology routine, it is the technician who acquires the image and observes it initially. It is often up to them – without ever asking the radiologist – whether or not another image must be taken due to poor quality acquisitions. It should also be noted that the technologists often acquire an image with certain parameters because they know which radiologists will evaluate the exam and what their personal preferences are. The classification of this "individual" choice depends on direct or indirect aspects. The person who makes this decision could be the administrator of the radiology service department, the radiologist itself, the technician or even the patient when refusing to perform a second or third exam in order to reacquire the image.
The basic parameters of decision theory involve some sets, probabilistic mechanisms, and decision rules .
The four main sets are:
1) a payoff set (p) – the offered benefit in a specific situation, consisting of all the possible consequences of the actions to be taken;
2) a set of "states of nature" (θ) – containing all the possible situations of the study; typically unknown parameters in the phenomena (i.e. if a film has a good or bad quality) involved;
3) a set of observations (x) – relating to the states of nature (i.e. spatial resolution, sensitivity of an image);
4) a set of possible courses of action (a) – to reject or accept the film.
The three main probabilistic mechanisms are:
1) The consequence function – P(p i |θ j , a k ). It shows the distribution probability of the payoffs for each pair of defined events – "state of nature" vs. action. It is necessary to consider the utility function, which represents the preferences of the decision maker in situations of uncertainty. In order to reach the model of utility function (u), a process of eduction is applied to prioritize the good and bad actions strategies incorporated into a unique evaluation dimension. This function is a polynomial one with a degree called r. The higher the r value, the most complex will be the utility function.
2) The distribution probability (likelihood function) – P(x i |θ j ). According to the determined observations of the state of nature, a film can have a good or bad quality including its own characteristics (contrast, speed, fog, digital exposition, latitude and other). The distribution probability considers the correlation between the observations made and the state of nature.
3) A prior distribution – π(θ j ). Possibly elicited from an expert, regarding to the state of nature. If the institution maintains a database of QC, it is possible to estimate a prior distribution to be applied to the test.
The deterministic choice is the most simple decision rule for each observation in which an ideal action should be taken.
The randomized decision rule yields two other types: 2.1) The randomization upon the non-randomized rules before any observation is made.
2.2) The behavioral randomized rule b(a j |x i ), in which a first observation is made. After this observation, a randomization of all possible actions is considered. The behavioral rule is more powerful, in the sense that it can access any risk; this is not the case regarding the randomized rule.
The risk function (Rd) compares the decision rules and the consequence function is defined by the process for obtaining the probability of a certain result related to the "state of nature" (in our case, a good or bad image). Based on the consequence function the decision-maker chooses, a certain proceeding or action is taken. This probabilistic mechanism should be better understood when analyzed together with the decision theory techniques. It can be inferred from an available database, from the elicitations of an expert or from both.
Herein, we suggest a general guideline, based on the decision-making paradigm, as proposed by the report from ICRU in 1996 . According to this report, the techniques of statistical DMT should be used as a standard procedure to treat problems of image detection, edge detection, noise and sharpness. The parameters referred by the present work are: inadequate use, inadequate sensibility and/or inadequate spatial resolution related to the radiological chain process. The application of the decision making theory was restricted to acquired images, not considering other parameters that could interfere before the image detection. The final outcome considered was to accept of reject a film lot. With this new approach, there are good consequences like: the patient's satisfaction, a correct diagnosis and stable financial costs.
This work does not aim on a new method proposition to improve the sensibility or specificity of the data acquisition system. The approach also applied a very well known concept of the radiology practice: the Receiver Operating Characteristics, also known as the ROC curve. This curve is determined according to a set of probabilities: true positive = P (positive test/positive for disease) vs. false positive = P (positive test/negative for disease). In other words, sensibility vs. a false positive or P (altered exam/has the disease) vs. P (altered exam/has not the disease) parameters that are part of most papers published in radiology journals, in which the accuracy of a diagnostic imaging test is verified.
This is the TPF (function of true positive probability) and FPF (function of false positive probability) ROC curve. It has a purely inferential character, and is a very particular case of a more general ROC curve, where the loss function is the ideal observer loss function (i.e., the negative of the utility function). We have four possibilities: if the film is bad, we reject it; if the film is good, we accept it; if the film is bad and we accept it, and finally, if the film is good and we reject it. In the first two cases, the target was reached. In the latest the target was missed, and they correspond to errors. In mathematical terms, the loss function is given by:
L(θ i , a j ) = -u(θ i , a j )
The ideal observer loss function is given by:
L(θ i , a j ) = 0 if a i = θ j ; L(θ j , a i ) = 1 if a j ≠ θ i , i, j = 0, 1.
This means that if we reach the target, we have no fine (the loss is null). If we miss the target, we pay a fine equal to one (the loss is equal to 1). But this rather peculiar utility function does not reflect the preferences of a decision-maker, since the negative of the ideal observer loss function is just an inference criterion.
When the ideal loss function is applied, the risks were calculated by:
Rd(θ j ) = -ΣP(x i |θ j )u(θ j , d(x i ))
These probabilities correspond to the specificity, sensitivity, false positive and false negative conditions. In these inferential contexts, the classical TPF and FPF ROC curve is the typically presented as the true positive versus the false positive conditions, with a 0–100% scale on both axes (concave function). In the current work, since we are dealing with preferences, the risks will not be considered as probabilities. The axes of the ROC curves will have a scale corresponding to the applied utility function, which should be educed from the decision-maker and represents his preferences (convex curve). It is the locus of all possible Neyman-Pearson Rules and not a TPF- FPF ROC curve.
The ROC curve axis reflects the preference dimension of the decision-maker regarding to the data acquisition system (i.e. technology or radiography type) and will imply in a likelihood function. This construction is independent from a considered database and the probability will usually be very similar.
In order to accept or reject a film lot according to this procedure is necessary to obtain the correct diagnosis, the cost optimization and the client's satisfaction. The client's satisfaction will be measured by the utility function, which is different from the one that corresponds to the ideal observer loss function. All values of probable observations were proposed based on previous information from real datasets and hypothetical predictions applications. The investigation researched some possibilities according to the dependency values related to the image quality assurance. In summary, this is a case in which the decision theory is used to establish a framework for radiographic film quality control protocol; this is done by studying the consequence of an action as a function of a specific probability distribution over possible consequences.
The decision making theory was implemented for specific computational tools. The data was stored in a Quattro Pro v. 9.0 (Borland Inc – USA) and in a Statistica v. 5.3 (StatSoft – USA). All data was obtained from densitometry data of the radiographic films of a phantom obtained by a digital densitometer (Model 07–443, Nuclear Associates Victoreen 07–443 Clamshell, United States). The values of the film characteristics were calculated from 15 different measures of radiation intensity.
The images were made in a conventional X-Ray, Medicor 500 mA and 125 kV, from the Hospital das Clínicas da Universidade Federal de Pernambuco (HC-UFPE), Brazil. The values were obtained from data analysis of the radiographic film (characteristic curve, modulated transfer function and the Wiener spectrum as exposed in ICRU ). Attributes such as contrast, latitude, speed and fog were determined according to our observations from each film condition (bad quality or good quality).
In case of two possible actions (accept/reject) and no prior probability, we applied the behavioral randomized  and Neyman-Pearson rules. In case of two possible actions, the observation sample space was partitioned into two regions, one of rejection and the other of acceptance. The model is very simple in compliance with Ockham's razor principle. The consequence function was calculated based on the subjective preference of the person making the decision.
p 0 = (incorrect diagnostic, high cost, low client satisfaction)
p 1 = (incorrect diagnostic, low cost, low client satisfaction)
p 2 = (incorrect diagnostic, high cost, high client satisfaction)
p 3 = (incorrect diagnostic, low cost, high client satisfaction)
p 4 = (correct diagnostic, high cost, low client satisfaction)
p 5 = (correct diagnostic, low cost, low client satisfaction)
p 6 = (correct diagnostic, high cost, high client satisfaction)
p 7 = (correct diagnostic, low cost, high client satisfaction)
Based on the parameters described above, a set of "states of nature" was established. A film lot is considered good if the percentage of bad units from a film box is below a certain pre-specified value. There is a natural doubts of which of these states reflect a true condition scenario.
This uncertainty establishes the decision problem. Where θ represents the real "state of nature" of the film, we have:
θ 0 = the film is a bad quality one;
θ 1 = the film is a good quality one.
The space or set of observations should consider a finite number of possibilities. We considered three groups of attributes: film usage, film sensibility and spatial resolution.
For each situation of film usage, film sensibility and spatial resolution, a dichotomy was considered, based on the limits of the values dependency. Another set (eight possibilities) was originated for the following observations:
x 0 = (inadequate for use, inadequate sensibility, inadequate spatial resolution)
x 1 = (inadequate for use, inadequate sensibility, adequate spatial resolution)
x 2 = (inadequate for use, adequate sensibility, inadequate spatial resolution)
x 3 = (inadequate for use, adequate sensibility, adequate spatial resolution)
x 4 = (adequate for use, inadequate sensibility, inadequate spatial resolution)
x 5 = (adequate for use, inadequate sensibility, adequate spatial resolution)
x 6 = (adequate for use, adequate sensibility, inadequate spatial resolution)
x 7 = (adequate for use, adequate sensibility, adequate spatial resolution)
The problem considered in this case had only two possibilities of action:
a 0 = to reject the film lot;
a 1 = to accept the film lot.
The effective action related to the condition of the state of nature reflects the probability of the feasible outcomes.
The film usage, film sensibility and spatial resolution attributes provide the likelihood function (P(x i |θ j )) of the analyzed images related to the values dependency.
The likelihood function P(x i |θ j ) consists of two values, in which i corresponds to the observed value and j to the true value. The value of x i (a set of observations) represents what is obtained from the equipment, and θ j (a set of "states of nature") is the "true" value: a good or bad film quality.
The probabilities are shown for each pair (θ i , a j ) of a consequence function.
(θ 0 , a 0 )
(θ 0 , a 1 )
(θ 1 , a 0 )
(θ 1 , a 1 )
The decision rules
The "state of nature" and the problem solution were obtained by the rule of Neyman-Pearson. The decision-maker utility function enunciated the level θ of tolerated risk when θ 0 was considered the true "state of nature".
We have applied two r values to simulate the decision-maker's influence or preference upon the decision rule to adopt an optimized action (ideal). The r value is actually the polynomial grade used in the mathematical formulation. We chose r = 2 and r = 4, in order to have the classical quadratic loss function (from Theory of Decision Making) and a more complex formulation. This was done in order to check whether the results would be the same, regardless of the decision-making opinion.
The observations were considered from a set of 8 possibilities as shown in Table 1. Thus, we have 256 possible decision rules (2 actions with 8 established observations). In order to determine a general utility function to access the decision risk, we have used just one parameter – r – in which the utility function considered was u(p) = p r .
Two values of r were used: r = 2 and r = 4. The higher values of r reflect an increased risk of the decision-maker. If the patient had to decide about the rejection or acceptance, he or she would tend to be more adverse to risk. We chose r = 2 as a conservative risk behavior simulation to the patient choice, so that, the values of P(x i |θ j ) = probability values of the signal/noise ratio were relayed on imaging equipments and its components (see Table 1) and P(p i |θ j , a k ) = once the nature condition of the chosen data θ 0(bad film) or θ 1(good film) regarding to a 0 = to reject or a 1 = to accept action (data not shown). The consequence function is defined as p i (payoffs), under probability of P(p i |θ j , a k ) (see Table 2).
Simulations with the model
The main purpose of the simulation is to analyze the influence of the utility function on the decision rule to accept or reject the film lot. There are several possibilities concerning the emphasis and accuracy on various aspects of the possible outcomes, contemplating the quality control protocol. In particular, the characteristic of risk aversion was evaluated. The probability of x 0 = (inadequate for use, inadequate sensibility, inadequate spatial resolution) when θ ("states of nature"), θ 0 (bad film) is 0.41177 (worst scenario) – case in which the report is incorrect and the patient re-exposure will be necessary – the probability of x 0 is 0.00977, if θ ("states of nature") is θ 1 (good film). On the other hand, the probability of when θ ("states of nature"), θ 0 (bad film) is 0.0084 makes the probability of x 7 be 0.4888 of θ 1 (good film).
Table 2 demonstrates all considered consequence functions. The highest probability of a correct diagnosis with an increased cost and low client satisfaction was represented by P(p 4 | θ 0, a 0) = 0.30 (Table 2).
Consider the utility function and . In order to calculate the risk function R d (θ 0) and R d (θ 1) corresponding to each utility function (polynomial degree) r, we used and , presented as R d (θ 0) × R d (θ 1) in figure 2, with r = 2 and in figure 3 with r = 4.
Representation of the results accomplished with the Neyman-Pearson decision rules for two values of r.
In figure 3, the decision rule d 16 corresponds to the vertex of the curve when the risk R d (θ 0) is set to r = 4. This rule says that if the observation was inadequate for use, the film must be rejected. All other options were accepted (see Table 3). This rule corresponds to R d (θ 0) = -0.18 and R d (θ 1) = -0.36. When r = 2 for the same value, we have R d (θ 0) = -0.18 and the minimum value in the curve corresponds to d 192 (R d (θ 1) = -0.44). This rule expresses that the only rejection condition for the film is when usage and sensibility are inadequate and spatial resolution is adequate (Figure 2).
However, when r = 4 (d 192) the risk of accepting a bad film is increased (R d (θ 0) = -0.18) and the risk of accepting a good film increases as well (compared to d 16). This rule was not the ideal one since it rejects the finest option: adequate for use, sensibility and spatial resolution.
The problem of accepting or rejecting a radiographic image based on the decision theory method was modeled here. The decision theory was applied on a hypothetical model. The ROC curve axis represent risks levels based on a preference concept or choice determined by the eduction of the utility function von Neumann-Morgenstern – this curve represents how a set of decision rules behave related to the risk of a consequence set. The ROC analysis for each risk set is a graphic representation, common to radiologists, and provides a manner to decide how a decision rule could be used, according to the false positive or sensitivity rate (Neyman-Pearson decision rule ).
In modern radiological services, it is extremely difficult to manage quality control processes. The mathematical approach can help, dealing with complex decisions such as reject or accept a radiographic image. The main result obtained in this modeling exercise was provided by a decision rules set that minimizes the risk of an undesired consequence. In addition, the solution proposed at this point can be applied in a practical radiology routine.
Residents and non-professionals staff of the radiology field have to learn how to make decisions, generally from gathering information from other more experienced professionals and from literature information. Decision Theory offers a way to arrange decisions in a rational manner, thus, avoiding many risks that may affect the patient health quality. If one radiologist considers an image appropriate for radiological appraisal, many possibilities may be influenced by this decision. Besides, the level of consequences depends on the type of the clinical question. If the radiography is performed to diagnose a broken arm, the presence of an insignificant spot in the image is generally not relevant as in the case of a slight spot in mammography exam performed for tumoral screening. The presence of micro-calcifications with specific morphological aspects has a direct impact on the next step, which may include or not additional invasive procedures. Therefore, the probability of a correct diagnosis could be directly affected by the quality of a simple X-ray image.
The expense of an image production is increased if a film does not have a good quality. X-ray re-exposure will be inevitable and the patient will certainly be unsatisfied at this point. This behavior variable should not be misguided with a simple level of comfort. The client contentment must be contemplated, not only by the correct diagnosis, but also by the number of necessary radiographic trials in the service. As a presumable consequence, the service could lose a representative number of clients, which would represent a substantial loss in profits.
All radiology services have an administrative design for quality control (QC) of the radiographic film. This is emphasized by the ICRU and is generally based on gold standard guidelines and limits of technical parameters among the used values to access the film quality. Although there are similar procedures reported in the literature , none of them is based on a formal quantitative model. Generally, the result of a QC program is measured by the number of exams correctly performed, costs involved and the patient's satisfaction. These variables are not easily measured beyond descriptive statistics. The decision is commonly not settle on a formal quantitative model, but in personal experience and historical background of the institution. The management team is responsible to decide upon the equipment choices variety, imaging procedures, questionnaires and personal training programs.
The use of Bayesian statistics allows one to give a closer look at the real based management information [14, 16, 17]. Historical information was considered and used for a prior distribution of the probabilities as a mean to avoid fake or unlikely occurrences [18, 19]. This approach can be applied in distinct radiological scenarios. This concept or perception adapted to other settings of variables could include another image modality, clinical context and/or strategies for a perspective research of decisions or administrative managing.
The main challenge concerning the decision theory implement is to define the state of nature (good or bad quality of the radiographic film). Therefore, a given action can lead to a favorable payoff of the decision-maker whatever the state of nature. If the state of nature is a good one, the preferred result would be more likely to occur. Even so, a determined action for a situation could probably unfold as a complete disaster in the case of a very unfavorable payoff, while the true state of nature is completely different or wrong. Note that in two other cases, still with a less significant extent, positive probabilities have appeared. The consequence of an action must be established through a numeric scale according to the decision-maker preferences .
Possible consequences were summarized based on the quality of an image in three distinct aspects: diagnosis (correct or incorrect), cost (high or low) and the patient satisfaction (high or low). In summary, the proposed decision problem is: having one of the eight possibilities (x) what decision should be made regarding the value of (θ) so that it can be considered true? The person responsible for the radiology department or the radiology management team (usually the decision-maker) will adopt a criterion for the decision process in order to guide the choice that will minimize the risk of having an undesired consequence. The choice of the vector x (a set of observations) and of the vector θ (a set of "states of nature") will depend on the considered precision degree as well as on the reliable technology. The consequence function P(p | θ, a) was estimated from the data obtained from the experiments or from a historical service database. It includes more probabilistic mechanisms than those found in a film. The utility function's eduction was not determined by any decision-maker. It is important to mention that the proposal of probabilistic mechanism in the likelihood function can be adjusted to any information provided. As a consequence the data consistency will depend on the source confidence. A consequence (p – a set of payoff), as described in the results, was related to the set of payoffs involved in a right or wrong diagnosis, a higher or lower cost and higher or lower client satisfaction, independently of the technology or of the radiologist that made the diagnosis. Each pair (θ, a) has a specific risk correlation to the preferences of the decision-maker. The decision-maker preferences, per se, were dictated by a behavior parameter in a risky circumstance. Decision theory application estimates the risks for all possible rational choices under uncertainty and allows a rational decision for all computed information during a specific procedure (Bayesian theory).
Again, the choice of the best decision rule can be found at the vertex region of the curve in the risk set R d (θ 0) with r = 4 which is a result of the 256 possible combinations, as described previously. The corresponding graphic (Figure 3) information corresponds to d 16, which means that the observation was inadequate for use (bad film), and the film was rejected (Table 3). The decision-maker option was to accept it. When adopting the criterion mentioned above, we found that the minimum risk combination of R d (θ 0) × R d (θ 1) corresponds to (Figure 3) R d (θ 0) = -0.18 when the state of nature of the film was bad (θ 0), and the risk of the film was considered good (θ 1) with r = 4 corresponding to R d (θ 1) = 0.36. In the case of r = 2, if the decision-maker had chosen the same value R d (θ 0) = -0.18, the decision rule would be different: the film is rejected when the state of nature and its technical quality are bad -d 192. Thus, the minimum risk combination R d (θ 0) × R d (θ 1) is not the best decision rule for this case. This model can be refined by constructing p, θ, x, values which may assume any mathematical (finite or infinite, discrete or continuous, scale or vectorial) representations. There are no restrictions for the current adopted classification.
The proposed model allows a cost-utility analysis of the radiographic film quality control. The decision theory addressed to this quality control issues established a systematic procedure, a well defined protocol approach and suggested a better use of the available imaging technologies acquisitions. Moreover, depending on the utility function, it allows us to analyze how the patient choices or the radiographic service decision-making preferences affect the decision rule. It constitutes a paradigm not only for a film quality control, but also for the radiographic service quality control in an integrated framework.
The first author was sponsored by a scholarship from Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco (FACEPE).
- ICRU: Medical imaging – the assessment of image quality. Report 54. 1996Google Scholar
- ICRP: Radiological Protection and Safety in Medicine: Pergamon, Publication 74. 1996Google Scholar
- Papp J: QUALITY MANAGEMENT in the Imaging Sciences. 2006, Mosby Elsevier, 3Google Scholar
- Nardi MTd: Avaliação dos rejeitos radiográficos no serviço de radiologia do Hospital de Clínicas de Porto Alegre. Revista do Hospital de Clínicas de Porto Alegre. 2004Google Scholar
- Monsees BS: The Mammography Quality Standards Act. An overview of the regulations and guidance. Radiol Clin North Am. 2000, 38 (4): 759-772. 10.1016/S0033-8389(05)70199-8.View ArticlePubMedGoogle Scholar
- Périard M, Chaloner P: Diagnostic X-ray imaging quality assurance: an overview. The Canadian Journal of Medical Radiation Technology. 1996, 27 (4): 171-177.Google Scholar
- Dubois D, Prade H: Towards Intelligent Systems table of contents, in Lecture Notes In Computer Science: Selected papers from the Workshop on Fuzzy Logic in Artificial Intelligence. 1995, Springer-Verlag: London, 240-251.Google Scholar
- Neumann Jv, Morgenstern O: Theory of Games and Economic Behavior(Commemorative Edition). 1944, Princeton: Princeton University Press, 704:Google Scholar
- Schafer C: Medical decision-making of the patient in the context of the family: results of a survey. Support Care Cancer. 2006, 14 (9): 952-959. 10.1007/s00520-006-0025-x.View ArticlePubMedGoogle Scholar
- Offredy M: Decision-making in primary care: outcomes from a study using patient scenarios. J Adv Nurs. 2002, 40 (5): 532-541. 10.1046/j.1365-2648.2002.02410.x.View ArticlePubMedGoogle Scholar
- Keeney R, Raiffa H: Decision with multiple objectives: preferences amd values tradeoffs. 1976, New York: Wiley, 1:Google Scholar
- Berger J: Statistical decision theory and bayesian analysis. 1985, New York: Springer-Verlag, SecondView ArticleGoogle Scholar
- Campello-de-Souza FM: Decisões Racionais em Situações de Incerteza. 2007, Universitária UFPE, RecifeGoogle Scholar
- Hornberger JC, BW Brown Jr, Halpern J: Designing a cost-effective clinical trial. Stat Med. 1995, 14 (20): 2249-2259. 10.1002/sim.4780142008.View ArticlePubMedGoogle Scholar
- Swensson RG: Using localization data from image interpretations to improve estimates of performance accuracy. Med Decis Making. 2000, 20 (2): 170-185. 10.1177/0272989X0002000203.View ArticlePubMedGoogle Scholar
- Kadane JB: Bayesian methods for health-related decision making. Stat Med. 2005, 24 (4): 563-567. 10.1002/sim.2036.View ArticlePubMedGoogle Scholar
- Etzioni RD, Kadane JB: Bayesian statistical methods in public health and medicine. Annu Rev Public Health. 1995, 16: 23-41. 10.1146/annurev.pu.16.050195.000323.View ArticlePubMedGoogle Scholar
- Ashby D: Bayesian statistics in medicine: a 25 year review. Stat Med. 2006, 25 (21): 3589-3631. 10.1002/sim.2672.View ArticlePubMedGoogle Scholar
- Fayers PM, Ashby D, Parmar MK: Tutorial in biostatistics Bayesian data monitoring in clinical trials. Stat Med. 1997, 16 (12): 1413-1430. 10.1002/(SICI)1097-0258(19970630)16:12<1413::AID-SIM578>3.0.CO;2-U.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/8/51/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.