How do you interpret a test result?

An autoimmunity test result includes several values. The most important performance values to interpret in autoimmune diagnostics are:

1.  Sensitivity and specificity

2.  ROC curve

3.  Positive and negative predictive value

4.  Positive and negative likelihood ratio

 

1. Sensitivity and specificity

Sensitivity is the number of true-positive test results.

 

TP

 

Number of true-positive test results

Sensitivity =

----------

=

-------------------------------------

 

TP+FN

 

all patients with disease

(TP = True positives, FN = False negatives)

 

Specificity is the number of negative results in persons who do not have the disease (true-negative test results in controls).

 

TN

 

Number of true-negative test results

Specificity =

----------

=

-------------------------------------

 

TN+FP

 

all patients without disease

(TN = True negatives, FP = False positives)

The control group could be a population of healthy individuals. However, to evaluate the specificity of a test realistically, the control population should consist of patients with diseases that are important in the differential diagnosis. For example, the specificity of a test for coeliac disease should be evaluated with a population of patients with other gastrointestinal diseases such as inflammatory bowel diseases, gastrointestinal infections, etc.

Example:

The prevalence of rheumatoid arthritis (RA) in the population tested is 2%. Therefore, 100 of 5,000 tested individuals will have rheumatoid arthritis. The remaining 4,900 are healthy or have a disease other than rheumatoid arthritis.

With the test in this example, 73 of 100 RA patients had a positive test result (true positives). Therefore 27 were not detected and had a negative test result (false negatives). In the control group of 4,900 individuals who do not have RA, 73 were positive (false positives) and 4,827 were negative (true negatives).

 Test positiveTest negativeTotal
RA 73 27 100
Non-RA 73 4,827 4,900
Total 146 4,854 5,000

The sensitivity of this test is 0.73 or 73% (number of true positives = 73/number of patients = 100) and the specificity is 0.985 or 98.5% (number of true negatives = 4,827/number of controls = 4,900).

Relevance of specificity

In the example above, 73 out of 5,000 individuals would test false-positive. This means 73 individuals are referred to a specialist or, in the worst case, are treated with toxic medication.

If the test has a lower specificity of, for example, 92% (which still sounds very reasonable) the false-positive results increase dramatically.

 Test positiveTest negativeTotal
RA 73 27 100
Non-RA 392 4,508 4,900
Total 465 4,535 5,000

There is a risk of incorrect diagnosis of RA in 392 individuals, more than five times more individuals than when the specificity is 98.5%.

Back to top 

 

2. ROC curve

The sensitivity of a test is higher when a low cut-off is chosen. A low cut-off leads directly to a lower specificity. The cut-off in autoimmune tests is always a balance between sensitivity and specificity. These values correlate inversely with each other; for every sensitivity value there is a corresponding specificity value. This relationship can be illustrated in an ROC curve.

Definition:

In signal detection theory, an ROC (receiver operating characteristic) curve is a graphical plot of the sensitivity, or true positives, vs. (1-specificity), or false positives.

Example:

A study by Bizzarro et al. (Clin Chem 2007; 53:1527-33), compared 11 tests for the detection of antibodies associated with rheumatoid arthritis. With the serum panel tested, the following ROC curve was true for EliA CCP:

ROC CCP

An optimal cut-off for a test is chosen to make sensitivity and specificity as high as possible. For EliA CCP, the optimal cut-off reveals a specificity of 98.5% and a sensitivity of 73%.

ROC RF

A classic RF test was compared in the same study using the same sera. Using the recommended cut-off, sensitivity and specificity were 54% and 86.1%, respectively.

For optimal comparability of different tests, the specificity can be set to a particular value and the respective sensitivities can be calculated using an ROC curve. In this example, the respective sensitivities were compared at defined specificities of 99%, 98% and 97%.

Assaysens. at recommended cut-offspec. at recommended cut-offsens. at 99% spec.sens. at 98% spec.sens. at 97% spec.
EliA CCP 73% 98.5% 69% 74% 74%
RF 54% 86.1% 13% 17% 17%

 

Back to top

 

3. Positive and negative predictive value

Definition:

The positive predictive value (PPV), or precision rate, or post-test probability of disease, is the proportion of patients with positive test results who have the disease. Predictive value is related to the sensitivity and specificity of the test or screening method.

 

TP

 

Number of patients positive

PPV =

----------

=

-------------------------------------

 

 TP+FP

 

all positive results

(TP = True positives, FP = False positives)

 

The negative predictive value (PPV) is the proportion of control patients with negative test results who are correctly diagnosed.

 

TN

 

Number of controls negative

NPV =

----------

=

-------------------------------------

 

 TN+FN

 

all negative results

(TN = True negatives, FN = False negatives)

Example:

 Test positiveTest negativeTotal
RA 73 27 100
Non-RA 73 4,827 4,900
Total 146 4,854 5,000

Taking the same example as in the paragraph about sensitivity and specificity (see above) for a test in rheumatoid arthritis (RA), the PPV is 50% (73 TP/146 P). The NPV is 99% (4,827 TN/4,854 N).

With this population, a clinician must be aware that half the positive results are in individuals that do not have RA. A positive result predicts the disease with a probability of 50%. On the other hand, a negative result predicts with a probability of 99% that the disease is not present.

Dependency on pre-test probability

The predictive values are highly dependent on the pre-test probability, which is rather small at 2% in this example. If the pre-test probability were 10% (e.g. because it is a specialist rheumatology laboratory), the data would change accordingly: 

 Test positiveTest negativeTotal
RA 370 130 500
Non-RA 60 3,940 4,000
Total 430 4,570 5,000

The performance of the test and the clinical features of the marker do not change with the pre-test probability. The sensitivity and specificity of the test are therefore relatively fixed. However, the positive predictive value increases with a five times higher pre-test probability from 50% to 86% (370 TP/430 P), and the negative predictive value falls from 99% to 86% (3,940 TN/4,570 N).

Back to top

 

4. Positive and negative likelihood ratio

Definition:

The likelihood ratio incorporates both the sensitivity and specificity of the test, and provides a direct estimate of how much a test result will change the odds of having a disease.

The likelihood ratio for a positive result (positive LR) tells you how much the odds of the disease increase when a test is positive.

 

TP/(TP+FN)

 

Sensitivity

pos LR =

-----------------

=

---------------

 

FP/(FP+TN)

 

1 – Specificity

(TP = True positives, TN = True negatives, FP = False positives, FN = False negatives)

The likelihood ratio for a negative result (negative LR) tells you how much the odds of the disease decrease when a test is negative.

 

FN/(TP+FN)

 

1-Sensitivity

neg LR =

-----------------

=

-------------------

 

TN/(FP+TN)

 

Specificity

(TP = True positives, TN = True negatives, FP = False positives, FN = False negatives)

Example:

 Test positiveTest negativeTotal
RA 73 27 100
Non-RA 73 4,827 4,900
Total 146 4,854 5,000

Taking the same example as in the paragraphs above for a test in rheumatoid arthritis (RA), the positive LR is very high, with (73/100)/(73/4,900) = 50. The likelihood is very high that a patient with a positive test result has RA. The negative LR is (27/100)/(4,827/4,900) = 0.27.

Interpretation of likelihood ratio:

 Negative LRPositive LR
no clinical value = test quality is not useful 1 1
small difference that may be relevant 0.2-0.5 2-5
modest, but substantial difference 0.1-0.2 5-10
clinically important difference = test quality is very useful <0.1> >10

 

The marker in the example above has a very high positive LR of 50. A positive result therefore indicates the disease with a high probability. However, a negative result does not exclude the disease. With a negative LR of 0.27, the clinical usefulness is only small.

Back to top