1. Regulatory-Industry Statistics Workshop , 2019
How “Gray Zone” is utilized in blood donor screening and diagnostic devices
Chunrong Cheng, Ph.D.
Zhen Jiang, Ph.D.
FDA/CBER/OBE/DB
Regulatory-Industry Statistics Workshop , 2019
Washington, DC

2. Disclaimer
Views expressed in this presentation are those of the speakers and not necessarily of the Food and Drug Administration.

3. Outlines Background Examples and analyses Summary Gray Zone (GZ)
In Vitro Diagnostics Devices (IVDs) at CBER
Examples and analyses
Summary

4. CBER (Center for Biologics Evaluation and Research)
IVDs for infectious diseases
Blood donor screening (e.g., HIV, HBV, HCV, Zika, Babesia, Chagas, CMV, HTLV, West Nile Virus, etc.)
Diagnostics (e.g., HIV and HTLV)

5. How GZ is used
Used in clinical investigation only, not in the real practice
Adjust the cutoff
Used in both clinical investigation and real practice as part of the interpretation
Intermediate:
Retest in duplicate for initial GZ, then classify the sample into either nonreactive or reactive
Final
Retest in duplicate for initial GZ, then classify the sample into either nonreactive, GZ, or reactive
There are two ways GZ is used in CBER submissions

6. Blood donor screening
Usually , GZ is used in clinical investigation, not at the blood centers.
Avoid the uncertainty regarding donor eligibility (go or no-go), with only one method.
The cutoff value is set to be conservative, due to the concern of recipient safety.
Intended population: healthy donors. It is rare to have reactive retest results for initial GZs , and they are likely to be false positives (e.g., cross-reactivity) which means waste of good blood products.
There is another cutoff for the GZ, such as or , which is not the focus of this talk.

7. Diagnostics
GZ is used in clinical investigation, and may also be used in real practice (point of care).
Intended population: subjects at higher risk or exposure, fewer false positives are expected.
GZ or positive results will be confirmed by other tests and clinical presentations.
No need to include GZ in the real practice if the clinical investigation shows it does not help identifying additional positive subjects.

8. GZ used in investigation only
S/CO<0.80
0.8≤S/CO<1
S/CO≥1.00
Nonreactive
Initial gray zone nonreactive
Initial reactive
No retesting
No retesting, go to supplementary test.
Retest in duplicate, if either is GZ or reactive, then go to supplementary testing: (0.9, 1.2, 0.7), (0.9, 0.9, 0.7)
Performance is based on initial GZ results as nonreactive
No retesting.
Retest in duplicate, repeatedly reactive if either is reactive.
GZ used in investigation only, which is the main focus of this talk. The lower end of GZ can be other value. In most cases, retesting is required for initial GZ or reactive.
For simplicity, the examples in this talk we just say GZ (could be initial or repeatedly).

9. Questions
For samples with a GZ result, should their results of supplementary testing be used in analysis? Yes
How does it affect specificity and sensitivity?
Does it affect donor screening and diagnostic devices differently?

10. Specificity Can be used to assess cutoff
Test: – (<0.8)
Comparator: –
Negative, no more testing
Test: +
Comparator: +
Test: +
Comparator –
Test: –(<0.8)
Comparator +
Supplementary testing
Test: – Comparator +
Test: GZ Comparator +
Can be used to assess cutoff
Test: GZ Comparator: –
Additional information to assess cutoff by including GZ in investigation

11. Healthy donors or low risk population (n=6,000)
Specificity
Healthy donors or low risk population (n=6,000)
Test –
Truth –
Test + Truth –
Truth +
Test +
Test: GZ
Comparator –
Truth?
5,968 10 2
Truth is determined by supplementary tests
For simplicity, omit the results of comparator
Theoretically, this could be >0, but we do not expect any for blood screening assay and very few for diagnostic, so put it 0 for simplicity,

12. Specificity Truth Action + - Blood screening Diagnostic GZ (n=10) 10
Truth
Action + -
Blood screening
Diagnostic
GZ (n=10) 10
Expected, do nothing
Do nothing or may need to adjust cutoff 1 9
Rarely seen, conduct risk analysis, or adjust cutoff 3 7
Unexpected, adjust cutoff
May need to adjust cutoff 5 ≥8 ≤2
These are general considerations, not guidelines.
Should be evaluated case by case.
For diagnostic devices, above table and similar analysis of the green box (Test: GZ; Comparator +) can be combined to determine whether keeping GZ in practice.

13. Specificity Truth Specificity + - Ignoring truth Counting truth
Truth
Specificity + -
Ignoring truth
Counting truth
GZ (n=10) 10
99.83%
[99.70%] (5,978/5,988)
[99.70%]
(5,978/5,988) 1 9
(5,977/5,987) 3 7
(5,975/5,985) 5
(5,973/5,983)
(5,968/5,978)
Decrease the point estimate and CI slightly, can be ignored if sample size is big enough.
Although sensitivity is not the focus, false negative results should be reported in the Package Insert (PI), if any.

14. Sensitivity
No impact on known positive samples, because samples either nonreactive or GZ will be tested by supplementary tests.

15. Results on high risk population (1)
High risk population with a low prevalence (n=1,000)
Test –
Truth –
Test + Truth –
Truth +
Test +
Test: GZ
Comparator –
Truth? 9
968 2 1 19 10
Same testing procedure as the donor population.
Specificity: decrease the point estimate and CI a little more because 1) the denominator is smaller, and 2) more false negative are expected: 99.80% [99.26%] (978/980) vs % [99.26%] (968/970).
False negative results should be reported in the PI, if any.

16. Results on high risk population (2)
High risk population with a high prevalence (n=1,000)
Test –
Truth –
Test + Truth –
Truth +
Test +
Test: GZ
Comparator –
Truth? 9
495 5 1
479 20
If nonreactive or reactive by both Test and comparator, supplementary testing is not done.
Specificity: decrease the point estimate and CI further more because denominator is even smaller, but can still be minimum
Sensitivity?

17. Results on high risk population (2)
Truth
Sensitivity + -
Ignoring truth
Counting truth
GZ (n=20) 20
99.79%
[98.84%]
(479/480)
99.79% [98.84%] (479/480) 5 15
98.76% [97.33%] (479/485) 10
97.76% [96.02%] (479/490)
96.77% [94.80%] (479/495)
95.8% [93.65%] (479/500)
Sensitivity: be reduced substantially since the numerator remains the same but the denominate increases.

18. Summary How does it affect specificity and sensitivity?
No impact on sensitivity on known population.
Slightly reduce specificity in low risk or donor population, or high risk population with a low prevalence.
May substantially reduce sensitivity in high risk population with a high prevalence.
Does it affect donor screening and diagnostic devices differently? Yes

19. Acknowledge
Dr. Pradip Akolkar from CBER/OBRR/DETTD for enormous valuble input.

20. Thank You!