1. Comparison of Performance of Laboratory Equipment
Dr. Angela A. Manginelli1, Dr. Walter Hoyer2, Dr. Gordon Brestrich3
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D Marburg, Germany.
Abstract
The Three Liquid Handlers
Within the validation of a new highly automated assay, the Clinical Laboratory Science tested several clinical samples in order to evaluate precision and linearity of the assay.
The equipment utilized for the testing includes three liquid handlers (highly automated pipetting robots).
The statistical evaluation of the data revealed a significantly poorer performance of one liquid handler compared to the other two.
To detect the root cause of the issue, an intensive work including complete dismantling of the machine with help of a technician of the vendor was conducted.
A tiny difference in the programming code of the liquid handlers was detected.
JMP was used for the statistical evaluation before and after the adjustment of the setting of the liquid handlers.
Maja
Bonnie
Kriemhild

2. Comparison of Performance of Laboratory Equipment
Dr. Angela A. Manginelli1, Dr. Walter Hoyer2, Dr. Gordon Brestrich3
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D Marburg, Germany.
How The Assay Works – Serial Dilution Scheme
Predilution and Titer Determination
Human
complement
Bacteria
Serum of vaccinated
donors
Step 2
....
384 Well – Micro titer plate
Buffer (diluent)
Automated
pipetting
Buffer
14 µL
Transfer 14 µL
Step 1
Mix
1:2
1:4
1:8
Step 3
384 Well –
Micro titer plate
Predilution
Sample +
Buffer Mix 1: µL
Sample 1
Sample 1  1:2
Sample 1  1:4
Sample 2
Sample 2  1:2
Sample 2  1:4
Normalized values
Log(Dilution)
Inflection Point
Serial dilution is used to determine titer.
The bactericidal antibody titer is the reciprocal dilution of the serum at which 50% of bacteria are killed.
After incubation time, the number of surviving bacteria is determined by means of measurement of fluorescence intensity.

3. Comparison of Performance of Laboratory Equipment
Dr. Angela A. Manginelli1, Dr. Walter Hoyer2, Dr. Gordon Brestrich3
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D Marburg, Germany.
Dilutional Linearity
Results
Error bars:
95% confidence interval of the mean
Samples are tested on three liquid handlers which are expected to deliver identical results.
Relative accuracy of the assay is evaluated by means of dilutional linearity.
Dilutional linearity is assessed by computing Recovery Rate (RR): ratio between the geometric mean (GM) of the two replicates at one predilution and the GM at previous predilution.
20 samples (serum)  40 RRs on each robot (one for 1:2 and one for 1:4 predilution).
Liquid Handler
Mean RR [%]
Std Dev RR
Bonnie 90
0.28
Kriemhild 98
0.10
Maja 97
0.12
Lower RR and higher variability on Bonnie.
Difference in RRs was significant (GLM) when removing the outlier circled in red (p = ).
Lab Manager
Statistician
Lab Operator
The reaction of the team

4. Comparison of Performance of Laboratory Equipment
Dr. Angela A. Manginelli1, Dr. Walter Hoyer2, Dr. Gordon Brestrich3
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D Marburg, Germany.
Investigation and Solution
Statistical Confirmation
Lab operator and technician of the vendor were locked for three weeks in the lab to investigate the issue.
They compared the setting of the three liquid handlers ...
... and found out that the pipet of Bonnie was set to a different mixing speed compared to the other two: ~16 µl/sec vs ~24 µl/sec
The setting was adjusted and the experiment was repeated:
24 Samples: undiluted, 1:2 and 1:4 prediluted 2 Replicates (to be averaged) for each predilution on each robot  18 measurement for each sample, 6 per robot 2 RRs for each sample on each robot (for predilutions 1:2 and 1:4)
Add Image
Conclusion
Stay calm, keep trying!
Never underestimate the importance of small things!
Luckily with JMP it is possible to write very nice and useful scripts!

5. Comparison of Performance of Laboratory Equipment
Dr. Angela A. Manginelli1, Dr. Walter Hoyer2, Dr. Gordon Brestrich3
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D Marburg, Germany.
JMP Platform/Script Used
Graph Builder and ANOVA model
Script to do the analysis BEFORE and AFTER the issue:
Log function
Import xlsx file
Lag function to create RR
Stack and adjust column names
Munger function