1. Testing Dose-Response with Multivariate Ordinal Data Bernhard Klingenberg Asst. Prof. of Statistics Williams College, MA Paper available at www.williams.edu/~bklingen In Collaboration with Aldo Solari, Luigi Salmaso and Fortunato Pesarin, University of Padova

2. Outline  Introduction Safety and Toxicity Data Notation and hypothesis of interest Stochastic Ordering Theorem (SMH IJD)  Testing SMH Simple Test statistics Permutation Approach Step-down methods for indiv. endpoint significance Increase power  Example Parallel, 5 dose group study with rats (8 rats per dose group) 25 Adverse Events from exposure to Perchlorethylene

3. Introduction: Safety and Toxicity Safety and Toxicity Data:  To capture large number of possible manifestations of a dose (exposure) effect on safety or toxicity: Multiple Endpoints  One such collection of endpoints to evaluate neurophysiological effects: Functional Observational Battery (FOB)  Others: Drug Safety, Disease progression

4. Introduction: FOB  Goal: Evaluation of neurophysiological effects to a toxin (Perchlorethylene)  Data 1 :  2 groups (No exposure vs. 1.5g/kg exposure)  8 rats in each group  Each evaluated at 25 endpoints (various effects), grouped into 6 domains  Response ordinal, on a scale from 1 (no effect) to 4 (most severe reaction) 1 Moser (1986) Journal of the American College of Toxicology

5. Introduction: FOB

6. Introduction: Notation  k-dimensional response vectors: ControlTreatment  Random Sample ControlTreatment  Hypothesis of interest: “No dose effect” d st

7. Introduction: No Toxicity  “ ”: For all response sequences Control Treatment  “ “: Stochastically larger 2 Control Treatment  Note: Rejection of H 0 does not lead to H 1 d st 2 Marschall & Olkin (1979) Inequalities: Theory of Majorization and Its Applications

8. SMH  Usually only interested if k margins are equal or not. I.e., for each adverse event,  Def.: Simultaneous Marginal Homogeneity (SMH) 3 : Vector of marginal probabilities are equal under the two exposures, for all adverse events simultaneously 3 Agresti and Klingenberg (2005) JRSS C, Klingenberg and Agresti (2006), Biometrics

9. SMH SMH with just two adverse events Control Treatment 1234 1212 … 3 4 Lacrimation Arousal Lacrimation Arousal 1234 1212 … 3 4

10. SMH  Theorem:  Prior assumption plausible when dealing with adverse events data (increase in exposure shift towards higher outcome categories) IJD SMH Cumulative marginal inhomogeneity:

11. Testing SMH  Consequence of Theorem: If prior assumption plausible, can use permutation approach to test hypothesis of SMH  Test for SMH: Modeling approach via cumulative logits (proportional odds form) 4  Estimation (ML, conditional ML, GEE,…) computationally impossible, Asymptotics invalid 3 4 Han, Catalano, Senchaudhuri, Metha (2004) Exact Analysis of Dose Response for Multiple Correlated Binary Outcomes, Biometrics.

12. Testing SMH  Let  Simple test statistic: Standardized differences in marginal sample proportions   given by (from multinomial assumption):

13. Testing SMH  To take advantage of ordinal nature: Consider scoring function  Let be score matrix  Look at difference in mean scores:  Estimate covariance matrix 4  under SMH  assuming working independence

14. Testing SMH  Test statistic for sparse data, ignoring correlation among adverse events: with  This gives global test of safety/toxicity  Permutation approach: 16!/(8!8!) = 12870 possible permutations, many leading to identical values of  Advantage of permutation approach: Incorporates dependence by resampling entire vectors; exact significance levels

15. Testing SMH  Example: Arousal Endpoint Computation with equally spaced scores: Note:

16. Testing SMH  Permutation Distribution: observed Perm. Distr. Asympt. Distr.

17. Testing SMH  Identifying which individual adverse events are significant leads to multiple hypotheses testing:  Use test statistic (standardized mean score difference) for individual tests  Multiplicity adjustments via step-down approach of Westfall &Young (1993), using distribution of maximum test statistic

18. Testing SMH  Permutation Distribution: Observed maximum Perm. Distr.

19. Testing SMH

20.  How sensitive are results to assigned scores?  Consider the scores that maximize (obtainable via isotonic regression; data-driven)  Appropriate for safety/toxicity data; maximizes the contrast btw. the mean score differences Equally spaced scores: With

21. Testing SMH

22. Testing Domains Domain effects?  Some endpoints may measure similar effects  Multiplicity adjustment at the endpoint level may be too conservative, leading to some false negatives  Adjusted P-value for domain less than or equal to smallest adjusted P-value within domain  “Proof”: Let endpoint h be in the first domain Dom 1 :

23. Testing Domains Important Consequence (Robustness Property):  Consonant domain test statistic:  Reject only (at domain level) if at least one endpoint within domain significant  If no significant endpoint, domain also not significant For domain significance, it is irrelevant how many, potentially non-significant endpoints are grouped into a domain! * * Provided the same test statistic is used for all intersection hypotheses

24. Testing Domains  Dissonant domain test statistic:  Accumulate effects over endpoints within domain  Even though no individual endpoint is significant, several marginally significant ones can result in significant domain P-value

25. Summary  Testing dose-response for multivariate ordinal data  Correlated ordinal responses (typical for toxicity or safety data) are often sparse and imbalanced use permutation approach  Instead of modeling dose-response, we focused on testing SMH vs. stochastic ordering  SMH IJD, but SMH IJD under  Test statistic: z h = (difference in mean scores) / s.e., for each endpoint, assuming working independence  s.e. derived from multinomial model and estimated under SMH  Multiplicity adjusted P-values for each endpoint from Westfall and Young’s step-down procedure st

26. Conclusions for FOB  Neuromuscular domain showed significant effect  Domain P-value 0.003 with dissonant test  Domain P-value 0.025 with consonant test  Adverse events in neuromuscular domain that show increased toxicity at the 1.5 g/kg exposure level when compared to control  Sensorimotor domain also shows increased level of toxicity, although no individual adverse event is significant  Gait (p= 0.025)  Hindelimb (p=0.044)  Forelimb (p=0.096)

27. Did not show  How methods extend to several dose levels  Effect of discreteness (used mid P-values throughout)  Combining P-values instead of test statistics, with functions other than the maximum Thank you and Go Gators