1. Identifying robust cancer treatment protocols from small experimental datasets
Jana Gevertz, Associate Professor
Department of Mathematics & Statistics
The College of New Jersey (TCNJ)

2. outline A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols
A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols

3. A mathematical biology paradigm
A fairly standard paradigm in mathematical biology:
Identify unresolved biological question
Develop a mathematical model to address the unresolved problem
Compare model predictions to biological data
Re-work model until it well-describes biological data
Use model to seek answers to unresolved biological question
Test model predictions
Biological question
Feedback
Develop mathematical model
Feedback
Compare to data and make predictions

4. Focus: model validation
And, this paradigm exists for a reason! Here let’s focus on the process of validating a mathematical model
Often, we have a small number of samples from a given experiment at our disposal
A model is typically fit to the average of the experimental data (with some consideration of the variance)
A model that well- describes the average data is deemed a “good” model for making biological predictions

5. Consequences for model predictions
Is a biological prediction that is good for the “average” of a sample useful for samples that deviate from the average?
DEPENDS!
If individual samples do not deviate significantly from the average, prediction may be applicable
Even if this is not the case, a prediction that is robust (not highly dependent on best-fit parameters) could still apply in samples that deviate from the average
Notice how no mice in the experiments I show here really behave like the “average” – some respond much better to the treatment protocol, and others respond much worse.

6. Project goals
Explore consequences of finding optimal treatment protocols in a mathematical model that is fit to the average of sample experimental data
Context: Melanoma response to treatment with oncolytic viruses, dendritic cell injections, and combinations of the two therapeutics
How:
Hierarchically build and fit model to (average) data
Find optimal drug combination, and quantitatively explore the protocol’s robustness
Expand robustness analysis to search for a robust optimal treatment combination

7. outline A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols
A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols

8. What data do we have? Treatment #1
Mice injected with melanoma cells may be treated with oncolytic viruses, viruses engineered to lyse and kill cancer cells
To date herpes virus is only virus clinically approved in the US for treatment of cancer, and that cancer happens to be the same one studied here – melanoma
Since late 1800s, doctors observed that some patients with cancer go into remission, if only temporarily, after a viral infection. In 1990s, idea expanded to allow these viruses to act as vectors delivering therapeutic molecules to tumor site. In 2000s, virotherapy began to be recognized as an immunotherapy in and of itself
These viruses are further designed so that they release molecules at the tumor site that enhance the immune system’s ability to recognize a tumor as “foreign” – immunotherapy

9. experimental data: Tumor volume for various treatment protocols
Melanoma cells are implanted in right abdomen of genetically identical mice. Treatment is initiated when tumor reaches a volume of ~100 mm3, with 8-9 mice per treatment cohort
Standard error = standard deviation/sqrt(sample size)
5x10^5 cell implanted. When tumor volume reached around 100 mm3, animals were sorted into groups with similar mean tumor volumes
Tumor growth was monitored everyday by measuring two perpendicular tumor diameters using calipers. Tumor volume was calculated by the following formula: volume = LW2, where L is length and W is width.
OV dose: either 1010 (Ad, Ad/4-1BBL) or 5x109 (Ad/IL-12, Ad/4-1BBL/IL-12) OVs on days 0, 2, 4
OV dose: 2.5x109 OVs on days 0, 2, 4
DC dose: 106 DCs on days 1, 3, 5
J.H. Huang et al., Mol. Ther. 18:

10. Hierarchical Model & Fit: Untreated tumor
U = uninfected tumor cells

11. Hierarchical Model & Fit: oncolytic virus
U = uninfected tumor cells
I = infected tumor cells
V = viruses

12. Biological considerations for further model development
So far, our model accounts for tumor growth and the lysing effects of oncolytic viruses. But, it does not account for the immune-boosting molecules added into the virus
Booster #1 (4-1BBL) and Booster #2 (IL-12): increase recognition of cancer cells by immune system (T cells), and make T cells more potent killers of cancer cells
IL-12 is thought to act through an intermediate cell type (naïve T cells) whereas 4-1BBL acts more directly on T cells
J.H. Huang et al., Mol. Ther. 18:

13. Hierarchical Model & Fit: Ovs with 4-1BBL (Ad/4-1BBL)
U = uninfected tumor cells
I = infected tumor cells
V = viruses
T = T cells

14. Hierarchical Model & Fit: Ovs with IL-12 (Ad/IL-12)
U = uninfected tumor cells
I = infected tumor cells
V = viruses
T = T cells
A = naïve T cells

15. Hierarchical Model & Fit: Ovs with 4-1BBL & IL-12 (Ad/4-1BBL/IL-12)
U = uninfected tumor cells
I = infected tumor cells
V = viruses
T = T cells
A = naïve T cells

16. What data do we have? Treatment #2
Mice injected with melanoma cells may be treated with dendritic cell injections, another type of immunotherapy
Dendritic cells are immune cells that present antigens (molecules capable of inducing an immune response) to other cells of the immune system
This antigen presentation eventually results in an adaptive immune response, where the immune system actively seeks out the antigen being presented by the dendritic cells
To be used for tumor treatments, dendritic cells are “taught” to present cancer antigens to the immune system, causing immune system to treat cancer cells as “foreign”
J.H. Huang et al., Mol. Ther. 18:

17. Hierarchical Model & Fit: add DCs (Ad/4-1BBL/IL-12+DCs)
U = uninfected tumor cells
I = infected tumor cells
V = viruses
T = T cells A = naïve T cells
D = dendritic cells (injected)

18. outline A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols
A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols

19. Typical approach to optimizing treatment
With a model that well- describes the AVERAGE experimental data, one typically uses optimization techniques to find the BEST protocol for the average mouse
Mathematically, we considered all 20 combinations of 3 virus doses and 3 DC doses
Optimal strategy: front- load OVs (VVVDDD)
But, response appears sensitive to dosing order 1 15 20
J.R. Wares et al., Math. Biosci. Eng. 12:

20. Impact of OV-DC Dosage on Optimal Treatment
Eliminate tumor means V<10^(-6) mm^3
Bifurcation values (smallest dose necessary to eliminate tumor) for different combinations of OV/DC doses
J.R. Wares et al., Math. Biosci. Eng. 12:

21. Impact of OV-DC Dosage on Optimal Treatment
Eliminate tumor means V<10^(-6) mm^3
Experimental dose is near a bifurcation point: changing the doses slightly can result in drastically different ranking of protocols
What about if mice deviate slightly from average? This certainly raises doubts about the robustness of our optimal protocol
J.R. Wares et al., Math. Biosci. Eng. 12:

22. outline A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols
A mathematical biology paradigm
Case study: modeling tumor response to oncolytic virotherapy and dendritic cell injections
A cautionary tale regarding optimal treatment protocols
The search for robust optimal treatment protocols
We define a robust protocol as one that elicits the same qualitative response to treatment in samples that deviate from the average
Whereas a fragile protocol is one that results in a qualitatively different treatment response in samples that deviate from the average

23. addressing robustness question: virtual populations
We use statistical techniques of bootstrapping and sampling to take our small dataset, and generate a virtual dataset of mice that contains many more mice than our experiments, but should be statistically similar to the mice in the experiments.
Sampling with replacement to create 1000 bootstrap replicates
Parameter fitting to average mouse in each bootstrap replicate
Posterior distributions on each of the fit parameters are estimated from the fitting done to the bootstrap replicates
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

24. Generate & analyze virtual populations
Pseudo-randomly sample the posterior distributions (only selecting parameters in 95% credible interval) to generate many virtual populations that are characterized by their parameterization of the full mathematical model
Determine the optimal protocol (among the 20 previously- discussed options) for each virtual population
Compare treatment response across virtual populations to assess robustness
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

25. So, what have we actually done to assess robustness?
So, instead of only having 8 mice in one experimental population to test whether a treatment is robust (what fraction of the population it is effective for)
We instead have 1000 “virtual” mice populations, which taken together represent the range of treatment responses we could expect in a much larger population of real mice .
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

26. Is the optimal for average robust?
Optimal treatment: V-V-V-D-D-D
Tumor shrunk to below the size of a single cell (effective) in only 30% of the “virtual populations”
Meaning, a treatment that causes the tumor to be eradicated in the “average mouse” (in the experimental population) is not predicted to be effective across a large fraction of virtual mice populations
Therefore, the optimal treatment we predicted is fragile (not robust)
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

27. Other regions of dosing space
Our optimal does not appear to be robust at experimental drug dose, but maybe a different protocol is a robust optimal? Or maybe another dose has a robust optimal?
To search for robust protocols, we have ranked the 20 treatment protocols for each virtual population (from fastest to eradication to largest 30- day volume) in 3 different regions of dosing space:
#1: Intermediate OV & DC (Experimental dose)
#2: High OV (+50%), Low DC (-50%)
#3: Low OV (-50%), High DC (+50%)
#1: 2.5x10^9 OV and 10^6 DC
#2: 10^9 OV and 1.5x10^6 DC
#3: 4x10^9 OV and 0.5x10^6 DC #3 #1 #2

28. dosing #1: Experi- mental OV+DC
“Top” protocol (the one effective in the largest fraction of virtual populations) front- loads OVs, consistent with optimal for average mouse!
But this “top” protocol is effective in only 30% of virtual populations ⇒ not a robust optimal protocol
: Effective
: Ineffective
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

29. dosing #1: experi- mental OV+DC
While the optimal protocol for the average ranks as top protocol 72% of the time, it is the WORST protocol in 14% of the virtual populations
Therefore the “optimal” treatment can actually be harmful to certain virtual populations
Conclude: no robust eradication response at experimental OV/DC dose ⇒ fragile
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

30. dosing #2: high OV and low DC
60% of protocols are ineffective in >90% of virtual populations
“Optimal” protocol is yet again OV-OV-OV-DC-DC- DC. But, it is only effective in 43% of virtual populations ⇒ still not a robust eradication response
: Effective
: Ineffective
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

31. dosing #3: Low OV and high DC
Prior optimal protocol is now the WORST protocol of the 20!
Optimal protocol is now front-loading DCs, which is the opposite of our optimal result in the other dosing regimes
This optimal is effective in 84% of virtual populations ⇒ robust protocol!
: Effective
: Ineffective
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

32. dosing #3: Low OV and high DC
Unlike at intermediate dose, the optimal of D-D-D-V-V-V now ranks as the #1 protocol in ALL virtual populations
This means that even when the protocol can’t eliminate the tumor, it is still the best protocol to give!
That is, we have a robust optimal protocol
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

33. Summary: fragile regions of dosing space
Using our virtual population methodology (VEPART), we were able to explore the robustness of the biological predictions drawn from our mathematical model
Predicted that the experimentally-utilized & high OV/low DC doses are fragile
Problem with fragility:
Significant variability (of increasing complexity) is observed between mice in experimental data
An optimal found in a fragile region of dosing space is not predicted to be effective in samples (mice) that deviate significantly from the average
Control Data
Ad/4-1BBL/IL-12 + DCs
Note we have other results too, including
Vary time between injections (optimal is to give doses closer together, converging to a giving the heaviest dose allowed all at one time, as it gives maximal initial immune response). However this ignores toxicity issues that could well arise from such an injection
Treating with different combinations of 6 doses of OVs and DCs (not constrained to be 3 and 3, so there are 2^6 = 64 possible combinations)
Found combinations with < 2 DC injections did the worst
Combinations with 4+ DC injections did best, many resulting in tumor remission
Using 6 DC doses eliminated the tumor, suggesting that DC vaccines may be the driving force … BUT, a growing body of data supports that DCs are generally inactive in the tumor microenvironment. Our data does not indicate this, and our model does not predict this …

34. Summary: robust regions of dosing space
Our analysis using VEPART found the first robust optimal combination of oncolytic viruses and dendritic cell injections (in the low OV/high DC region of dosing space)
This protocol has the potential to be effective in mice (samples) that deviate some- what significantly from the average!
Note we have other results too, including
Vary time between injections (optimal is to give doses closer together, converging to a giving the heaviest dose allowed all at one time, as it gives maximal initial immune response). However this ignores toxicity issues that could well arise from such an injection
Treating with different combinations of 6 doses of OVs and DCs (not constrained to be 3 and 3, so there are 2^6 = 64 possible combinations)
Found combinations with < 2 DC injections did the worst
Combinations with 4+ DC injections did best, many resulting in tumor remission
Using 6 DC doses eliminated the tumor, suggesting that DC vaccines may be the driving force … BUT, a growing body of data supports that DCs are generally inactive in the tumor microenvironment. Our data does not indicate this, and our model does not predict this …
S. Barish, M.F. Ochs, E.D. Sontag and J.L. Gevertz, Proc. Natl. Acad. Sci. 114: E6277-E6286.

35. Current work: robust vs personalized therapy
The goal of personalized medicine is to tailor treatment protocols to individual patients
While robust protocols are predicted to be effective across individuals that vary from the average, they are not guaranteed to be effective in any particular individual
Personalizing therapy is quite challenging from a clinical and mathematical perspective
For instance, our model has 6 variables whose values depend on parameters embedded in 14D parameter space
This is a high-dimensional parameter space given the limited amount of clinical data available per patient
Before personalizing therapy in our model, we’ve rigorously explored whether our model structure is the minimal one needed to well-describe the data

36. A minimal model for personalizing therapy
Explored impact each parameter has on model dynamics
95% credible intervals (* indicates optimal parameter value): larger the interval, less certainty we have in parameter value
Local sensitivity analysis (find those parameters that give a fit within 10% of optimal fit): larger the % deviation, the less certainty we have in value of parameter
Global sensitivity analysis using Sobol method: the smaller the Sobol index, the less sensitive the model is to value of that parameter
Max % Deviation from Optimal
95% Credible Interval
Sobol Index r
3.0%
[0.93r*,1.14r*] -- β
0.7%
[0.94β*, 1.05β*] cA
17300%
[0.54cA*, 3321cA*]
0.0046 cT
47.2%
[0.01cT*, 2.45cT*]
0.5825 ck
46.3%
[0.0002ck*, 1.90ck*]
0.2631 χD
7.0%
[0.86χD*, 1.80χD*]

37. A minimal model for personalizing therapy
Model exhibits extreme insensitivity to choice of cA parameter, with many points with cA = 0 giving fits within 10% of optimal ⇒ can remove A from model!

38. A minimal model for personalizing therapy
Best fits using original model
Best fits using minimal model
Ad/4-1BBL/IL-12 model, get 3.05% worse fit
Ad/4-1BBL/IL-12 + DCs model, get 0.08% better fit
Fit individual mice with minimal model (after reducing dimensionality of parameter space using Sobol indices)

39. Preliminary results: robust vs personalized therapy
With individualized fits, can test treatment response of each mouse to predicted robust optimal of D-D-D-V-V-V, and to other protocols (in low OV/high DC region of dosing space)
DDDVVV is optimal in 75% of mice (& eradicates tumors in all but Mouse 6)
It is sub-optimal in 25% of mice, but no protocols tested at this dose could eradicate these tumors (Mouse #2, #7)
: Effective
: Ineffective

40. Acknowledgements Thank you! Collaborators on Modeling Work
Lab of Chae-Ok Yun, Hanyang University
Arum Yoon
Il-Kyu Choi (Harvard Medical School)
Joanna Wares, University of Richmond
Peter Kim, University of Sydney
Collaborator on Robustness Work
Michael Ochs, TCNJ
Syndi Barish, TCNJ (former undergrad now at Yale University)
Eduardo Sontag, Northeastern University
Thank you!