1. Bill Lotter, Harvard Biophysics PhD Candidate
PredNet Code Overview
Bill Lotter, Harvard Biophysics PhD Candidate

2. Review of Model

3. Code Overview Implemented in Keras (DL library by Francois Chollet)
Keras doesn’t allow cyclical graphs => Had to implement as it’s own layer

4. Parameters of Model

5. Parameters of Model
stack_sizes: number of channels in targets (A) and predictions (Ahat) in each layer of the architecture
Ex. (3, 16, 32) => 3 layer model with RGB input and 16 and 32 channels in the second and third layers

6. Parameters of Model
R_stack_sizes: number of channels in the representation (R) modules

7. Parameters of Model
[A][Ahat][R]_filt_sizes: size of filters in each of the layers
Ex. R_filt_sizes = (3,3,3) =>
3 layer model with 3x3 convolutions for all filters in convLSTM

8. Parameters of Model
extrap_start_time: time step for which model will start extrapolating Starting at this time step, the prediction from the previous time step will be treated as the "actual"
Ex. if set to 5, then after 5 timesteps, the model won’t look at incoming frame and will iterate over predictions

9. Parameters to Potentially Experiment With
Number of layers in model
Number of filters (R_stack_sizes, stack_sizes)
Filter Sizes

10. Hyperparameters from Previous Models

11. Hyperparameter Suggestions
Number of layers: 3, 4 or 5
Number of filters (stack sizes):
increase by 2x or 1.5x per layer
start at 16, 32, or 48
Filter Sizes: 3x3 convolutions are pretty standard
could try 5x5 on lowest layer if want