1. Algorithms in Bioinformatics
ANNs with Keras API
Algorithms in Bioinformatics

2. Who am I? PhD student with Morten Graduated in March
Warning! I am no expert
Who am I?

3. What’s up for today? Play with Keras API Why is Keras smart?
Other frameworks?

4. How to build an Artificial Neural Network?
Invariant parameters
Input dimension
Output dimension
Semi-variable
Loss-function
Variable
# hidden layers
# hidden neurons
Activation functions
GD optimizations
Add-ons
Normalization
Regularization
Dropout

5. How do I build the best network?
You most likely won’t

6. Load Data.
Define Model.
Compile Model.
Fit Model.
Evaluate Model.
Getting started

7. Load data Peptides in amino acid sequence
Encode AA using BLOSUM : independend variable (X)
Binding affinity : dependend variable (y)
We will take a look at it later

8. Define model
from keras.models import Sequential from keras.layers import Dense import numpy as np # fix random seed for reproducibility np.random.seed(7) # create model model = Sequential() model.add(Dense(12, input_dim=INPUT_DIMENSIONS, activation='relu')) model.add(Dense(8, activation='relu')) model.add(Dense(1, activation='sigmoid'))

9. Compile model
# Compile model model.compile(loss='binary_crossentropy’, optimizer='adam', metrics=['accuracy'])

10. Fit Model
# Fit the model model.fit(X, Y, epochs=EPOCHS, batch_size=BATCH_SIZE, validation_data=(X_val, y_val))

11. Evaluate model
# evaluate the model scores = model.evaluate(X, Y) print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

12. Avoid overfitting Regularization Dropout Batch normalization
from keras.regularizers import l2
model.add(Dense(number_of_neurons, activation = 'relu’, kernel_regularizer=l2(0.001)))
Dropout
from keras.layers import Dropout
model.add(Dropout(0.2))
Batch normalization
from keras.layers.normalization import BatchNormalization
model.add(BatchNormalization())
Regularizer
l1: Activity is calculated as the sum of absolute values.
l2: Activity is calculated as the sum of the squared values.
l1_l2: Activity is calculated as the sum of absolute and sum of the squared values.
Dropout
small dropout value of 20%-50% of neurons
Use a larger network. You are likely to get better performance when dropout is used on a larger network, giving the model more of an opportunity to learn independent representations.
Use dropout on incoming (visible) as well as hidden units
Use a large learning rate with decay and a large momentum.
Constrain the size of network weights. A large learning rate can result in very large network weights. Imposing a constraint on the size of network weights such as max-norm regularization with a size of 4 or 5 has been shown to improve results.

13. Give it a go!
Exercise can be found on padawan in /home/people/herpov/algorithms_in_bioinformatics/ Contains a notebook, the relevant data, and exercise instructions