1. Algorithms and data structures
Simulated annealing
Tabu search
Genetic algorithms
Ant algorithms
Neural networks

2. Simulated annealing
SA heuristic considers some neighboring state s* of the current state s, and probabilistically decides between moving the system to state s* with probability or staying in-state s
When T tends to zero, the probability P ( e , e ′ , T ) must tend to zero if e ′ > e and to a positive value otherwise. A B

3. Simulated annealing state = initial_state k = 0 while k < kmax :
T = temperature(k ∕ kmax)
snew = pick_a_random_neighbour(state)
if P(E(state), E(snew), T) ≥ rand():
state = snew
k += 1
T(k) = t0e-ak

4. SA – sample application
Graph coloring:
Initial_state, state – valid coloring
pick_a_random_neighbour (s) – a coloring that was obtained by changing a color (or a number of colors dependent on T) in state s to another colors
P: E(s) –> number of colors in s
random factor e(T)

5. Tabu search
Improving changes (moves) are accepted. The potential solutions that have been previously visited within a certain short-term period or have violated a rule, are marked as "tabu" (forbidden).

6. Tabu search bestCandidate = state = initial_state
tabuList = [initial_state]
while (not stoppingCondition()):
sNeighborhood = getNeighbors(bestCandidate)
bestCandidate = sNeighborHood.firstElement
for sCandidate in sNeighborHood:
if (not tabuList.contains(sCandidate) and \ (E(sCandidate) > E(bestCandidate)) ) :
bestCandidate = sCandidate
if E(bestCandidate) > E(state): state = bestCandidate
tabuList.push(bestCandidate)
if tabuList.size > maxTabuSize : tabuList.removeFirst()

7. TS – sample application
Graph coloring:
Initial_state, state – valid coloring
E(s) –> number of colors in s

8. Genetic algorithm Initial population of randomly generated individuals
In each generation, the fitness of every individual in the population is evaluated;
The more fit individuals are stochastically selected from the current population, and each individual's genome is modified (recombined and possibly randomly mutated) to form a new generation.
The new generation of candidate solutions is then used in the next iteration of the algorithm.
The fitness is usually the value of the objective function for the optimization problem being solved.

9. GA - sample appliction Salesman route:
Recombination: for parent-individuals some parts of solution are exchanged between parent-individuals
Mutation: a part of soluton (a single edge in the way?) is randomly replaced by another route

10. Ant colony algorithm
"simulated ants" walking around the graph representing the problem
ants (initially) wander randomly, and upon finding food return to their colony while laying down pheromone trail
If other ants find such a path, they are likely not to keep travelling at random, but instead to follow the trail, returning and reinforcing it if they eventually find food

11. ACA – sample application
Salesman route
It must visit each city exactly once;
A distant city has less chance of being chosen (the visibility);
The more intense the pheromone trail laid out on an edge between two cities, the greater the probability that that edge will be chosen;
Having completed its journey, the ant deposits more pheromones on all edges it traversed, if the journey is short;
After each iteration, trails of pheromones evaporate.

12. Neuron model Discrete y = 1 if xiwi >= P or 0 otherwise
Contineous y = 2/(1-e-net), where net = xiwi y w1 wk
..... x1 xk

13. Neural network O1 O2
Output layer
Hidden layer(s)
Input layer I1 I2 I3

14. Topology Input/output sizes depend on imput size/categories count
The number and size of hidden layers should be determined experimentally
There are possible inter-layes connections, slow-down connections, loop-back connections, different transition functions, convolutions etc.

15. Interesting properties
Supervised learning
Preparation of training set – correct answers:
Back propagation algorithm
Interesting properties
Error resistance
Intrinsic recall

16. A Classifier O1 O2 Output layer Hidden layer(s) ..... Input layer I1

17. Sample applications
Hand written text recognition and image classification
Diagnistic skin deeseases
Risk odf bank loan estimation
Detection of the natural deposits
Real estates values apraisal
Fraud detection