1. Digital Optimization
Martynas Vaidelys

2. Optimization
Mathematical optimization (optimization or mathematical programming) is the selection of a best element 𝑠 ∗ (with regard to some criteria) from some set of available alternatives 𝑆 ∗ .
𝑠 ∗ ∈ 𝑆 ∗ = 𝑠 𝑠 = arg min 𝑠∈𝑆 𝑓 𝑠 .
Optimization includes finding “best available” values of some objective function given a defined domain (or a set of constraints), including a variety of different types of objective functions and different types of domains.

3. Objective function The function 𝑓 is called, variously:
an objective function
a loss function
cost function (minimization)
indirect utility function (minimization)
a utility function (maximization)
a fitness function (maximization)
an energy function
A feasible solution that minimizes (or maximizes, if that is the goal) the objective function is called an optimal solution.

4. Continuous vs. Digital
Continuous optimization – the variables used in the objective function can assume real values, e.g., values from intervals of the real line.
Analytical optimization methods
Gradient descent, linear and non-linear optimization…
Digital (discrete) optimization – the variables used in the mathematical program are restricted to assume only discrete values, such as the integers.
Combinatorial optimization
Integer programing

5. Continuous to Digital
Some continuous problems can be transformed to digital optimization problems by:
Changing objective function’s argument type to binary form
Rounding real values to integers

6. Optimization problems
Local minimums (optimums)
Constraints
Feasibility
Existence
Multi-objective, multi-variable optimization
Multi-modal optimization
Computing speed, some discrete optimization problems are NP-complete

7. Classical optimization
Solution is identified by means of enumeration or differential calculus
Allows an analytical solution (e.g. LS estimation)
Existence of (unique) solution presumed
Convergence of classical optimization methods for the solution of the corresponding first-order conditions
Solution can be approximated reasonably well by standard algorithms (e.g. gradient methods)
Straightforward application of standard methods, in general, will not always provide a good approximation of the global optimum

8. Heuristic methods Based on concepts found in nature
Have become feasible as a consequence of growing computational power
Although aiming at high quality solution, they cannot pretend to produce the exact solution in every case with certainty
Nevertheless, a stochastic high-quality approximation of a global optimum is probably more valuable than a deterministic poor-quality local minimum provided by a classical method or no solution at all.
Easy to implement to different problems
Side constraints on the solution can be taken into account at low additional costs

9. Classical vs. Heuristic

10. Optimization methods Exhaustive search Simulated annealing Tabu search
Ant colonies
Memetic Algorithms
Scatter Search

11. Genetic algorithm
Imitates evolutionary process of species that reproduces
Do not operate on a single current solution, but on a set of current solutions (population)
New individuals generated with cross-over:
combines part of genetic patrimony of each parent and applies a random mutation
If new individual (child), inherits good characteristics from parents → higher probability to survive

12. Genetic algorithm

13. Particle swarm optimization
PSO algorithm works by having a population (called a swarm) of candidate solutions (called particles).
Particles are moved around in the search-space according to a few simple formulae.
The movements of the particles are guided by their own best known position in the search-space as well as the entire swarm's best known position.
When improved positions are being discovered these will then come to guide the movements of the swarm.
The process is repeated and by doing so it is hoped, but not guaranteed, that a satisfactory solution will eventually be discovered.

14. Applications Task scheduling
Identification of an optimal set of time lags
Shortest path search
Sequential ordering
Identification of parameters

15. Conclusions
In digital optimization variables are restricted to assume only discrete values
Objective function is the main component in optimization process
Local minimums, constraints and optimization speed are the main problems
Heuristic algorithms are not guaranteed to find an optimal solution, nor are they guaranteed to run quickly. However they are still better then a low-quality deterministic local minimum provided by a classical method

16. Klausimai Tikslo funkcijų svarba ir jų pavyzdžiai.
Kuo skiriasi skaitmeninis ir tolydus optimizavimas?
Kokios optimizavimo problemos?
Euristinių metodų privalumai lyginant su klasikiniais?
Genetinio algoritmo principai.
Skaitmeninio optimizavimo pritaikymo sritys.