MODULAR SYSTEMS & COMBINATORIAL OPTIMIZATION (based on course “System design”, 2004…2008, MIPT) Mark Sh. Levin Inst. for Inform. Transmission Problems, Russian Acad. of Sci. Dept. ‘Technology for Complex System Modeling”, Division of Applied Mathematics&Informatics, HSE, Moscow, Russia, Dec. 19, 2012 PLAN: 1.Preliminaries (about me, about you) 2.Modularity (applications, basic technological problem) 3.Decision cycle (problem, model, algorithm, computing, decisions) 4.Combinatorial optimization problems 5.Four-layer framework (basic combinatorial problems/models, composite models, framework of problems, applied layer) 6.Composite problems – research projects 7.Conclusion (about novelty)

PRELIMINARIES About me: A. Education: (a) Radio Engineering (MTUSI, 1970) (b) Mehmat (MSU,1975) (c) Faculty of Economics (PhD-studies, MSU)(1981) (d) PhD-engineering & CS (RAS) B. Works: (i) system design, (ii) software engineering: information software packages, (iii)management systems (iv)decision making + COMBINATORIAL OPTIMIZATION Applications: (a) special systems, (b) governmental organizations (c) manufacturing, (d) geology, (e) house-building, (f) machine-building, (g) communication C. Recent teaching: Moscow Univs.; MIPT ( ): “System Design” About you: 1.Your future objectives (i) Work in bank, consulting company, etc. (ii) Establishing new company (Google, Microsoft, Facebook, etc.) (iii) Academic research & educational work: *BS level: conf. papers *MS level: 1…3 conf. papers, 1…3 journal articles (WoS) PhD level: 3…5 conf. papers, 3…5 journal articles (WoS) Assistant Prof. level: 8…15 conf. papers, 5…7, journal articles (WoS) Associate Prof. level: 30 journal articles (WoS), 30 conf. papers Full Prof. level: 60 journal articles (WoS), 60 conf. papers GOALS: 1.Extending your thinking 2.Possible joint research 3.Usage of my materials

GLANCE Applications (engineering, IT&CS, economics, geology, biomedicine, etc.) System approach, system design, systems engineering Combinatorial optimization Hierarchical system model

MODULARITY APPLICATIONS: (1)House-building (2)Computers (3)Machine-building (4)Bioinformatics (5)Software, (6)Combinatorial chemistry, ETC BASIC EXAMPLE: linguistics (NB!) BASIC SYSTEM PROBLEMS: 1. System model (system hierarchy, architecture) 2. System design (system configuration) 3. System improvement/upgrade (adaptation, modification, reconfiguration) 4. Multi-stage system design (design of system trajectory) 5. Combinatorial system evolution (combinatorial modeling) 6. System forecasting SYSTEM: 1.Basic (e.g., physical) system (e.g., computer, machine/car, house, communication system, software, algorithm system, personnel) 2.Plan: medical treatment, plan of system improvement, economical plan, rules 3.Requirements (system of requirements) 4.Standard (standard as system, system of standard)

COMBINATORIAL OPTIMIZATION PROBLEMS COMBINATORIAL PROBLEMS (basic problems): There is a set of elements. Find: Basic Problem 1. Ordering of elements Basic Problem 2. Grouping of elements Basic Problem 3. Assignment of elements to certain “places” PROBLEMS: 1.Ordering/scheduling 2.Ranking 3.Knapsack problem 4.Multiple choice problem 5.Assignemnt/allocation/location problems (including marriage problem) 6.Clustering (grouping) 7.TSP 8.Graph coloring 9. Covering problems 10.Spanning trees (minimum spanning tree, Steiner tree) 11.SAT etc. BASIC ISSUES: 1.Complexity (Polynomial algorithm exists or does not exist) 2.Design of algorithms: (a) enumerative methods (B-A-B, dynamic programming) (b) polynomial algorithms (c) simple (e.g., greedy) heuristics (c) heuristics (d) approximation algorithms (e) genetic algorithms (evolutionary optimization)

Multicriteria ranking/choice Initial set of alternatives Linear ranking Choice Ranking

Clustering / Classification Problem Initial set of objects/ alternatives Clusters Goals: 1.To decrease the dimension 2.To design a hierarchy

Simple structures (chains, trees, parallel-series graphs) CHAIN TREE PARALLEL-SERIES GRAPH

Simple structures (hierarchy) Level 4 Level 3 Level 1 Level 2

Optimization problems on graphs: illustrations a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 BASIC GRAPH (DIGRAPH): weights for arcs (or edges) a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a Shortest Path for : L = = 8

Optimization problems on graphs: illustrations a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 Spanning tree (length = 19): Traveling Salesman Problem : L = a4a4 a3a3 a7a7 a1a1 a0a0 a2a2 a5a5 a6a6 a9a9 a8a8 a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a

PACKING PROBLEM (illustration) REGION FOR PACKING ELEMENTS GOALS: *Maximum of packed elements *Minimum of free space

BIN-PACKING PROBLEM (illustration) CONTAINERS FOR PACKING ELEMENTS GOAL: Usage of minimal number of containers

SCHEDULING: illustrative example for assembly process (algorithm of longest tails) GOAL: Minimal total complete time Tasks & precedence constraints 1 (distance from corner) processors: t

MAXIMA CLIQUE PROBLEM (illustration) Initial graph G = (R, E), R is set of vertices, E is set of edges Problem is: Find the maximal (by number of vertices) clique (i.e., complete subgraph) G = (R,E) Clique consisting of 6 vertices (maximal complete subgraph)

Spanning (illustration): 1-connected graph a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 Steiner tree (example): a4a4 a3a3 a7a7 a1a1 a0a0 a2a2 a5a5 a6a6 a9a9 a8a8 2 a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 Spanning tree (length = 19): a4a4 a3a3 a7a7 a1a1 a0a0 a2a2 a5a5 a6a6 a9a9 a8a8 2

Knapsack problem max  m i=1 c i x i s.t.  m i=1 a i x i  b x i  {0, 1}, i = 1, …, m possible additional constraints  m i=1 a ik x i  b k, k = 1, …, l... 1 i m (index) a 1 a i a m (required resource) c 1 c i c m (utility / profit) x 1 x i x m (Boolean variable)

Multiple choice problem max  m i=1  qi j=1 c ij x ij s.t.  m i=1  qi j=1 a ij x ij  b  qi j=1 x ij  1, i = 1, …, m x ij  {0, 1}, i = 1, …, m, j = 1, …, qi... J 1 J i J m...  i | J i | = qi, j = 1, …, qi

Assignment/Allocation problem Allocation (assignment, matching, location): matrix of weights c ij BIPARTITE GRAPH a b c d e f g h a b c d e f g h Positions Set of elements

Assignment/allocation problem a3a3 a1a1 a2a2 anan b1b1 FORMULATION (algebraic): Set of elements: A = { a 1, …, a i, …, a n } Set of positions: B = { b 1, …, b j, …. b m } (now let n = m) Effectiveness of pair a i and b j is: c ( a i, b j ) x ij = 1 if a i is located into position b j and 0 otherwise ( x ij  { 0,1 } ) The problem is: max  n i=1  n j=1 c ij x ij s.t.  n i=1 x ij = 1  j  n j=1 x ij = 1  i b2b2 b3b3 bmbm... ELEMENTSPOSITIONS

Multiple matching problem A = { a 1, … a n }B = { b 1, … b m } C = { c 1, … c k } EXAMPLE: 3-MATCHING (3-partitie graph)

Graph coloring problem (illustration) Initial graph G = (A, E), A is set of vertices, E is set of edges Problem is: Assign a color for each vertex with minimal number of colors under constraint: neighbor vertices have to have different colors G = (A,E) Right coloring

VERTEX COVERING PROBLEM Vertex set A = { a 1, … a n }, edge set E={e 1, …,e k }, graph G = (A, E) PROBLEM: find vertex covering (A’  A) that covers  e  E

Satisfiability problem: illustration for application in software / electronic systems SYSTEM x1x1 xmxm x m-1 x2x2... y (0 or 1) c1c1 c2c2 c n-1 cncn Example: c 1 = not x 1 OR x 2 OR x 4 OR not x 5 OR x 7 c 2 = x 1 OR not x 2 OR not x 3 OR x 5 OR x 7 c 3 = not x 1 OR not x 2 OR x 3 OR not x 5 OR not x n c 4 = not x 2 OR x 3 OR x 7 OR x n-2 OR x n-1... y = c 1 &c 2 & … &c n Literal: x i / not x i PROBLEM: Exist x o =(x 1,…,x n ) that y(x o ) =1 OR not

Satisfiability Problem SATISFIABILITY 3-SATISFIABILITY 3-MATCHINGVERTEX COVERING PARTITIONING (about knapsack) HAMILTONIAN CYCLECLIQUE BASIC 6 NP-COMPLETE PROBLEMS AND DIAGRAM

Alignment (illustration) CASE OF 2 WORDS: A ABBDX A DACXZ Word 1 Word 2 A ABB DXZ Superstructure C A ABBD X A D A CXZ

Morphological clique PART 1 PART 2 PART 3 Vertices (design alternatives) Edges (compatibility) NOTE: about k-matching

COMPLEXITY OF COMBINATORIAL OPTIMIZAITON PROBLEMS Polynomial solvable problems NP-hard problems Approximate polynomial solvable Problems (FPTAS) Knapsack problem Multiple choice problem Quadratic assignment problem Morphological clique problem Clique problem TSP

DECISION CYCLE & Support Components Applied modular problem(s) Solving process (e.g., computing) DECISION Frame of problems/ mathematical models Library of problems/ models Solving schemes Algorithms Procedures Program (software)/ interactive procedure

Four-layer Framework Multicriteria ranking Knapsack problem Multiple choice problem Cluste- ring Assignment/ allocation Design of hierarchy (clustering, multicriteria spanning) Morpholo- gical clique (synthesis) ETC Layer 1: Basic problems/models Multi- criteria knapsack Multicriteria multiple choice problem Multi- criteria assignment/ allocation ETC Layer 2: Composite models/procedures Four-stage composite framework Hierarchical morpho- logical design System upgrade/ improve- ment System evolution/ forecasting ETC Layer 3: Basic (typical) solving framework Design/ planning of testing Modular design of software Informa- tion retrieval Design of marke- ting strategy ETC Layer 4: Domain-oriented frameworks Planning of mainte- nance Planning of medical treatment Improve- ment of network Evolution of require- ments Span- ning tree Steiner tree problem Shortest path problem Multistage design

S=X*Y*Z Y Z=P*Q*U*V Z 1 =P 2 *Q 3 *U 1 *V 5 Z 2 =P 1 *Q 2 *U 3 *V 1 Y1Y2Y3Y1Y2Y3 AB A1A2A3A1A2A3 B1B2B3B4B1B2B3B4 C1C2C3C4C5C1C2C3C4C5 D=I*J I1I2I3I1I2I3 J1J2J3J4J1J2J3J4 P1P2P3P1P2P3 Q1Q2Q3Q4Q1Q2Q3Q4 U1U2U3U1U2U3 V1V2V3V4V5V6V1V2V3V4V5V6 C X=A*B*C*D P QU V X 1 =A 1 *B 2 *C 4 *D 3 X 2 =A 3 *B 4 *C 2 *D 1 D 1 =I 1 *J 1 D 2 =I 1 *J 2 D 3 =I 3 *J 4 S 1 =X 2 *Y 3 *Z 2 S 2 =X 1 *Y 2 *Z 1 J I Illustration for HMMD approach Compatibility for I,J Compatibility for P,Q,U,V Compatibility for A,B,C,D Compatibility for X,Y,Z

Product trajectory Stage 1... T0 Stage 3... Stage 2... Trajectory

a b c d e f g h Groups of testers SYSTEMS FRAMEWORK: Clustering, Assignment, Multiple Choice Problem TESTERS 2 Groups of systems X X XTest actions: A 1 (no test) A 2 (simple test) A 3 (analysis test) A 4 (structure research) A 5 (new test)

MPEG 1 T 0 MPEG 2 MPEG 4 Forecast Improvement Evolution as Generations of MPEG-like standard, Forecasting

SYSTEM LIFE CYCLE (product, etc.) R & D t ManufacturingTestingMarketing Utilization & Maintenance Recycling 0 T T: about 12 years (submarines, airplanes, nuclear technology, etc.) TENDENCY: increasing T (2 years, 6 months) SYSTEM: 1.Basic (e.g., physical) system (e.g., computer, machine/car, house, communication system, software, algorithm system, personnel) 2.Plan: medical treatment, plan of system improvement, economical plan, rules 3.Requirements (system of requirements) 4.Standard (standard as system, system of standard) Car: 1.Body 2.Motor 3.Drive system 4.Electronics 5.Safety MY COMBINATORIAL TECHNOLOGICAL SYSTEMS PROBLEMS: 1.Design of system model 2.System design 3.System improvement 4.System evolution 5.System forecasting

Research Opportunity (with Novelty) Novelty: new approach at each level/for each component: 1.Applied problem (engineering, CS, economics) 2.Model/Model framework 3.Algorithm / algorithm framework 4.Software (program, program package) EXAMPLES OF STUDENT RESEARCH PROJECTS: ABOUT 30 PUBLISHED PAPERS (my site) EXAMPLES OF MY RESEARCH PROJECTS: MY PUBLISHED PAPERS (my site)

That’s All Thanks! Mark Sh. Levin