1. MODULAR SYSTEMS & COMBINATORIAL OPTIMIZATION (based on course “System design”, 2004…2008, MIPT) Mark Sh. Levin Inst. for Inform. Transmission Problems, Russian Acad. of Sci. Email: mslevin@acm.org Http://www.mslevin.iitp.ru/ Dept. ‘Technology for Complex System Modeling”, Division of Applied Mathematics&Informatics, HSE, Moscow, Russia, Dec. 19, 2012 PLAN: 1.Preliminaries (about me, about you) 2.Modularity (applications, basic technological problem) 3.Decision cycle (problem, model, algorithm, computing, decisions) 4.Combinatorial optimization problems 5.Four-layer framework (basic combinatorial problems/models, composite models, framework of problems, applied layer) 6.Composite problems – research projects 7.Conclusion (about novelty)

2. PRELIMINARIES About me: A. Education: (a) Radio Engineering (MTUSI, 1970) (b) Mehmat (MSU,1975) (c) Faculty of Economics (PhD-studies, MSU)(1981) (d) PhD-engineering & CS (RAS) B. Works: (i) system design, (ii) software engineering: information software packages, (iii)management systems (iv)decision making + COMBINATORIAL OPTIMIZATION Applications: (a) special systems, (b) governmental organizations (c) manufacturing, (d) geology, (e) house-building, (f) machine-building, (g) communication C. Recent teaching: Moscow Univs.; MIPT (2004..2008): “System Design” About you: 1.Your future objectives (i) Work in bank, consulting company, etc. (ii) Establishing new company (Google, Microsoft, Facebook, etc.) (iii) Academic research & educational work: *BS level: 1...3 conf. papers *MS level: 1…3 conf. papers, 1…3 journal articles (WoS) PhD level: 3…5 conf. papers, 3…5 journal articles (WoS) Assistant Prof. level: 8…15 conf. papers, 5…7, journal articles (WoS) Associate Prof. level: 30 journal articles (WoS), 30 conf. papers Full Prof. level: 60 journal articles (WoS), 60 conf. papers GOALS: 1.Extending your thinking 2.Possible joint research 3.Usage of my materials

3. GLANCE Applications (engineering, IT&CS, economics, geology, biomedicine, etc.) System approach, system design, systems engineering Combinatorial optimization Hierarchical system model

4. MODULARITY APPLICATIONS: (1)House-building (2)Computers (3)Machine-building (4)Bioinformatics (5)Software, (6)Combinatorial chemistry, ETC BASIC EXAMPLE: linguistics (NB!) BASIC SYSTEM PROBLEMS: 1. System model (system hierarchy, architecture) 2. System design (system configuration) 3. System improvement/upgrade (adaptation, modification, reconfiguration) 4. Multi-stage system design (design of system trajectory) 5. Combinatorial system evolution (combinatorial modeling) 6. System forecasting SYSTEM: 1.Basic (e.g., physical) system (e.g., computer, machine/car, house, communication system, software, algorithm system, personnel) 2.Plan: medical treatment, plan of system improvement, economical plan, rules 3.Requirements (system of requirements) 4.Standard (standard as system, system of standard)

5. COMBINATORIAL OPTIMIZATION PROBLEMS COMBINATORIAL PROBLEMS (basic problems): There is a set of elements. Find: Basic Problem 1. Ordering of elements Basic Problem 2. Grouping of elements Basic Problem 3. Assignment of elements to certain “places” PROBLEMS: 1.Ordering/scheduling 2.Ranking 3.Knapsack problem 4.Multiple choice problem 5.Assignemnt/allocation/location problems (including marriage problem) 6.Clustering (grouping) 7.TSP 8.Graph coloring 9. Covering problems 10.Spanning trees (minimum spanning tree, Steiner tree) 11.SAT etc. BASIC ISSUES: 1.Complexity (Polynomial algorithm exists or does not exist) 2.Design of algorithms: (a) enumerative methods (B-A-B, dynamic programming) (b) polynomial algorithms (c) simple (e.g., greedy) heuristics (c) heuristics (d) approximation algorithms (e) genetic algorithms (evolutionary optimization)

6. Multicriteria ranking/choice Initial set of alternatives Linear ranking Choice Ranking

7. Clustering / Classification Problem Initial set of objects/ alternatives Clusters Goals: 1.To decrease the dimension 2.To design a hierarchy

8. Simple structures (chains, trees, parallel-series graphs) CHAIN TREE PARALLEL-SERIES GRAPH

9. Simple structures (hierarchy) Level 4 Level 3 Level 1 Level 2

10. Optimization problems on graphs: illustrations a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 BASIC GRAPH (DIGRAPH): weights for arcs (or edges) 2 1 2 2 4 4 1 34 3 2 4 3 2 a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 2 1 2 2 4 4 1 34 3 2 4 3 2 Shortest Path for : L = 2+1+1+2+2 = 8

11. Optimization problems on graphs: illustrations a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 Spanning tree (length = 19): 2 1 2 4 4 1 34 3 2 4 3 2 Traveling Salesman Problem : L = 2+1+3+4+2+2+3+4+4+4 a4a4 a3a3 a7a7 a1a1 a0a0 a2a2 a5a5 a6a6 a9a9 a8a8 a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 2 1 2 4 4 1 34 3 2 4 3 2 2 2

12. PACKING PROBLEM (illustration) 1 1 2 2 3 3 4 4 5 5 7 6 6 7 8 8 9 9 10 11 12 13 14... REGION FOR PACKING ELEMENTS GOALS: *Maximum of packed elements *Minimum of free space

13. BIN-PACKING PROBLEM (illustration) CONTAINERS FOR PACKING ELEMENTS... 1 1 2 2 3 3 4 4 5 5 6 6 GOAL: Usage of minimal number of containers

14. SCHEDULING: illustrative example for assembly process (algorithm of longest tails) GOAL: Minimal total complete time 1 3 2 6 5 4 9 8 710 11 12 13 14 15 16 17 18 19 Tasks & precedence constraints 1 (distance from corner) 7 2 2 3 3 3 4 4 4 5 5 6 6 6 67 7 7 3 processors: t 1 2 3 0 17 18 19 12 16 14 13 15 9 10 11 6 7 8 3 4 521 8

15. MAXIMA CLIQUE PROBLEM (illustration) Initial graph G = (R, E), R is set of vertices, E is set of edges Problem is: Find the maximal (by number of vertices) clique (i.e., complete subgraph) G = (R,E) Clique consisting of 6 vertices (maximal complete subgraph)

16. Spanning (illustration): 1-connected graph a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 Steiner tree (example): 2 1 2 4 4 1 34 3 2 4 3 2 a4a4 a3a3 a7a7 a1a1 a0a0 a2a2 a5a5 a6a6 a9a9 a8a8 2 a0a0 a1a1 a2a2 a3a3 a4a4 a6a6 a5a5 a7a7 a8a8 a9a9 Spanning tree (length = 19): 2 1 2 4 4 1 34 3 2 4 3 2 a4a4 a3a3 a7a7 a1a1 a0a0 a2a2 a5a5 a6a6 a9a9 a8a8 2

17. Knapsack problem max  m i=1 c i x i s.t.  m i=1 a i x i  b x i  {0, 1}, i = 1, …, m possible additional constraints  m i=1 a ik x i  b k, k = 1, …, l... 1 i m (index) a 1 a i a m (required resource) c 1 c i c m (utility / profit) x 1 x i x m (Boolean variable)

18. Multiple choice problem max  m i=1  qi j=1 c ij x ij s.t.  m i=1  qi j=1 a ij x ij  b  qi j=1 x ij  1, i = 1, …, m x ij  {0, 1}, i = 1, …, m, j = 1, …, qi... J 1 J i J m...  i | J i | = qi, j = 1, …, qi

19. Assignment/Allocation problem Allocation (assignment, matching, location): matrix of weights c ij BIPARTITE GRAPH 1 2 3 4 5 6 7 8 a b c d e f g h........ a b c d e f g h 1234567812345678 Positions Set of elements

20. Assignment/allocation problem a3a3 a1a1 a2a2 anan b1b1 FORMULATION (algebraic): Set of elements: A = { a 1, …, a i, …, a n } Set of positions: B = { b 1, …, b j, …. b m } (now let n = m) Effectiveness of pair a i and b j is: c ( a i, b j ) x ij = 1 if a i is located into position b j and 0 otherwise ( x ij  { 0,1 } ) The problem is: max  n i=1  n j=1 c ij x ij s.t.  n i=1 x ij = 1  j  n j=1 x ij = 1  i b2b2 b3b3 bmbm... ELEMENTSPOSITIONS

21. Multiple matching problem A = { a 1, … a n }B = { b 1, … b m } C = { c 1, … c k } EXAMPLE: 3-MATCHING (3-partitie graph)

22. Graph coloring problem (illustration) Initial graph G = (A, E), A is set of vertices, E is set of edges Problem is: Assign a color for each vertex with minimal number of colors under constraint: neighbor vertices have to have different colors G = (A,E) Right coloring

23. VERTEX COVERING PROBLEM Vertex set A = { a 1, … a n }, edge set E={e 1, …,e k }, graph G = (A, E) PROBLEM: find vertex covering (A’  A) that covers  e  E

24. Satisfiability problem: illustration for application in software / electronic systems SYSTEM x1x1 xmxm x m-1 x2x2... y (0 or 1) c1c1 c2c2 c n-1 cncn Example: c 1 = not x 1 OR x 2 OR x 4 OR not x 5 OR x 7 c 2 = x 1 OR not x 2 OR not x 3 OR x 5 OR x 7 c 3 = not x 1 OR not x 2 OR x 3 OR not x 5 OR not x n c 4 = not x 2 OR x 3 OR x 7 OR x n-2 OR x n-1... y = c 1 &c 2 & … &c n Literal: x i / not x i PROBLEM: Exist x o =(x 1,…,x n ) that y(x o ) =1 OR not

25. Satisfiability Problem SATISFIABILITY 3-SATISFIABILITY 3-MATCHINGVERTEX COVERING PARTITIONING (about knapsack) HAMILTONIAN CYCLECLIQUE BASIC 6 NP-COMPLETE PROBLEMS AND DIAGRAM

26. Alignment (illustration) CASE OF 2 WORDS: A ABBDX A DACXZ Word 1 Word 2 A ABB DXZ Superstructure C A ABBD X A D A CXZ

27. Morphological clique PART 1 PART 2 PART 3 Vertices (design alternatives) Edges (compatibility) NOTE: about k-matching

28. COMPLEXITY OF COMBINATORIAL OPTIMIZAITON PROBLEMS Polynomial solvable problems NP-hard problems Approximate polynomial solvable Problems (FPTAS) Knapsack problem Multiple choice problem Quadratic assignment problem Morphological clique problem Clique problem TSP

29. DECISION CYCLE & Support Components Applied modular problem(s) Solving process (e.g., computing) DECISION Frame of problems/ mathematical models Library of problems/ models Solving schemes Algorithms Procedures Program (software)/ interactive procedure

30. Four-layer Framework Multicriteria ranking Knapsack problem Multiple choice problem Cluste- ring Assignment/ allocation Design of hierarchy (clustering, multicriteria spanning) Morpholo- gical clique (synthesis) ETC Layer 1: Basic problems/models Multi- criteria knapsack Multicriteria multiple choice problem Multi- criteria assignment/ allocation ETC Layer 2: Composite models/procedures Four-stage composite framework Hierarchical morpho- logical design System upgrade/ improve- ment System evolution/ forecasting ETC Layer 3: Basic (typical) solving framework Design/ planning of testing Modular design of software Informa- tion retrieval Design of marke- ting strategy ETC Layer 4: Domain-oriented frameworks Planning of mainte- nance Planning of medical treatment Improve- ment of network Evolution of require- ments Span- ning tree Steiner tree problem Shortest path problem Multistage design

31. S=X*Y*Z Y Z=P*Q*U*V Z 1 =P 2 *Q 3 *U 1 *V 5 Z 2 =P 1 *Q 2 *U 3 *V 1 Y1Y2Y3Y1Y2Y3 AB A1A2A3A1A2A3 B1B2B3B4B1B2B3B4 C1C2C3C4C5C1C2C3C4C5 D=I*J I1I2I3I1I2I3 J1J2J3J4J1J2J3J4 P1P2P3P1P2P3 Q1Q2Q3Q4Q1Q2Q3Q4 U1U2U3U1U2U3 V1V2V3V4V5V6V1V2V3V4V5V6 C X=A*B*C*D P QU V X 1 =A 1 *B 2 *C 4 *D 3 X 2 =A 3 *B 4 *C 2 *D 1 D 1 =I 1 *J 1 D 2 =I 1 *J 2 D 3 =I 3 *J 4 S 1 =X 2 *Y 3 *Z 2 S 2 =X 1 *Y 2 *Z 1 J I Illustration for HMMD approach Compatibility for I,J Compatibility for P,Q,U,V Compatibility for A,B,C,D Compatibility for X,Y,Z

32. Product trajectory Stage 1... T0 Stage 3... Stage 2... Trajectory

33. 1 2 3 4 a b c d e f g h Groups of testers SYSTEMS FRAMEWORK: Clustering, Assignment, Multiple Choice Problem TESTERS 2 Groups of systems X X XTest actions: A 1 (no test) A 2 (simple test) A 3 (analysis test) A 4 (structure research) A 5 (new test)

34. MPEG 1 T 0 MPEG 2 MPEG 4 Forecast Improvement Evolution as Generations of MPEG-like standard, Forecasting

35. SYSTEM LIFE CYCLE (product, etc.) R & D t ManufacturingTestingMarketing Utilization & Maintenance Recycling 0 T T: about 12 years (submarines, airplanes, nuclear technology, etc.) TENDENCY: increasing T (2 years, 6 months) SYSTEM: 1.Basic (e.g., physical) system (e.g., computer, machine/car, house, communication system, software, algorithm system, personnel) 2.Plan: medical treatment, plan of system improvement, economical plan, rules 3.Requirements (system of requirements) 4.Standard (standard as system, system of standard) Car: 1.Body 2.Motor 3.Drive system 4.Electronics 5.Safety MY COMBINATORIAL TECHNOLOGICAL SYSTEMS PROBLEMS: 1.Design of system model 2.System design 3.System improvement 4.System evolution 5.System forecasting

36. Research Opportunity (with Novelty) Novelty: new approach at each level/for each component: 1.Applied problem (engineering, CS, economics) 2.Model/Model framework 3.Algorithm / algorithm framework 4.Software (program, program package) EXAMPLES OF STUDENT RESEARCH PROJECTS: ABOUT 30 PUBLISHED PAPERS (my site) EXAMPLES OF MY RESEARCH PROJECTS: MY PUBLISHED PAPERS (my site)

37. That’s All Thanks! http://www.mslevin.iitp.ru/ Mark Sh. Levin