1. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000. Challenges Promoting Theoretical EC Research William E. Hart Optimization Uncertainty/Estimation Dept Sandia National Laboratories wehart@sandia.gov http://www.cs.sandia.gov/~wehart

2. EC Theory - GECCO 2002 Slide 2 Overview A reminder: EC theory is relevant! Publishing theoretical conference papers is difficult … some personal reflections Some ideas/suggestions

3. EC Theory - GECCO 2002 Slide 3 The Role of EC Theory I. Modeling and Abstraction Goal: identify/validate essential features of successful EAs Simple EC methods allow us to explore their limitations/possibilities Analysis of abstract EC methods enables these results to be applied to many specific implementations Q: What are effective abstract EA formulations?

4. EC Theory - GECCO 2002 Slide 4 The Role of EC Theory II.Unification and Linkage Goal: develop linkages between different disciplines Transferring ideas between disciplines is easier in simple EC algorithms Reformulating EC methods as more traditional algorithmic techniques clarifies their strengths and weaknesses Analytic tools from other disciplines can be validated on simple EC models Q: Can analysis of direct search methods be applied to EAs?

5. EC Theory - GECCO 2002 Slide 5 The Role of EC Theory III.Assessing Fundamental Algorithmic Capabilities Goal: what EC methods can solve what classes of problems? Rigorous evaluation of EC methods is required Negative results must be formulated mathematically Mathematical analysis provides the strongest assurance of robustness in EC methods Q: Is randomization an essential feature of EC methods?

6. EC Theory - GECCO 2002 Slide 6 The Role of EC Theory IV.Identifying New Algorithmic Directions Goal: anticipate and evaluate potential EC methods Can consider EC methods for which practical implementations are not readily available Abstract EC models can identify promising research directions and desirable EC capabilities The design of practical EC methods can be guided by our analysis of abstract EC models Q: How can EC methods effectively exploit derivative information?

7. EC Theory - GECCO 2002 Slide 7 What Theory (Probably) Cannot Do Guarantee fast global convergence Explain why your code does not seem to work on your application Provide convergence guarantees for your new EC heuristic Prove that a particular EC heuristics will not work in general

8. EC Theory - GECCO 2002 Slide 8 Publishing Theoretical EC Results - Journals EC journals generally encourage theoretical results Journal articles are usually long enough to provide a complete analysis Reviewers do not always appreciate the strategic value of theoretical results Providing reasonable turnaround to authors of theoretical papers is challenging!

9. EC Theory - GECCO 2002 Slide 9 Publishing Theoretical EC Results - Conferences Conference papers have very tight space limitations! Short presentation of theoretical results is quite difficult –What background can you assume of an EC audience? –How can you explain, state and discuss the results? Conferences typically have a very loose review process –Quick reviews needed –Rough match of papers to reviewers –Limited feedback to authors

10. EC Theory - GECCO 2002 Slide 10 My Own Experiences Publishing in: –ICGA, GECCO, CEC, FOGA –EC, IEEE TEC General theme: local convergence on –Evolutionary pattern search EA framework –Self-adaptive EAs Challenges: –Effectively meeting page limitations! –Poor reviewer feedback –Distinguishing between conference/journal publications

11. EC Theory - GECCO 2002 Slide 11 Meeting Conference Page Limits Idea 1: State the results without proofs –Usually can condense analysis to fit page limits –Some reviewer always complains about missing proofs –Note: any negative feedback can make it difficult to publish a conference paper!

12. EC Theory - GECCO 2002 Slide 12 Meeting Conference Page Limits(contd) Idea 2: Include proofs in an optimal appendix for the reviewers –Reviewers get confused –It is easy to ding a paper for exceeding page limits –Organizing and writing this appendix well can be challenging!

13. EC Theory - GECCO 2002 Slide 13 Poor Reviewer Feedback Typical reviewer comments: –I dont believe these results –Im confused by these results… –The analysis considers an algorithm/problem that is too simple –The authors should rework their paper to … Poor review of theoretical results: –A simple bug in my ICGA97 paper was not caught –Many reviewers fail to appreciate the difference between an approximate analysis of an EAs search dynamics and an exact characterization of an EAs search dynamics

14. EC Theory - GECCO 2002 Slide 14 Publication in Journals EC Publication Policy Dilemma for EC theory –Can simple theoretical conference papers be republished in journals? –How about more extended conference papers? (e.g. FOGA?) We should encourage theoretical analyses to be published in journals (they get more careful reviews there). Current policies effectively discourage publication of theoretical papers in conferences! Papers should describe mature work that is original in nature and has not been published elsewhere.

15. EC Theory - GECCO 2002 Slide 15 Some Ideas/Suggestions Encourage the submission of Extended Abstracts for conferences –Idea: a paper that explicitly omits technical details to focus on the big picture –Commonly used in theoretical computer science conferences STOC, FOCS, SODA Special Sessions/Minisymposiums –Idea: sessions of presenters who do not prepare a paper –Organized about a specific topic or theme –Approved by the conference organizing committee –Commonly used within mathematical conferences (e.g. SIAM)

16. EC Theory - GECCO 2002 Slide 16 Some Ideas/Suggestions(contd) Incorporate EC Theory tracks into EC conferences –Idea: explicitly encourage theoretical papers –This would hopefully help focus the reviewers –Problem: this track probably cuts across other tracks Work to clarify editorial policies for EC journals –Idea: allow revised conference papers –Example: SIAM J Computing In general, only original papers will be considered. However, papers previously presented in conference proceedings of limited circulation may be submitted in revised form.

17. EC Theory - GECCO 2002 Slide 17 Outstanding Issues How do we ensure that qualified reviewers are available for theoretical papers? What constitutes an adequate review of theoretical papers? How can we best educate the EC community about the impact of EC theory?

18. EC Theory - GECCO 2002 Slide 18 References Challenges for Theoretical Computer Science http://www.research.att.com/~dsj/nsflist.html SIAM: Society for Industrial and Applied Mathematics http://www.siam.org/ ACM SIGACT http://sigact.acm.org/sigact Advice to Authors of Extended Abstracts http://www.acm.org/sigsoft/conferences/pughadvice.html http://www.acm.org/sigsoft/conferences/pughadvice.html