Database Publication Practices Surajit Chaudhuri Microsoft Research

Coping with Growth u Issue: Higher submission rate u We are not alone in facing this problem –What are others thinking? –ACM SGB Task Force on the Impact of Increasing Conference Submissions (Chair: Alexander L. Wolf) – next 4 slides u My analysis and some painful suggestions

Basis of SGB Task Force Data u ASE ( ) u DAC ( ) u DATE ( ) u EComm ( ) u FOCS ( ) u FSE ( ) u GLSVLSI ( ) u ICSE ( ) u ICSM ( ) u ISSTA ( ) u Middleware ( ) u MobiCom ( ) u MobiHoc ( ) u MobiSys ( ) u PODC ( ) u PODS ( ) u POPL ( ) u SCG ( ) u SIGCOMM ( ) u SODA ( ) u SPAA ( ) u STOC ( ) u UML ( ) u UAI ( )

MobiCom ( )

Observations/Perceptions: Stature/Workload u Conferences with lowering acceptance rates receiving more polished papers on narrower topics u Pressure to publish in top venues –CS argued that conferences more important than journals; now we are suffering for it –grad students expected to publish in top places u Between a rock and hard place –grow PC, but lose coordination –shrink PC, but lose quality of reviews u Vicious cycle –people agreeing to do more PCs (can’t say no) –people do less per PC

Most Radical Idea u Rethink the role of conferences –reduce importance w.r.t. journals –reduce number and increase acceptance rates –conference presentations derived from “best” journal submissions (rather than vice versa) –make tenure evaluation based on quality of top five papers, not number of papers u Journals have better scale properties –larger reviewer pool –less time pressure on authors and reviewers

My Analysis and Suggestions (No implication for SIGMOD06)

Request to the Steering Committees u Experimenting with procedural changes –Observe –Make only one significant change for a while –Observe u Avoid second-order changes that do not address the pain point

Second Order Issues u Double-blind reviewing –Expected to impact on selection of “border-line” papers –Many ACM SIGs follow »SIGGRAPH, AAAI, SIGCOMM u Author Feedback –Good for “venting” –Tight timeline for reviewing makes it ineffective

Accommodating Growth u Acceptance rate –Acceptance rate is 15%, 20% or 39%? »Do the math! –Not lower than many other large conferences –Mandated changes seem unreasonable u Diversity and Narrowing of Topics –AAAI: “Big Ideas”, Tech papers, Abstracts –Independent conferences as tracks (like WWW) –Reuse journals as publication u Program Constraints –Presentation decoupled from acceptance –Posters and Plenary (old KDD style)

Impact on Reviewing Infrastructure u Choice of PC members crucial –Quality of Papers highly correlated with PC members u “Everyone is a PC member” –VLDB05: 610 reviewers u Large PC has likely to have high variability –VLDB05: 610 reviewers –But, otherwise 3-review load is too high u Suggestions (next 2 slides) –Short-cuts in reviewing process –Throttle the “flow” of papers

Reviewing Quality/Load u 2-level PC (but reduce dependence on external reviewers) [like AAAI, SIGGRAPH] –Senior PC member nominates –Handles 4/5 papers –Good training for future PC experience u Early Rejection –If a paper gets 2 “weak” rejects, it is rejected without a third review u General Chairs should yield more time to reviewing –Electronic PC requires it –Electronic Proceedings make it easy u We should learn/coordinate with broader CS community

Throttle the flow u No other ACM SIG engineers pipeline –SIGGRAPHICS or EuroGraphics –Mobicom or SIGCOMM –Takes motivation away to make serious changes u Selectively “break the pipeline” –VLDB-> ICDE –SIGMOD -> VLDB (except for roll-over papers) u Send the right (not polite) message –Weak Reject” vs. “Reject” –Very short reviews for bottom 10% - 20%

More Serious Issues beyond Publication Practices u Legacy of mid’80s – mid 90’s –Parallel DBMS, Query Optimization –Debate between deductive DB and OODB washed away! u Jury is still out on the last decade? u Today: –Are there too many problem statements? –Everyone working on “personalized” problems –Many “fuzzy” problems with “fuzzy” yardstick for solutions? –Worry: Weakening link to systems/hard engineering