A Template for Producing IT Research and Publication (for top CS and MIS journals) Hsinchun Chen, 2019

Agenda Choosing Publishable Research Abstract Introduction Literature Review (including IS Positioning) Research Questions System and Research Design Research Testbed System/Research Experiment Findings and Discussions (including Contributions to IS) Conclusions and Future Directions References Acknowledgement

Choosing Publishable Research A publishable paper needs: an interesting research topic, good lit review, new algorithm/technique, relevant testbed, systematic evaluation (comparing with existing approaches and with statistical tests) Writing needs to be precise, concise, professional and entirely error-free. Carefully develop slides before writing the paper.

Choosing Publishable Research Research topic needs to be new and interesting. Avoid old and well-studied topic. Research could be technique/algorithm driven or application driven.  “Computational Design Science” Read a lot. Understand the current trends and directions. Need comprehensive lit review. Use well grounded methodologies. Compare with existing techniques/approaches with data sets.

Title 8 words or less. Develop a title after finishing the paper. Title needs to reflect the essence of the research. Don’t use cute title, e.g., “To aggregate or not to aggregate” Use project/system acronym with clear relevant meaning, e.g., COPLINK, BioPortal; not ALOHA.

Abstract Most important part of a paper – the first impression! Abstract should reflect the entire paper. 200-300 words in one paragraph. 2-3 sentences to summarize problem motivation. 2-3 sentences to describe proposed method or algorithm. 2-3 sentences to summarize evaluation method. 3-4 sentences to summarize key findings. Write abstract after finishing the entire paper. Select key sentences from paper.

Introduction Motivate the research topic. 4-6 paragraphs. Less than 2 pages. Describe the importance of the research topic, current approaches, proposed methods, and structure of the paper.

Literature Review A major part of the paper. Need to show that you know the field. Read and digest many papers before writing. Need to review seminal works and critical new works (past 3-4 years in major journals and conferences). 3-6 pages. 3-4 major subsections. Never do a laundry list review. Never do too much tutorial.

Literature Review Select only closely relevant works to review. Best to present as a taxonomy with 2-3 critical sub-categories (e.g., problems, techniques, results) to classify works. Use a detailed table to summarize and compare past works. Sometimes a good comprehensive review paper can stand on its own. Need to critique past works (critique not criticize!). Need to summarize research gaps that would lead into your proposed research questions.

Literature Review (only for MIS) For major MIS journals (MISQ and ISR), you need to present a valid research framework or methodological foundation, especially for “Computational Design Science” Summarize and discuss prior papers in MISQ, ISR and JMIS that are relevant to your work. What is your research approach and what is your potential contribution to IS knowledge base?

Research Questions Summarize research gaps in the lit review section. Need 3-5 research questions that show the focus of research. Need to demonstrate potential novelty for both CS (algorithm) and MIS (application). Half a page.

Research Testbed Use research testbed to validate designs and approaches. What data sets will be used in the experiment or evaluation? Testbed should be interesting, relevant, and significant. For new emerging critical applications, research testbed could become the focus of research. 2-4 pages, in detail.

Research & System Design If possible, present your research methodological foundation (or theory), e.g., Systemic Functional Linguistic Theory, Kernel Theory.  For MIS Describe how the overall system/design works and its key components. Use an overall system or framework diagram with boxes and arrows. Explain the rationale of each component. 3-6 pages. Need to describe key technical innovation/novelty (CS) and technical details (algorithms, mathematical notations). Even for MIS, need to demonstrate novelty inspired by applications, not just simply application of existing techniques (e.g., LSTM, CNN, RNN).

System Evaluation/Experiment A paper won’t get accepted by a top-tier journal without a detailed, systematic evaluation. Present research hypotheses: focused and measurable. Focus on the experiment and evaluation. Summarize evaluation methodologies adopted in past research. Compare with existing benchmark techniques. Describe experimental procedure, conditions, tasks, subjects. 3-4 pages, in detail. Need statistical tests (t-test, F-test).

Research Findings and Discussions Summarize key findings in a clear and understandable format. You may group your findings in chunks, each of which starts with a bold summarizing sentence. Present examples and discussions right after each finding to bring insight and better understanding. User screen shots to illustrate. 4-8 pages, in detail.

Research Findings and Discussions Tables and figures are critical. Need to be consistent and neat. Highlight interesting numbers. In caption, you may use 3-4 sentences to describe more details about a figure or a table. Use a small paragraph in text to explain the essence about a figure or a table. Need statistical tests (T-test, F-test). P value (significance level) at 5%. Reject or accept hypotheses postulated.

Research Findings and Discussions Tables and figures are critical. Need to be consistent and neat. Highlight interesting numbers. In caption, you may use 3-4 sentences to describe more details about a figure or a table. Use a small paragraph in text to explain the essence about a figure or a table. Need statistical tests (T-test, F-test). P value (significance level) at 5%. Reject or accept hypotheses postulated.

Contributions to IS Knowledge Base (MIS only) Provide a “generalization” or “abstraction” of your overall contributions to IS knowledge base. Link to prior literature. What class of problems or applications are you solving? Develop “Novel Design Principles” for future researchers or practitioners: what application or data characteristics, what analytical methods, what operational/engineering considerations?

References Where has similar work been published? What kind of articles are accepted by the target journal? Reference related papers that were published in the target journal. Must have 5-10 key journals, key conferences, and key authors in the field. Number the references in a consistent format. Include a few of your own works. Avoid having too many self-citations. 15-50 citations for most journals.

Acknowledgement Funding sources and grant number. Research partners and participants.

Additional Notes Conference papers only need to have about 30% of the above coverage. Target only major professional conferences to report work in progress and solicit feedback. Conference papers need to be quickly expanded to target at top-tier journals.

Additional Notes Select the right journals for publications. Research recent journal scope, past publications, and AE background. Some may change over time. Consult senior colleagues for suggestion. Top MIS: MISQ, ISR (careful IS positioning) Algorithm-oriented: IEEE TKDE, ACM TOIS, IEEE SMC (algorithm novelty) OM-oriented: MS, JOC System-oriented: JMIS, JASIST, DSS Application-oriented: CACM, IEEE Computer Health-oriented: JAMIA, JBI