Samira Khan University of Virginia Aug 23, 2017 COMPUTER ARCHITECTURE CS 6354 Samira Khan University of Virginia Aug 23, 2017 The content and concept of this course are adapted from CMU ECE 740

ROTUNDA, PAVILLIONS AND THE LAWN

A DESIGN BASED ON PRINCIPLES 1856 engraving Designed by Thomas Jefferson Unique design at that time (1819) University surrounded by a library Separation of church and education Students and professors living in the pavilions A holistic approach to training and education

A KEY QUESTION How was Jefferson able to design a school that was so different from precedents? Can have many guesses Visionary Passion for architecture and education (Ultra) hard work, perseverance, dedication (over decades) Experience of decades Creativity Out-of-the-box thinking Principled design A good understanding of past designs Good judgment and intuition Strong combination of skills (architecture, art, law, mathematics, horticulture, philosophy …) … (You will be exposed to and hopefully develop/enhance many of these skills in this course)

MAJOR HIGH-LEVEL GOALS OF THIS COURSE Understand the principles Understand the precedents Based on such understanding: Enable you to evaluate tradeoffs of different designs and ideas Enable you to develop principled designs Enable you to develop novel, out-of-the-box designs The focus is on: Principles, precedents, and how to use them for new designs In Computer Architecture

ROLE OF THE (COMPUTER) ARCHITECT from Yale Patt’s lecture notes

ROLE OF THE (COMPUTER) ARCHITECT Look backward (to the past) Understand tradeoffs and designs, upsides/downsides, past workloads. Analyze and evaluate the past Look forward (to the future) Be the dreamer and create new designs. Listen to dreamers Push the state of the art. Evaluate new design choices Look up (towards problems in the computing stack) Understand important problems and their nature Develop architectures and ideas to solve important problems Look down (towards device/circuit technology) Understand the capabilities of the underlying technology Predict and adapt to the future of technology (you are designing for N years ahead). Enable the future technology

TAKEAWAYS Being an architect is not easy You need to consider many things in designing a new system + have good intuition/insight into ideas/tradeoffs But, it is fun and can be very technically rewarding And, enables a great future E.g., many scientific and everyday-life innovations would not have been possible without architectural innovation that enabled very high performance systems E.g., your mobile phones This course will teach you how to become a good computer architect

INSTRUCTOR Samira Khan Post Doctorate in CMU PhD from UT San Antonio Assistant Professor samirakhan@virginia.edu Post Doctorate in CMU PhD from UT San Antonio Worked at Intel, AMD, EPFL Research Interest Computer Architecture and Systems Memory System Design New Emerging Technologies Hardware-Software Co-Design Performance, Power, Reliability

TEACHING ASSISTANT Marzieh Lenjani Masters from University of Tehran PhD Student ml2au@virginia.edu Masters from University of Tehran Research Interest Computer Architecture and Systems Memory Approximation

AGENDA Course Overview How to Jump Into Research How to Review Papers First Homework (for Next Week) First Review Assignment (for Next Week)

WHERE TO GET UP-TO-DATE COURSE INFO? Website Piazza Your email: start the subject line with CS 6354 Office Hours

WHAT WILL YOU LEARN Computer Architecture: The science and art of designing, selecting, and interconnecting hardware components and designing the hardware/software interface to create a computing system that meets functional, performance, energy consumption, cost, and other specific goals. Traditional definition: “The term architecture is used here to describe the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior as distinct from the organization of the dataflow and controls, the logic design, and the physical implementation.” Gene Amdahl, IBM Journal of R&D, April 1964

LEVELS OF TRANSFORMATION Interface between Software and Hardware Problem Operating System Interface between Software and Hardware Architecture Circuits

THE POWER OF ABSTRACTION Isolation A higher level only needs to know about the interface to the lower level, not how the lower level is implemented For example, high-level language programmer does not really need to know about the architecture Productivity No need to worry about decisions made in underlying levels For example, programming in Java vs. C vs. assembly vs. binary vs. by specifying control signals of each transistor every cycle

CROSSING THE ABSTRACTION LAYERS Should we always focus on our own layer? As long as everything goes well, not knowing what happens in the underlying level (or above) is not a problem What if One of the layers reach a limit, there is no way to improve There is a new disruptive change in technology that cannot be contained in a layer New Applications that are too slow for today’s system

LEVELS OF TRANSFORMATION Computer Architecture Problem Broader Scope of Computer Architecture Operating System Architecture Circuits

SCOPE OF THE COURSE This course will take a broad view of architecture Beyond the ISA+microarchitecture levels E.g., system-architecture interfaces and interactions E.g., application-architecture interfaces and interactions Out-of-the-box thinking is greatly encouraged E.g., research projects and readings on architectures that challenge the current dominant paradigms processing in memory, approximate systems, asymmetry everywhere, … E.g., readings on topics that are traditionally covered less in computer architecture courses

WHAT WILL YOU LEARN? Hardware/software interface, major components, and programming models of a modern computing platform State-of-the-art as well as research proposals (lots of them) Tradeoffs and how to make them Emphasis on cutting-edge (research & state-of-the-art) Hands-on research in a computer architecture topic Semester-long research project Focus: How to design better architectures (not an intro course) How to dig out information No textbook really required But, see the syllabus

TWO EXAMPLES New Challenge: DRAM Scaling Problem Emerging Technology: Non-Volatile Memory

DRAM SCALING CHALLENGE Technology Scaling DRAM Cells DRAM Cells So now let’s see how we have achieved more memory in till now? The first DRAM chip had only a KB of memory, and now we can have gigabytes on memory in the system. For the last few decades technology scaling has enabled higher capacity. Similar to processor scaling, DRAM vendors have packed more cells in the same die area to achieve higher capacity at a lower cost. Now let’s see how DRAM chip capacity changed with technology scaling over the years. DRAM scaling enabled high capacity

DRAM scaling is getting difficult DRAM SCALING TREND 2X/1.5 YEARS 2X/3 YEARS Here I show the trend of DRAM chip capacity over the years. In the x axis I show the year of mass production and in the y axis we have the capacity in megabits per chip. So in the last 30 years, chip capacity grown from 1 Mb to 8Gb. However, if we look at the growth, previously capacity used to double every two year, but the scaling trend has slowed down and capacity is doubling in every three years. It is very clear from the trend that DRAM scaling is getting difficult. DRAM scaling is getting difficult Source: Flash Memory Summit 2013, Memcon 2014

DRAM SCALING CHALLENGE Technology Scaling DRAM Cells DRAM Cells Manufacturing reliable cells at low cost is getting difficult This is due to the fact that, as the cells are shrinking down there are more failures in DRAM cells and Manufacturing reliable cells at low cost in getting difficult.

TRADITIONAL APPROACH TO ENABLE DRAM SCALING Unreliable DRAM Cells Reliable DRAM Cells Make DRAM Reliable Reliable System Manufacturing Time System in the Field Traditional way to enable scaling is to use circuit-level optimizations and testing to to make sure every DRAM cell operates correctly. So when the chips are used in our systems, systems assumes that DRAM is error-free. Currently, Manufacturers have to provide a strict reliability guarantee for DRAMs. DRAM has strict reliability guarantee

Shift the responsibility to systems NEW APPROACH Unreliable DRAM Cells Reliable DRAM Cells Make DRAM Reliable Reliable System Manufacturing Time Manufacturing Time System in the Field System in the Field In my work, I show that in stead of the manufacturers, if the system becomes responsible for ensuring the reliability of the DRAM cells, manufactures can shrink the cells to smaller technology nodes without paying the cost for providing reliability guarantee. By shifting the responsibility of providing realizable DRAM operation from manufactures to systems, we can enable DRAM scaling. Shift the responsibility to systems

VISION: SYSTEM-LEVEL DETECTION AND MITIGATION Unreliable DRAM Cells Detect and Mitigate Reliable System Detect and mitigate errors after the system has become operational ONLINE PROFILING Reduces cost, increases yield, and enables scaling

TOWARDS AN ONLINE PROFILING SYSTEM Initially Protect DRAM with Strong ECC Periodically Test Parts of DRAM 2 1 Test Test ECC Test Based on these observations, we propose that in an online profiling system, 1 row (8KB) 1 round 667 ns + 64 ms + 667 ns = 64 ms 256 rows 1 round ( 2048 KB = 1MB = -.05% ) 0.17 ms + 64 + 0.17 ms = 64.34 ms Full memory 1 round 1024 times Full memory 7000 rounds 1024 * 7000 times Mitigate errors and reduce ECC 3 Run tests periodically after a short interval at smaller regions of memory

BREAKING THE ABSTRACTION Architecture Circuits Operating System Problem OS needs to know about testing and tested pages Need to implement the testing in the hardware Need to know the circuit-level characteristics of the failures Samira Khan+, "The Efficacy of Error Mitigation Techniques for DRAM Retention Failures: A Comparative Experimental Study”, SIGMETRICS 2014

TWO EXAMPLES New Challenge: DRAM Scaling Problem Emerging Technology: Non-Volatile Memory

TWO-LEVEL STORAGE MODEL CPU Ld/St VOLATILE MEMORY FAST DRAM BYTE ADDR FILE I/O STORAGE NONVOLATILE SLOW BLOCK ADDR

TWO-LEVEL STORAGE MODEL CPU Ld/St VOLATILE MEMORY FAST DRAM BYTE ADDR NVM FILE I/O STORAGE PCM, STT-RAM NONVOLATILE SLOW BLOCK ADDR Non-volatile memories combine characteristics of memory and storage

VISION: UNIFY MEMORY AND STORAGE CPU Ld/St PERSISTENTMEMORY NVM Provides an opportunity to manipulate persistent data directly

memory-storage system DRAM IS STILL FASTER CPU CPU Ld/St PERSISTENTMEMORY MEMORY DRAM NVM A hybrid unified memory-storage system

UNIFY MEMORY AND STORAGE Opportunity to update data in-place in memory with Ld/St interface Do not need to move data from disk to memory, translate file to data structure and transfer to disk again Eliminates wasted work to locate, transfer, and translate data Improves both energy and performance Simplifies programming model as well

BREAKING THE ABSTRACTION Architecture Circuits Operating System Problem Applications manipulate persistent data directly Who provides the consistency and persistency guarantee? How does hardware manage the hybrid memory? Need to know the circuit-level characteristics of the underlying technology Jinglei Ren+, “Dual-Scheme Checkpointing: A Software-Transparent Mechanism for Supporting Crash Consistency in Persistent Memory Systems”, MICRO 2015 Justin Meza+, “A Case for Efficient Hardware-Software Cooperative Management of Storage and Memory”, WEED 2013

COURSE GOALS Goal 1: To familiarize you with both fundamental design tradeoffs and recent research issues/trends in processor, memory, and platform architectures in today’s and future systems Strong emphasis on fundamentals and design tradeoffs. Goal 2: To provide the necessary background and experience to advance the state-of-the-art in computer architecture by performing cutting-edge research Strong emphasis on Critical analysis of research papers (through reading and literature review assignments) Developing new mechanisms that advance the state of the art (through the course research project).

THIS IS A GRADUATE-LEVEL CLASS Required background: basic architecture (3330) basic compilers basic OS programming skills spirit, excitement, and dedication for deep exploration of a topic in computer architecture

WHAT DO I EXPECT FROM YOU? Learn the material & dig deeper Work hard Ask questions, take notes, participate in discussion Critically review the assigned research papers & readings Discuss/critique them online with peers and us Use Piazza and Review Site frequently… Start the research project early and focus Remember “Chance favors the prepared mind.” (Pasteur)

HOW WILL YOU BE EVALUATED? Research Project: 40% Critical Reviews and Paper Presentation: 25% Exam(s): 30% Homework, class participation, and my evaluation of your performance: 5% Participation+discussion is very important Grading will be back-end heavy. Most of your grade will be determined late. How you prepare and manage your time is important But grades should not be the reason for taking this course

RESEARCH PROJECT Your chance to explore in depth a computer architecture topic that interests you Perhaps even publish your innovation in a top computer architecture conference. Start thinking about your project topic from now! Interact with me and mentors Read the project topics handout well Groups of 2-3 students (will finalize this later) Proposal due: Sep 20, 2017

AGENDA Course Overview How to Jump Into Research How to Review Papers First Homework (for Next Week) First Review Assignment (for Next Week)

HOW TO DO RESEARCH AND ADVANCED DEV We will talk a lot about this in this course Learning by example Reading and evaluating strong and seminal papers Learning by doing Semester-long research project Learning by open, critical discussions Recitation sessions, online discussion of papers & ideas on Piazza and the Paper Review Site

WHAT IS THE GOAL OF RESEARCH? To generate new insight that can enable what previously did not exist Research (in engineering) is a hunt for insight that can eventually impact the world

SOME BASIC ADVICE FOR GOOD RESEARCH Choose great problems to solve: Have great taste Difficult Important High impact Read heavily and critically Think big (out of the box) Do not restrain yourself to tweaks Aim high Write and present really well

Looking here for lost keys

Lost keys here Looking here

Current Architecture Practice

Aim Here 5-10 years

Enable this point 5-10 years

The Research Formula

Reward If you are wildly successful, what difference will it make?

Effort Learn as much as possible with as little work as possible

Effort Do the minimum analysis and experimentation necessary to make a point

Research is a hunt for insight Need to get off the beaten path to find new insights

RECOMMENDED TALK Bill Dally, Moving the needle: Effective Computer Architecture Research in Academy and Industry ISCA 2010 Keynote Talk. Acknowledgment: Past few slides are from this talk

“The purpose of computing is insight, not numbers” MORE GOOD ADVICE “The purpose of computing is insight, not numbers” Richard Hamming

HOW TO DO THE PAPER REVIEWS 1: Brief summary What is the problem the paper is trying to solve? What are the key ideas of the paper? Key insights? What is the key contribution to literature at the time it was written? What are the most important things you take out from it? 2: Strengths (most important ones) Does the paper solve the problem well? 3: Weaknesses (most important ones) This is where you should think critically. Every paper/idea has a weakness. This does not mean the paper is necessarily bad. It means there is room for improvement and future research can accomplish this. 4: Can you do (much) better? Present your thoughts/ideas. 5: What have you learned/enjoyed/disliked in the paper? Why? 6: Detailed comment on the contribution of the paper Review should be short and concise (~half a page to a page)

ADVICE ON PAPER REVIEWS When doing the reviews, be very critical Always think about better ways of solving the problem or related problems Do background reading Reviewing a paper/talk is the best way of learning about a research problem/topic Think about forming a literature survey topic or a research proposal based on the paper (for future studies)

PAPER PRESENTATION Group of 3-4 students Each class two groups will present two papers I will provide a list of papers and you can choose Have to submit the slides one week before Everyone else will submit reviews on those papers Ask questions during/after the presentation Remember 5% grade is on class participation The presenters will also initiate discussion on piazza Each person must have at least one post for each paper on piazza

PROJECT More information to come… In the meantime: Read a lot of papers; find focused problem areas to survey papers on We will provide a list of project ideas and papers associated with them A good way of finding topics to survey or do projects on is: Examining the provided project ideas and papers Reading assigned papers in lectures Examining papers from recent conferences (ISCA, MICRO, HPCA, ASPLOS, …)

PROJECT TIMELINE Project Proposal Due: Sep 20, 2017 Project Proposal Presentation: Sep 25-27, 2017 Project Milestone Meetings Slides Due: Nov 1, 2017 Project Milestone Meetings: Nov 6-7, 2017 (By appointment) Final Presentation: Dec 14, 2017 (2.00-5.00 pm) Final Report Due: Dec 14, 2017

AGENDA Course Overview How to Jump Into Research How to Review Papers First Homework (for Next Week) First Review Assignment (for Next Week)

HOMEWORK0 Due on next Monday Aug 28, 2017 Student information Want to know about you What to know about your prior computer architecture background All future grading is dependent on this homework Submit a pdf file in Collab

AGENDA Course Overview How to Jump Into Research How to Review Papers First Homework (for Next Week) First Review Assignment (for Next Week)

REQUIRED REVIEWS Due on next Wednesday Aug 30, 2017 Enter your reviews in Collab Start discussing ideas and thoughts on Piazza

REVIEW PAPER 1 (REQUIRED) Onur Mutlu and Lavanya Subramanian, “Research Problems and Opportunities in Memory Systems" Invited Article in Supercomputing Frontiers and Innovations (SUPERFRI), 2015.

Samira Khan University of Virginia Aug 23, 2017 COMPUTER ARCHITECTURE CS 6354 Samira Khan University of Virginia Aug 23, 2017 The content and concept of this course are adapted from CMU ECE 740