1. Samira Khan University of Virginia August 23, 2016
ADVANCED RESEARCH IN PROCESSOR & MEMORY DESIGN
CS 6501
Samira Khan
University of Virginia
August 23, 2016
The content and concept of this lecture is adapted from CMU ECE 740

2. ROTUNDA, PAVILLIONS AND THE LAWN

3. A DESIGN BASED ON PRINCIPLES
1856 engraving
Designed by Thomas Jefferson
Unique design at that time (1819)
University surrounded by a library
Separation of church and education
Students and professors living in the pavilions
A holistic approach to training and education

4. A KEY QUESTION
How was Jefferson able to design a school that was so different from precedents?
Can have many guesses
Visionary
Passion for architecture and education
(Ultra) hard work, perseverance, dedication (over decades)
Experience of decades
Creativity
Out-of-the-box thinking
Principled design
A good understanding of past designs
Good judgment and intuition
Strong combination of skills (architecture, art, law, mathematics, horticulture, philosophy …) …
(You will be exposed to and hopefully develop/enhance many of these skills in this course)

5. MAJOR HIGH-LEVEL GOALS OF THIS COURSE
Understand the principles
Understand the precedents
Based on such understanding:
Enable you to evaluate tradeoffs of different designs and ideas
Enable you to develop principled designs
Enable you to develop novel, out-of-the-box designs
The focus is on:
Principles, precedents, and how to use them for new designs
In Computer Architecture

6. ROLE OF THE (COMPUTER) ARCHITECT
from Yale Patt’s lecture notes

7. ROLE OF THE (COMPUTER) ARCHITECT
Look backward (to the past)
Understand tradeoffs and designs, upsides/downsides, past workloads. Analyze and evaluate the past
Look forward (to the future)
Be the dreamer and create new designs. Listen to dreamers
Push the state of the art. Evaluate new design choices
Look up (towards problems in the computing stack)
Understand important problems and their nature
Develop architectures and ideas to solve important problems
Look down (towards device/circuit technology)
Understand the capabilities of the underlying technology
Predict and adapt to the future of technology (you are designing for N years ahead). Enable the future technology

8. TAKEAWAYS Being an architect is not easy
You need to consider many things in designing a new system + have good intuition/insight into ideas/tradeoffs
But, it is fun and can be very technically rewarding
And, enables a great future
E.g., many scientific and everyday-life innovations would not have been possible without architectural innovation that enabled very high performance systems
E.g., your mobile phones
This course will teach you how to become a good computer architect

9. INSTRUCTOR Samira Khan Post Doctorate in CMU PhD from UT San Antonio
Assistant Professor
Post Doctorate in CMU
PhD from UT San Antonio
Worked at Intel, AMD, EPFL
Research Interest
Computer Architecture and Systems
Memory System Design
New Emerging Technologies
Hardware-Software Co-Design
Performance, Power, Reliability

10. AGENDA Course Overview Example Research Problems
How to Jump Into Research

11. WHERE TO GET UP-TO-DATE COURSE INFO?
Website
Piazza
Your start the subject line with CS 6501
Office Hours

12. WHAT IS COMPUTER ARCHITECTURE
Computer Architecture: The science and art of designing, selecting, and interconnecting hardware components and designing the hardware/software interface to create a computing system that meets functional, performance, energy consumption, cost, and other specific goals.
Traditional definition: “The term architecture is used here to describe the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior as distinct from the organization of the dataflow and controls, the logic design, and the physical implementation.” Gene Amdahl, IBM Journal of R&D, April 1964

13. LEVELS OF TRANSFORMATION Interface between Software and Hardware
Problem
Operating System
Interface between Software and Hardware
Architecture
Circuits

14. THE POWER OF ABSTRACTION
Isolation
A higher level only needs to know about the interface to the lower level, not how the lower level is implemented
For example, high-level language programmer does not really need to know about the architecture
Productivity
No need to worry about decisions made in underlying levels
For example, programming in Java vs. C vs. assembly vs. binary vs. by specifying control signals of each transistor every cycle

15. CROSSING THE ABSTRACTION LAYERS
Should we always focus on our own layer?
As long as everything goes well, not knowing what happens in the underlying level (or above) is not a problem
What if
One of the layers reach a limit, there is no way to improve
There is a new disruptive change in technology that cannot be contained in a layer
New Applications that are too slow for today’s system

16. LEVELS OF TRANSFORMATION Computer Architecture
Problem
Broader Scope of
Computer Architecture
Operating System
Architecture
Circuits

17. SCOPE OF THE COURSE
This course will take a broad view of processor and memory design
Beyond the ISA+microarchitecture levels
E.g., system-architecture interfaces and interactions
E.g., application-architecture interfaces and interactions
Out-of-the-box thinking is greatly encouraged
E.g., research projects and readings on architectures that challenge the current dominant paradigms
processing in memory, approximate systems, persistent memory, neuromorphic computing, …
E.g., readings on topics that are traditionally covered less in computer architecture courses

18. WHAT WILL YOU LEARN?
Hardware/software interface, major components, and programming models of a modern computing platform
State-of-the-art as well as research proposals (lots of them)
Tradeoffs and how to make them
Emphasis on cutting-edge (research & state-of-the-art)
Hands-on research in a computer architecture topic
Semester-long research project
Focus: How to design better architectures (not an intro course)
Broaden your research vision
Will read papers not only from computer architecture
But, from circuits, programming language, operating systems, graphics, networking, …

19. COURSE GOALS
Goal 1: Rethink and redesign our computing model focusing on minimizing data movement and storage.
Understand the importance of cross layer research
Gain new insight from VLSI circuits, architecture design, systems, programming languages, and new application domains
Strong emphasis on
Critical analysis of research papers (through reading and literature review assignments)
Goal 2: To provide the necessary background and experience to advance the state-of-the-art in computer architecture by performing cutting-edge research
Developing new mechanisms that advance the state of the art (through the course research project).

20. THIS IS A GRADUATE-LEVEL CLASS
Required background:
basic architecture
basic compilers
basic OS
programming skills
spirit, excitement, and dedication for deep exploration of a topic in computer architecture

21. WHAT DO I EXPECT FROM YOU?
Work hard
Ask questions, think critically, participate in discussion
Critically review the assigned research papers & readings
Discuss/critique them online with peers and us
Use Piazza and Review Site frequently…
Start the research project early and focus
Remember “Chance favors the prepared mind.” (Pasteur)

22. HOW WILL YOU BE EVALUATED?
Class Participation 10%
Reviews 25%
Class Presentations 30%
Project 35%
Participation+discussion is very important
Grading will be back-end heavy. Most of your grade will be determined late. How you prepare and manage your time is important
But grades should not be the reason for taking this course

23. HOW TO DO THE PAPER REVIEWS
1: Brief summary
What is the problem the paper is trying to solve?
What are the key ideas of the paper? Key insights?
What is the key contribution to literature at the time it was written?
What are the most important things you take out from it?
2: Strengths (most important ones)
Does the paper solve the problem well?
3: Weaknesses (most important ones)
This is where you should think critically. Every paper/idea has a weakness. This does not mean the paper is necessarily bad. It means there is room for improvement and future research can accomplish this.
4: Can you do (much) better? Present your thoughts/ideas.
5: What have you learned/enjoyed/disliked in the paper? Why?
Review should be short and concise (~half a page to a page)

24. ADVICE ON PAPER REVIEWS
When doing the reviews, be very critical
Always think about better ways of solving the problem or related problems
Do background reading
Reviewing a paper/talk is the best way of learning about a research problem/topic
Think about forming a literature survey topic or a research proposal based on the paper (for future studies)

25. PRESENTATION Summary of the Work Strengths and Weaknesses Discussion
Problem, key ideas or insights, detailed mechanisms, and results.
Strengths and Weaknesses
Detailed discussion on both sides
Discussion
Future research directions
Alternative ways of solving the problem
Importance of the problem
Anything interesting about the research direction
What did you like? What did you not like?

26. PRESENTATION Each class two paper presentations
One main paper from the list (Review required)
One related paper (Review encouraged)
From the list or
From the reference list or
You can pick, but needs to be approved
Sign up for the papers and slots by Friday
Read the abstracts, think about the topic
Select papers/topics you find interesting
Can select you own topic, but needs to be approved

27. RESEARCH PROJECT More information to come… In the meantime:
Read a lot of papers; find focused problem areas to survey papers on
We will provide a list of project ideas and papers associated with them
A good way of finding topics to survey or do projects on is:
Examining the provided project ideas and papers
Reading assigned papers in lectures
Examining papers from recent conferences (ISCA, MICRO, HPCA, ASPLOS, …)

28. RESEARCH PROJECT
Your chance to explore in depth a computer architecture topic that interests you
Perhaps even publish your innovation in a top computer architecture conference.
Start thinking about your project topic from now!
Interact with me
Read the project topics handout well
Proposal due: Sep 15, 2016

29. AGENDA Course Overview Example Research Problems
How to Jump Into Research

30. TWO EXAMPLES New Challenge: DRAM Scaling Problem
Emerging Technology: Non-Volatile Memory

31. DRAM SCALING CHALLENGE
Technology
Scaling
DRAM Cells
DRAM Cells
So now let’s see how we have achieved more memory in till now? The first DRAM chip had only a KB of memory, and now we can have gigabytes on memory in the system. For the last few decades technology scaling has enabled higher capacity. Similar to processor scaling, DRAM vendors have packed more cells in the same die area to achieve higher capacity at a lower cost. Now let’s see how DRAM chip capacity changed with technology scaling over the years.
DRAM scaling enabled high capacity

32. DRAM scaling is getting difficult
DRAM SCALING TREND
2X/1.5 YEARS
2X/3 YEARS
Here I show the trend of DRAM chip capacity over the years. In the x axis I show the year of mass production and in the y axis we have the capacity in megabits per chip. So in the last 30 years, chip capacity grown from 1 Mb to 8Gb. However, if we look at the growth, previously capacity used to double every two year, but the scaling trend has slowed down and capacity is doubling in every three years. It is very clear from the trend that DRAM scaling is getting difficult.
DRAM scaling is getting difficult
Source: Flash Memory Summit 2013, Memcon 2014

33. DRAM SCALING CHALLENGE
Technology
Scaling
DRAM Cells
DRAM Cells
Manufacturing reliable cells at low cost
is getting difficult
This is due to the fact that, as the cells are shrinking down there are more failures in DRAM cells and Manufacturing reliable cells at low cost in getting difficult.

34. TRADITIONAL APPROACH TO ENABLE DRAM SCALING
Unreliable
DRAM Cells
Reliable
DRAM Cells
Make
DRAM
Reliable
Reliable System
Manufacturing Time
System
in the Field
Traditional way to enable scaling is to use circuit-level optimizations and testing to to make sure every DRAM cell operates correctly. So when the chips are used in our systems, systems assumes that DRAM is error-free. Currently, Manufacturers have to provide a strict reliability guarantee for DRAMs.
DRAM has strict reliability guarantee

35. Shift the responsibility to systems
NEW APPROACH
Unreliable
DRAM Cells
Reliable
DRAM Cells
Make
DRAM
Reliable
Reliable System
Manufacturing Time
Manufacturing Time
System
in the Field
System
in the Field
In my work, I show that in stead of the manufacturers, if the system becomes responsible for ensuring the reliability of the DRAM cells, manufactures can shrink the cells to smaller technology nodes without paying the cost for providing reliability guarantee. By shifting the responsibility of providing realizable DRAM operation from manufactures to systems, we can enable DRAM scaling.
Shift the responsibility to systems

36. VISION: SYSTEM-LEVEL DETECTION AND MITIGATION
Unreliable
DRAM Cells
Detect
and
Mitigate
Reliable System
Detect and mitigate errors after
the system has become operational
ONLINE PROFILING
Reduces cost, increases yield,
and enables scaling

37. TOWARDS AN ONLINE PROFILING SYSTEM
Initially Protect DRAM
with Strong ECC
Periodically Test
Parts of DRAM 2 1
Test
Test
ECC
Test
Based on these observations, we propose that in an online profiling system,
1 row (8KB) 1 round 667 ns + 64 ms ns = 64 ms
256 rows 1 round ( 2048 KB = 1MB = -.05% ) 0.17 ms ms = ms
Full memory 1 round 1024 times
Full memory 7000 rounds 1024 * 7000 times
Mitigate errors and
reduce ECC 3
Run tests periodically after a short interval
at smaller regions of memory

38. BREAKING THE ABSTRACTION
Architecture
Circuits
Operating System
Problem
OS needs to know about testing and
tested pages
Need to implement the testing
in the hardware
Need to know the circuit-level characteristics
of the failures
Samira Khan+, "The Efficacy of Error Mitigation Techniques for DRAM Retention Failures: A Comparative Experimental Study”, SIGMETRICS 2014

39. TWO EXAMPLES New Challenge: DRAM Scaling Problem
Emerging Technology: Non-Volatile Memory

40. TWO-LEVEL STORAGE MODEL
CPU
Ld/St
VOLATILE
MEMORY
FAST
DRAM
BYTE ADDR
FILE
I/O
STORAGE
NONVOLATILE
SLOW
BLOCK ADDR

41. TWO-LEVEL STORAGE MODEL
CPU
Ld/St
VOLATILE
MEMORY
FAST
DRAM
BYTE ADDR
NVM
FILE
I/O
STORAGE
PCM, STT-RAM
NONVOLATILE
SLOW
BLOCK ADDR
Non-volatile memories combine characteristics of memory and storage

42. VISION: UNIFY MEMORY AND STORAGE
CPU
Ld/St
PERSISTENTMEMORY
NVM
Provides an opportunity to manipulate persistent data directly

43. memory-storage system
DRAM IS STILL FASTER
CPU
CPU
Ld/St
PERSISTENTMEMORY
MEMORY
DRAM
NVM
A hybrid unified
memory-storage system

44. UNIFY MEMORY AND STORAGE
Opportunity to update data in-place in memory with Ld/St interface
Do not need to move data from disk to memory, translate file to data structure and transfer to disk again
Eliminates wasted work to locate, transfer, and translate data
Improves both energy and performance
Simplifies programming model as well

45. BREAKING THE ABSTRACTION
Architecture
Circuits
Operating System
Problem
Applications manipulate persistent
data directly
Who provides the consistency and
persistency guarantee?
How does hardware manage the
hybrid memory?
Need to know the circuit-level characteristics
of the underlying technology
Jinglei Ren+, “Dual-Scheme Checkpointing: A Software-Transparent Mechanism for Supporting Crash Consistency in Persistent Memory Systems”, MICRO 2015
Justin Meza+, “A Case for Efficient Hardware-Software Cooperative Management of Storage and Memory”, WEED 2013

46. AGENDA Course Overview Example Research Problems
How to Jump Into Research

47. HOW TO DO RESEARCH AND ADVANCED DEV
We will talk a lot about this in this course
Learning by example
Reading and evaluating strong and seminal papers
Learning by doing
Semester-long research project
Learning by open, critical discussions
Recitation sessions, online discussion of papers & ideas on Piazza and the Paper Review Site

48. WHAT IS THE GOAL OF RESEARCH?
To generate new insight
that can enable what previously did not exist
Research (in engineering) is a hunt for insight that can eventually impact the world

49. SOME BASIC ADVICE FOR GOOD RESEARCH
Choose great problems to solve: Have great taste
Difficult
Important
High impact
Read heavily and critically
Think big (out of the box)
Do not restrain yourself to tweaks
Aim high
Write and present really well

50. Looking here for lost keys

51. Lost keys here
Looking here

52. Current Architecture Practice

54. Aim Here
5-10 years

55. Enable this point
5-10 years

56. The Research Formula

57. Reward
If you are wildly successful, what difference will it make?

58. Effort
Learn as much as possible with as little work as possible

59. Effort
Do the minimum analysis and experimentation necessary to make a point

60. Research is a
hunt for insight
Need to get off the beaten path to find new insights

61. RECOMMENDED TALK
Bill Dally, Moving the needle: Effective Computer Architecture Research in Academy and Industry
ISCA 2010 Keynote Talk.
Acknowledgment: Past few slides are from this talk

62. “The purpose of computing is insight, not numbers”
MORE GOOD ADVICE
“The purpose of computing is insight, not numbers”
Richard Hamming

63. Samira Khan University of Virginia August 23, 2016
ADVANCED RESEARCH IN PROCESSOR & MEMORY DESIGN
CS 6501
Samira Khan
University of Virginia
August 23, 2016
The content and concept of this lecture is adapted from CMU ECE 740