1. Samira Khan University of Virginia Aug 29, 2018
COMPUTER ARCHITECTURE
CS 6354
Samira Khan
University of Virginia
Aug 29, 2018
Some content of this course is adapted from CMU ECE 740

2. ROTUNDA, PAVILLIONS AND THE LAWN

3. A DESIGN BASED ON PRINCIPLES
1856 engraving
Designed by Thomas Jefferson
Unique design at that time (1819)
University surrounded by a library
Separation of church and education
Students and professors living in the pavilions
A holistic approach to training and education

4. A KEY QUESTION
How was Jefferson able to design a school that was so different from precedents?
Can have many guesses
Visionary
Passion for architecture and education
(Ultra) hard work, perseverance, dedication (over decades)
Experience of decades
Creativity
Out-of-the-box thinking
Principled design
A good understanding of past designs
Good judgment and intuition
Strong combination of skills (architecture, art, law, mathematics, horticulture, philosophy …) …
(You will be exposed to and hopefully develop/enhance many of these skills in this course)

5. MAJOR HIGH-LEVEL GOALS OF THIS COURSE
Understand the principles
Understand the precedents
Based on such understanding:
Enable you to evaluate tradeoffs of different designs and ideas
Enable you to develop principled designs
Enable you to develop novel, out-of-the-box designs
The focus is on:
Principles, precedents, and how to use them for new designs
In Computer Architecture

6. ROLE OF THE (COMPUTER) ARCHITECT
from Yale Patt’s lecture notes

7. ROLE OF THE (COMPUTER) ARCHITECT
Look backward (to the past)
Understand tradeoffs and designs, upsides/downsides, past workloads. Analyze and evaluate the past
Look forward (to the future)
Be the dreamer and create new designs. Listen to dreamers
Push the state of the art. Evaluate new design choices
Look up (towards problems in the computing stack)
Understand important problems and their nature
Develop architectures and ideas to solve important problems
Look down (towards device/circuit technology)
Understand the capabilities of the underlying technology
Predict and adapt to the future of technology (you are designing for N years ahead). Enable the future technology

8. TAKEAWAYS Being an architect is not easy
You need to consider many things in designing a new system + have good intuition/insight into ideas/tradeoffs
But, it is fun and can be very technically rewarding
And, enables a great future
E.g., many scientific and everyday-life innovations would not have been possible without architectural innovation that enabled very high performance systems
E.g., your mobile phones
This course will teach you how to become a good computer architect

9. INSTRUCTOR Samira Khan Post Doctorate in CMU PhD from UT San Antonio
Assistant Professor
Post Doctorate in CMU
PhD from UT San Antonio
Worked at Intel, AMD, EPFL
Research Interest
Computer Architecture and Systems
Memory System Design
New Emerging Technologies
Hardware-Software Co-Design
Performance, Power, Reliability

10. TEACHING ASSISTANT Yizhou Wei Undergrad Nanjing University
CS PhD Student
Undergrad Nanjing University
Research Interests
Computer Architecture and Systems
Memory
Processing in Memory

11. AGENDA Course Overview How to Review Papers
First Homework (for Next Week)
First Review Assignment (for Next Week)

12. WHERE TO GET UP-TO-DATE COURSE INFO?
Website
Piazza
piazza.com/virginia/fall2018/cs6354/home
Your start the subject line with CS 6354
Fastest Response
Office Hours
By appointment

13. WHAT WILL YOU LEARN
Computer Architecture: The science and art of designing, selecting, and interconnecting hardware components and designing the hardware/software interface to create a computing system that meets functional, performance, energy consumption, cost, and other specific goals.
Traditional definition: “The term architecture is used here to describe the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior as distinct from the organization of the dataflow and controls, the logic design, and the physical implementation.” Gene Amdahl, IBM Journal of R&D, April 1964

14. LEVELS OF TRANSFORMATION Interface between Software and Hardware
Problem
Operating System
Interface between Software and Hardware
Architecture
Circuits

15. THE POWER OF ABSTRACTION
Isolation
A higher level only needs to know about the interface to the lower level, not how the lower level is implemented
For example, high-level language programmer does not really need to know about the architecture
Productivity
No need to worry about decisions made in underlying levels
For example, programming in Java vs. C vs. assembly vs. binary vs. by specifying control signals of each transistor every cycle

16. CROSSING THE ABSTRACTION LAYERS
Should we always focus on our own layer?
As long as everything goes well, not knowing what happens in the underlying level (or above) is not a problem
What if
One of the layers reach a limit, there is no way to improve
There is a new disruptive change in technology that cannot be contained in a layer
New Applications that are too slow for today’s system

17. LEVELS OF TRANSFORMATION Computer Architecture
Problem
Broader Scope of
Computer Architecture
Operating System
Architecture
Circuits

18. SCOPE OF THE COURSE This course will take a broad view of architecture
Beyond the ISA+microarchitecture levels
E.g., system-architecture interfaces and interactions
E.g., application-architecture interfaces and interactions
Out-of-the-box thinking is greatly encouraged
E.g., research projects and readings on architectures that challenge the current dominant paradigms
processing in memory, approximate systems, asymmetry everywhere, …
E.g., readings on topics that are traditionally covered less in computer architecture courses

19. WHAT WILL YOU LEARN?
Hardware/software interface, major components, and programming models of a modern computing platform
State-of-the-art as well as research proposals (lots of them)
Tradeoffs and how to make them
Emphasis on cutting-edge (research & state-of-the-art)
Hands-on research in a computer architecture topic
Semester-long research project
Focus: How to design better architectures (not an intro course)
How to dig out information
No textbook really required
But, see the syllabus

20. TWO EXAMPLES Emerging Technology: Non-Volatile Memory
New Challenge: Memory Wall

21. TWO-LEVEL STORAGE MODEL
CPU
Ld/St
VOLATILE
MEMORY
FAST
DRAM
BYTE ADDR
FILE
I/O
STORAGE
NONVOLATILE
SLOW
BLOCK ADDR

22. TWO-LEVEL STORAGE MODEL
CPU
Ld/St
VOLATILE
MEMORY
FAST
DRAM
BYTE ADDR
NVM
FILE
I/O
STORAGE
PCM, STT-RAM
NONVOLATILE
SLOW
BLOCK ADDR
Non-volatile memories combine characteristics of memory and storage

23. VISION: UNIFY MEMORY AND STORAGE
CPU
Ld/St
PERSISTENTMEMORY
NVM
Provides an opportunity to manipulate persistent data directly

24. memory-storage system
DRAM IS STILL FASTER
CPU
CPU
Ld/St
PERSISTENTMEMORY
MEMORY
DRAM
NVM
A hybrid unified
memory-storage system

25. UNIFY MEMORY AND STORAGE
Opportunity to update data in-place in memory with Ld/St interface
Do not need to move data from disk to memory, translate file to data structure and transfer to disk again
Eliminates wasted work to locate, transfer, and translate data
Improves both energy and performance
Simplifies programming model as well

26. END-TO-END SYSTEM FOR PERSISTENT MEMORY
PROBLEM
How to write consistent code?
How to allocate objects in persistent memory?
CPU
PERSISTENTMEMORY
Ld/St
APPLICATION
Software
How to annotate and verify applications?
COMPILER
PERSISTENT MANAGER
How to manage unified memory and storage support?
ARCHITECTURE
How to provide hardware support?
Hardware
CIRCUITS
How to reduce wear-out?
GOAL: A full stack support for persistent memory applications
Jinglei Ren+, “Dual-Scheme Checkpointing: A Software-Transparent Mechanism for Supporting Crash Consistency in Persistent Memory Systems”, MICRO 2015
Justin Meza+, “A Case for Efficient Hardware-Software Cooperative Management of Storage and Memory”, WEED 2013
Sihang Liu+, “Crash Consistency for Encrypted Non-Volatile Main Memory Systems”, HPCA 2018

27. TWO EXAMPLES Emerging Technology: Non-Volatile Memory
New Challenge: Memory Wall

28. DRAM Latency-Capacity Trend
16X
-20%
The figure shows the historical trends of DRAM during the last 12-years, from 2000 to 2011.
We can see that, during this time, DRAM capacity has increased by 16 times.
On the other hand, during the same period of time, DRAM latency has reduced by only 20%.
As a result, the high DRAM latency continues to be a critical bottleneck for system performance.
DRAM latency continues to be a critical bottleneck

29. Memory is the main bottleneck
Memory Wall
1) High latency
Memory L3
CPU L2 L1 MC
2) Low bandwidth
Memory is the main bottleneck

30. Memory is the main bottleneck
Processing in Memory
Calculate in memory
Smart
Memory L3
CPU L2 L1 MC
Compute
Only send back
the result
Memory is the main bottleneck

31. IN-MEMORY COMPUTING Offload computation to appropriate computing model
SPECIALIZED CORES
ARCHITECTURE
CIRCUITS
RUNTIME
PROBLEM
APPLICATION OS
Software
Hardware
Which functions to offload?
What is the interface?
IN-MEMORY
LOGIC
Who provides data consistency?
How to schedule data access and offloading?
How to enable computation?
MEMORY WITH LOGIC
Offload computation to appropriate computing model
based on application, runtime, and accelerator characteristics
Image: Micron Technology, 2015

32. COURSE GOALS
Goal 1: To familiarize you with both fundamental design tradeoffs and recent research issues/trends in processor, memory, and platform architectures in today’s and future systems
Strong emphasis on fundamentals and design tradeoffs.
Goal 2: To provide the necessary background and experience to advance the state-of-the-art in computer architecture by performing cutting-edge research
Strong emphasis on
Critical analysis of research papers (through reading and literature review assignments)
Developing new mechanisms that advance the state of the art (through the course research project).

33. THIS IS A GRADUATE-LEVEL CLASS
Required background:
basic architecture (3330)
basic compilers
basic OS
programming skills
spirit, excitement, and dedication for deep exploration of a topic in computer architecture

34. WHAT DO I EXPECT FROM YOU?
Learn the material & dig deeper
Work hard
Ask questions, take notes, participate in discussion
Critically review the assigned research papers & readings
Discuss/critique them online with peers and us
Use Piazza frequently…
Start the research project early and focus
Remember “Chance favors the prepared mind.” (Pasteur)

35. HOW WILL YOU BE EVALUATED?
Research Project: 40%
Critical Reviews and Paper Presentation: 25%
Exam(s): 30%
Homework, class participation, and my evaluation of your performance: 5%
Participation+discussion is very important
Grading will be back-end heavy. Most of your grade will be determined late. How you prepare and manage your time is important
But grades should not be the reason for taking this course

36. RESEARCH PROJECT
Your chance to explore in depth a computer architecture topic that interests you
Perhaps even publish your innovation in a top computer architecture conference.
Start thinking about your project topic from now!
Interact with me and mentors
Read the project topics handout well
Groups of 2-3 students (will finalize this later)
Proposal due: Sep 26, 2018

37. AGENDA Course Overview How to Review Papers
First Homework (for Next Week)
First Review Assignment (for Next Week)

38. HOW TO DO THE PAPER REVIEWS
1: Brief summary
What is the problem the paper is trying to solve?
What are the key ideas of the paper? Key insights?
What is the key contribution to literature at the time it was written?
What are the most important things you take out from it?
2: Strengths (most important ones)
Does the paper solve the problem well?
3: Weaknesses (most important ones)
This is where you should think critically. Every paper/idea has a weakness. This does not mean the paper is necessarily bad. It means there is room for improvement and future research can accomplish this.
4: Can you do (much) better? Present your thoughts/ideas.
5: What have you learned/enjoyed/disliked in the paper? Why?
6: Detailed comment on the contribution of the paper
Review should be short and concise (~half a page to a page)

39. ADVICE ON PAPER REVIEWS
When doing the reviews, be very critical
Always think about better ways of solving the problem or related problems
Do background reading
Reviewing a paper/talk is the best way of learning about a research problem/topic
Think about forming a literature survey topic or a research proposal based on the paper (for future studies)

40. PAPER PRESENTATION Group of 2-3 students
Each class two groups will present two papers
I will provide a list of papers and you can choose
Have to submit the slides one week before
Everyone else will submit reviews on those papers
Ask questions during/after the presentation
Remember 5% grade is on class participation
The presenters will also initiate discussion on piazza
Each person must have at least one post for each paper on piazza

41. PROJECT More information to come… In the meantime:
Read a lot of papers; find focused problem areas to survey papers on
We will provide a list of project ideas and papers associated with them
A good way of finding topics to survey or do projects on is:
Examining the provided project ideas and papers
Reading assigned papers in lectures
Examining papers from recent conferences (ISCA, MICRO, HPCA, ASPLOS, …)

42. PROJECT TIMELINE Project Proposal Due: Sep 26, 2018
Project Proposal Presentation: Oct 1-3, 2018
Project Milestone Meetings Slides Due: Oct 24, 2018
Project Milestone Meetings: Oct 24-26, 2018 (By appointment)
Final Presentation: Dec 14, 2018 ( pm)
Final Report Due: Dec 16, 2018

43. AGENDA Course Overview How to Jump Into Research How to Review Papers
First Homework (for Next Week)
First Review Assignment (for Next Week)

44. HOMEWORK0 Due on next Monday Sep 3, 2018 Student information
Want to know about you
What to know about your prior computer architecture background
All future grading is dependent on this homework
Submit a pdf file in Collab

45. AGENDA Course Overview How to Review Papers
First Homework (for Next Week)
First Review Assignment (for Next Week)

46. REQUIRED REVIEWS Due on next Wednesday Sep 5, 2018
Enter your reviews in Collab
Start discussing ideas and thoughts on Piazza

47. REVIEW PAPER 1 (REQUIRED)
Onur Mutlu and Lavanya Subramanian, “Research Problems and Opportunities in Memory Systems" Invited Article in Supercomputing Frontiers and Innovations (SUPERFRI), 2015.

48. Samira Khan University of Virginia Aug 28, 2018
COMPUTER ARCHITECTURE
CS 6354
Samira Khan
University of Virginia
Aug 28, 2018
Some content of this course is adapted from CMU ECE 740