Project 4. “File System Implementation”

Slides:



Advertisements
Similar presentations
Chapter 4 Memory Management Basic memory management Swapping
Advertisements

4.4 Page replacement algorithms
Tutorial 8 March 9, 2012 TA: Europa Shang
Chapter 4 Memory Management Page Replacement 补充:什么叫页面抖动?
1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.
Discussion Week 7 TA: Kyle Dewey. Overview Midterm debriefing Virtual memory Virtual Filesystems / Disk I/O Project #3.
File Systems.
Caching and Virtual Memory. Main Points Cache concept – Hardware vs. software caches When caches work and when they don’t – Spatial/temporal locality.
Memory Management (II)
Virtual Memory. Why do we need VM? Program address space: 0 – 2^32 bytes –4GB of space Physical memory available –256MB or so Multiprogramming systems.
Virtual Memory Management B.Ramamurthy. Paging (2) The relation between virtual addresses and physical memory addres- ses given by page table.
1 Virtual Memory Management B.Ramamurthy Chapter 10.
EECS 370 Discussion 1 SMBC. EECS 370 Discussion Exam 2 Solutions posted online Will be returned in next discussion (12/9) – Grades hopefully up on CTools.
Caching and Virtual Memory. Main Points Cache concept – Hardware vs. software caches When caches work and when they don’t – Spatial/temporal locality.
1 Design and Performance of a Web Server Accelerator Eric Levy-Abegnoli, Arun Iyengar, Junehwa Song, and Daniel Dias INFOCOM ‘99.
CSE 451: Operating Systems
« Performance of Compressed Inverted List Caching in Search Engines » Proceedings of the International World Wide Web Conference Commitee, Beijing 2008)
1 Memory Management 4.1 Basic memory management 4.2 Swapping 4.3 Virtual memory 4.4 Page replacement algorithms 4.5 Modeling page replacement algorithms.
Virtual Memory Expanding Memory Multiple Concurrent Processes.
Kyu Ho Park Sept. 22, Lecture 4 proc file system(procfs) (Project 2 included) Sept.19,4pm.
Buffers Let’s go for a swim. Buffers A buffer is simply a collection of bytes A buffer is simply a collection of bytes – a char[] if you will. Any information.
Lecture 10 Page 1 CS 111 Summer 2013 File Systems Control Structures A file is a named collection of information Primary roles of file system: – To store.
Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the same chip.
Project 4. “File System Implementation”
11.1 Silberschatz, Galvin and Gagne ©2005 Operating System Principles 11.5 Free-Space Management Bit vector (n blocks) … 012n-1 bit[i] =  1  block[i]
CSCI 156: Lab 11 Paging. Our Simple Architecture Logical memory space for a process consists of 16 pages of 4k bytes each. Your program thinks it has.
ICOM 6005 – Database Management Systems Design Dr. Manuel Rodríguez-Martínez Electrical and Computer Engineering Department Lecture 7 – Buffer Management.
Project 3. “System Call and Synchronization” By Dongjin Kim
Silberschatz, Galvin and Gagne ©2011 Operating System Concepts Essentials – 8 th Edition Chapter 9: Virtual Memory.
1 Section 8: File Systems Project 3. 2 Questions?
Chapter 19 Translation Lookaside Buffer
Translation Lookaside Buffer
EE516: Embedded Software Project 1
Memory Hierarchy Ideal memory is fast, large, and inexpensive
Computer Organization
Jonathan Walpole Computer Science Portland State University
Memory COMPUTER ARCHITECTURE
Lecture: Large Caches, Virtual Memory
Chapter 11: File System Implementation
The Buffer Cache.
Project 3. “System Call and Synchronization”
Virtual Memory Use main memory as a “cache” for secondary (disk) storage Managed jointly by CPU hardware and the operating system (OS) Programs share main.
Lecture: Cache Hierarchies
File System Structure How do I organize a disk into a file system?
Database Management Systems (CS 564)
Introduction to C Topics Compilation Using the gcc Compiler
Consider a Direct Mapped Cache with 4 word blocks
Lecture: Large Caches, Virtual Memory
Lecture: Cache Hierarchies
Main Memory Management
Lecture 10: Buffer Manager and File Organization
Chapter 8: Main Memory.
Chapter 9 Virtual Memory
File Systems Directories Revisited Shared Files
EECE.4810/EECE.5730 Operating Systems
Chapter 9: Virtual-Memory Management
Lecture 6 Memory Management
Memory and cache CPU Memory I/O.
Performance metrics for caches
Performance metrics for caches
Performance metrics for caches
CSE 451: Operating Systems Autumn 2003 Lecture 10 Paging & TLBs
CSE 451: Operating Systems Autumn 2003 Lecture 10 Paging & TLBs
Performance metrics for caches
CS703 - Advanced Operating Systems
Lecture 9: Caching and Demand-Paged Virtual Memory
COMP755 Advanced Operating Systems
Performance metrics for caches
Mr. M. D. Jamadar Assistant Professor
Virtual Memory 1 1.
Presentation transcript:

Project 4. “File System Implementation” 2016. 11. 15 Dong Jin Kim, PhD student

Introduction to Projects Project 1: Setting up Environment for Project Project 2: Linux Fundamental Project 3: System Call and Synchronization Project 4: File System Project 5: Device Driver for Embedded H/W Especially, including “BeagleBoard Development”

Contents of Project 4 Task 1. Data encryption on FUSE file system (30%) Task 2. Buffer cache implementation on FUSE file system (30%) Task 3. File system performance evaluation (10%) Report (30%) Explanation about your work Answer to questions

FUSE File System What is FUSE? FUSE : File System in User space General File System is in the Kernel Space Our Goal Simple Buffer Cache Management in FUSE

FUSE Install Download provided “fuse-2.9.4.tar.gz” from the course home page # wget http://core.kaist.ac.kr/~EE516/resources/fuse-2.9.4.tar.gz # tar xf fuse-2.9.4.tar.gz # cd fuse-2.9.4 # ./configure # make # make install

FUSE Tools # wget http://core.kaist.ac.kr/~EE516/resources/fuse-tutorial.tgz # tar xf fuse-tutorial. # cd fuse-tutorial-2014-06-12 # ./configure # make

Use FUSE File System # cd src # mkdir /mnt/fuse # mkdir /root/fuse # ./bbfs -o direct_io /mnt/fuse /root/fuse # mount Check information Mounting Point Real Dir.

General File System File system operation in kernel space User Program read() write() User Space Kernel Space VFS File System

Current FUSE File System File system operation in user space FUSE bb_read bb_write User Program read() write() read() write() User Space System Call Kernel Space VFS File System

Data Encryption Encrypt data using “keys” FUSE User Program read() Task 1 Data Encryption Encrypt data using “keys” FUSE bb_read bb_write User Program Encrypted data read() write() read() write() User Space System Call Kernel Space VFS File System

Simple Encryption and Decryption Method Task 1 Simple Encryption and Decryption Method Using encryption, only users with correct keys can take correct data Incorrect keys will give incorrect data For each “char” or “byte”, encrypt with … Addition: add a value to original data Right shift: after addition, an amount of bit-shift operation to the data Decryption = reverse of encryption 1 1 2-bit right shift

Modify FUSE for Encryption and Decryption Task 1 Modify FUSE for Encryption and Decryption Modify 2 functions in bbfs.c bb_read Decrypt after pread called bb_write Encrypt before pwrite called With correct keys, correct data should be available With incorrect keys, correct data should not be available How to give encryption keys Create “fuse-tutorial-2014-06-12/src/ee516.conf” file First line: two non-negative integer, separated by space or tab(\t) First integer: key for addition The number that should be added to original data Second integer: key for shift The number of bit-shifts for each byte When you mount fuse file system, read this file and take keys If you want to change keys, unmount fuse  modify ee516.conf file  mount fuse

Example Result File written in (addition, shift) = (3, 5) Task 1 Example Result File written in (addition, shift) = (3, 5) Read with (4, 5), (3, 4), and (4, 4), respectively

Buffer Cache Implementation Task 2 Buffer Cache Implementation Add buffer cache Add this FUSE bb_read bb_write Buffer Cache User Program read() write() read() write() User Space System Call Kernel Space VFS File System

Modify FUSE for Buffer Cache Implementation Task 2 Modify FUSE for Buffer Cache Implementation Modify 3 functions in bbfs.c bb_read bb_write bb_release Read and write operations Read Buffer hit Return contents Buffer miss If buffer is full, evict some contents Read from disk and write to buffer Write Write into the buffer Release If there are dirty data of corresponding file in the buffer, write them to the disk

Eviction Algorithms to Implement Task 2 Eviction Algorithms to Implement Implement 2 eviction algorithms 1. Random eviction Choose victim randomly 2. LRU Least Recently Used For buffer hit, entry moves to the head of the queue

Specification to Implement the Buffer Cache Task 2 Specification to Implement the Buffer Cache Do not use O_DIRECT flags for testing the given FUSE file system FUSE doesn’t support O_DIRECT yet “Buffer Cache” should have an array of “chunks” and a linked list Each “chunk” stores 4KB data You can assume that request size is fixed to 4KB The linked list will manage eviction algorithm Buffer Cache Buffer Cache Size : 5MB Don’t care about consistency (for just 1 user) Eviction policy will given as an integer in the second line of “ee516.conf” file 0: no buffer / 1: random eviction / 2: LRU

Performance Evaluation Task 3 Performance Evaluation Use benchmark tool of project 1 Remove all O_DIRECT flags for FUSE Use FILESIZE 10MB Use request size (second parameter of execution) 4096 You can use http://core.kaist.ac.kr/~EE516/resources/task3_fsbench.c Run benchmark and analyze Current Linux file system Same as project 1 FUSE with 0. No encryption and no buffer 1. No encryption and random eviction buffer 2. No encryption and LRU buffer 3. Encryption and no buffer 4. Encryption and random eviction buffer 5. Encryption and LRU buffer

Questions for the Report Find answers from your brain, web site, book, Q1. Pros. and Cons. of FUSE file system (against file system in kernel space) Q2. In our encryption, can different key pairs give same encrypted / decrypted data? Why? Q3. List several encryption methods and analyze them. Q4. Why performances of Linux and FUSE file systems are different? Q5. Why performances of different eviction algorithms are different? In which cases each eviction algorithm can have advantage? Specify the references

Submission Contents Submission Source codes Report Implemented and modified codes Include comments in the source codes Report Key points of your source code Do not attach entire source codes in the report Result screenshots Answers of questions Submission Due date: Nov. 28., PM 23:59 Delay penalty 10% per day (AM 00:00) E-mail: Joo Kyung Ro eu8198@kaist.ac.kr Title: [EE516 Project 4] student_number Attached file: student_number.zip Various compression format is allowed