Ayman El-Ghazali Senior Microsoft.

Slides:

Advertisements

Similar presentations

Overview of Mass Storage Structure

Advertisements

Hard Drives Storing Information 1 Byte at a Time.

Storing Data: Disks and Files: Chapter 9

Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Mass-Storage Systems Revised Tao Yang.

Denny Cherry Manager of Information Systems MVP, MCSA, MCDBA, MCTS, MCITP.

CS4432: Database Systems II Data Storage - Lecture 2 (Sections 13.1 – 13.3) Elke A. Rundensteiner.

IELM 230: File Storage and Indexes Agenda: - Physical storage of data in Relational DB’s - Indexes and other means to speed Data access - Defining indexes.

1 CS143: Disks and Files. 2 System Architecture CPU Main Memory Disk Controller... Disk Word (1B – 64B) ~ x GB/sec Block (512B – 50KB) ~ x MB/sec System.

CS4432: Database Systems II Lecture 2 Timothy Sutherland.

Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Mass-Storage Systems Revised Tao Yang.

CPSC 231 Secondary storage (D.H.)1 Learning Objectives Understanding disk organization. Sectors, clusters and extents. Fragmentation. Disk access time.

Introduction to Database Systems 1 The Storage Hierarchy and Magnetic Disks Storage Technology: Topic 1.

Solid State Drive Feb 15. NAND Flash Memory Main storage component of Solid State Drive (SSD) USB Drive, cell phone, touch pad…

Layers of a DBMS Query optimization Execution engine Files and access methods Buffer management Disk space management Query Processor Query execution plan.

Operating Systems CMPSC 473 I/O Management (2) December Lecture 24 Instructor: Bhuvan Urgaonkar.

CS4432: Database Systems II Data Storage (Better Block Organization) 1.

Lecture 11: DMBS Internals

ISDMR:BEIT VIII: CHAP2:MADHU N 1 ISM - Course Organization.

Disk Access. DISK STRUCTURE Sector: Smallest unit of data transfer from/to disk; 512B 2/4/8 adjacent sectors transferred together: Blocks Read/write heads.

Introduction to Database Systems 1 Storing Data: Disks and Files Chapter 3 “Yea, from the table of my memory I’ll wipe away all trivial fond records.”

CS 111 – Aug – 1.3 –Information arranged in memory –Types of memory –Disk properties Commitment for next day: –Read pp , In other.

Architecture Rajesh. Components of Database Engine.

Hardware Trends. Contents Memory Hard Disks Processors Network Accessories Future.

Chapter 12: Mass-Storage Systems Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Chapter 12: Mass-Storage.

ICS 321 Fall 2011 Overview of Storage & Indexing (i) Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 11/9/20111Lipyeow.

DBI313. MetricOLTPDWLog Read/Write mixMostly reads, smaller # of rows at a time Scan intensive, large portions of data at a time, bulk loading Mostly.

External Storage Primary Storage : Main Memory (RAM). Secondary Storage: Peripheral Devices –Disk Drives –Tape Drives Secondary storage is CHEAP. Secondary.

Chapter 8 External Storage. Primary vs. Secondary Storage Primary storage: Main memory (RAM) Secondary Storage: Peripheral devices  Disk drives  Tape.

DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.

12/18/20151 Operating Systems Design (CS 423) Elsa L Gunter 2112 SC, UIUC Based on slides by Roy Campbell, Sam.

Disk Basics CS Introduction to Operating Systems.

CS 101 – Sept. 28 Main vs. secondary memory Examples of secondary storage –Disk (direct access) Various types Disk geometry –Flash memory (random access)

DMBS Internals I February 24 th, What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the.

DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.

DMBS Architecture May 15 th, Generic Architecture Query compiler/optimizer Execution engine Index/record mgr. Buffer manager Storage manager storage.

CPSC 231 Secondary storage (D.H.)1 Learning Objectives Understanding disk organization. Sectors, clusters and extents. Fragmentation. Disk access time.

Programmer’s View of Files Logical view of files: –An a array of bytes. –A file pointer marks the current position. Three fundamental operations: –Read.

Computer Performance. Hard Drive - HDD Stores your files, programs, and information. If it gets full, you can’t save any more. Measured in bytes (KB,

COSC 6340: Disks 1 Disks and Files DBMS stores information on (“hard”) disks. This has major implications for DBMS design! » READ: transfer data from disk.

Optimizing SQL Server and Databases for large Fact Tables =tg= Thomas Grohser, NTT Data SQL Server MVP SQL Server Performance Engineering SQL Saturday.

1 Lecture 16: Data Storage Wednesday, November 6, 2006.

Storage Tuning for Relational Databases Philippe Bonnet – Spring 2015.

SQL Server Internals 101 AYMAN SENIOR MICROSOFT.

Select Operation Strategies And Indexing (Chapter 8)

1 Components of the Virtual Memory System  Arrows indicate what happens on a lw virtual address data physical address TLB page table memory cache disk.

CS422 Principles of Database Systems Disk Access Chengyu Sun California State University, Los Angeles.

I/O Management and Disk Scheduling

Chapter 10: Mass-Storage Systems

What Is a Latch? …and Why Do I Care? Eddie Wuerch, mcm

A Day in the Life of a Row Eddie Wuerch, mcm

What Is a Latch? …and Why Do I Care? Eddie Wuerch, mcm

Lecture 16: Data Storage Wednesday, November 6, 2006.

Database Management Systems (CS 564)

Computer Science 210 Computer Organization

I/O Resource Management: Software

CPSC-608 Database Systems

Chapter Overview Understanding the Database Architecture

Hustle and Bustle of SQL Pages

Disks and Files DBMS stores information on (“hard”) disks.

File Processing : Storage Media

Lecture 11: DMBS Internals

Lecture 9: Data Storage and IO Models

What Is a Latch? …and Why Do I Care? Eddie Wuerch, mcm

File Processing : Storage Media

Chapter 11: Mass-Storage Systems

Mass-Storage Systems (Disk Scheduling)

CS 295: Modern Systems Storage Technologies Introduction

Memory Management & Virtual Memory

Hybrid Buffer Pool The Good, the Bad and the Ugly

Presentation transcript:

Ayman El-Ghazali Senior Microsoft

Page Header (96 Bytes) Data Row 1 (100 Bytes) Data Row 3 (80 Bytes) Data Row 2 (200 Bytes) 321 Warning: Actual colors may vary! Row offset: Distance from the first byte in the row to the start of the page For this example: Row Offset 1 would have the value 96 Row Offset 2 would have the value 196 Row Offset 3 would have the value 396

GAM Extent 0 Extent 250 Tracks 64,000 Extents (4GB) 1 Represents if that extent is free GAM Page

Any Free Storage Space Available?

Uniform ExtentsMixed Extent Extent 1 (Table1) Extent 2 (Table2) Extent 3 (Table1) IAM1 Allocation Bitmaps Table3 SGAM GAM PFS IAM2

Client Query Engine Storage Engine Storage (RAM or HD) Query Engine

FSB CPU North Bridge (Memory Control) South Bridge (I/O Control) VIA DMA SATA/SAS PCI-E RAM

Refer to the book: Professional SQL Server 2008 Internals and Troubleshooting

1 Platter(s) 5 Drive Head 6 Track(s) 7 Sector(s) 8 Cluster(s) * Sector sizes in many modern disks are now 4K in size (previously 512 bytes)

Track Sectors Drive Head

Track Sectors Drive Head

64KB Allocation Units 4KB Allocation Units

Partition1 content/uploads/2012/02/Hard_Disk1.png

LUN1 LUN2 LUN3 LUN4

128 KB of Data in pages that are full takes up 128KB (Fill Factor 100%) 64 KB of Data in pages that are half full takes up 128KB (Fill Factor 50%)

64KB pages compressed down to 32KB (50%) 32KB pages decompressed to 64KB (if all pages had to be decompressed) Transfer 32KB Decompress

ZYNZ1GD3ZYNZ1GD4ZYNZ1GD5ZYNZ1GD6 ZYNZ1GD7ZYNZ1GD8ZYNZ1GD9ZYNZ1GD10 Page Info Buffer Cache Address 1:1:200XYNPUG19 3:7:98AZX444ST1 5:1:199ZYNZ1GD8 2:2:17XYAPUG10 1:1:21TRW56F7F Request for Page #199 on File #1 from DatabaseID 5 Stored in reverse DatabaseID : FileNumber : PageNumber 5:1:199 5:1: 199 Hash Bucket Data Cache (RAM)

MetricDD4 RAMDDR3 RAMSSD (Solid State Drive)HDD (Hard Disk Drive) Random Access Speed or Latency (RAM) Slightly higher than DDR3 10ns (0.01 Microsecond) 100 Microseconds (0.1 ms) 100,000 Nanoseconds (ns) 4-9 milliseconds (ms) Microseconds Cost per GB$4+ (was $18+ in 2015) $4+$0.5-$3.00 (Enterprise PCI-E $2+)$0.05-$0.15 Power Consumption~40% < DDR32.5-5W0.6W-3W (PCI-E ~20W)5-20 Watts Data Transfer Rate (Peak) SATA III (600 MB/s)N/A MB/sLess than 15K RPM PCI-E rev 3 x16 (~16 GB/s)N/A MB/sN/A DIMM17,000-34,100 MB/s6,400-24,000 MB/sN/A *DDR4 Memory peak transfer rate may increase; Current Server RAM up to 25,000 MB/s *Drive price and specs for Single Drives not SAN Storage *SATA III and SAS are very close in performance *Fibre Channel connection on a SAN Speeds are between MB/s for transfer rates *SSD sizes max out around 1TB for mainstream drives; HDD sizes max around 4TB *Prices from

HDD 10,000 Pages SSD DDR3 RAM SSD PCI-E 50,000 Pages 250,000 Pages

10,000 Pages DDR4 RAM 50,000 Pages 250,000 Pages

RAM SSD HDD 1 Cookie 10,000 Cookies 1,000,000 Cookies

FSB CPU North Bridge (Memory Control) South Bridge (I/O Control) VIA DMA SATA/SAS PCI-E RAM CPU does all the “Thinking” Requests data from disk Disk returns data to Memory via Direct Memory Access Memory pushes data to CPU via Front Side Bus For SQL Server, when the pages are kept in memory, they can be brought back from memory without having to go to disk.

Ayman El-Ghazali -