Data consistency & data store FESA Advanced. Agenda 2 Problems with previous framework More problems with the new CPUs FESA 3 solutions Fields types Field.

Slides:



Advertisements
Similar presentations
Processes and Threads Chapter 3 and 4 Operating Systems: Internals and Design Principles, 6/E William Stallings Patricia Roy Manatee Community College,
Advertisements

B-Trees. Motivation When data is too large to fit in the main memory, then the number of disk accesses becomes important. A disk access is unbelievably.
5.1 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts with Java – 8 th Edition Chapter 5: CPU Scheduling.
 A quantum is the amount of time a thread gets to run before Windows checks.  Length: Windows 2000 / XP: 2 clock intervals Windows Server systems: 12.
IBM OS/2 CS450 Section 2 5:00PM Perpetrators: © Dario Berini, Derek Cole, Drew Hayes, Leigh Johann, Matt Kesler.
Slides 8d-1 Programming with Shared Memory Specifying parallelism Performance issues ITCS4145/5145, Parallel Programming B. Wilkinson Fall 2010.
Avishai Wool lecture Priority Scheduling Idea: Jobs are assigned priorities. Always, the job with the highest priority runs. Note: All scheduling.
Chapter 7 Interupts DMA Channels Context Switching.
Figure 1.1 Interaction between applications and the operating system.
CS533 - Concepts of Operating Systems
Computer Organization and Architecture
Java ThreadsGraphics Programming Graphics Programming: Java Threads.
DoublePlay: Parallelizing Sequential Logging and Replay Kaushik Veeraraghavan Dongyoon Lee, Benjamin Wester, Jessica Ouyang, Peter M. Chen, Jason Flinn,
A. Frank - P. Weisberg Operating Systems Introduction to Cooperating Processes.
Chapter 1 An Overview of Database Management. 1-2 Topics in this Chapter What is a Database System? What is a Database? Why Database? Data Independence.
Give an example to show the advantages to using multithreaded Clients See page 142 of the core book (Tanebaum 2002).
Layers and Views of a Computer System Operating System Services Program creation Program execution Access to I/O devices Controlled access to files System.
I/O Systems ◦ Operating Systems ◦ CS550. Note:  Based on Operating Systems Concepts by Silberschatz, Galvin, and Gagne  Strongly recommended to read.
Chapter 18 Multicore Computers
Ranga Rodrigo. Class is central to object oriented programming.
Previously Fetch execute cycle Pipelining and others forms of parallelism Basic architecture This week we going to consider further some of the principles.
Object Oriented Analysis & Design SDL Threads. Contents 2  Processes  Thread Concepts  Creating threads  Critical sections  Synchronizing threads.
Programming in Java Unit 2. Class and variable declaration A class is best thought of as a template from which objects are created. You can create many.
Operating Systems ECE344 Ashvin Goel ECE University of Toronto Threads and Processes.
Chapter 2 (PART 1) Light-Weight Process (Threads) Department of Computer Science Southern Illinois University Edwardsville Summer, 2004 Dr. Hiroshi Fujinoki.
Operating Systems ECE344 Ashvin Goel ECE University of Toronto OS Design.
Operating Systems ECE344 Ding Yuan Paging Lecture 8: Paging.
© Janice Regan, CMPT 300, May CMPT 300 Introduction to Operating Systems Introduction to Concurrency.
CE Operating Systems Lecture 3 Overview of OS functions and structure.
Lecture 8 February 29, Topics Questions about Exercise 4, due Thursday? Object Based Programming (Chapter 8) –Basic Principles –Methods –Fields.
Code Migration Russell T. Potee, III. Overview Why Code Migration? Code Migration Models Migration and Handling Resources Heterogeneous Systems Migration.
Computer Organization & Assembly Language © by DR. M. Amer.
SE-2840 Dr. Mark L. Hornick1 Servlet Threads and Sessions.
Java Thread and Memory Model
Multiprocessor Cache Consistency (or, what does volatile mean?) Andrew Whitaker CSE451.
December 1, 2006©2006 Craig Zilles1 Threads and Cache Coherence in Hardware  Previously, we introduced multi-cores. —Today we’ll look at issues related.
Operating Systems CSE 411 CPU Management Sept Lecture 10 Instructor: Bhuvan Urgaonkar.
Page 1 Printing & Terminal Services Lecture 8 Hassan Shuja 11/16/2004.
Concurrent Computing CSCI 201L Jeffrey Miller, Ph.D. HTTP :// WWW - SCF. USC. EDU /~ CSCI 201 USC CSCI 201L.
Chapter 13 – I/O Systems (Pgs ). Devices  Two conflicting properties A. Growing uniformity in interfaces (both h/w and s/w): e.g., USB, TWAIN.
CS Introduction to Data Structures Spring Term 2004 Franz Hiergeist.
CS399 New Beginnings Jonathan Walpole. 2 Concurrent Programming & Synchronization Primitives.
Design Patterns Software Engineering CS 561. Last Time Introduced design patterns Abstraction-Occurrence General Hierarchy Player-Role.
Processes, Threads, and Process States. Programs and Processes  Program: an executable file (before/after compilation)  Process: an instance of a program.
Copyright © Curt Hill Parallelism in Processors Several Approaches.
CS 740: Advanced Computer Networks IP Lookup and classification Supplemental material 02/05/2007.
1 FESA architecture v.1.0 Framework Configuration & Data-entry Tool 5 th December 2003.
Threaded Programming Lecture 1: Concepts. 2 Overview Shared memory systems Basic Concepts in Threaded Programming.
Lecture 12 Page 1 CS 111 Online Using Devices and Their Drivers Practical use issues Achieving good performance in driver use.
Silberschatz, Galvin and Gagne ©2009Operating System Concepts – 8 th Edition Chapter 4: Threads.
Software tools for digital LLRF system integration at CERN 04/11/2015 LLRF15, Software tools2 Andy Butterworth Tom Levens, Andrey Pashnin, Anthony Rey.
ECE 297 Concurrent Servers Process, fork & threads ECE 297.
December 1, 2006©2006 Craig Zilles1 Threads & Atomic Operations in Hardware  Previously, we introduced multi-core parallelism & cache coherence —Today.
Lucas De Marchi sponsors: co-authors: Liria Matsumoto Sato
Event Sources and Realtime Actions
Lecture 12: Real-Time Scheduling
Atomic Operations in Hardware
Atomic Operations in Hardware
Unit OS2: Operating System Principles
CS490 Windows Internals Quiz 2 09/27/2013.
Data-Types.
Use of Multiple Devices
Threads & multithreading
Process Description and Control
Background and Motivation
Top Half / Bottom Half Processing
LHC BLM Software audit June 2008.
Fabric metrics Qi Zhang(IBM Research – T.J. Watson) Hardware Software
Access Control What’s New?
Creating and Using Classes
Presentation transcript:

Data consistency & data store FESA Advanced

Agenda 2 Problems with previous framework More problems with the new CPUs FESA 3 solutions Fields types Field type decision trees

Typical priorities we typically give a lower prio to the server threads E.g. RT around 50 and server around 25 Sometimes, we give even lower prio for some properties (FESA 2.10 background props) 3

Settings problem 4 Datastore Server writes new settings RT writes settings in HW The RT uses inconsistent settings Inconsistent Datastore Server writes new settings RT writes settings in HW Inconsistent The RT uses consistent settings  set

Acquisition problem 5 Datastore Server updates clients RT writes acquisition Datastore Server updates clients RT writes acquisition The server sends consistent data The server sends inconsistent data  Inconsistent

Agenda 6 Problems with previous framework More problems with the new CPUs FESA 3 solutions Fields types Field type decision trees

New CPU New problems Intel Core 2 DUO L7400 – 2 cores w/o hyperthreading 2 cores  2 threads run in parallel whatever the prio 7 Datastore Server writes new settings RT writes settings in HW Inconsistent The RT uses inconsistent settings  Inconsistent Similar problem for the acquisition case Playing with prio no longer helps with data races set

Agenda 8 Problems with previous framework More problems with the new CPUs FESA 3 solutions Fields types Field type decision trees

FESA 3 solutions Settings Settings flow as defined in FESA 3 is High-level  Server part  RT part  Hardware Double-buffer mechanism to protect against data races 9 Datastore Server writes new settings RT writes settings in HW set Inconsistent OK RT sync OK The RT uses consistent settings RT sync done in the RT part Very cheap pointer swapping  very little jitter introduced pending buffer (server) active buffer (RT)

FESA 3 solutions Acquisition 10 Acquisition flow as defined in FESA 3 is Hardware  RT part  Server part  High-level Rolling-buffer mechanism to protect against data races Uses the cycle-stamp to select the buffer Implementation hidden  you don’t have access to the other buffers Datastore Updates for X Server updates clients RT writes acquisition Data for cystamp X Cystamp Y Z Discards data for X The clients receive consistent acquisitions

Agenda 11 Problems with previous framework More problems with the new CPUs FESA 3 solutions Fields types Field type decision trees

Field types There are 5 types of fields Configuration Setting Unshared setting Acquisition Generic Configuration field Read-only from anywhere Default field in the configuration section 12

Field types Setting field Shared and data-consistent (double-buffer) Read-write from server Read-only from RT Default field in the setting section Buffer swap by RT part Unshared setting field Not visible from RT Setting field’s attribute “shared” = false 13

Field types Acquisition field Shared and data-consistent (rolling-buffer) Read-only from server Read-write from RT Default field in the acquisition section Works with the cycle-stamp in the context Generic field Accessible from anywhere and without any protection Any field’s attribute “data-consistent” = false Should be used for private fields only 14

An example 6 fields – all int32_t 1 configuration field – configField 3 setting fields serverField – a server only field (i.e. not shared) sharedField – a double-buffer field unprotectedField – a generic field w/o protection 2 acquisition fields rollingField – a rolling-buffer field rtField – a generic field w/o protection (RT use only) 15

An example Design 16

An example device.h class Device : public fesa::AbstractDevice { public: Device(); ConfigFieldScalar configField; SettingFieldScalar serverField; SettingFieldScalar sharedField; GenericFieldScalar unprotectedField; AcqFieldScalar rollingField; GenericFieldScalar rtField; };

An example Device constructor Device::Device() : configField("configField",this), serverField("serverField",true, false, this, true, false), sharedField("sharedField",true, false, this, true, true), unprotectedField("unprotectedField",true, this, true), rollingField("rollingField",true, this, false), rtField("rtField",true, this, false) { } Note: double-buffer fields and unshared fields have same impl. Only the last arg is different

Rolling buffer depth configuration Rolling buffer’s size in the instantiation file Minimum 3 slots Maximum deps on the FEC memory

Agenda 20 Problems with previous framework More problems with the new CPUs FESA 3 solutions Fields types Field type decision trees

Which setting field? Do I need to access the field from the RT part? Setting field Do I need to write into the field from the RT part? Is the field private to the RT part? Setting field Setting field with default values N Y NY NY You must provide your own data protection mechanism ǃ

Which acquisition field? Is the field private to the RT part? Acquisition field Do I need to keep the value from the previous execution? Acquisition field with default values 2 separate fields: 1 private & 1 w/o history Y N YN start over for each field