1 Copyright © 2014 M. E. Kabay. All rights reserved. Application Controls CSH5 Chapter 52 “Application Controls” Myles Walsh.

Slides:



Advertisements
Similar presentations
Managing Multi-User Databases (2) IS 240 – Database Management Lecture #19 – Prof. M. E. Kabay, PhD, CISSP Norwich University
Advertisements

Managing Multi-User Databases (1) IS 240 – Database Management Lecture #18 – Prof. M. E. Kabay, PhD, CISSP Norwich University
Database Management System MIS 520 – Database Theory Fall 2001 (Day) Lecture 13.
Database Systems, 8 th Edition Concurrency Control with Time Stamping Methods Assigns global unique time stamp to each transaction Produces explicit.
Database Administration Chapter Six DAVID M. KROENKE’S DATABASE CONCEPTS, 2 nd Edition.
Transaction Management and Concurrency Control
10 1 Chapter 10 Transaction Management and Concurrency Control Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
Transaction Management and Concurrency Control
Transaction Management and Concurrency Control
DAVID M. KROENKE’S DATABASE PROCESSING, 10th Edition © 2006 Pearson Prentice Hall 4-1 David M. Kroenke Database Processing Chapter 9 Managing Multi- User.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Transaction Management and Concurrency Control.
What is a Transaction? Logical unit of work
Chapter 8 : Transaction Management. u Function and importance of transactions. u Properties of transactions. u Concurrency Control – Meaning of serializability.
DBMS Functions Data, Storage, Retrieval, and Update
DAVID M. KROENKE’S DATABASE PROCESSING, 10th Edition © 2006 Pearson Prentice Hall 8-1 COS 346 Day 18.
Chapter 9 Transaction Management and Concurrency Control
9 Chapter 9 Transaction Management and Concurrency Control Hachim Haddouti.
Database Administration Part 1 Chapter Six CSCI260 Database Applications.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Transaction Management and Concurrency Control.
Transaction Management WXES 2103 Database. Content What is transaction Transaction properties Transaction management with SQL Transaction log DBMS Transaction.
Transaction Management and Concurrency Control
Transaction Management Chapter 9. What is a Transaction? A logical unit of work on a database A logical unit of work on a database An entire program An.
INTRODUCTION TO TRANSACTION PROCESSING CHAPTER 21 (6/E) CHAPTER 17 (5/E)
1 IT420: Database Management and Organization Transactions 31 March 2006 Adina Crăiniceanu
© 2013 Pearson Education, Inc. Publishing as Prentice Hall 1 CHAPTER 11: DATA AND DATABASE ADMINISTRATION Modern Database Management 11 th Edition Jeffrey.
Database Integrity and Security HAP 709 – Healthcare Databases George Mason University Janusz Wojtusiak, PhD Fall, 2010.
Concepts of Database Management, Fifth Edition
Database Management System Module 5 DeSiaMorewww.desiamore.com/ifm1.
BIS Database Systems School of Management, Business Information Systems, Assumption University A.Thanop Somprasong Chapter # 10 Transaction Management.
Chapterb19 Transaction Management Transaction: An action, or series of actions, carried out by a single user or application program, which reads or updates.
Concurrency and Transaction Processing. Concurrency models 1. Pessimistic –avoids conflicts by acquiring locks on data that is being read, so no other.
Chapter 15 Recovery. Topics in this Chapter Transactions Transaction Recovery System Recovery Media Recovery Two-Phase Commit SQL Facilities.
Lecture 12 Recoverability and failure. 2 Optimistic Techniques Based on assumption that conflict is rare and more efficient to let transactions proceed.
1 CS 430 Database Theory Winter 2005 Lecture 16: Inside a DBMS.
Copyright © Curt Hill Database Function What should every database do?
1 Transactions Chapter Transactions A transaction is: a logical unit of work a sequence of steps to accomplish a single task Can have multiple.
1 IT420: Database Management and Organization Session Control Managing Multi-user Databases 24 March 2006 Adina Crăiniceanu
Concurrency Control in Database Operating Systems.
11/7/2012ISC329 Isabelle Bichindaritz1 Transaction Management & Concurrency Control.
Chapter 15 Recovery. Copyright © 2004 Pearson Addison-Wesley. All rights reserved.15-2 Topics in this Chapter Transactions Transaction Recovery System.
Transactions and Locks A Quick Reference and Summary BIT 275.
© 2002 by Prentice Hall 1 Database Administration David M. Kroenke Database Concepts 1e Chapter 6 6.
The Relational Model1 Transaction Processing Units of Work.
Fundamentals, Design, and Implementation, 9/e Chapter 9 Managing Multi-User Databases.
MBA 664 Database Management Dave Salisbury ( )
KROENKE and AUER - DATABASE CONCEPTS (3 rd Edition) © 2008 Pearson Prentice Hall 6-1 Chapter Objectives Understand the need for and importance of database.
Transaction Management Transparencies. ©Pearson Education 2009 Chapter 14 - Objectives Function and importance of transactions. Properties of transactions.
1 Intro stored procedures Declaring parameters Using in a sproc Intro to transactions Concurrency control & recovery States of transactions Desirable.
1 Advanced Database Concepts Transaction Management and Concurrency Control.
Module 11: Managing Transactions and Locks
9 1 Chapter 9_B Concurrency Control Database Systems: Design, Implementation, and Management, Rob and Coronel.
10 1 Chapter 10_B Concurrency Control Database Systems: Design, Implementation, and Management, Rob and Coronel.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Transaction Management and Concurrency Control.
10 Transaction Management and Concurrency Control MIS 304 Winter 2005.
©Bob Godfrey, 2002, 2005 Lecture 17: Transaction Integrity and Concurrency BSA206 Database Management Systems.
3 Database Systems: Design, Implementation, and Management CHAPTER 9 Transaction Management and Concurrency Control.
10 1 Chapter 10 - A Transaction Management Database Systems: Design, Implementation, and Management, Rob and Coronel.
Module 14: Managing Transactions and Locks. Overview Introducing Transactions and Locks Managing Transactions Understanding SQL Server Locking Architecture.
David M. Kroenke and David J. Auer Database Processing Fundamentals, Design, and Implementation Chapter Nine: Managing Multiuser Databases.
18 September 2008CIS 340 # 1 Last Covered (almost)(almost) Variety of middleware mechanisms Gain? Enable n-tier architectures while not necessarily using.
Chapter 13 Managing Transactions and Concurrency Database Principles: Fundamentals of Design, Implementation, and Management Tenth Edition.
SYSTEMS IMPLEMENTATION TECHNIQUES TRANSACTION PROCESSING DATABASE RECOVERY DATABASE SECURITY CONCURRENCY CONTROL.
Copyright © 2016 Pearson Education, Inc. CHAPTER 12: DATA AND DATABASE ADMINISTRATION Modern Database Management 12 th Edition Jeff Hoffer, Ramesh Venkataraman,
Transaction Management and Concurrency Control
Multi-User Databases Chapter 9.
Database Processing: David M. Kroenke’s Chapter Nine: Part One
Chapter 10 Transaction Management and Concurrency Control
Introduction of Week 13 Return assignment 11-1 and 3-1-5
Database Administration
Transactions, Properties of Transactions
Presentation transcript:

1 Copyright © 2014 M. E. Kabay. All rights reserved. Application Controls CSH5 Chapter 52 “Application Controls” Myles Walsh

2 Copyright © 2014 M. E. Kabay. All rights reserved. Topics  Protection in Development  Protecting Databases  Protecting Batch Files  Ensuring that Information in the System is Valid

3 Copyright © 2014 M. E. Kabay. All rights reserved. Protection in Development  Software Quality Assurance (QA)  Focuses on methods for preventing, catching and correcting errors in source code and other operational instructions  Application Controls  Specific subset of methods for preventing data corruption in production systems  Databases  Primary mechanism for storing and manipulating data in today’s systems  Batch vs Online  Live interaction (online)  Automated processing (batch)

4 Copyright © 2014 M. E. Kabay. All rights reserved. Types of Data Corruption  Physical  Caused by hardware failures  Errors do not correlate with (respect) Applications Files / datasets / databases Why not?  Logical  Caused by software failures  Errors may correlate with Applications Files / datasets / databases  Why only “may”?

5 Copyright © 2014 M. E. Kabay. All rights reserved. DBMS Controls  Referential Integrity  Uniqueness Constraints  Locking  Transaction Controls  Database Recovery For those who have completed IS240, this section should be a review.

6 Copyright © 2014 M. E. Kabay. All rights reserved. Referential Integrity  In designing databases, may stipulate that certain records may not exist without pre- existing indexes  E.g., cannot normally enter  Order-detail without entering order-header  Prescription data without entering Patient data Doctor data Pharmacist data Drug data Admission data

7 Copyright © 2014 M. E. Kabay. All rights reserved. Referential Integrity (cont’d)  DBMS will prevent deletion of records when others are dependent on them  E.g.,  Cannot delete order-header if there are orders with the order-number of that header  Cannot delete patient-master record if there are admission records for that patient

8 Copyright © 2014 M. E. Kabay. All rights reserved. DBMS Controls  Referential Integrity  Uniqueness Constraints  Locking  Transaction Controls  Database Recovery

9 Copyright © 2014 M. E. Kabay. All rights reserved. Uniqueness Constraints  In relational DBMS (RDBMS), every record in a table (dataset) must be unique  If there is no natural key or index field (or combination of fields) that guarantees uniqueness, can create one automatically  DBMS can enforce uniqueness of specific fields in records using the unique key characteristic  E.g.,  Order-number  Patient-ID  Student-ID….

10 Copyright © 2014 M. E. Kabay. All rights reserved. DBMS Controls  Referential Integrity  Uniqueness Constraints  Locking  Transaction Controls  Database Recovery

11 Copyright © 2014 M. E. Kabay. All rights reserved. Locking  Concurrency Control  Basic Concepts of Locking  Serializable Transactions  Deadlock (Deadly Embrace)  Locking Strategies

12 Copyright © 2014 M. E. Kabay. All rights reserved. Concurrency Control  Multi-Step Transactions  Resource Locking  Consistent Transactions  Transaction Isolation Level  Cursor Type

13 Copyright © 2014 M. E. Kabay. All rights reserved. Multi-Step Transactions Are Fragile  Think about order-entry system  Create order-header  Includes total of cost of line-items (details) Updated at END of detail data entry  Begin entering line-items  Enter 3 records … have not yet finished  System crashes  What is the value in the order-header’s total field?

14 Copyright © 2014 M. E. Kabay. All rights reserved. Concurrency Causes New Problems E.g., The Lost Update Problem:  User A reads inventory: finds 20 widgets.  User B reads inventory: also finds 20 widgets.  User A subtracts 10 widgets from 20, writes total ____ widgets back into inventory  User B subtracts 5 widgets from ____, writes total ____ widgets back into inventory  But actually, there are only ____ widgets left in the real inventory 20 widgets A B

15 Copyright © 2014 M. E. Kabay. All rights reserved. Atomic Transactions  We want to complete  All the steps of a transaction or  None of the steps  ATOMIC  Greek “a” for “not” & “tomos” for “cut”  Thus “atomic” means “can not be cut.”  We mark atomic transactions with boundaries  Start transaction  Commit transaction  If necessary, can reverse steps taken  Rollback transaction ατομος

16 Copyright © 2014 M. E. Kabay. All rights reserved. Resource Locking  Basic Concepts of Locking  Lock Terminology  Serializable Transactions  Deadlocks  Optimistic vs Pessimistic Locking  Declaring Lock Characteristics

17 Copyright © 2014 M. E. Kabay. All rights reserved. Basic Concepts of Locking  Locking is used in inter-process communication (IPC)  A lock is a form of semaphore (signal)  Locks allow processes to  Coordinate their access to resources  Prevent inconsistencies  In DBMS, primarily used to serialize data access  One process gets control of data at a time

18 Copyright © 2014 M. E. Kabay. All rights reserved. Lock Terminology  Implicit vs explicit  Automatic locks placed by DBMS: implicit  Programmatically ordered: explicit  Lock granularity  Large: database, dataset  Fine: records  Exclusive vs shared locks  Exclusive: One process READ/WRITE No other processes allowed at all  Shared: One process has R/W Other processes can only READ

19 Copyright © 2014 M. E. Kabay. All rights reserved. Conditional vs Unconditional Locking  Conditional locking  Process 1 locks resource A  Process 2 locks resource A Receives error condition Lock fails and process 2 continues  Unconditional locking  Process 1 locks resource A  Process 2 locks resource A Does not receive a condition report Process 2 waits in suspense (hangs) until lock is granted

20 Copyright © 2014 M. E. Kabay. All rights reserved. Serializable Transactions  Prevent transactions affecting same records from overlapping  Two-phase locking  Can accumulate locks  But once any lock is released, cannot get more until all are released  Defines growing phase and shrinking phase  More restrictive (and more common) strategy  No locks released until COMMIT or ROLLBACK

21 Copyright © 2014 M. E. Kabay. All rights reserved. Deadlock (Deadly Embrace) A A B B Process 1 locks resource A unconditionally Process 2 locks resource B unconditionally 1 locks B unconditionall y 2 locks A unconditionally

22 Copyright © 2014 M. E. Kabay. All rights reserved. Preventing Deadlocks  Deadlock is example of a race condition  Will not necessarily occur  Occurs by chance when specific events happen at specific time  Always ensure that processes in applications  LOCK RESOURCES IN SAME ORDER  UNLOCK RESOURCES IN REVERSE ORDER  Apply these principles to example on previous slide to see how they absolutely prevent deadlock

23 Copyright © 2014 M. E. Kabay. All rights reserved. Pessimistic Locking Strategy  Assume collisions will occur and prevent conflicts  Lock records  Process transaction  Release locks  But very dangerous for performance if processing involves human interaction  Not controllable  Operator can leave resources locked and hang system Operator could go to lunch! DO NOT LOCK AROUND HUMAN INTERVENTION!

24 Copyright © 2014 M. E. Kabay. All rights reserved. Optimistic Locking Strategy  Assume collisions will be rare and recover if they happen  Read original data records  Process transaction using buffers  Lock original data records  Check to see if original data have changed If no change, commit transaction & unlock If change, unlock & start over

25 Copyright © 2014 M. E. Kabay. All rights reserved. Optimistic Locking (1) Process Value1 Value2 Value1 DB Data buffers Value2 Observe events when there is no change in initial data during processing Same ?

26 Copyright © 2014 M. E. Kabay. All rights reserved. Optimistic Locking (2) Process Value1 Value3 Value2 Value3 Value1 DB Data buffers If data change while process is preparing new buffer, start over. Same ? Someone else changed the data

27 Copyright © 2014 M. E. Kabay. All rights reserved. Optimistic vs Pessimistic Strategies  Optimistic locking advantages  Does not lock resources around human intervention  Appropriate for Web / Internet transactions  Especially important if lock granularity is large (e.g., entire DB or entire tables)  Optimistic locking disadvantages  If specific resource is in high demand (much contention for specific records) then can cause repeated access (thrashing)  Can degrade individual and system performance

28 Copyright © 2014 M. E. Kabay. All rights reserved. Declaring Lock Characteristics  Older programs often used specific calls to locking routines  E.g., “DBLOCK”  Passed parameters to set exact type of lock Conditional or not, granularity etc.  Modern programming using DBMS uses transaction markers  BEGIN, COMMIT, ROLLBACK  Allows global definition of locking strategy  DBMS handles details  Can thus change locking globally without reprogramming

29 Copyright © 2014 M. E. Kabay. All rights reserved. DBMS Controls  Referential Integrity  Uniqueness Constraints  Locking  Transaction Controls  Database Recovery

30 Copyright © 2014 M. E. Kabay. All rights reserved. ACID Transactions  Transactions sometimes described as ideally ACID  Atomic: all changes in the multi-step transaction are committed or none is  Consistent: all records involved in the transaction are changed or none is  Isolated: concurrency does not harm integrity  Durable: not reversible once committed except through normal transaction processing of a new transaction

31 Copyright © 2014 M. E. Kabay. All rights reserved. Consistency  Statement-level consistency  If change is supposed to apply to group of records, then no changes to any of those records will be permitted until all records have been changed  Transaction-level consistency  Same principle applied to multiple steps  Not always easy to achieve  If locking applied around very long processes, will see performance / throughput degradation for other users  May want to limit long updates to batch processing during off-hours

32 Copyright © 2014 M. E. Kabay. All rights reserved. Transaction Isolation Level  Can have difficulties / inconsistencies when concurrent processes access intermediate results during transactions  Dirty read: access a record changed by another process but not yet committed  Nonrepeatable read: some other process has altered the original record (e.g., during optimistic locking)  Phantom read: a new movie by George Lucas – NO NO – means new records inserted or deleted since last read

33 Copyright © 2014 M. E. Kabay. All rights reserved. ANSI SQL Isolation Levels  Can specify degree of protection desired

34 Copyright © 2014 M. E. Kabay. All rights reserved. DBMS Controls  Referential Integrity  Uniqueness Constraints  Locking  Transaction Controls  Database Recovery

35 Copyright © 2014 M. E. Kabay. All rights reserved. Database Recovery  Transactions  Application Logging  Transactions and Log Files  Backups & Log Files  Recovery from Backups  Recovery from Log Files

36 Copyright © 2014 M. E. Kabay. All rights reserved. Transactions  What are transactions?  Why should we care if a transaction were interrupted by a DBMS failure or a system failure? CLASS DISCUSSION

37 Copyright © 2014 M. E. Kabay. All rights reserved. Application Logging  Benefits of logging  Audit trail for security / investigations  Performance data  Debugging  What might a logging process write into the log file when a process is  Adding a record?  Changing a record?  Deleting a record? CLASS DISCUSSION

38 Copyright © 2014 M. E. Kabay. All rights reserved. Transactions and Log Files  Why would it matter to anyone that a log file keep a distinction among different types of transactions?  How does a log file mark completion of an atomic transaction? CLASS DISCUSSION

39 Copyright © 2014 M. E. Kabay. All rights reserved. Backups & Log Files Distinguish among the following types of backups:  System vs application  Full (everything)  Differential (aka “Partial”) (everything changed since last full)  Incremental (everything changed since last incremental) (aka “Partial”)  Delta (only changed data) (aka “Partial”)  Log files (only the information about the changes)

40 Copyright © 2014 M. E. Kabay. All rights reserved. Backup Types Do not use the term “partial backup.”

41 Copyright © 2014 M. E. Kabay. All rights reserved. Recovery from Log Files  Roll-backward recovery  Use log file to identify interrupted (incomplete) transactions using checkpoints Look for start marker without end marker  Remove all changes that are part of those incomplete transactions  Roll-forward recovery  Start with valid backup  Use log file to re-apply all completed transactions  Leave out the incomplete transactions Which kind of recovery is faster?

42 Copyright © 2014 M. E. Kabay. All rights reserved. Protecting Batch Files  Batch processing as defined in Computer Desktop Encyclopedia:  (1) Performing a particular operation automatically on a group of files all at once rather than manually opening, editing and saving one file at a time. For example, graphics software that converts a selection of images from one format to another would be a batch processing utility.  (2) Processing a group of transactions at one time. Transactions are collected and processed against the master files (master files updated) at the end of the day or some other time period. Contrast with transaction processing.

43 Copyright © 2014 M. E. Kabay. All rights reserved. Batch Processing (2) © Computer Desktop Encyclopedia

44 Copyright © 2014 M. E. Kabay. All rights reserved. Batch Processing (3)  Normal batch processing automatically keeps original files as default backups  Process master file + transaction file(s)  Copy unchanged records into new file  Copy modified records from transactions  Don’t copy deleted records  End up with  Original master file  New master file  Transaction (activity) files  Typically keep several generations of masters

45 Copyright © 2014 M. E. Kabay. All rights reserved. Assuring that Information in the System is Valid  Validation Controls: catching data input errors  Check digits in input stream  Hash totals  Digital signatures  Range checks  Table lookups (including combinations)  Diagnostic Utilities: catching data corruption or tampering  Edit checks  Business rules  Exception reports  Statistical Quality Control (SQC) methods (anomaly detection)

46 Copyright © 2014 M. E. Kabay. All rights reserved. Review Questions (1) 1.Distinguish between SQA and application controls. 2.Why should we pay attention to applications when planning our security procedures? 3.Why are databases of such concern in application security discussions? 4.Name and distinguish between the two fundamental types of data corruption (by cause). 5.Explain the concept of referential integrity using examples. 6.How do DBMSs enforce uniqueness constraints?

47 Copyright © 2014 M. E. Kabay. All rights reserved. Review Questions (2) 7.Why do concurrently-accessed databases require locking strategies? 8.What’s a transaction? 9.What is meant by atomic transactions? 10.How is a transaction marked in a log file? 11.Which has finer granularity, locking an entire dataset or locking a set of records? 12.Distinguish between exclusive and shared locks. 13.Distinguish between conditional and unconditional locks. 14.What’s a deadlock and how can you prevent it?

48 Copyright © 2014 M. E. Kabay. All rights reserved. Review Questions (3) 15.Distinguish between pessimistic and optimistic locking strategies. 16.What does ACID mean in discussions of transactions? Explain each of the components. 17.Why do production applications normally include log files as part of their design? 18.Explain how roll-backward recovery works. 19.Explain how roll-forward recovery works. 20.Discuss the security features of batch processing. 21.Explain how applying each of the validation controls described in slide 45 could help check the validity of stored information in a database.

49 Copyright © 2014 M. E. Kabay. All rights reserved. DISCUSSION