2. Recovery in Main Memory Databases -Le Gruenwald, Jing Huang, Margaret H. Dunham el al - Engineering Intelligent Systems, Vol.4, No. 3, September 1996 이 인선 97/08/21

3. Introduction s General MMDB Architecture –Main Memory (MM) in RAM memory –Stable Memory(SM) u optional nonvolatile memory u used to hold log buffers(log tail) u avoid I/O actions when transaction are committed u essential to performance –Archive Memory(AM) holds a backup of the entire database s focus on logging, checkpointing, reloading

4. MMDB Logging(1) –physical logging u the state of the database modified by an operation are logged u it is recommended for MMDB systems –logical logging u contains descriptions of higher level operations and records the state transition of the database u the idempotent property does not hold

5. MMDB Logging(2) s Logging rules –Write Ahead Rule u undo-log data must be written to a nonvolatile memory prior to the updating in the database –Commit rule u if a DBMS allows a transaction to commit, the redo-log data of it should be ensured in nonvolatile storage –Logging After Writing u the after image of an updated item should be written to the log after its corresponding update is propagated to the database u simplifies the log processing with a fuzzy checkpointing MMDB

6. MMDB Logging(3) s MMDB logging differs from DRDB logging in three ways –a nonvolatile log buffer should be used to satisfy WAL without requiring I/O prior to transaction commit –physical logging is recommended as it is easier to use with fuzzy checkpointing –to reduce the amount of the log needed to redo transactions after a system failure, the LAW policy should be followed

7. Checkpointing DRDB s Commit consistent checkpointing –periodically stop processing transactions –flush all dirty cache slots and mark the log s cache consistent checkpointing s fuzzy checkpointing –only flushes those dirty slots that have not been flushed since before the previous checkpoint –normal replacement activity will flush most cache slots that were dirty since before the previous checkpoint –checkpoint won’t have much flushing to do and won’t delay active transaction for very long.

8. Checkpointing MMDBs(1) s Focuses on low-interference with normal transactions and supporting efficient recovery s Fuzzy checkpointing –Hagmann u first suggested using fuzzy checkpointing for MMDBs u “a crash recovery scheme for a memory-resident database system” u IEEE transactions on computers. Vol. C-35, No. 9, september 1986 u the checkpointer does not need to obtain the locks on the data items to be checkpointed u the database is dumped in sections u after dumping a section, the checkpointer writes a log record to the log u a section must not overwrite its previous image (sliding monoplexed backups)

9. LAW with fuzzy checkpointing

10. Checkpointing MMDBs(2) –Salem and Garcia-Molina u “checkpointing memory-resident databases”(‘89) u compared the fuzzy checkpointing scheme with two-non- fuzzy checkpointing schemes u fuzzy checkpointing is the most efficient one u ping-pong scheme –each dirty page is flushed twice –Lin and Dunham u “segmented fuzzy checkpointing for main memory databases”(‘94) u checkpoints one segment at a time in a round-robin fashion u automatically changes the segment boundaries based on the distribution of update operations

11. Checkpointing MMDBs(3) Redo log size in the Segmented fuzzy checkpointing –Li et al u “checkpointing and recovery in partitioned main memory databases(‘95) u the database is divided into partitions, each of which has its own log disks u the time to recover from a system failure is reduced BC1BC1 a1b1c1 BC2BC2 a2b2c2 1 2 34

12. Checkpointing MMDBs(4) s Non-Fuzzy Checkpointing –overhead comes from locking the checkpointed objects to ensure transaction-consistency or action-consistency –Lehman and Carey u “a recovery algorithm for a high-performance memory- resident database system”(‘87) u transaction-consistent(at relation level)scheme u no need to maintain undo-log-records in nonvolatile storage u checkpointing increases the data contention with normal transaction

13. Checkpointing MMDBs(5) –Salem and Garcia-Molina u “checkpointing memory-resient databases” (‘89) u discuss two non-fuzzy checkpointing approaches –the first(black and white) one aborts some update transactions –the second(Copy-On-Update) one requires some update transactions storing the original values of data items to be updated –both have severe impact on the system performance –Jagadish et al u “recovering from main-memory lapses” (‘93) u propose an action-consistent checkpointing scheme u the undo-logs of active transactions are first written to the log, and then dirty pages are flushed to disk u during normal processing, the redo-logs of the committed transactions are written to the log u ping-pong update u this approach was originally used in Dali

14. Checkpointing MMDBs(6) s Log-driven checkpointing –applies the log to a previous dump to generate a new dump –originally used to generate remote backup of the database –is adopted to “incremental recovery in main memory database systems” (‘92) –with high transaction processing rate in MMDBs, the size of the log can increase rapidly –it is quite inefficient compared to fuzzy checkpointing

15. MMDB Reloading(1) s Issues –occurrence frequency of the reload process u on average, a system failure occurs once every few weeks u media failure, MM page faults –when the system should resume its execution after a failure u 28.43 minutes are needed to recover 1Giga DB [?] u if the system is not available at all during recovery, many transactions will be backlogged –reload prioritization u reload priority can be determined based on access frequency, transaction deadline(“MMDB reload algorithms”) or temporal data interval from real-time applications[?]

16. MMDB Reloading(2) s Existing reload schemes –simple reloading u the system can not be brought online until the entire database is memory-resident –concurrent reloading u Grenwald –“mmdb reload algorithms” (‘91) –two processors(RP & DP), nonvolatile shadow memory(SM) and dual address translation mechanism in the MARS system –ordered reload with prioritization/ smart reload/ frequency reload –the differences lie in the structure of AM, utilization of data access frequency, reload prioritization, and reload granularity –the frequency reload yields the best transaction response time and system throughput

17. MMDB Reloading(3) u Lehman –“a recovery algorithm for a high-preformance” –after the system catalogs and their indices are reloaded then regular transaction processing is allowed to resume u Levy and Silberschatz –“incremental recovery in main memory database systems”, (‘92) –resume transaction processing immediately after a system failure and recovers pages individually according to the demand of post-crash transaction. –Stale/fresh marking technique –in order to implement a page-based recovery, log records must be grouped together on a page basis during normal operation

18. Recovery with Existing MMDB Systems(1) s Dali from AT&T –the original recovery manager was implemented according to “recovering from main-memory lapses” (‘93) u logging only redo records during normal execution u segment-level action-consistent checkpoints u checkpointer write to the disk relevant parts of the undo log u recovery has only a single pass over the log u require no special h/w to preserve the data –test led to a restructuring of its recovery manager u “multi-level recovery in the Dali storage manager” (‘95) u multi-level logging, post-commit actions, dirty page detection, and fuzzy checkpoints

19. Recovery with Existing MMDB Systems(2) s Fast Path –supports the memory-resident data and disk- resident data –performs updates to memory resident data at commit time –no undo operations are required when a failure occurs –a group commit is adopted –transaction-consistent backup copy of the database is refreshed during system shutdown or infrequently checkpoints. –Two backup database with ping-pong backups

20. Recovery with Existing MMDB Systems(3) s two real-time system examples u NEC Real-Time DBMS u Stone RTDB –NEC RTDBMS has several features to ensure high throughput and accurate predictability u no page fault u in-memory log buffer is nonvolatile u physical logging using deferred update u fuzzy checkpointing u no real-time characteristics such as transaction deadline and criticalness are utilized in the recovery components

21. Summary and Conclusion –Discussed 3 logging rules u nonvolatile log buffer should be used to satisfy WAL without requiring I/O prior to transaction commit u LAW should be followed to reduce the amount of log needed to redo transactions after a system failure –described three groups of checkpointing –identified 3 issues about reloading u data should be prioritized for reload purposes –future research u investigate how real-time requirements such as transaction deadline and temporal data intervals can be incorporated into MMDB recovery

22. a crash recovery scheme for a memory-resident database system Robert B. Hagmann IEEE transactions on computers. Vol. C-35, No. 9, september 1986

23. overview s Presents a method of doing recovery that uses the existing techniques of fuzzy dumps and log compression s design requirement –small system example u 2 pages/transaction *100 transactions/s * 3600s /h * 8h = 5,760,000 pages written to the log –transaction size must be short –checkpointed periodically every five minutes

24. Overview(2) –The principal requirement of the system is “fast” recovery from a system crash u critical factor : transfer rate of the disk u can be improved by using several parallel processors s design overview –fuzzy dump u simply a copy of the database taken without any synchronization –If a DBMS uses a nonvolatile storage, some log compression can occur –else precommitting and group commits can be used to increase performance

25. overview s Design details