Topics  In-memory Spatial Database based on key-value pair  Building Information Model based on relationship inference.

Slides:



Advertisements
Similar presentations
Limitations of the relational model 1. 2 Overview application areas for which the relational model is inadequate - reasons drawbacks of relational DBMSs.
Advertisements

Chapter 10: Designing Databases
Phil Bernstein Microsoft Research May 29, 2008 © 2008 Microsoft Corporation.
IS 4420 Database Fundamentals Chapter 6: Physical Database Design and Performance Leon Chen.
Geographic Information Systems
Data Management I DBMS Relational Systems. Overview u Introduction u DBMS –components –types u Relational Model –characteristics –implementation u Physical.
File Systems and Databases Hachim Haddouti
Physical design. Stage 6 - Physical Design Retrieve the target physical environment Create physical data design Create function component implementation.
Attribute databases. GIS Definition Diagram Output Query Results.
Chapter 1 An Overview of Database Management. 1-2 Topics in this Chapter What is a Database System? What is a Database? Why Database? Data Independence.
Object-Oriented Methods: Database Technology An introduction.
Module 14: Scalability and High Availability. Overview Key high availability features available in Oracle and SQL Server Key scalability features available.
BIS310: Week 7 BIS310: Structured Analysis and Design Data Modeling and Database Design.
10. Creating and Maintaining Geographic Databases.
How WebMD Maintains Operational Flexibility with NoSQL Rajeev Borborah, Sr. Director, Engineering Matt Wilson – Director, Production Engineering – Consumer.
Chapter 3 Data Models.
Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe Slide 1- 1 Chapter 1 - Introduction: Databases and Database Users - Outline Types of Databases and.
Chapter 5: Modeling Systems Requirements: Events and Things
Word Wide Cache Distributed Caching for the Distributed Enterprise.
Database System Concepts and Architecture Lecture # 3 22 June 2012 National University of Computer and Emerging Sciences.
24. April 1998 Dutch Cadastre 1 Efficient Storage And Retrieval for Large Spatial Data Set in a Relational DBMS Andrew U. Frank Dept. of Geoinformat ion.
CS370 Spring 2007 CS 370 Database Systems Lecture 2 Overview of Database Systems.
Ihr Logo Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Turban, Aronson, and Liang.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Database Performance Tuning and Query Optimization.
1 © Prentice Hall, 2002 Physical Database Design Dr. Bijoy Bordoloi.
Database and Database Users. Outline Database Introduction An Example Characteristics of the Database Actors on the Scene Advantages of using the DBMS.
1 Overview of Databases. 2 Content Databases Example: Access Structure Query language (SQL)
1 Introduction to Database Systems. 2 Database and Database System / A database is a shared collection of logically related data designed to meet the.
Software School of Hunan University Database Systems Design Part III Section 5 Design Methodology.
9/14/2012ISC329 Isabelle Bichindaritz1 Database System Life Cycle.
TM 7-1 Copyright © 1999 Addison Wesley Longman, Inc. Physical Database Design.
Physical Database Design Chapter 6. Physical Design and implementation 1.Translate global logical data model for target DBMS  1.1Design base relations.
6 Chapter Databases and Information Management. File Organization Terms and Concepts Bit: Smallest unit of data; binary digit (0,1) Byte: Group of bits.
Copyright 2002 Prentice-Hall, Inc. Modern Systems Analysis and Design Third Edition Jeffrey A. Hoffer Joey F. George Joseph S. Valacich Chapter 20 Object-Oriented.
© 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang 5-1 Chapter 5 Business Intelligence: Data.
Databases and Statistical Databases Session 4 Mark Viney Australian Bureau of Statistics 5 June 2007.
Chapter 6 1 © Prentice Hall, 2002 The Physical Design Stage of SDLC (figures 2.4, 2.5 revisited) Project Identification and Selection Project Initiation.
Massively Distributed Database Systems - Distributed DBS Spring 2014 Ki-Joune Li Pusan National University.
Information System Development Courses Figure: ISD Course Structure.
Methodology - Conceptual Database Design. 2 Design Methodology u Structured approach that uses procedures, techniques, tools, and documentation aids to.
1/26/2004TCSS545A Isabelle Bichindaritz1 Database Management Systems Design Methodology.
Oleh Munawar Asikin. Principles of Information Systems, Seventh Edition 2  Database management system (DBMS): group of programs that manipulate database.
1 CS 430 Database Theory Winter 2005 Lecture 16: Inside a DBMS.
10/10/2012ISC239 Isabelle Bichindaritz1 Physical Database Design.
Methodology - Conceptual Database Design
Database Management COP4540, SCS, FIU Physical Database Design (ch. 16 & ch. 3)
Chapter 9 Database Systems Introduction to CS 1 st Semester, 2014 Sanghyun Park.
Introduction to Database AIT632 Chapter 1 Sungchul Hong.
Data resource management
Part4 Methodology of Database Design Chapter 07- Overview of Conceptual Database Design Lu Wei College of Software and Microelectronics Northwestern Polytechnical.
Creating and Maintaining Geographic Databases. Outline Definitions Characteristics of DBMS Types of database Relational model SQL Spatial databases.
Esri UC 2014 | Technical Workshop | Editing Versioned Geodatabases : An Introduction Cheryl Cleghorn and Shawn Thorne.
INTRODUCTION TO DBS Database: a collection of data describing the activities of one or more related organizations DBMS: software designed to assist in.
INTRODUCTION TO ORACLE DATABASE ADMINISTRATION Lynnwood Brown President System Managers LLC Introduction – Lecture 1 Copyright System Managers LLC 2003.
Physical Database Design Purpose- translate the logical description of data into the technical specifications for storing and retrieving data Goal - create.
Introduction.  Administration  Simple DBMS  CMPT 454 Topics John Edgar2.
Assoc. Prof. Dr. Ahmet Turan ÖZCERİT.  The concept of Data, Information and Knowledge  The fundamental terms:  Database and database system  Database.
Towards Unifying Vector and Raster Data Models for Hybrid Spatial Regions Philip Dougherty.
 Distributed Database Concepts  Parallel Vs Distributed Technology  Advantages  Additional Functions  Distribution Database Design  Data Fragmentation.
1 Chapter 2 Database Environment Pearson Education © 2009.
Distributed Cache Technology in Cloud Computing and its Application in the GIS Software Wang Qi Zhu Yitong Peng Cheng
ITD1312 Database Principles Chapter 5: Physical Database Design
Chapter 2 Database Environment Pearson Education © 2009.
Data Base System Lecture : Database Environment
Database Systems Instructor Name: Lecture-3.
Metadata Framework as the basis for Metadata-driven Architecture
The Physical Design Stage of SDLC (figures 2.4, 2.5 revisited)
Chapter 2 Database Environment Pearson Education © 2009.
Chapter 2 Database Environment Pearson Education © 2009.
Presentation transcript:

Topics  In-memory Spatial Database based on key-value pair  Building Information Model based on relationship inference

In-memory Spatial Database based on key-value pair  Background  Architectural Overview  Features

Background This research is supported to solve kernel issues during the State Grid information construction. The achievements have been applied to the power information systems.

Deploy view State Grid Shanghai Grid … User1User2… Hubei Province Grid Beijing Grid User1User2… User1User2… User1User2…

NetworkStorage Software… Physical ResourceGraphics … Vector Database ManageAnalysis … Spatial Power Grid Services ERPMarking GIS… Systems PlanningSchedule Sale… Businesses Security Data Network System … Application Integrate Product Monitor Emergency … Technical Architecture

Background Store Data Organize Data Access Data Others Large scale data Key-value Transaction Locks Lazy write to DBMS High concurrency High throughout Batch to submit data long period, offline data edit Increment data query Can’t be accessed directly by user hybrid index based on spatial and topology Change notify Reuse existing power database, such as Oracle Spatial, SDE, DB2 Spatial Extender Data synchronize … Key problems Partition based on spatial and topology logical data model Version History Database

Background Why not traditional spatial database?  Why not DBMS, such as Oracle?  Why not SDE or ArcServer?  Why not object-oriented, such as Oracle spatial?  Why not traditional spatial index, such as Quad-tree, R*- tree? Bad I/O throughout Bad performance when high concurrency and batch edit Can’t provide necessary features Bad performance There are some bugs in Linux Can’t provide necessary features More slow than raw data type, such as BLOB Spatial index has bugs and has to be rebuilt at fixed period Spatial index performance is bad Precise filter is even slow, such as sdo_relate Spatial function is not necessary No. We need a hybrid index, including spatial, topology and attributes information

Background Why in-memory spatial database on key-pair  Why store data in memory? How is the data persistence?  Why use key-pair? What is your spatial model?  Do you use spatial index?  Do you support condition query like SQL? in-memory database is much fast than disk database The changed data will be written into DBMS Simple means high performance! In-memory database doesn’t need complex search We use the hybrid index, including spatial, topology and attribute information We support simple compare function,such as greater, equal, less…

Architectural-Logical View

Architectural-Framework

Features-Data Model Indeed, e very record is a memory block!  Typical data struct  Index for attributes B-tree index, Hash index  Spatial index Spatial index is a hybrid index: B-tree + grid index + topology based index protocol head description field value 1 field value 2 … field value n

Features-Cluster In my research, cluster is a group of virtual, loosely coupled in- memory database that work together closely.  In a cluster, all in-memory databases have the same data.  For different group, maybe they have intersected data.  How to partition the power data for cluster is based on the spatial, topology and attribute information  When some data are updated in one in-memory database, other in-memory database which has these data will be synchronized almost real-time.

Features-Change notify When data is update, clients will receive the notify. TThe notification granularity is based on object, not record, which help to TThe notify is based on subscribe & notify model. UUser can only download changed records.

Features-Version & History DB Provide version mechanism and history database based on time axis.  Sometimes, work maybe needs several days, even a month to finish. These data will be managed as a version and other client can’t see them before finishing the work.  At any time, in-memory can create database snapshot for archive or future comparative analysis.

BIM based on relationship inference  Background  Roadmap

Background  In common, the life cycle of a building includes 3 phases:  IFC is a vendor-neutral BIM data repository for the semantic information of building objects, including geometry, associated properties, and relationships, to facilities.

Background Impression  IFC is a complex, relatively complete and huge specification.  From the OO point of view, IFC is a well designed and loose coupled class set.

Background Problems and Limitations for IFC  For OO design, message or event driven is an import design pattern, but it seems there is no definition about it.  IFC doesn’t define something about analysis, which is important for management phase during the life cycle of a building.  The ability of relationship description is much weak and can not describe the implicit relationship automatically, which is import for analysis and modeling. GIST belong to OTB Liu belong to GIST Liu belong to OTB

Roadmap Goals  Provide the ability of relationship inference.  Improve the degree of automation during the process of building information modeling.  Provide a mechanism for building information analysis

Roadmap-relationship inference Classification based on the number of relation unary Binary triple Classification based on type subsumption(subClass- of, superClass-of, …) mereology(is-part-of,) topology(next-to, touch- with, …) …. transitive symmetric inverse equality hasValue exists at occur at maximum minimum

Roadmap-analysis based on event driven Initial thoughts  Information transmission is modeled a virtual pipeline.  Every object will get information from pipeline and analyze them. The result is also pushed into the pipeline.  The purpose: 1) object is self-contain; 2) object is independent of other object; 3) combine different analysis

Thank you