Dr. Kurt Stockinger Data Warehouse & Business Intelligence Architect Credit Suisse, Zurich May 28, 2013 The Parallel Universe Effect between Data-Intensive.

Slides:



Advertisements
Similar presentations
Supervisor : Prof . Abbdolahzadeh
Advertisements

Big Data: Analytics Platforms Donald Kossmann Systems Group, ETH Zurich 1.
+ Hbase: Hadoop Database B. Ramamurthy. + Introduction Persistence is realized (implemented) in traditional applications using Relational Database Management.
Senior Project Manager & Architect Love Your Data.
Big Data Management and Analytics Introduction Spring 2015 Dr. Latifur Khan 1.
Paula Ta-Shma, IBM Haifa Research 1 “Advanced Topics on Storage Systems” - Spring 2013, Tel-Aviv University Big Data and.
Running Hadoop-as-a-Service in the Cloud
Evaluation of distributed open source solutions in CERN database use cases HEPiX, spring 2015 Kacper Surdy IT-DB-DBF M. Grzybek, D. L. Garcia, Z. Baranowski,
Data Warehousing - 3 ISYS 650. Snowflake Schema one or more dimension tables do not join directly to the fact table but must join through other dimension.
Fraud Detection in Banking using Big Data By Madhu Malapaka For ISACA, Hyderabad Chapter Date: 14 th Dec 2014 Wilshire Software.
Hadoop tutorials. Todays agenda Hadoop Introduction and Architecture Hadoop Distributed File System MapReduce Spark 2.
Panel Summary Andrew Hanushevsky Stanford Linear Accelerator Center Stanford University XLDB 23-October-07.
Big Data – Are You Ready? December 15 th, 2011 ICT Forum December 15 th, 2011 Frank Zervos Technology Director SEE Oracle Hellas.
+ Hbase: Hadoop Database B. Ramamurthy. + Motivation-1 HDFS itself is “big” Why do we need “hbase” that is bigger and more complex? Word count, web logs.
TITLE SLIDE: HEADLINE Presenter name Title, Red Hat Date For Red Hat, it's 1994 all over again Sarangan Rangachari VP and GM, Storage and Big Data Red.
Business Intelligence: The Next Big Thing (Really!) John Bair CTO, Ajilitee Sep 14, 2012 Presented to TDWI St. Louis Chapter.
CERN IT Department CH-1211 Geneva 23 Switzerland t XLDB 2010 (Extremely Large Databases) conference summary Dawid Wójcik.
This presentation was scheduled to be delivered by Brian Mitchell, Lead Architect, Microsoft Big Data COE Follow him Contact him.
S EEQ C ORPORATION Big Data Oregon Connections Telecommunications Conference Dustin Johnson October 23, 2014.
BIG DATA OFF-SHORE SERVICES:. Off-Shore “Big Data” Center: Modern Facilities in Bangalore’s Central Business District 60,000 Sqft. Space  Capacity for.
MapReduce April 2012 Extract from various presentations: Sudarshan, Chungnam, Teradata Aster, …
Data Warehouse Overview September 28, 2012 presented by Terry Bilskie.
Introduction to Hadoop and HDFS
Contents HADOOP INTRODUCTION AND CONCEPTUAL OVERVIEW TERMINOLOGY QUICK TOUR OF CLOUDERA MANAGER.
An Introduction to HDInsight June 27 th,
+ Hbase: Hadoop Database B. Ramamurthy. + Motivation-0 Think about the goal of a typical application today and the data characteristics Application trend:
Big Data Analytics Large-Scale Data Management Big Data Analytics Data Science and Analytics How to manage very large amounts of data and extract value.
IT Architectures for Handling Big Data in Official Statistics: the Case of Scanner Data in Istat Gianluca D’Amato, Annunziata Fiore, Domenico Infante,
CS525: Big Data Analytics MapReduce Computing Paradigm & Apache Hadoop Open Source Fall 2013 Elke A. Rundensteiner 1.
Business Intelligence Transparencies 1. ©Pearson Education 2009 Objectives What business intelligence (BI) represents. The technologies associated with.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Big Data Directions Greg.
Nov 2006 Google released the paper on BigTable.
CISC 849 : Applications in Fintech Namami Shukla Dept of Computer & Information Sciences University of Delaware iCARE : A Framework for Big Data Based.
Big Data Tools Hadoop S.S.Mulay Sr. V.P. Engineering February 1, 2013.
Harnessing Big Data with Hadoop Dipti Sangani; Madhu Reddy DBI210.
Andy Roberts Data Architect
BIG DATA. Big Data: A definition Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database.
What is it and why it matters? Hadoop. What Is Hadoop? Hadoop is an open-source software framework for storing data and running applications on clusters.
CPSC8985 FA 2015 Team C3 DATA MIGRATION FROM RDBMS TO HADOOP By Naga Sruthi Tiyyagura Monika RallabandiRadhakrishna Nalluri.
Data Analytics and Hadoop Service in IT-DB Visit of Cloudera - April 19 th, 2016 Luca Canali (CERN) for IT-DB.
Big Data Analytics Hadoop is here to Stay!. What is Big Data? Large databases which are hard to dealComplex and Unstructured dataNeed for Parallel ProcessingHigh.
Microsoft Ignite /28/2017 6:07 PM
A Suite of Products that allow you to Predict Outcomes, Prescribe Actions and Automate Decisions.
Large Scale Semantic Data Integration and Analytics through Cloud: A Case Study in Bioinformatics Tat Thang Parallel and Distributed Computing Centre,
BI 202 Data in the Cloud Creating SharePoint 2013 BI Solutions using Azure 6/20/2014 SharePoint Fest NYC.
A Better Way Check Image Data Management Solution Success Story.
Hadoop Introduction. Audience Introduction of students – Name – Years of experience – Background – Do you know Java? – Do you know linux? – Any exposure.
Big Data-An Analysis. Big Data: A definition Big data is a collection of data sets so large and complex that it becomes difficult.
How can SQL on Hadoop assist with Big Data Evaluation?
Data Analytics (CS40003) Introduction to Data Lecture #1
Supervisor : Prof . Abbdolahzadeh
Big Data & Test Automation
Big Data Analytics on Large Scale Shared Storage System
The value of data science standards in manufacturing analytics Soundar Srinivasan Bosch data Mining solutions and services.
Big Data Enterprise Patterns
5/9/2018 7:28 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS.
Big Data A Quick Review on Analytical Tools
Big Data –An Overview
Hadoopla: Microsoft and the Hadoop Ecosystem
Operational & Analytical Database
Modern Data Management
Azure's Performance, Scalability, SQL Servers Automate Real Time Data Transfer at Low Cost MINI-CASE STUDY “Azure offers high performance, scalable, and.
Data Warehouse Overview September 28, 2012 presented by Terry Bilskie
Azure Data Lake for First Time Swimmers
Mary Ledbetter, Systems Sales Engineer
Dep. of Information Technology By: Raz Dara Mohammad Amin
SQL Server 2019 Bringing Apache Spark to SQL Server
Architecture of modern data warehouse
Big Data.
Presentation transcript:

Dr. Kurt Stockinger Data Warehouse & Business Intelligence Architect Credit Suisse, Zurich May 28, 2013 The Parallel Universe Effect between Data-Intensive Physics and Banking

Kurt Stockinger Date: May 28, 2013, Slide 2 Typical Data Warehouse Reference Architecture Customers Products (CH) Products (US) Transactions (CH)

Kurt Stockinger Date: May 28, 2013, Slide 3 Main Characteristics of Major Industries in DWH Source: M. Ehrenmann, R. Pieringer, K. Stockinger. Is there a Cure-All for Business Analytics? Business Intelligence Journal, Vol. 17, No. 3, Sept. 2012

Kurt Stockinger Date: May 28, 2013, Slide 4 Commonalities and Differences Between Banking and Physics in Data Management BankingPhysics Data volumeLarge (PB range)Very large (multi PB) Data sourcesLarge (thousands)Small (<10) Data typesNumerical + textNumerical + images Data StorageRelational databases (DB2, Oracle, Teradata) + Cubes (Microsoft) Open-source (Root, HDF5, Flat files + FastBit) Data IntegrationETL (PowerCenter,…)Typically not the main challenge Data AnalysisBusiness intelligence tools (BusinessObjects, Microstrategy, SAS,…) + data mining tools (Clementine) Root, parallel R, … Data VisualizationSee aboveRoot, VisIt Main challengeData modeling & integrationManaging the volume

Kurt Stockinger Date: May 28, 2013, Slide 5  Physics: – Most of the software has been home-grown due to lack of scaling of commercial alternatives (bleeding edge + cost effective)  Banking: – Most of the software has been bought (early followers, stable solution)  Big data technology (scalable + cost effective): – Hadoop (HDFS), Pig, Hive, Mahout (Machine Learning)  Both physics and banking can benefit from new technology  However, both areas seem to be skeptical and adoption rate seems modest – Physics:  CHEP 2012 showed “promising” results with Hadoop (experiment log analysis, HDFS as new storage) – Banking:  Early success stories, e.g. DWH query log analysis:  Still struggling with integration of DWH+BI solutions with Big Data technology The Grand Unification? Big Data Technology