1 One-stop shopping for time series data David Williams (London), Thomas Araujo (Houston) & Stuart Reed (London)

Slides:



Advertisements
Similar presentations
DIGIDOC A web based tool to Manage Documents. System Overview DigiDoc is a web-based customizable, integrated solution for Business Process Management.
Advertisements

Copyright Hub Software Engineering Ltd 2010All rights reserved Hub Document Exchange Product Overview Secure Transmission for Transaction-based Documents.
Test Case Management and Results Tracking System October 2008 D E L I V E R I N G Q U A L I T Y (Short Version)
Vice President of Facilities
Microsoft Excel 2003 Illustrated Complete Excel Files and Incorporating Web Information Sharing.
SOFTWARE PRESENTATION ODMS (OPEN SOURCE DOCUMENT MANAGEMENT SYSTEM)
Connect. Communicate. Collaborate Click to edit Master title style MODULE 1: perfSONAR TECHNICAL OVERVIEW.
Environmental Council of States Network Authentication and Authorization Services The Shared Security Component February 28, 2005.
Active Directory: Final Solution to Enterprise System Integration
Access 2007 Product Review. With its improved interface and interactive design capabilities that do not require deep database knowledge, Microsoft Office.
Security Management IACT 918 July 2004 Gene Awyzio SITACS University of Wollongong.
SESSION 9 THE INTERNET AND THE NEW INFORMATION NEW INFORMATIONTECHNOLOGYINFRASTRUCTURE.
Security Management IACT 418/918 Autumn 2005 Gene Awyzio SITACS University of Wollongong.
ASP.NET 2.0 Chapter 6 Securing the ASP.NET Application.
Bar|Scan ® Asset Inventory System The leader in asset and inventory management.
Definitions Collaboration – working together on team projects and sharing information, often through ad-hoc processes, to accomplish project goals. Document.
Federated Searching Pre-Conference Workshop - The federated searching cookbook Qin Zhu HP Labs Research Library February 18, 2007.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Agenda Overview 2.What is SharePoint? 3.NCDOT Websites 4.Roles 5.Search 6.SharePoint Interface.
November 10, 2010 Washington, D.C.. Agenda Secure Component tool Secure Component tool Contacts and Messaging Contacts and Messaging Workspaces Workspaces.
Trimble Connected Community
©Kwan Sai Kit, All Rights Reserved Windows Small Business Server 2003 Features.
OCLC Online Computer Library Center CONTENTdm ® Digital Collection Management Software Ron Gardner, OCLC Digital Services Consultant ICOLC Meeting April.
1 Quality Assurance In moving information from statistical programs into the hands of users we have to guard against the introduction of error. Quality.
Business Solutions Using Microsoft ® Office SharePoint ® Server ROADSHOW.
Security Architecture
A Guide to Getting Started. 2 Free social learning network for teachers, students, schools and districts Safe and easy way to connect Exchange ideas Share.
Section 1: Introducing Group Policy What Is Group Policy? Group Policy Scenarios New Group Policy Features Introduced with Windows Server 2008 and Windows.
Module 7: Fundamentals of Administering Windows Server 2008.
| e n a b l i n g | i n t e r a c t i v e | a d a p t i v e | O V E R V I E W Providing secure access to real-time data via the Internet Focused on delivering.
Best of Both Worlds: Information Management Solutions SmartCore Management Dashboards.
Electronic Records Management: A Checklist for Success Jesse Wilkins April 15, 2009.
© 2008 IBM Corporation ® IBM Cognos Business Viewpoint Miguel Garcia - Solutions Architect.
Combining survey and administrative data to create a new input data file for National Accounts processes Shaun McLaughlin Central Statistics Office, Ireland.
1 Introduction to Microsoft Windows 2000 Windows 2000 Overview Windows 2000 Architecture Overview Windows 2000 Directory Services Overview Logging On to.
Introduction to Databases Trisha Cummings. What is a database? A database is a tool for collecting and organizing information. Databases can store information.
1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015.
Copenhagen, 7 June 2006 Toolkit update and maintenance Anton Cupcea Finsiel Romania.
Digital Commons & Open Access Repositories Johanna Bristow, Strategic Marketing Manager APBSLG Libraries: September 2006.
Lesson Overview 3.1 Components of the DBMS 3.1 Components of the DBMS 3.2 Components of The Database Application 3.2 Components of The Database Application.
Database Security and Auditing: Protecting Data Integrity and Accessibility Chapter 1 Security Architecture.
MSF Design Example Conceptual Design Logical Design Physical Design.
3-Tier Client/Server Internet Example. TIER 1 - User interface and navigation Labeled Tier 1 in the following graphic, this layer comprises the entire.
Database Environment Chapter 2. Data Independence Sometimes the way data are physically organized depends on the requirements of the application. Result:
ECI – electronic Commerce Infrastructure “ An application to the Shares Market ” Demetris Zeinalipour ( Melinos Kyriacou
Database Concepts Track 3: Managing Information using Database.
INTRUSION DETECTION SYSYTEM. CONTENT Basically this presentation contains, What is TripWire? How does TripWire work? Where is TripWire used? Tripwire.
Database Management Systems (DBMS)
A Guide to Getting Started. 2 Free social learning network for teachers, students, schools and districts Safe and easy way to connect Exchange ideas Share.
A Guide to Getting Started BCPS – Office of Instructional Technology.
Database Security and Auditing: Protecting Data Integrity and Accessibility Chapter 1 Security Architecture.
The world leader in serving science Overview of Thermo 21 CFR Part 11 tools Overview of software used by multiple business units within the Spectroscopy.
The Online World ONLINE DOCUMENTS. Online documents Online documents (such as text documents, spreadsheets, presentations, graphics and forms) are any.
Copyright (c) 2014 Pearson Education, Inc. Introduction to DBMS.
MCSE Guide to Microsoft Exchange Server 2003 Administration Chapter One Introduction to Exchange Server 2003.
The overview How the open market works. Players and Bodies  The main players are –The component supplier  Document  Binary –The authorized supplier.
Directory Services CS5493/7493. Directory Services Directory services represent a technological breakthrough by integrating into a single management tool:
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
EBay Searcher Brian Payton, Jason Nowakoski, Justin Szeluga, Salvatore Siragusa, David Wolkiser.
Renee R. Henderson Instructional Technology Specialist Technology Training Team, PGCPS
E-Business Infrastructure PRESENTED BY IKA NOVITA DEWI, MCS.
SharePoint 101 – An Overview of SharePoint 2010, 2013 and Office 365
Set up your own Cloud The search for a secure and acceptable means of gaining access to your files stored at the office from a remote location.
Amazon Storage- S3 and Glacier
Lecture 1: Multi-tier Architecture Overview
Database Environment Transparencies
Information Technology Ms. Abeer Helwa
Reportin Integrates with Microsoft Office 365 to Provide an End-to-End Platform for Financial Teams That Simplifies Report Creation and Management OFFICE.
Metadata The metadata contains
Presentation transcript:

1 One-stop shopping for time series data David Williams (London), Thomas Araujo (Houston) & Stuart Reed (London)

2 Current Status Enron’s market data is spread across different locations, databases and technology platforms. Data is held behind different security layers, and is described inconsistently. Currently there is no mechanism to locate and download time series data from a single user interface. Users need to know multiple search techniques, and have access to multiple network drives/databases. Enron pays for the same data multiple times, because there is no central index of market data. There is little or no monitoring or auditing of data usage. There is little connectivity between Enron’s systems. To provide additional data feeds to current systems is IT-intensive and slow.

3 Purpose of MKM To provide a simple and flexible global one-stop shop for time series data where the location of the data is transparent to the end user; To reduce the risk of using ambiguous data to make commercial decisions through improved data descriptions and a central register of time series metadata. To provide a standard library of data manipulation and validation functions, thus reducing time spent formatting data in spreadsheets and time lost through using flawed data; To provide a backbone which allows “public” data to be freely shared, and “private” data to be kept secure. To improve Enron’s bottom line by only paying external publishers for the data once. To allow users greater access to data without taking away their existing systems. To reduce the time required by developers to source and connect to market data.

4 History of MKM MKM started as a project called RateServer which was an initiative to move market data into one database. It became apparent that this would be unworkable, as all databases store their data in vastly different formats. Furthermore, to migrate the data to a single source would be high risk and exposed to error. Instead, Enron could take advantage of a one-stop shop for data without having to physically relocate data. This can be done using a messaging infrastructure comprising TIBCO software and XML documents. This initiative became known as MKM.

5 Why use a pilot? Whilst collecting user requirements in November/December 1999 it became apparent that without a very simple demonstration of the concept, it was difficult for users to state their exact business requirements. By developing a pilot, we can not only prove the underlying technologies such as the messaging backbone, but we can return to the users with a “straw man”, and subsequently collect and collate a firm set of user requirements for the first production version of MKM.

6 What is in the pilot version? The pilot version of MKM includes: The Browser: provides a user interface for publishing and retrieving a limited selection of price and volume data for gas and power business units; The Time Series Index: stores time series metadata; The Message Station: an inbox which holds inbound time series data. The TIBCO messaging backbone: underpins MKM and is transparent to the user. Connectivity to two market data servers - LIM (Houston) and PR1P (London).

7 What is planned for future versions? Calculation Services Security Services Validation Services Notification Services Auditing Services

8 Calculation Services Using calculation services, users will be able to manipulate time series data from multiple data sources. For example, it will be possible to: Aggregate time series Splice time series Apply calculations/pre-defined algorithms Apply filters (e.g. remove weekends) Perform functions that are dependant upon a library of pre-set calendars Export data to Microsoft Office applications if preferred.

9 Security Services The Security Services component will underpin MKM and ensure that only authorised and authenticated users can publish and retrieve data. Other issues addressed by this component will include: Encryption, to ensure that data is not “sniffed” from the network Regions - some data providers only allow data to be used in certain geographic regions.

10 Validation Services Using Validation Services, users will be able to gain comfort over the integrity and accuracy of their data. Specifically, they will be able to: Apply predefined checks on market data. Create customized checks. Create system/user alerts to notify the necessary user via or a pager. View a log of known errors. Validate data by cross system checks, choosing multiple symbols, or choosing a specific publication.

11 Notification Services The Notification Services component will allow data publishers and system administrators to alert users with respect to almost anything relating to MKM, namely: System issues New /updated data Ongoing initiatives Security Violations Validation Results

12 Auditing Services The Auditing Services component adds an element of control to the system. This component will allow administrators to identify which users are using which data, and will allow those responsible for data economics to improve Enron’s bottom line through purchasing only data which is used. Administrators will be able to: Monitor system availability and performance Monitor data usage Keep track of who uses what data, and when data is used. View audit logs.

13 LIM Phoenix PR1P EnPower GasDesk Browser Time Series Index Time Series IndexCalculation Notification Validation Auditing Security Layer TIBCO Backbone etc. Spreadsheets Flat files XML Market Data Servers Browsing & Indexing Services Client Applications Interim Solution

14 LIM Phoenix PR1P EnPower GasDesk Browser Time Series Index Time Series IndexCalculation Notification Validation Auditing Security Layer TIBCO Backbone etc. Spreadsheets Flat files XML Market Data Servers Browsing & Indexing Services Client Applications End Game Solution XML

15 What do we need from you? All projects have to meet the requirements of the business. We need you to tell us: Which market data servers would you like to see connected to MKM? What else would you like MKM to do for you? How we should prioritise future developments?