Implementing Unified Access to Scientific Data from.NET Platform Sergey B. Berezin Dmitriy V. Voitsekhovskiy Vilen M. Paskonov Moscow State University.

Slides:



Advertisements
Similar presentations
웹 서비스 개요.
Advertisements

Pulan Yu School of Informatics Indiana University Bloomington Web service based Varuna.Net.
A Workflow Engine with Multi-Level Parallelism Supports Qifeng Huang and Yan Huang School of Computer Science Cardiff University
GIS Tool in Drupal PNW Drupal Summit - Seattle October 20th, 2012 Mack Hardy & Tom
Caching the MDSPlus Data via Hibernate By Ajith M Jose Comp6703 Project Client: Raju Karia Supervisor: Dr. Henry Gardner (Development of “WebScope”)
A New Computing Paradigm. Overview of Web Services Over 66 percent of respondents to a 2001 InfoWorld magazine poll agreed that "Web services are likely.
15 Chapter 15 Web Database Development Database Systems: Design, Implementation, and Management, Fifth Edition, Rob and Coronel.
2006 IEEE International Conference on Web Services ICWS 2006 Overview.
Computer Science 101 Web Access to Databases Overview of Web Access to Databases.
Client-Server Processing and Distributed Databases
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Internet GIS. A vast network connecting computers throughout the world Computers on the Internet are physically connected Computers on the Internet use.
1 Introduction to Web Development. Web Basics The Web consists of computers on the Internet connected to each other in a specific way Used in all levels.
INTRODUCTION TO WEB DATABASE PROGRAMMING
A Free sample background from © 2001 By Default!Slide 1.NET Overview BY: Pinkesh Desai.
Chapter 10 Architectural Design
Cizer.NET Reporting Forum for Business Intelligence Copyright © 2005 Cizer Software OR
A Scalable Application Architecture for composing News Portals on the Internet Serpil TOK, Zeki BAYRAM. Eastern MediterraneanUniversity Famagusta Famagusta.
Beyond DHTML So far we have seen and used: CGI programs (using Perl ) and SSI on server side Java Script, VB Script, CSS and DOM on client side. For some.
Databases and the Internet. Lecture Objectives Databases and the Internet Characteristics and Benefits of Internet Server-Side vs. Client-Side Special.
Oracle8 JDBC Drivers Section 2. Common Features of Oracle JDBC Drivers The server-side and client-side Oracle JDBC drivers provide the same basic functionality.
ASP.NET + Ajax Jesper Tørresø ITNET2 F08. Ajax Ajax (Asynchronous JavaScript and XML) A group of interrelated web development techniques used for creating.
C Copyright © 2009, Oracle. All rights reserved. Appendix C: Service-Oriented Architectures.
CIS 375—Web App Dev II Microsoft’s.NET. 2 Introduction to.NET Steve Ballmer (January 2000): Steve Ballmer "Delivering an Internet-based platform of Next.
Institute for Mathematical Modeling RAS 1 Visualization in distributed systems. Overview. Remote visualization means interactive viewing of three dimensional.
Active Monitoring in GRID environments using Mobile Agent technology Orazio Tomarchio Andrea Calvagna Dipartimento di Ingegneria Informatica e delle Telecomunicazioni.
A Metadata Based Approach For Supporting Subsetting Queries Over Parallel HDF5 Datasets Vignesh Santhanagopalan Graduate Student Department Of CSE.
1 HKU CSIS DB Seminar: HKU CSIS DB Seminar: Web Services Oriented Data Processing and Integration Speaker: Eric Lo.
Global Land Cover Facility The Global Land Cover Facility (GLCF) is a member of the Earth Science Information Partnership (ESIP) Federation providing data,
Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.
Indo-US Workshop, June23-25, 2003 Building Digital Libraries for Communities using Kepler Framework M. Zubair Old Dominion University.
Web Services Kanda Runapongsa Dept. of Computer Engineering Khon Kaen University.
1 Computer Programming (ECGD2102 ) Using MATLAB Instructor: Eng. Eman Al.Swaity Lecture (1): Introduction.
Javascript Cog Kit By Zhenhua Guo. Grid Applications Currently, most grid related applications are written as separate software. –server side: Globus,
P2P-Based Best Resolution Match Image Access, Retrieval and Delivery P Contreras, S Johnstone, F Murtagh CS, QUB.
Integrated Grid workflow for mesoscale weather modeling and visualization Zhizhin, M., A. Polyakov, D. Medvedev, A. Poyda, S. Berezin Space Research Institute.
Chapter 18 Object Database Management Systems. McGraw-Hill/Irwin © 2004 The McGraw-Hill Companies, Inc. All rights reserved. Outline Motivation for object.
DataNet – Flexible Metadata Overlay over File Resources Daniel Harężlak 1, Marek Kasztelnik 1, Maciej Pawlik 1, Bartosz Wilk 1, Marian Bubak 1,2 1 ACC.
ICDL 2004 Improving Federated Service for Non-cooperating Digital Libraries R. Shi, K. Maly, M. Zubair Department of Computer Science Old Dominion University.
Chapter 10 Intro to SOAP and WSDL. Objectives By study in the chapter, you will be able to: Describe what is SOAP Exam the rules for creating a SOAP document.
GEON2 and OpenEarth Framework (OEF) Bradley Wallet School of Geology and Geophysics, University of Oklahoma
RSISIPL1 SERVICE ORIENTED ARCHITECTURE (SOA) By Pavan By Pavan.
CSIT 220 (Blum)1 Remote Procedure Calls Based on Chapter 38 in Computer Networks and Internets, Comer.
NOVA A Networked Object-Based EnVironment for Analysis “Framework Components for Distributed Computing” Pavel Nevski, Sasha Vanyashin, Torre Wenaus US.
Information Technology: GrADS INTEGRATED USER INTERFACE Maps, Charts, Animations Expressions, Functions of Original Variables General slices of { 4D Grids.
Architecture View Models A model is a complete, simplified description of a system from a particular perspective or viewpoint. There is no single view.
Data Integration in Bioinformatics Using OGSA-DAI The BioDA Project Shirley Crompton, Brian Matthews (CCLRC) Alex Gray, Andrew Jones, Richard White (Cardiff.
Module: Software Engineering of Web Applications Chapter 2: Technologies 1.
12 Oct 2003VO Tutorial, ADASS Strasbourg, Data Access Layer (DAL) Tutorial Doug Tody, National Radio Astronomy Observatory T HE US N ATIONAL V IRTUAL.
GraDS MacroGrid Carl Kesselman USC/Information Sciences Institute.
Intro to Web Services Dr. John P. Abraham UTPA. What are Web Services? Applications execute across multiple computers on a network.  The machine on which.
Chapter 18 Object Database Management Systems. Outline Motivation for object database management Object-oriented principles Architectures for object database.
.NET Mobile Application Development XML Web Services.
Electronic Commerce Semester 1 Term 1 Lecture 7. Introduction to the Web The Internet supports a variety of important tools, such as file transfer, electronic.
Copyright 2007, Information Builders. Slide 1 iWay Web Services and WebFOCUS Consumption Michael Florkowski Information Builders.
From Use Cases to Implementation 1. Structural and Behavioral Aspects of Collaborations  Two aspects of Collaborations Structural – specifies the static.
A Use Case for GEON 1 A user request of the form: “For a given region (i.e. lat/long extent, plus depth), return a 3D structural model with accompanying.
CEN6502, Spring Understanding the ORB: Client Side Structure of ORB (fig 4.1) Client requests may be passed to ORB via either SII or DII SII decide.
Getting Started with Datazen Microsoft’s New Mobile Dashboard Platform.
Submitted by: Moran Mishan. Instructed by: Osnat (Ossi) Mokryn, Dr.
From Use Cases to Implementation 1. Mapping Requirements Directly to Design and Code  For many, if not most, of our requirements it is relatively easy.
XML 1. Chapter 8 © 2013 Pearson Education, Inc. Publishing as Prentice Hall SAMPLE XML SCHEMA (XSD) 2 Schema is a record definition, analogous to the.
1 Chapter 1 INTRODUCTION TO WEB. 2 Objectives In this chapter, you will: Become familiar with the architecture of the World Wide Web Learn about communication.
ODBC, OCCI and JDBC overview
.NET Framework 2.0 .NET Framework 3.0 .NET Framework 3.5
WEB SERVICES.
Flanders Marine Institute (VLIZ)
University of Technology
Serpil TOK, Zeki BAYRAM. Eastern MediterraneanUniversity Famagusta
L. Glimcher, R. Jin, G. Agrawal Presented by: Leo Glimcher
Presentation transcript:

Implementing Unified Access to Scientific Data from.NET Platform Sergey B. Berezin Dmitriy V. Voitsekhovskiy Vilen M. Paskonov Moscow State University Department of Computational Mathematics and Cybernetics Supported by Student Laboratory of Microsoft Technologies and RFBR grants

Different languages, common tools Viscous fluid flow visualization via vector fields and color maps ( Seismic data visualization via isosurfaces ( Tensor field visualization for diffusion through biological tissue (

Scientific data access requirements Viscous fluid flow visualization via vector fields and color maps ( Seismic data visualization via isosurfaces ( Tensor field visualization for diffusion through biological tissue ( We need to: Retrieve typed data object without regard where it is stored and how it is stored. Physical data independence Retrieve partial data when needed Filtering & Caching Retrieve data description Metadata support We don’t want to: Rewrite existing computational software Use existing formats Install new system software Use existing protocols

Scientific data access today

What’s so special in scientific data? Scientific data … Have a complex structure; Parameterized by …  Time  Sampling point coordinates  More complex parameters; Stored in many files of various formats Have very large size of individual data items Don’t fit well to relational model!

What is DataSet?

Example: Accessing data in C# // Retrieve the DataSet object from a server by GUID DataSet dataset = DataSet.Open(" "767c57b1-801e-4784-bbd fd0ec2"); // Fetching DataItem by name. // DataItem may be either simple or composite, it doesn’t matter DataItem xVelocity = dataset.DataItems["u-values"]; // Creating parameter corresponding to time moment = 0.0 CompositeParameter param = new CompositeParameter( new ParameterValue("time", 0.0d) ); // Fetching DataItemSlice for the parameter. // It is an instance of DataItem for specified parameter value. DataItemSlice dataVelocity = xVelocity[param]; // Getting required data: velocity array for time = 0.0 ScalarArray3d data = dataVelocity.GetData() as ScalarArray3d;

DataRequest: communicating with server The following DataRequest is sent to a server as the result of the previous example: <dataRequest dataSource="…" dataSet="767c57b1-801e-4784-bbd fd0ec2" … > <dataSource sourceName="u0000.cdf" sourceType="netCDF" sourceParameters="u" /> The following DataRequest is received from the server: <dataRequest dataSource="…" dataSet="767c57b1-801e-4784-bbd fd0ec2" … >

Complex structures in DataSet file with scalar array file with scalar array file with scalar array file with spatial grid scalar array data item scalar array data item scalar array data item vector array constructor vector array data item spatial grid data item data field constructor vector field data item x X,Y, Z

Example: Accessing composite data in C# // Retrieve the DataSet object from a server by GUID DataSet dataset = DataSet.Open(" "767c57b1-801e-4784-bbd fd0ec2"); // Fetching DataItem by name. // DataItem may be either simple or composite, it doesn’t matter DataItem velocity = dataset.DataItems["uvw-values"]; // Creating parameter corresponding to time moment = 0.0 CompositeParameter param = new CompositeParameter( new ParameterValue("time", 0.0d) ); // Fetching DataItemSlice for the parameter. // It is an instance of DataItem for specified parameter value. DataItemSlice dataVelocity = velocity[param]; // Getting required data: velocity array for time = 0.0 Vector3dArray3d data = dataVelocity.GetData() as Vector3dArray3d;

DataRequest: composite data items The following DataRequest is sent to a server as the result of the previous example execution: … The following DataRequest is received from server:

Filtering Filtering allows transfer of only required data from server to client Filtering may be performed both by a client-side and a server- side of the system. Examples of the filtering are cropping and thinning of large vector fields x d vectors 108 МB x d vectors 12 MB x 100 2d vectors 120KB 0.4 cropping filter [0.4,0.76] x [0.4,0.76] thinning filter (0.1,0.1)

Example: Filtering data in C# // Initializing the DataSet object from a server by its GUID DataSet dataset = DataSet.Open(" "767c57b1- 801e-4784-bbd fd0ec2"); // Fetching DataItem by its name. It may be either simple or composite DataItem velocity = dataset.DataItems["uvw-values"]; // Creating parameter corresponding to time moment = 0.0 CompositeParameter param = new CompositeParameter( new ParameterValue("time", 0.0d) ); // Fetching DataItemSlice for the parameter. DataItemSlice dataVelocity = velocity[param]; // Creating filter "Thinner" for a type of the velocity data item // and setting up its parameters IThinner3dFilter filter = FilterFactory.GetFilter("Thinner", dataVelocity.TypeDescriptor) as IThinner3dFilter; filter.PercentageX = 0.05; filter.PercentageY = 0.05; filter.PercentageZ = 0.05; // Getting required data: thinned out velocity array for time = 0.0 Vector3dArray3d data = dataVelocity.GetData(filter) as Vector3dArray3d;

DataRequest: communicating with server The following DataRequest is sent to a server as the result of the previous example execution: … The returned DataRequest is similar in this case to the returned DataRequest from the previous example.

Caching Both server-side and client-side of the system cache the results of a successful DataRequest execution. Server-side cache filtering results Client-side cache retrieved data items and results of DataRequest

How DataRequest is performed?

Deployment Scenario The simplest scenario is as follows: More sophisticated scenario includes development of distributed data sources that provide scientific data. Dedicated servers will act as data processor performing data filtering and transformations Dedicated servers will act as data registries allowing DataSet enumeration and querying in entire global network. This will make possible to create dynamic data libraries of researches and enables easy data publishing

Why.NET? Object-oriented data access requires an object-oriented platform to be built on. High extensibility is based on CLR dynamic nature  New data types  New filters  New parsers  No built in data types, filters, parsers.NET opens new horizons with LINQ, WPF,…

Future work Transferring files from remote server is just an example of DataProvider  Extend architecture for new types of data providers LINQ technology will make data access from C# much more elegant. Development of easy-to-use data management applications for the proposed approach. Development of an innovative visualization system, highly extensible and customizable  Or integrate our approach with existing one

Future visualization system Control A Control B Control C A BABA C View 1 View 1.1 Control D D View 1.2 View Step 1. Choose object of interest Step 2. Choose data transform Step 3. Choose visualization algorithm Example

Questions? Visit: Mail to: