Lsevlet: web-based processing tool for LiDAR data J Ramon Arrowsmith December 14, 2004.

Slides:



Advertisements
Similar presentations
LAS format and processing DTMs from tins and local binning Arrowsmith.
Advertisements

XSEDE 13 July 24, Galaxy Team: PSC Team:
Terrain Datasets: How good are they? Celso Ferreira¹, Francisco Olivera¹, Dean Djokic² ¹Texas A & M University ² Environmental Systems Research Institute.
Celso Ferreira¹, Francisco Olivera², Dean Djokic³ ¹ PH.D. Student, Civil Engineering, Texas A&M University ( ² Associate.
The GEON LiDAR Workflow: An Internet-Based Tool for the Distribution and Processing of LiDAR Point Cloud Data Christopher J. Crosby, J Ramón Arrowsmith,
2. Point Cloud x, y, z, … Complete LiDAR Workflow 1. Survey 4. Analyze / “Do Science” 3. Interpolate / Grid USGS Coastal & Marine.
Update on ASU GEON Activities J Ramon Arrowsmith GEON PI Meeting Reston, VA May 13, 2006.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Active tectonics, Digital Elevation Model analysis, and remote sensing in GEON PI Meeting update.
Lidar data processing update October 15, 2005 Ramon Arrowsmith Chris Crosby Department of Geological Sciences Arizona State University Efrat Frank, Ashraf.
SAN DIEGO SUPERCOMPUTER CENTER Developing a CUAHSI HIS Data Node, as part of Cyberinfrastructure for the Hydrologic Sciences David Valentine Ilya Zaslavsky.
A Kepler-based Three Tier Architecture applied to LiDAR Interpolation and Analysis Efrat Frank, Ilkay Altintas San Diego Supercomputer Center, UCSD Configuration.
Presented by High-Performance Visualization of Geographic Data Budhendra Bhaduri Alexandre Sorokine Geographic Information Science and Technology Computational.
UNIVERSITY of MARYLAND GLOBAL LAND COVER FACILITY High Performance Computing in Support of Geospatial Information Discovery and Mining Joseph JaJa Institute.
Hands-On Microsoft Windows Server 2008 Chapter 1 Introduction to Windows Server 2008.
OpenTopography 2: A Cyberinfrastructure-Enabled Facility for High-Resolution Topographic Data and Tools PI: Chaitan Baru (SDSC) Co-PIs: J Ramon Arrowsmith.
TeraGrid Gateway User Concept – Supporting Users V. E. Lynch, M. L. Chen, J. W. Cobb, J. A. Kohl, S. D. Miller, S. S. Vazhkudai Oak Ridge National Laboratory.
Web-Based Tool and Why Cross Platform Support Multi-User No special software to install… just a browser Offload real work to server No worrying about versions.
Scientific Data Infrastructure in CAS Dr. Jianhui Scientific Data Center Computer Network Information Center Chinese Academy of Sciences.
TPB Models Development Status Report Presentation to the Travel Forecasting Subcommittee Ron Milone National Capital Region Transportation Planning Board.
The Creation of a Big Data Analysis Environment for Undergraduates in SUNY Presented by Jim Greenberg SUNY Oneonta on behalf of the SUNY wide team.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Image processing, LiDAR and high resolution 2D interpolation and 3D visualization for data integration.
ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University SCOTGRID: A PROTOTYPE TIER-2 CENTRE Steve Thorn Authors: A. Earl, P. Clark, S.
GEON Science Application Demos
Appendix B Planning a Virtualization Strategy for Exchange Server 2010.
Running Climate Models On The NERC Cluster Grid Using G-Rex Dan Bretherton, Jon Blower and Keith Haines Reading e-Science Centre Environmental.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES High Performance Computing applications in GEON: From Design to Production Dogan Seber.
Cloud Computing in NASA Missions Dan Whorton CTO, Stinger Ghaffarian Technologies June 25, 2010 All material in RED will be updated.
Presented by: Sanketh Beerabbi University of Central Florida COP Cloud Computing.
VO Sandpit, November 2009 e-Infrastructure to enable EO and Climate Science Dr Victoria Bennett Centre for Environmental Data Archival (CEDA)
Service Computation 2010November 21-26, Lisbon.
3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.
MURI Hardware Resources Ray Garcia Erik Olson Space Science and Engineering Center at the University of WI - Madison.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop Objectives.
IPlant cyberifrastructure to support ecological modeling Presented at the Species Distribution Modeling Group at the American Museum of Natural History.
Enabling Access to High-Resolution LiDAR Topography through Cyberinfrastructure-Based Data Distribution and Processing Christopher J. Crosby, J Ramón Arrowsmith.
CBEO Portal Presentation 2/6/2008, 4:30pm EST SDSC Or link from
| nectar.org.au NECTAR TRAINING Module 3 Common use cases.
Efrat Frank, Ashraf Memon, Vishu Nandigam, Chaitan Baru
Batch Scheduling at LeSC with Sun Grid Engine David McBride Systems Programmer London e-Science Centre Department of Computing, Imperial College.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
Instrumentation of the SAM-Grid Gabriele Garzoglio CSC 426 Research Proposal.
S&T IT Research Support 11 March, 2011 ITCC. Fast Facts Team of 4 positions 3 positions filled Focus on technical support of researchers Not “IT” for.
Event-Based Hybrid Consistency Framework (EBHCF) for Distributed Annotation Records Ahmet Fatih Mustacoglu Advisor: Prof. Geoffrey.
August 2003 At A Glance VMOC-CE is an application framework that facilitates real- time, remote cooperative work among geographically dispersed mission.
Where to find LiDAR: Online Data Resources.
Using free and/or open source tools to build workflows to manipulate and process LiDAR data Christopher Crosby.
DataNet – Flexible Metadata Overlay over File Resources Daniel Harężlak 1, Marek Kasztelnik 1, Maciej Pawlik 1, Bartosz Wilk 1, Marian Bubak 1,2 1 ACC.
Tool Integration with Data and Computation Grid GWE - “Grid Wizard Enterprise”
Quick Introduction to NorduGrid Oxana Smirnova 4 th Nordic LHC Workshop November 23, 2001, Stockholm.
IPlant Collaborative Hands-on Cyberinfrastructure Workshop - Part 1 R. Walls University of Arizona Biodiversity Information Standards (TDWG) Sep. 28, 2015,
1 CYBERINFRASTRUCTURE FOR THE GEOSCIENCES IGEON 2007 at the University of Hyderabad, India, August Web Services – The Motivation.
INTRODUCTION TO GIS  Used to describe computer facilities which are used to handle data referenced to the spatial domain.  Has the ability to inter-
TeraGrid Gateway User Concept – Supporting Users V. E. Lynch, M. L. Chen, J. W. Cobb, J. A. Kohl, S. D. Miller, S. S. Vazhkudai Oak Ridge National Laboratory.
Managing Enterprise GIS Geodatabases
Open Source GIS The Ecotrust mission to utilize new innovative technology Aaron Racicot – GIS Programmer
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Active tectonics, Digital Elevation Model analysis, and remote sensing in GEON J Ramón Arrowsmith.
All Hands Meeting 2005 BIRN-CC: Building, Maintaining and Maturing a National Information Infrastructure to Enable and Advance Biomedical Research.
TeraGrid Capability Discovery John-Paul “JP” Navarro TeraGrid Area Co-Director for Software Integration University of Chicago/Argonne National Laboratory.
CNAF - 24 September 2004 EGEE SA-1 SPACI Activity Italo Epicoco.
Data Analytics Challenges Some faults cannot be avoided Decrease the availability for running physics Preventive maintenance is not enough Does not take.
Accessing the VI-SEEM infrastructure
Tools and Services Workshop
Joslynn Lee – Data Science Educator
OpenMosix, Open SSI, and LinuxPMI
How to prepare for the End of License of Windows Server 2012/R2
Overview Introduction VPS Understanding VPS Architecture
TeraScale Supernova Initiative
Frieda meets Pegasus-WMS
L. Glimcher, R. Jin, G. Agrawal Presented by: Leo Glimcher
Presentation transcript:

Lsevlet: web-based processing tool for LiDAR data J Ramon Arrowsmith December 14, 2004

PART 1: LiDAR data handling and processing - the challenges Huge datasets: –8.79 million pts –Files getting larger with higher pulse rate instruments How do we grid and distribute these data? –ArcGIS can’t handle it easily –Expensive commercial software not an option for most data consumers Hector Mine Earthquake ALSM coverage (Mojave Desert)

ALSM data schematic workflow with partial implementation -possible scenario for NCALM data handling system? -incorporation of greater flexibility into system as user community grows -Crosby and Arrowsmith, 2004 watershed analysis (?)

GRASS as a processing tool for LiDAR GRASS: Open source GIS –Interpolation commands designed for large data sets –Splines use local pt density to segment data into rectangular areas for interpolation –Can control spline tension and smoothness Modular configuration could be easily implemented with in the GEON work flow –EX: User uploads point data to remote site where GRASS interpolation module runs on super computer and returns user a raster file. Ultimately a gridding utility for all large, computationally intensive data - gravity

What’s next? Build projection library Add functionality - TIN, Kriging, “Oskin”-type local plane fits, etc. More performance testing Comparison of interpolation algorithms for different landscapes Migrate to SDSC HP cluster for improved performance (?) Ties to NCALM and Earthscope Web-based front end for data distribution

A quick primer on… From geongrid.org: “Our mission is to prototype cyberinfrastructure for the Earth Sciences, based on close collaboration among IT and Earth Science researchers.” “We have deployed a distributed system consisting of “nodes” that host datasets and/or tools and applications programs, called the GEONgrid.” “…Such an environment enables new modes of science and can transform the day-to-day conduct of science.”

A quick primer on…

Data node:  CPU Xeon 2.8 Ghz MHz Hyperthreaded. 2GB Ram  2TB of Storage Agassiz (Point of Presence Node):  CPU Xeon 2.8 Ghz MHz Hyperthreaded. 2GB Ram roughly  550GB of Storage Compute-0-* (* = 1-4):  CPU Xeon 2.8 Ghz) MHz Hyperthreaded. 2GB Ram  roughly 35GB of Storage Linux platform GEON computing resources