Issues with servers availablity and/or capacity

Slides:



Advertisements
Similar presentations
Copyright © 2011 by the Commonwealth of Pennsylvania. All Rights Reserved. Load Test Report.
Advertisements

Attie Naude 14 May 2013 Windows Azure Mobile Services.
13,000 Jobs and counting…. Advertising and Data Platform Our System.
Line Balancing Problem A B C 4.1mins D 1.7mins E 2.7 mins F 3.3 mins G 2.6 mins 2.2 mins 3.4 mins.
Service Models Example Server has 50 movies, 100 min. each Request rate: 1 movie/min Max. capacity: 20 streams Random Access Model –Case 1: after 20 movies,
EHarmony in Cloud Subtitle Brian Ko. eHarmony Online subscription-based matchmaking service Available in United States, Canada, Australia and United Kingdom.
Slide 1 of 10 Job Event Basics A Job Event is the name for the collection of components that comprise a scheduled job. On the iSeries a the available Job.
Software Performance Engineering Steve Chenoweth CSSE 375, Rose-Hulman Tues, Oct 23, 2007.
CGIWrap CGIWrap is a gateway program that allows general users to use CGI scripts and HTML forms without compromising the security of the http server.
SERVICE BROKER. SQL Server Service Broker SQL Server Service Broker provides the SQL Server Database Engine native support for messaging and queuing applications.
SQL Server Management Studio Introduction
A Technical Overview of Microsoft SQL Server 2005 Melville Thomson IT Pro Evangelist (UK)
Testing Dr. Andrew Wallace PhD BEng(hons) EurIng
TIBCO Designer TIBCO BusinessWorks is a scalable, extensible, and easy to use integration platform that allows you to develop, deploy, and run integration.
Managing Data Interoperability with FME Tony Kent Applications Engineer IMGS.
ConfidentialPA Testing Mobile Applications A Model for Mobile Testing.
Functions Lesson 10. Skills Matrix Function A function is a piece of code or routine that accepts parameters and stored as an object in SQL Server. The.
KISTI’s Activities on the NA4 Biomed Cluster Soonwook Hwang, Sunil Ahn, Jincheol Kim, Namgyu Kim and Sehoon Lee KISTI e-Science Division.
JavaScript, Fourth Edition
Integration Broker PeopleTools Integration Broker Steps –Introduction & terminologies –Application Server PUB/SUB services (Application Server)
Data Interoperability Basics Bruce Harold & Dale Lutz.
Block1 Wrapping Your Nugget Around Distributed Processing.
Database Change Management One solution to an often complex problem Kevin Hurwitz Headspring Systems
Introduction to RtReports – Tony Fenn & Chris Nelson Introduction to RtReports Chris Nelson - Senior Developer Tony Fenn - Product Manager.
Scale Fail or, how I learned to stop worrying and love the downtime.
Activatio n limit. maximum number of process instances that can concurrently be loaded into memory.
23. October 2014 WG DIS, Brx WFD 2016 Tools Andres Stæhr Bastholm.
AMH001 (acmse03.ppt - 03/7/03) REMOTE++: A Script for Automatic Remote Distribution of Programs on Windows Computers Ashley Hopkins Department of Computer.
PERFORMANCE AND ANALYSIS WORKFLOW ISSUES US ATLAS Distributed Facility Workshop November 2012, Santa Cruz.
Build and Deployment Process Understand NCI’s DevOps and continuous integration requirements Understand NCI’s build and distribution requirements.
WISE Working Group D September 2009, Brussels Jon Maidens.
1 Integration of the LCP Reporting Into the E-PRTR Scope and Technical proposal November 3rd.
1 Integration of the LCP Reporting Into the E-PRTR Scope and Technical specifications December 3rd.
Pre-Conference Training Demos & Discussion Technical Tools Don Thomson, CEO.
StoRM + Lustre Proposal YAN Tian On behalf of Distributed Computing Group
1 Integration of the LCP Reporting Into the E-PRTR Current status and Technical proposal August 4th.
Oracle Database Architectural Components
Overview on the work performed during EPIKH Training Faiza MEDJEK /INFN, CATANIA 1.
Everything you've ever wanted to know about using Control-M to integrate any application workload September 9, 2016 David Fernandez Senior Presales Consultant.
Jean-Philippe Baud, IT-GD, CERN November 2007
Mastering Person Data:
Netscape Application Server
Chapter 2 Database System Concepts and Architecture
Service Restore Flow Receives/retrives input of list of server involved in the process Flow Performs multi level health check like process status, replication.
Open Source distributed document DB for an enterprise
JZOS (Java Batch Launcher and Toolkit for z/OS) Quick Start Guide
Example of a page header
Recent DBS Developments & Works Ahead
Pre-Conference Training Demos & Discussion Technical Tools
Workflow Best Practices
Solving ETL Bottlenecks with SSIS Scale Out
X in [Integration, Delivery, Deployment]
Armando Lacerda
Migrating your SQL Server Instance
Google App Engine Ying Zou 01/24/2016.
ASP.NET Module Subtitle.
COGNOS 8 BI - Introduction and Architecture Cognos CoE
Armando Lacerda
Introduction to VSTS Database Professional
Armando Lacerda
WISE Distributed Architecture
Armando Lacerda
Reportnet 3.0 Database Feasibility Study – Approach
Cost Effective Presto on AWS
Mark Quirk Head of Technology Developer & Platform Group
IIS and .NET Security Application Pools Pamella Smith June 18, 2009.
GGF10 Workflow Workshop Summary
Dimension Load Patterns with Azure Data Factory Data Flows
Presentation transcript:

Issues with servers availablity and/or capacity ReportNet situation

ReportNet QC Engine failing Reasons A combination of several dataflows caused the XQuery engines to fail. Code lists exceeded the designed limits. (max ~5000 towards ~500.000) Communication between ReportNet and FME Server Solutions A new QC engine (Xquery) is deployed. Significant performance gains on large data sets. Several dataflows have optimised there QC scripts in ReportNet. Split ReportNet core from QC engines. (WFD has it’s own instance) Renovate the sub system for code lists. In future EEA plans a complete makeover for ReportNet EEA will maintain and optimise the old system while develop and design ReportNet 2.0

WFD-Specific

WFD is highly spatial We integrated FME (Feature Manipulation Engine) into ReportNet. FME allows complex spatial validation and can natively read/write GML,XML,SQL,...

Problems during communication Firewall cuts connections After 30+ minutes ReportNet FME SERVER QC execution at envelope and dataset level FME QC for WFD Waits until validation is performed 6 isolated engines The firewall would cut the connection and not inform the servers (security reason against hackers) -> Leads to memory leaks crashing FME server.

Solution Firewall cuts connections After 30+ minutes ReportNet FME Server will close streaming connections that run for 29 minutes ReportNet FME SERVER QC execution at envelope and dataset level FME Process Initiate or check results DB Store Results Maximum open for 5 min 3 isolated engines Background job’s Continue validation Store results Job queue ReportNet checks every 20min (During four hours) 6 isolated engines 80+ Workspaces 11 schemas on spatial QC -> Heavy!

Testing

Extensive monitoring