100K ACTIVE CLIENTS SOLUTION Version: 18 May 2015.

Slides:



Advertisements
Similar presentations
Testing Relational Database
Advertisements

Logically Centralized Control Class 2. Types of Networks ISP Networks – Entity only owns the switches – Throughput: 100GB-10TB – Heterogeneous devices:
Case Tools Trisha Cummings. Our Definition of CASE  CASE is the use of computer-based support in the software development process.  A CASE tool is a.
ManageEngine TM Applications Manager 8 Monitoring Custom Applications.
Beyond data modeling Model must be normalised – purpose ? Outcome is a set of tables = logical design Then, design can be warped until it meets the realistic.
Applying Wireless in Legacy Systems
MSDN subscriptions Stand- alone tool Team collaboration Release management Visual Studio Team Foundation Server Device CAL Visual Studio Team Foundation.
Data Warehousing: Defined and Its Applications Pete Johnson April 2002.
SIP Routing Proxy Integrated Routing solution. General functionality  The Routing Proxy is based on Session Initial Protocol(SIP) and is acting as a.
Dynamics AX Technical Overview Application Architecture Dynamics AX Technical Overview.
Introduction to Database using Microsoft Access 2013 Part 1 November 4, 2014.
Pricing Changes MSDN subscriptions Stand- alone tool Team collaboration Release management Visual Studio Team Foundation Server Device CAL Visual Studio.
©2015 Cleo. All rights reserved. Company confidential. Managing Chaos: Frank Toscano Director, Product Management 2 Enterprise Data Movement.
PPOUG, 05-OCT-01 Agenda RMAN Architecture Why Use RMAN? Implementation Decisions RMAN Oracle9i New Features.
Securing Legacy Software SoBeNet User group meeting 25/06/2004.
WP6: Grid Authorization Service Review meeting in Berlin, March 8 th 2004 Marcin Adamski Michał Chmielewski Sergiusz Fonrobert Jarek Nabrzyski Tomasz Nowocień.
Internet Service Provisioning Phase - I August 29, 2003 TSPT Web:
AICT5 – eProject Project Planning for ICT. Process Centre receives Scenario Group Work Scenario on website in October Assessment Window Individual Work.
Database System Development Lifecycle © Pearson Education Limited 1995, 2005.
1DMG Confidential. Problem #1  Development and maintenance Huge demand for DMG services plus focus on short-term benefits led to shortcuts in code development.
Lecture On Database Analysis and Design By- Jesmin Akhter Lecturer, IIT, Jahangirnagar University.
STEALTH Content Store for SharePoint using Caringo CAStor  Boosting your SharePoint to the MAX! "Optimizing your Business behind the scenes"
QuickBooks, hosted by Reckon Online Catie Cotcher.
Yokogawa Electric Corporation Copyright © Yokogawa Electric Corporation Release 2.10 Functionality Overview September 2004.
Update on Selective Editing and Implications for Staff Skills International Trade Conference September 2008 Ken Smart.
Orbited Scaling Bi-directional web applications A presentation by Michael Carter
The Data Warehouse “A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile collection of “all” an organisation’s data in support.
Copyright © Yokogawa Electric Corporation Release 2.10 Functionality Overview September 2004.
The Client/Server Database Environment Ployphan Sornsuwit KPRU Ref.
Personal Computer - Stand- Alone Database  Database (or files) reside on a PC - on the hard disk.  Applications run on the same PC and directly access.
Module 10 Administering and Configuring SharePoint Search.
What is Sure Stats? Sure Stats is an add-on for SAP that provides Organizations with detailed Statistical Information about how their SAP system is being.
CENTRALISED AND CLIENT / SERVER DBMS. Topics To Be Discussed………………………. (A) Centralized DBMS (i) IntroductionIntroduction (ii) AdvantagesAdvantages (ii)
Module 7 Planning and Deploying Messaging Compliance.
ESRI User Conference 2004 ArcSDE. Some Nuggets Setup Performance Distribution Geodatabase History.
1 © 2003, Cisco Systems, Inc. All rights reserved. CCNA 3 v3.0 Module 9 Virtual Trunking Protocol.
11 CLUSTERING AND AVAILABILITY Chapter 11. Chapter 11: CLUSTERING AND AVAILABILITY2 OVERVIEW  Describe the clustering capabilities of Microsoft Windows.
MIS 7003 MBA Core Course in MIS Professor Akhilesh Bajaj The University of Tulsa Introduction to S/W Engineering © All slides in this presentation Akhilesh.
1DMG Confidential. Problem #1  Scalability Ingest and export processes not able to handle burst traffic loads Exponential growth in storage usage and.
ERCOT PMO Update Robert Connell Director Program Management May Board of Directors May 17, 2005 (Through 4/30/05)
Doing a CIM Project. 22 CIM Design Center  A rule I learned about applying technology:  Understand the design center of the technology.  Use extreme.
Full and Para Virtualization
Chapter 12 The Network Development Life Cycle
1 Object-Oriented Analysis and Design with the Unified Process Figure 13-1 Implementation discipline activities.
Power Guru: Implementing Smart Power Management on the Android Platform Written by Raef Mchaymech.
T Project Review Muuntaja I1 Iteration
Introduction to Performance Testing Performance testing is the process of determining the speed or effectiveness of a computer, network, software program.
QlikView System Overview Applications, Architecture, Configurations and Modules.
QlikView Architecture Overview
HPHC - PERFORMANCE TESTING Dec 15, 2015 Natarajan Mahalingam.
Cofax Scalability Document Version Scaling Cofax in General The scalability of Cofax is directly related to the system software, hardware and network.
Chapter 9 Database Planning, Design, and Administration Transparencies © Pearson Education Limited 1995, 2005.
QlikView Sizing Walkthru - RAM. Total RAM on the QlikView Server One or more QlikView documents (QVWs) loaded on Server Unaggregated QVW data model QlikView.
Understanding and Improving Server Performance
Extended Operating System Support
© 2002, Cisco Systems, Inc. All rights reserved.
InGenius Connector Enterprise Microsoft Dynamics CRM
Recipes for Use With Thin Clients
Affinity Depending on the application and client requirements of your Network Load Balancing cluster, you can be required to select an Affinity setting.
N-Tier Architecture.
SQL Server Monitoring Overview
The Client/Server Database Environment
EIN 6133 Enterprise Engineering
Conditions Data access using FroNTier Squid cache Server
Software Architecture in Practice
Performance Load Testing Case Study – Agilent Technologies
Dev Test on Windows Azure Solution in a Box
Chapter 2: Scaling VLANs
LO2 – Understand Computer Software
AICT5 – eProject Project Planning for ICT
Presentation transcript:

100K ACTIVE CLIENTS SOLUTION Version: 18 May 2015

Some Constraints (to simplify / de-risk) 100K ENABLED clients … … but not 100K clients all posting 24 x 7. –We will support 100K enabled clients on the assumption that, on average, a Android client does 30 sessions per day with a total duration of 13 hours. –The peak may exceed this, but the system may temporarily get behind. –We will NOT support 200K enabled, such that 100K are posting. Maximum of 25 QlikView users. –Any more than this then we are no longer single QVW server & SMB licencing, but into multiple servers and Enterprise licencing costs. This has not been allowed for in Elisa pricing. –Performance, and what those users are doing, may well introduce a lower usability limit.

Caveats There is no intention to remove any existing functionality or measurements –with the exception of fine-grained detail from LARs. But there is no intention to provide additional functionality or measurements either … –No significant changes to data model, no new correlations, … –… except enriched measurements from the clients will be tagged with geo-location to avoid costly correlation with Location Observer table. –Service Measurements for Elisa are not included. –We must avoid entangling “data feeds” with the 100K. Some functionality / measurements may be provided in a different way. –E.g. source of Max Throughput from periodic Connection Observations instead from LARs.

100K System We aim to achieve 100K enabled clients by software improvements only. However a front-end proxy may be appropriate to provide added security and take the Loss Metric Processor CPU load. Database sharding will be a contingency if the single CIP server cannot handle the load. This will require the CIP server plus a minimum of 2 database shard servers. We will not know the performance for certain until the system is built and performance tested using PSL’s simulator.

Phased plan Phase 0: 15,000 Clients for IDAEnd April –Smart Phone Client functionality as now. –1.8 Database schema, but with DC Observer tables in RAM –No Session Insight GUI –Clean install of datacarddb; retain Reporting database for QlikView –New Export interface Phase 1: > 20,000 Clients for ElisaMid June –New system install within Elisa IT –New 100K ready Database Schema –Session Insight GUI update to support new database. –Retain Reporting database for QlikView Phase 2: 100,000 ClientsEnd August –New Smart Phone clients doing enrichment. –Server support for enrichment on the client. –QlikView scaling address as much as possible

Phase 0: 15,000 Enabled Clients IDA Upgrade Refresh of existing IDA Android and iOS clients (non-enriching) –Appsee integration Database – existing 1.8 schema, except … –Put selected DC Observer tables into RAM and delete data after enrichment. Reporting Database retained … –Loading of the Reporting Database via stored procedures –Retain validation in Reporting Database. –No change to QVW load via the Reporting Database. New Export Interface.

Phase 1: > 20,000 Enabled Clients First phase for Elisa … Android Oma Elisa –No change to client functionality. Database – full100K database schema –Must avoid another database schema upgrade later. –Use Tokudb engine for performance and reduced table size. –Limited database tuning. Change to GUI to access ENR tables Fix Loss Metric Processor (if required). New Export Interface. Reporting Database retained … –Loading of the Reporting Database via stored procedures –Retain validation in Reporting Database. –No change to QVW load via the Reporting Database.

Phase 1: > 20,000 Enabled Clients QlikView changes to optimise current product … –May need incremental load on long-load docs only –Avoid incremental loading development for all docs in Phase 1 –Implement the most cost-effective data model fixes –Implement the most cost-effective chart optimisation routes –Produce more accurate, evidence-based server sizing estimates (System Monitoring improvements).

Phase 2: 100,000 clients Enrichment on the Android and iOS clients turned on –Addition of location to all ENR tables (to avoid Location correlation). –Enriched Connection Updates (with location). –Server acquisition changes for receiving enriched records from clients –Directly load into the ENR tables in the database. –Existing server enrichment retained for legacy clients.. No update to Windows or OSX clients planned. –Since only a few thousand of them, they have little impact on overall performance. Removal of LARS (except legacy Windows and OSX clients). –Use Enriched Connection Update records instead. Remove reporting databases. –Load QlikView direct from the replicated ENR tables. Move validation currently done on the Reporting database. –Not quite sure where yet …

Phase 2: Ready for 100,000 clients QlikView – all apps optimised to support 50K+ clients –(Dependent on the revised sizing estimates from Phase 1) –Implement new UDA (Universal Data Architecture) –Base QVD Loads as standard (replacement of E & T loaders) –All data model improvements implemented –All application design improvements implemented –Define data model requirements –Feasibility of 100k support to be evidenced through even more accurate estimates and possibly a load simulation –Acknowledge that 100k support of the current system breadth and depth may not be possible on one Small Business Server –Accurate evidence-based server sizing will be crucial if we are to justify 100k on one server or on more than one server – “guestimates” are not suitable at this scale given the cost implications of any such decision

Phase 2a If performance test on Phase 2 shows we can’t get to 100K enabled clients then we will have to shard the database. We should not need more than 2 or 3 shards – which means an additional two or three servers in addition to the current CIP server. One replication server should cope – it’s not doing much.

Elisa In-house System Servers Database Server Replication Server QlikView Server Enablement / Speed Test Server Session Insight GUI QlikViewExport Interface Customer devices Feed Interface to 3 rd party system (outside current project scope) Shard 1 Server Shard 2 Server Traffic Manager (Load Balancer) CIP Server