Allscripts Enterprise EHR Message Servers Advanced Edition September 18, 2018

Agenda Message Server Functions Core Connectivity Characteristics Monitoring Message Servers Changes in 11.4 AHS CSS Configuration Manager Message Server Functions – cover basic functionality of message servers (quickly) Regular monitoring guidelines (SES, system cert guidelines, perfmons, drive utilization) Core Connectivity Characteristics – load balancer utilization, using multiple message servers, MSO configuration Changes in Message Server Role in 11.4 – CED/CCD server specs, other new spec changes September 18, 2018

Agenda FaxTester Tray Sync Utility TWAdmin Workspace Location-Aware Printing Delta Application AHS CSS Configuration Manager – running as service account; filtering; multiple print servers (polling) FaxTester – testing lines, setting the “driver” Tray Sync Utility – where to get, how to use TWAdmin Printing Workspace – using a Windows print server IP while using AHS message servers; using multiple instances of the same server to easily break out by site; 200 printer limit (get case number); searching in the print queue Location-Aware Printing – a Galen-developed EHR add-on Delta Application – running as service account; changing the operational directory; adding additional Works DB instances September 18, 2018

Message Server Functions Print Jobs Fax Jobs Script Jobs Eligibility Jobs CED Processing Event Bus Message Jobs Delta Process Poll 2 – asking about managing functionality Message Center servers are in place to unburden the end-user’s PC from the overhead of Crystal Reports formatting and spooling to Network printers. By forwarding these responsibilities to the Message Center server, the end-user immediately regains operational control of their PC. The Message Center server can also provide faxing capabilities utilizing Black Ice fax components, which are built into the Allscripts Enterprise application. The message server can handle 6 types of messages Script – Eligibility - The eligibility checking function will retrieve a patient Pharmacy Benefit Management (PBM) information and mail order pharmacy that the patient should use, if the patient’s plan participates (CareMark, Wellspan, ESI, and Medco) (more info on AHS KB000006933 TouchWorks v11 Eligibility Verification) CED – to be covered later September 18, 2018

Core Connectivity Characteristics Print Spoolers point at web servers Load balancer ability Web Server provides database connectivity Print spooler can be configured to point at a single web server, but it should always point at a load balancer address if using more than one web server If a single web server fails, the print server will still function properly Should never list more than one web server here from the same environment The web server handles the communication to the database, not the print server; with Enterprise EEHR, all database communication is meant to go through the data access layer, the web server September 18, 2018

Core Connectivity Characteristics Multiple servers can be configured for individual functions One server can handle multiple clients (MSO configuration) Uses TWCSS Spooler service Service account Message server functionality can be consolidated into one server, broken into multiple servers by message type, or even have several servers for individual message types The Allscripts Message Server depends on the TWCSS Spooler Windows service to function Should be configured to start automatically Should be configured to use the service account, in order to ensure that proper access permissions are used September 18, 2018

Print Job Workflow Diagram Architecture Diagram Flow: End users submits a request, in this case, a print request The request goes to the web server, which adds the job to a table in the Works database on the database server The print server, using its defined polling interval polls the web server for print requests The web server check the table in the database server for print requests The database server returns the requested jobs, if any, to the web server The web server returns these jobs to the print server The print server processes the jobs and sends them to the desired printer September 18, 2018

Monitoring Message Servers Is my server compliant? SES document System certification document Windows updates September 18, 2018

Monitoring Message Servers Does my server have problems? Perfmons (used in ICP) Memory\Pages/Sec < 40 Processor(_Total)\% Processor Time < 40% average System\Processor Queue Length < 1, with no sustained spikes Pages/Sec - Memory \ Pages/sec is the sum of Pages Input/sec and Pages Output/sec and reports the number of requested pages that were not immediately available in RAM and had to be read from the disk (resulting in hard page faults) or that had to be written to the disk to make room in RAM for other pages. Monitoring the working sets of processes enables you to correlate a particular applications memory usage with page faulting. When memory is in short supply, working sets are continuously trimmed and page faults are frequent. Acceptable rates for Memory\Pages/sec range from 40 per second on older portable computers to 150 per second for the newest disk systems. Use a shorter monitoring period on client computers than on server computers (updating once per second is appropriate) because paging activity can occur in bursts on clients. Paging activity tends to reach a steady state on server computers; therefore, longer-term monitoring is appropriate. % Processor Time - The Processor\% Processor Time counter determines the percentage of time the processor is busy by measuring the percentage of time the thread of the Idle process is running and then subtracting that from 100 percent. This measurement is the amount of processor utilization. Although you might sometimes see high values for the Processor\% Processor Time counter (70 percent or greater depending on your workload and environment), it might not indicate a problem; you need more data to understand this activity. For example, high processor-time values typically occur when you are starting a new process and should not cause concern. Processor Queue Length - A collection of one or more threads that is ready but not able to run on the processor due to another active thread that is currently running is called the processor queue. The clearest symptom of a processor bottleneck is a sustained or recurring queue of more than two threads. Although queues are most likely to develop when the processor is very busy, they can develop when utilization is well below 90 percent. This can happen if requests for processor time arrive randomly and if threads demand irregular amounts of time from the processor. For more information about monitoring and adjusting thread scheduling, see "Threads in a Bottleneck" later in this chapter. September 18, 2018

September 18, 2018

Monitoring Message Servers Does my server have problems? Drive space utilization Event log monitoring Message queue Printer errors 2008 won’t show the “Status” column by default – View -> Choose Details… Can do all this manually or automated with PowerShell, or even with using an add-on like SolarWinds Server & Application Monitor (SAM) or Microsoft’s System Center Operations Manager (SCOM) September 18, 2018

September 18, 2018

Changes to Message Server in 11.4 SES Hardware requirements unchanged OS requirements: 11.2.3 11.4 2003 SP2 x86 with KB2641690 2008 SP2 x86 2003 SP2 x86 with KB2641690 2008 SP2 x86 or x64 2008 R2 x64 September 18, 2018

Changes to Message Server in 11.4 SES Software prerequisites: 11.2.3 11.4 IIS .NET 2.0 SP2 and 3.5 SP1 MDAC 2.8 Crystal 2008 Redistributable SQL Server Client Tools IIS .NET 4.0 Visual C++ 2010 Redistributable Package September 18, 2018

New CCD/CED Processing Server Introduced in 11.2.3 (later HFs) Used for auto-submission feature (dbMotion) Metric – one server per 1600 patient encounters per hour VM support - yes Clinical Exchange Document (CED) Continuity of Care Document (CCD) Implementation of the community CCD/CED auto-submission feature brings upon the need for a set of dedicated servers to handle the workload of the CCD/CED generation and transmission tasks. The auto-submit dbMotion configuration creates a CCD/CED for every patient visit or patient health record change, resulting in high system processing demands placed upon these servers. Separating this CCD/CED processing function out to a dedicated set of servers eliminates the performance impact of these tasks from affecting standard EHR web server performance. This set of servers is only necessary in the auto-submission configuration and is not necessary in an on-demand CCD/CED implementation. September 18, 2018

New CCD/CED Processing Server 2 cores, 2 GB RAM 2003 SP2 x86 with KB2641690 or 2008 SP2 x86 (requires 5.02 framework) IIS, IE 6.0 SP2, 7.0, or 8.0 (requires 5.02 framework), MDAC 2.8 .NET 2.0 SP2, .NET 3.5 SP1 Crystal 2008 redistributable, Silverlight, Windows Identity Foundation September 18, 2018

AHS CSS Configuration Manager CSS - Communication SubSystem Control Panel RightFax integration possible Run as service account Used when printing scanned images CSS stands for communication subsystem Allscripts Enterprise EHR message server configuration takes place in a custom control panel applet called CSS Configuration Manager This is where RightFax integration is set up RightFax is a third-party software solution for faxing, e-mailing and printing The spooler sends the fax image to RightFax, and the client configured their RightFax server to process the message how they see fit If using RightFax, can have fax message server role virtual within Enterprise environment RightFax allows a single fax server to be used by multiple applications across the customer’s facility September 18, 2018

AHS CSS Configuration Manager Options Compression Reduces size of messages Optimistic Printer Installation Assumes every printer is connected to server Force local printing Used when multiple print servers in use Not used when printers spanning multiple servers not identical Posting center – used when server located outside network – indicates the system is connected to the printers via IP, not by a server (does not need the Allscripts application installed) (indicates a legacy system formerly hosted at an ASP site; not used going forward) Optimistic print installation: ** By default, any print job that we attempt to process the print center computer will enumerate the list of printers installed on the computer and get information from these printers (port, status, etc). If the printer isn’t installed (network printer), we then will attempt to establish the connection. All this work getting exercised on every job can be expensive and taxing to the WinSpool process (sometimes it even hangs). Optimistic Printer Installation tells the print engine that we are confident that every printer is installed (connected) to this print center computer. Each job we attempt to process we simply try to print to that given name and bypass all the validation logic. Force local printing: Force Local Printing Always will bypass the destination server name in the job and assume the print queue named in the job exists locally on the same server as the TWCSSSPooler service. This is the fastest end-to-end printing method and in cases where multiple servers are present can offer automatic fault tolerance. The important implication of this method, however, is that printer queue administration must be highly disciplined and the installed printers and their names match across servers as appropriate. Allscripts recommends that Compression and Optimistic Printing are ALWAYS enabled. Force local printing should only be used when using multiple print servers AND the printers listed on each server are exactly duplicated September 18, 2018

Screenshot of the main CSS Configuration Manager screen, showing all workstation options September 18, 2018

Only need to check TouchWorks App Group. September 18, 2018

Validating Web Server September 18, 2018

Print servers – physical versus virtual September 18, 2018

September 18, 2018

September 18, 2018

FaxTester Acquired from Galen or Allscripts Used to test fax line connectivity Also used to set the “modem type” Stores this information in the registry September 18, 2018

September 18, 2018

Viewing FaxTester Registry Information September 18, 2018

Tray Sync Utility Used to sync tray usage between the printer and what is stored in the EHR Part of the standard Message Server installation September 18, 2018

Tray Sync Utility Usage Instructions Log onto print server. Open the CSSPrinterTrays.exe utility (C:\Program Files\Allscripts Healthcare Solutions\TouchWorks Web\Touchworks Print Center\) Ensure a valid Web Server is listed and that TouchWorks is listed as the AppGroup, and then click “Connect.” Select the desired Print Server, and click “Edit.” Select the desired printer, and click Synch. September 18, 2018

September 18, 2018

TWAdmin Printing Workspaces Print Queue Shows all jobs: previewed, printed, faxed, etc. Filter display Personalize vs. Search Tailored for each login Poll 1 – asking about message server administration The Print Queue page by default shows all jobs for all message types in all statuses Do DEMO of print queue page to show filtering, searching capability, and tabs, rerouting, cancelling, and resubmitting jobs September 18, 2018

TWAdmin Printing Workspaces Print Queue Jobs archive daily at 7 am SQL job CH Printing Queue Purge - Works September 18, 2018

TWAdmin - Print Queue September 18, 2018

TWAdmin Printing Workspaces (cont.) Print Servers Configure print/fax servers Set up “virtual” print server listings AutoPrint Defaults Defines actions for distribution and printing Print servers tab is used to set up virtual print server listings Pulls a listing of the printers at the IP/server name you give Name is totally arbitrary, can be anything you want Can list same server multiple times, used for segregating printers by site, etc. Also used to configure these virtual servers Autoprint defaults tab is where default actions are set for printing items Can set by site, provider, etc. Can set up automatic carbon copies September 18, 2018

TWAdmin - Print Servers by Location Note that all 4 servers listed have the same server name September 18, 2018

TWAdmin - Print Servers by Location Note that all 4 servers listed have the same server name Enable faxing checkbox will allow end-users to fax. Without it being checked on any server, end-users won’t be able to fax at all. September 18, 2018

TWAdmin - Print Servers September 18, 2018

TWAdmin - Print Servers September 18, 2018

TWAdmin - Print Servers September 18, 2018

TWAdmin - AutoPrint Defaults September 18, 2018

Location-Aware Printing Printing – user-based versus location-based Prevents printing sensitive patient data to wrong location Excellent for patient-centric workflows The EHR's user-based Print Defaults may cause printing of sensitive patient data to the wrong printer as providers and staff often move from room to room LAP enables Print Defaults to be based on each PCs' location rather than EHR User's Print Defaults LAP allows you to print based on where you are vs who you are. Groups have various reasons for printing based on location, particularly to exam rooms as providers and staff move throughout the office as they see patients. The most critical need for LAP comes with groups using Patient Centric Workflows, where providers and staff go to the patient, moving room to room, rather than the patient moving to where staff are in the office. September 18, 2018

LAP Example Set the location (computer name) Select the calling point Select the subtype Select the desired printer to default the print dialog to September 18, 2018

Delta Application Handles transfer of information between client and Allscripts SSL One Delta process per environment New Delta released in version 11.2.2 Doesn’t require SQL Client Tools Communicates over SSL to Allscripts Updates content in reference databases Sends utilization data to Allscripts - The Delta Export utilization data is typically sent out by your account/relationship manager on a monthly basis. If you aren’t receiving it regularly, please follow up with them. New Delta appears the same but is optimized; no longer uses BCP (SQL Client Tools) September 18, 2018

Delta Application Verify PASS daily Day Close Export (production only) Transfer Import Manage log files Move them to a new drive Auto-archive September 18, 2018

Delta Update Manager September 18, 2018

September 18, 2018

September 18, 2018

September 18, 2018

September 18, 2018

September 18, 2018

Thank you for joining us today. For additional assistance: You can contact us through our website at www.galenhealthcare.com Poll 3 – asking about Galen services