1. Allscripts Enterprise EHR Message Servers
Advanced Edition
September 18, 2018

2. Agenda Message Server Functions Core Connectivity Characteristics
Monitoring Message Servers
Changes in 11.4
AHS CSS Configuration Manager
Message Server Functions – cover basic functionality of message servers (quickly)
Regular monitoring guidelines (SES, system cert guidelines, perfmons, drive utilization)
Core Connectivity Characteristics – load balancer utilization, using multiple message servers, MSO configuration
Changes in Message Server Role in 11.4 – CED/CCD server specs, other new spec changes
September 18, 2018

3. Agenda FaxTester Tray Sync Utility TWAdmin Workspace
Location-Aware Printing
Delta Application
AHS CSS Configuration Manager – running as service account; filtering; multiple print servers (polling)
FaxTester – testing lines, setting the “driver”
Tray Sync Utility – where to get, how to use
TWAdmin Printing Workspace – using a Windows print server IP while using AHS message servers; using multiple instances of the same server to easily break out by site; 200 printer limit (get case number); searching in the print queue
Location-Aware Printing – a Galen-developed EHR add-on
Delta Application – running as service account; changing the operational directory; adding additional Works DB instances
September 18, 2018

4. Message Server Functions
Print Jobs
Fax Jobs
Script Jobs
Eligibility Jobs
CED Processing
Event Bus Message Jobs
Delta Process
Poll 2 – asking about managing functionality
Message Center servers are in place to unburden the end-user’s PC from the overhead of Crystal
Reports formatting and spooling to Network printers. By forwarding these responsibilities to the
Message Center server, the end-user immediately regains operational control of their PC.
The Message Center server can also provide faxing capabilities utilizing Black Ice fax components,
which are built into the Allscripts Enterprise application.
The message server can handle 6 types of messages
Script –
Eligibility - The eligibility checking function will retrieve a patient Pharmacy Benefit Management (PBM) information and mail order pharmacy that the patient should use, if the patient’s plan participates (CareMark, Wellspan, ESI, and Medco) (more info on AHS KB TouchWorks v11 Eligibility Verification)
CED – to be covered later
September 18, 2018

5. Core Connectivity Characteristics
Print Spoolers point at web servers
Load balancer ability
Web Server provides database connectivity
Print spooler can be configured to point at a single web server, but it should always point at a load balancer address if using more than one web server
If a single web server fails, the print server will still function properly
Should never list more than one web server here from the same environment
The web server handles the communication to the database, not the print server; with Enterprise EEHR, all database communication is meant to go through the data access layer, the web server
September 18, 2018

6. Core Connectivity Characteristics
Multiple servers can be configured for individual functions
One server can handle multiple clients (MSO configuration)
Uses TWCSS Spooler service
Service account
Message server functionality can be consolidated into one server, broken into multiple servers by message type, or even have several servers for individual message types
The Allscripts Message Server depends on the TWCSS Spooler Windows service to function
Should be configured to start automatically
Should be configured to use the service account, in order to ensure that proper access permissions are used
September 18, 2018

7. Print Job Workflow Diagram
Architecture Diagram
Flow:
End users submits a request, in this case, a print request
The request goes to the web server, which adds the job to a table in the Works database on the database server
The print server, using its defined polling interval polls the web server for print requests
The web server check the table in the database server for print requests
The database server returns the requested jobs, if any, to the web server
The web server returns these jobs to the print server
The print server processes the jobs and sends them to the desired printer
September 18, 2018

8. Monitoring Message Servers
Is my server compliant?
SES document
System certification document
Windows updates
September 18, 2018

9. Monitoring Message Servers
Does my server have problems?
Perfmons (used in ICP)
Memory\Pages/Sec < 40
Processor(_Total)\% Processor Time < 40% average
System\Processor Queue Length < 1, with no sustained spikes
Pages/Sec - Memory \ Pages/sec is the sum of Pages Input/sec and Pages Output/sec and reports the number of requested pages that were not immediately available in RAM and had to be read from the disk (resulting in hard page faults) or that had to be written to the disk to make room in RAM for other pages. Monitoring the working sets of processes enables you to correlate a particular applications memory usage with page faulting. When memory is in short supply, working sets are continuously trimmed and page faults are frequent.
Acceptable rates for Memory\Pages/sec range from 40 per second on older portable computers to 150 per second for the newest disk systems. Use a shorter monitoring period on client computers than on server computers (updating once per second is appropriate) because paging activity can occur in bursts on clients. Paging activity tends to reach a steady state on server computers; therefore, longer-term monitoring is appropriate.
% Processor Time - The Processor\% Processor Time counter determines the percentage of time the processor is busy by measuring the percentage of time the thread of the Idle process is running and then subtracting that from 100 percent. This measurement is the amount of processor utilization. Although you might sometimes see high values for the Processor\% Processor Time counter (70 percent or greater depending on your workload and environment), it might not indicate a problem; you need more data to understand this activity. For example, high processor-time values typically occur when you are starting a new process and should not cause concern.
Processor Queue Length - A collection of one or more threads that is ready but not able to run on the processor due to another active thread that is currently running is called the processor queue. The clearest symptom of a processor bottleneck is a sustained or recurring queue of more than two threads. Although queues are most likely to develop when the processor is very busy, they can develop when utilization is well below 90 percent. This can happen if requests for processor time arrive randomly and if threads demand irregular amounts of time from the processor. For more information about monitoring and adjusting thread scheduling, see "Threads in a Bottleneck" later in this chapter.
September 18, 2018

10. September 18, 2018

11. Monitoring Message Servers
Does my server have problems?
Drive space utilization
Event log monitoring
Message queue
Printer errors
2008 won’t show the “Status” column by default – View -> Choose Details…
Can do all this manually or automated with PowerShell, or even with using an add-on like SolarWinds Server & Application Monitor (SAM) or Microsoft’s System Center Operations Manager (SCOM)
September 18, 2018

12. September 18, 2018

13. Changes to Message Server in 11.4 SES
Hardware requirements unchanged
OS requirements:
11.2.3
11.4
2003 SP2 x86 with KB SP2 x86
2003 SP2 x86 with KB SP2 x86 or x R2 x64
September 18, 2018

14. Changes to Message Server in 11.4 SES
Software prerequisites:
11.2.3
11.4
IIS .NET 2.0 SP2 and 3.5 SP1 MDAC 2.8 Crystal 2008 Redistributable SQL Server Client Tools
IIS .NET Visual C Redistributable Package
September 18, 2018

15. New CCD/CED Processing Server
Introduced in (later HFs)
Used for auto-submission feature (dbMotion)
Metric – one server per 1600 patient encounters per hour
VM support - yes
Clinical Exchange Document (CED)
Continuity of Care Document (CCD)
Implementation of the community CCD/CED auto-submission feature brings upon the need for a set of dedicated servers to handle the workload of the CCD/CED generation and transmission tasks.
The auto-submit dbMotion configuration creates a CCD/CED for every patient visit or patient health record change, resulting in high system processing demands placed upon these servers.
Separating this CCD/CED processing function out to a dedicated set of servers eliminates the performance impact of these tasks from affecting standard EHR web server performance.
This set of servers is only necessary in the auto-submission configuration and is not necessary in an on-demand CCD/CED implementation.
September 18, 2018

16. New CCD/CED Processing Server
2 cores, 2 GB RAM
2003 SP2 x86 with KB or 2008 SP2 x86 (requires 5.02 framework)
IIS, IE 6.0 SP2, 7.0, or 8.0 (requires 5.02 framework), MDAC 2.8
.NET 2.0 SP2, .NET 3.5 SP1
Crystal 2008 redistributable, Silverlight, Windows Identity Foundation
September 18, 2018

17. AHS CSS Configuration Manager
CSS - Communication SubSystem
Control Panel
RightFax integration possible
Run as service account
Used when printing scanned images
CSS stands for communication subsystem
Allscripts Enterprise EHR message server configuration takes place in a custom control panel applet called CSS Configuration Manager
This is where RightFax integration is set up
RightFax is a third-party software solution for faxing, ing and printing
The spooler sends the fax image to RightFax, and the client configured their RightFax server to process the message how they see fit
If using RightFax, can have fax message server role virtual within Enterprise environment
RightFax allows a single fax server to be used by multiple applications across the customer’s facility
September 18, 2018

18. AHS CSS Configuration Manager Options
Compression
Reduces size of messages
Optimistic Printer Installation
Assumes every printer is connected to server
Force local printing
Used when multiple print servers in use
Not used when printers spanning multiple servers not identical
Posting center – used when server located outside network – indicates the system is connected to the printers via IP, not by a server (does not need the Allscripts application installed) (indicates a legacy system formerly hosted at an ASP site; not used going forward)
Optimistic print installation: ** By default, any print job that we attempt to process the print center computer will enumerate the list of printers installed on the computer and get information from these printers (port, status, etc). If the printer isn’t installed (network printer), we then will attempt to establish the connection. All this work getting exercised on every job can be expensive and taxing to the WinSpool process (sometimes it even hangs). Optimistic Printer Installation tells the print engine that we are confident that every printer is installed (connected) to this print center computer. Each job we attempt to process we simply try to print to that given name and bypass all the validation logic.
Force local printing: Force Local Printing Always will bypass the destination server name in the job and assume the print queue named in the job exists locally on the same server as the TWCSSSPooler service. This is the fastest end-to-end printing method and in cases where multiple servers are present can offer automatic fault tolerance. The important implication of this method, however, is that printer queue administration must be highly disciplined and the installed printers and their names match across servers as appropriate.
Allscripts recommends that Compression and Optimistic Printing are ALWAYS enabled. Force local printing should only be used when using multiple print servers AND the printers listed on each server are exactly duplicated
September 18, 2018

19. Screenshot of the main CSS Configuration Manager screen, showing all workstation options
September 18, 2018

20. Only need to check TouchWorks App Group.
September 18, 2018

21. Validating Web Server
September 18, 2018

22. Print servers – physical versus virtual
September 18, 2018

23. September 18, 2018

24. September 18, 2018

25. FaxTester Acquired from Galen or Allscripts
Used to test fax line connectivity
Also used to set the “modem type”
Stores this information in the registry
September 18, 2018

26. September 18, 2018

27. Viewing FaxTester Registry Information
September 18, 2018

28. Tray Sync Utility
Used to sync tray usage between the printer and what is stored in the EHR
Part of the standard Message Server installation
September 18, 2018

29. Tray Sync Utility Usage Instructions
Log onto print server.
Open the CSSPrinterTrays.exe utility (C:\Program Files\Allscripts Healthcare Solutions\TouchWorks Web\Touchworks Print Center\)
Ensure a valid Web Server is listed and that TouchWorks is listed as the AppGroup, and then click “Connect.”
Select the desired Print Server, and click “Edit.”
Select the desired printer, and click Synch.
September 18, 2018

30. September 18, 2018

31. TWAdmin Printing Workspaces
Print Queue
Shows all jobs: previewed, printed, faxed, etc.
Filter display
Personalize vs. Search
Tailored for each login
Poll 1 – asking about message server administration
The Print Queue page by default shows all jobs for all message types in all statuses
Do DEMO of print queue page to show filtering, searching capability, and tabs, rerouting, cancelling, and resubmitting jobs
September 18, 2018

32. TWAdmin Printing Workspaces
Print Queue
Jobs archive daily at 7 am
SQL job CH Printing Queue Purge - Works
September 18, 2018

33. TWAdmin - Print Queue
September 18, 2018

34. TWAdmin Printing Workspaces (cont.)
Print Servers
Configure print/fax servers
Set up “virtual” print server listings
AutoPrint Defaults
Defines actions for distribution and printing
Print servers tab is used to set up virtual print server listings
Pulls a listing of the printers at the IP/server name you give
Name is totally arbitrary, can be anything you want
Can list same server multiple times, used for segregating printers by site, etc.
Also used to configure these virtual servers
Autoprint defaults tab is where default actions are set for printing items
Can set by site, provider, etc.
Can set up automatic carbon copies
September 18, 2018

35. TWAdmin - Print Servers by Location
Note that all 4 servers listed have the same server name
September 18, 2018

36. TWAdmin - Print Servers by Location
Note that all 4 servers listed have the same server name
Enable faxing checkbox will allow end-users to fax. Without it being checked on any server, end-users won’t be able to fax at all.
September 18, 2018

37. TWAdmin - Print Servers
September 18, 2018

38. TWAdmin - Print Servers
September 18, 2018

39. TWAdmin - Print Servers
September 18, 2018

40. TWAdmin - AutoPrint Defaults
September 18, 2018

41. Location-Aware Printing
Printing – user-based versus location-based
Prevents printing sensitive patient data to wrong location
Excellent for patient-centric workflows
The EHR's user-based Print Defaults may cause printing of sensitive patient data to the wrong printer as providers and staff often move from room to room
LAP enables Print Defaults to be based on each PCs' location rather than EHR User's Print Defaults
LAP allows you to print based on where you are vs who you are. Groups have various reasons for printing based on location, particularly to exam rooms as providers and staff move throughout the office as they see patients. The most critical need for LAP comes with groups using Patient Centric Workflows, where providers and staff go to the patient, moving room to room, rather than the patient moving to where staff are in the office.
September 18, 2018

42. LAP Example Set the location (computer name) Select the calling point
Select the subtype
Select the desired printer to default the print dialog to
September 18, 2018

43. Delta Application
Handles transfer of information between client and Allscripts
SSL
One Delta process per environment
New Delta released in version
Doesn’t require SQL Client Tools
Communicates over SSL to Allscripts
Updates content in reference databases
Sends utilization data to Allscripts - The Delta Export utilization data is typically sent out by your account/relationship manager on a monthly basis. If you aren’t receiving it regularly, please follow up with them.
New Delta appears the same but is optimized; no longer uses BCP (SQL Client Tools)
September 18, 2018

44. Delta Application Verify PASS daily Day Close Export (production only)
Transfer
Import
Manage log files
Move them to a new drive
Auto-archive
September 18, 2018

45. Delta Update Manager
September 18, 2018

46. September 18, 2018

47. September 18, 2018

48. September 18, 2018

49. September 18, 2018

50. September 18, 2018

51. Thank you for joining us today. For additional assistance:
You can contact us through our website at
Poll 3 – asking about Galen services