1. XenClient Enterprise 4.5 Diagnostics

2. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 2 Table of Contents Engine and Synchronizer Problem ReportingPage 3 Problem Reports and Technical Support Service RequestsPage 4 Problem Reporting FAQPage 5 Engine Problem Reporting AppletPage 6 Engine Problem Report FormPage 7 Engine Problem Report StatusPage 8 Engine Diagnostics and SynchronizerPage 9 Requesting Computer Diagnostics from SynchronizerPage 10 Collecting Computer Diagnostics from SynchronizerPage 11 Engine Debug LevelsPage 12 Engine TCP DumpsPage 13 Synchronizer Problem ReportsPage 14 Synchronizer Problem Report FormPage 15 Synchronizer Problem Report TasksPage 16 Generating Synchronizer DiagnosticsPage 17 Synchronizer Publish DiagnosticsPage 18 Synchronizer Log FilesPage 19 Synchronizer Installation and Upgrade LogsPage 20 Windows Crash DumpsPage 21 Manual Crash Dump GenerationPage 22 Log Files in Windows Virtual MachinesPage 23 Windows Ring Buffer TracingPage 24

3. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 3 Engine and Synchronizer Problem Reporting The XenClient Enterprise Engine and Synchronizer both have built-in features for reporting problems directly to the Citrix support and product teams. Here is how it works: XCE Synchronizer XCE Engine User submits report from Engine or Synchronizer. XCE Report Server Report is uploaded to a cloud-based report server over HTTPS. Reports are available to Citrix support and product engineers. In order to submit a problem report, the Engine or Synchronizer must be connected to a network with Internet access. Any firewalls or HTTP proxies in the network must allow HTTPS uploads to the host “nxbr.virtualcomputer.com” (this hostname has historical significance).

4. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 4 Problem Reports and Technical Support Service Requests Submitting a problem report from Engine or Synchronizer does not automatically create or update a Citrix Technical Support service request (SR). If you require Technical Support assistance with a XenClient Enterprise issue, a service request should be opened before submitting the problem report. The service request ID (or SR number) should be included in the subject of the problem report. If it is not practical to open a SR before submitting the problem report, then you can submit the problem report first. When you open the SR, indicate that a problem report was submitted, and include the following information so it can be found: Whether it was an Engine or Synchronizer problem report. Approximate date and time the problem report was submitted. Subject of the problem report. Email address of the person submitting the problem report.

5. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 5 Problem Reporting FAQ What is contained within a problem report? Mostly log files and configuration data. If you are curious you can request diagnostics from the Synchronizer and look at the contents. Ensure to exclude personal data from the problem reports. Does a problem report open a Service Request with Citrix technical support? No. See here for details. If you require technical assistance with XenClient Enterprise, you must open a Service Request through the usual channels. Problem reports enhance, but do not replace, the Service Request process.here Does Citrix really look at the problem reports? Yes. Every problem report is reviewed to see if it is something we must investigate further or follow up on. What kind of response should I expect from Citrix after submitting a problem report? It depends. We receive many problem reports and it is not possible to respond to every one. But we might ask for more information, or suggest a possible fix or workaround for the reported problem. We can only respond if a valid email address is provided in the problem report. How do I track the status of my problem report? If you are submitting a problem report for an issue that must be tracked to resolution, open a service request with Citrix Technical Support. The service request can be used to track the issue. There is currently no way for the report submitter to update or review the report after it is submitted. Why does the Engine problem report get stuck in “Pending” status? Usually this means the computer is not connected to the internet, or something in the network (like a HTTP proxy) is blocking the upload. You can collect Engine diagnostics from the Synchronizer console instead (details), or move the computer to a network with less restrictive access to the internet.details Why is the report server “nxbr.virtualcomputer.com”? This name has historical significance, as the XenClient Enterprise product evolved from the NxTop product developed by Virtual Computer, Inc.

6. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 6 Engine Problem Reporting Applet Click the “Problem Reporting” applet. Start the XCE Engine control panel. Select “Tools by Category”.

7. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 7 Engine Problem Report Form Subject Provide a brief but descriptive subject. If the problem report is related to a support case or service request, include the case or service request number. Description The description should include: Detailed description of the problem Summary of events leading up to the problem Error messages displayed in the Engine What was done to recover from the problem? Who requested the problem report? Email Enter your email address so we know who submitted the report. Priority The desired priority and should be based on how the problem is impacting your business operations. Reproducibility Indicate how reproducible the problem is. Submit Click Submit for submitting the problem report.

8. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 8 Engine Problem Report Status Pending Status When a bug report is initially submitted, it will first go into “Pending” status. This means the Engine is processing the problem report and uploading it to the report server. Success Status The problem report will transition to “Success” status after the report has been generated and uploaded to the report server. Wait For Success Before Restarting If you are required to restart or shutdown the computer, wait for the problem report to reach “Success” status first. Otherwise the problem report would not get submitted, and interesting information about the state of the computer might get lost. Reports Stuck in Pending Status Problem reports can get stuck in “Pending” status if the computer does not have network access, or if a network firewall blocks uploads to the report server. The Engine might also have problems with some HTTP proxy servers.

9. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 9 Engine Diagnostics and Synchronizer Engine diagnostics can be requested from the Synchronizer console. The content of the diagnostics is the same as an Engine problem report. This might be useful in the following circumstances. The computer is connected to a network that does not have internet access, or that has a firewall or HTTP proxy that might block the report from being uploaded to the report server over the internet. The engine policy is configured to block access to the launcher screen and control panel, so the problem reporting applet is unavailable. The computer is in a remote location and cannot be easily accessed (the remote access feature might be helpful here).

10. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 10 Requesting Computer Diagnostics from Synchronizer Select the computer in the navigation panel. Click “Request Diagnostics”. Select the “Diagnostics” tab to see the status of the diagnostics request.

11. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 11 Collecting Computer Diagnostics from Synchronizer After requesting the computer diagnostics, the request goes into a pending state, during which the following steps occur. Synchronizer waits for Engine to check for updates. Engine gets the message that it should generate diagnostics. Engine generates a diagnostics package. Engine uploads the diagnostics to Synchronizer. After the Engine uploads diagnostics to Synchronizer, a link appears which can be used to download them. Ensure that the request state is “No Request Pending” otherwise you might get an old set of diagnostics.

12. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 12 Engine Debug Levels The debug level for Engine logging is controlled in the Problem Reporting section of the control panel. The available settings are: OFF: This is the default setting and is usually sufficient for most troubleshooting purposes. VERBOSE: Occasionally requested, usually for wireless networking issues. VERY VERBOSE: This debug level can result in unusably verbose logs and should only be set when specifically requested. If verbose logs are requested, the debug level should be set before reproducing the problem. The debug level is reset automatically after a period of time, or when the Engine restarts.

13. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 13 Engine TCP Dumps A TCP dump in the Engine can be started from the problem reporting control panel. The TCP data cannot be extracted directly from the Engine, but it is included in Engine problem reports or diagnostics. To collect TCP dump data for a networking problem: Turn on the TCP dump as shown. Reproduce the problem. The TCP dump buffer file has a limited size so the problem should be reproduced quickly. Submit a problem report or collect Engine diagnostics from Synchronizer. The TCP dump is automatically reset after a period of time, or when the Engine restarts. TCP dumps from the Engine are often requested in conjunction with TCP dumps taken in a Virtual Machine using a tool like WireShark (http://www.wireshark.org).http://www.wireshark.org

14. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 14 Synchronizer Problem Reports Synchronizer diagnostics can be generated for the primary Synchronizer server, or a remote Synchronizer server. First, select the server. Click “Submit Bug Report”. Select the “Submit a Bug Report” option if you want to generate Synchronizer diagnostics and upload them to the Citrix report server. Select “Generate Diagnostics Only” if you want to generate Synchronizer diagnostics without uploading them to the Citrix report server.

15. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 15 Synchronizer Problem Report Form Subject Provide a brief but descriptive subject. If the problem report is related to a support case or service request, include the case or service request number. Description The description should include: Detailed description of the problem Summary of events leading up to the problem Error messages displayed in the console What was done to recover from the problem Who requested the problem report Email Enter your email address so we know who submitted the report. Priority The desired priority and should be based on how the problem is impacting your business operations. Reproducibility Indicate how reproducible the problem is. Date and Time Indicate the date and approximate time when the error occurred.

16. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 16 Synchronizer Problem Report Tasks When a request for Synchronizer diagnostics is submitted, a task is created to handle the request. The initial task state is “Queued” and it might take up to a minute to start running. When the task is complete, the diagnostics are ready. When the task is complete, the diagnostics have been uploaded to the Citrix report server. They can also be downloaded from the Synchronizer.

17. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 17 Generating Synchronizer Diagnostics To generate Synchronizer diagnostics without uploading them to the Citrix report server: Select the primary or remote server in the console navigation panel. Click “Submit Bug Report”. Choose the “Generate Diagnostics Only” option. A background task is created to handle the request. When the task is complete, the diagnostics are available to download from the Synchronizer console.

18. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 18 Synchronizer Publish Diagnostics There is a special set of diagnostics available from the Synchronizer for publish failures. When a publish operation fails, these diagnostics can be downloaded from the failed publish task. Locate the failed publish task in the Synchronizer console. Click the indicated icon to display the task details. The icon looks like this: Click the link to download publish diagnostics. The publish log might also be useful to review. Publish diagnostics cannot be uploaded to the Citrix report server from the Synchronizer. If Citrix support asks for publish diagnostics, you must retrieve them manually then send them through email or upload to ShareFile or a FTP server.

19. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 19 Synchronizer Log Files Whenever possible, Synchronizer diagnostics should be generated through the Synchronizer console using the methods outlined previously. But sometimes it might not be possible to collect Synchronizer diagnostics, for example, if a problem with the Synchronizer prevents the console from starting properly. In these cases the Synchronizer log files can be collected directly. The default folder for Synchronizer log files is: C:\Program Files (x86)\Apache Software Foundation\Tomcat 6.0\Logs If you are asked to collect Synchronizer diagnostics, but cannot collect them through the Synchronizer console, zip up this folder instead and send it in. This folder might be several megabytes in size so you might be required to use ShareFile or a FTP server to submit them.

20. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 20 Synchronizer Installation and Upgrade Logs When running the Synchronizer installation program to install or upgrade Synchronizer, the following log files are created which might be useful in troubleshooting install or upgrade problems. The Synchronizer install program will create a log file in the Temp folder for the user running the installer. Files beginning with “bitrock_installer” are Synchronizer install log files. The installer can also generate log files in the Synchronizer installation folder. But if the initial installation of Synchronizer fails, this folder is usually deleted as part of the post-install cleanup process.

21. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 21 Windows Crash Dumps If a Windows Virtual Machine crashes or blue-screens, it should generate a crash dump file into one of the following locations: L:\NxTop\KERNEL.DMP C:\KERNEL.DMP C:\Windows\MEMORY.DMP Note that “L:\NxTop” is a hidden folder. The dump file might have a different name but it should end in a “.DMP” suffix. These files can be large so they should be uploaded to FTP or ShareFile, not sent as email attachments.

22. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 22 Manual Crash Dump Generation Sometimes it might be useful to force Windows to crash and generate a dump file. This can be done by enabling the hotkey labeled “Platform: Crash VM for Debug” in the Engine control panel. Open the Engine control panel and choose “Tools by Category view”. Open the “Hot Keys” applet. Enable the hotkey labeled “Platform: Crash VM for Debug”. Bring the Virtual Machine you want to crash to the foreground. Enter the hotkey, this should cause the Virtual Machine to blue-screen.

23. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 23 Log Files in Windows Virtual Machines The following log files are available in Windows Virtual Machines running on XCE Engine. If you are asked for “Windows VM log files”, collect all of these. If you are only asked for the “NxTopMonitor.log” file, the others are optional. L:\NxTop\Logs\NxTopMonitor.log This file contains lots of useful information about how the Virtual Machine is interacting with the Engine and can be very helpful in troubleshooting issues with monitors, networking, optical drives, and many other problems within a Windows Virtual Machine. Note: L:\NxTop\ is a hidden folder. Some Virtual Machines might have the local drive mapped to a different drive letter. C:\Program Files\NxTop PV Drivers\ This folder is in “Program Files (x86)” on 64-bit Windows. Get these files: setupapi.dev.log install.log DPINST.LOG C:\Windows\inf\setupapi.dev.log This file contains the latest record of device installs in Windows.

24. XenClient Enterprise 4.5 Diagnostics Copyright © 2013 Citrix Page 24 Windows Ring Buffer Tracing The NxTop PV Drivers package includes a utility that can be used to dump the Windows kernel ring buffer. This is often useful when troubleshooting issues in a deployed Virtual Machine. In particular it can be useful when USB devices are not working as expected. 1.Start a command prompt with Administrator privileges. 2.Navigate to one of these folders: 32-bit Windows: C:\Program Files\NxTop PV Drivers 64-bit Windows: C:\Program Files (x86)\NxTop PV Drivers 3.Change directory into the “winxp”, “winlh”, or “win7x64” folder. 4.Change directory into the “vcilog” folder. 5.Run this command to start the trace: vcirb.exe > L:\ringbuffer.log 6.Reproduce the problem while the trace is active. 7.To stop tracing, press ENTER in the command prompt window. 8.Send L:\ringbuffer.log to whoever requested the tracing. The trace output file “L:\ringbuffer.log” is an example. You can save it wherever you like.