Copyright 2009 Trend Micro Inc. Classification 9/23/ Troubleshooting TMSP Marks Shen Senior Engineer – QA Evan Wang Engineer - QA

Copyright 2009 Trend Micro Inc. Classification 9/23/ Agenda  Frequent Case  Debug log and information  Troubleshooting  Q&A

Copyright 2009 Trend Micro Inc. No report was generated Logs need to collect on Daemon Server –/opt/TrendMicro/tdss/tdes/log/iae_log.txt –/opt/TrendMicro/tdss/tdes/reports/tdes.log(Since 2.5R3) –/var/log/cron Information –Customer expiration date –Device register to TMSP

Copyright 2009 Trend Micro Inc. No report was generated Cont. Normal debug log for log correlation and report generation Crontab task cron_iae.sh will be executed at 2:15 am every day

Copyright 2009 Trend Micro Inc. No report was generated Cont. Report will not be generated if Customer service get expired Customer without device registered

Copyright 2009 Trend Micro Inc. No incident in report Logs need to collect on Daemon Server –/opt/TrendMicro/tdss/tdes/log/iae_log.txt Information –Check if TDA log has been uploaded Latest log time

Copyright 2009 Trend Micro Inc. Cannot access Admin console (err 404) Logs that need to collect on Daemon Server –/var/log/httpd/access_log –/var/log/httpd/error_log Information –ps –ef | grep httpd –netstat –anp | grep httpd

Copyright 2009 Trend Micro Inc. No Rsync log uploaded Logs need to collect on Access Server –Log receiver /var/log/messages /home/tdalog/log/pre-post-exec.log /home/tdalog/log/db_import_tda.log /home/tdalog/log/db_import_tdm.log –Authentication (describe in next sides) Information –ps –ef | grep tmsshd –netstat –anp | grep tmsshd Listen on port 22

Copyright 2009 Trend Micro Inc. No Rsync log uploaded Cont. normal log of tmsshd and rsync Classification 9/23/2015 9

Copyright 2009 Trend Micro Inc. No Rsync log uploaded Cont. Normal debug log of TDA log processing –/home/tdalog/log/pre-post-exec.log

Copyright 2009 Trend Micro Inc. CAS server case Problems caused by CAS failure : –Device register to TMSP fail –Customer portal login fail (only before R3) –Log uploading fail through RSYNC Log on Access Server: –/var/log/messages –/var/log/cas_8000.log –/var/log/cas_8001.log –/var/log/cas_8002.log Information: ps –ef | grep pound ps –ef | grep rubcasd Normal log of CAS authentication –/var/log/cas_8000.log

Copyright 2009 Trend Micro Inc. CAS server case – Service down TDA register fail Check /var/log/messages Recover –If pound or cas service is down /etc/init.d/pound start /etc/init.d/rubcasd start

Copyright 2009 Trend Micro Inc. Data Gateway Case Problems caused by Data Gateway failure : –OCS Heartbeat / OCS log cannot be handled –T2 / T3 mitigation request cannot be delivered to TMTM –SIC sample cannot be handled Logs on Access Server: –/opt/TrendMicro/dg/apache-tomcat /webapps/dg/WEB- INF/logs/dg.log –$APACHE_HOME/logs/ssl_request_log_dg –$APACHE_HOME/logs/error_log –/opt/TrendMicro/dg/apache-tomcat /logs/catalina.out Information: ps –ef | grep httpd ps –ef | grep tomcat netstat –anp | grep 443 netstat –anp | grep 8009 netstat –anp | grep 8080

Copyright 2009 Trend Micro Inc. Data Gateway Case – DB disconnect TDA register fail Check apache error log: /usr/apache/logs/error_log

Copyright 2009 Trend Micro Inc. Customer portal cannot login Logs need to collect on Access Server: –/opt/TrendMicro/dg/apache-tomcat /webapps/tms2/WEB-INF/logs/tms.log –$APACHE_HOME/logs/ssl_request_log_portal –$APACHE_HOME/logs/error_log –/opt/TrendMicro/dg/apache-tomcat /logs/catalina.out Information ps –ef | grep httpd ps –ef | grep tomcat netstat –anp | grep 443 netstat –anp | grep 8009 netstat –anp | grep 8080

Copyright 2009 Trend Micro Inc. Cannot get notification Exclude Mail server problem, collect debug logs: –/root/infomation.log –/var/log/cron No Subscription DB connection fail

Copyright 2009 Trend Micro Inc Classification FAQ Why no daily report can be found from web UI? 1. Check TDES log from #tail -n 100 /opt/TrendMicro/tdss/tdes/log/iae_log.txt If content like "Daily report: customer_ID, JP" cannot be found, that mean the scheduled job has not started so far. 2. Daily report is auto generated at 7:15 am every day, so check the system time of TDES: 3. Report generation need take some time, please check if the report is generating: #ps -ef | grep php if some php process is running, it means the reports are under generating. 4. Check if customer has expired for TMSP service, find the profile “expire time” from admin console

Copyright 2009 Trend Micro Inc. FAQ Why there is no data in report? 1. Check if the log has been uploaded to TMS and imported into Database Login log receiver machine and check the file last modification time #ll /home/tdalog/userdata/USERID/DEVICE_GUID/*.db If not latest data, that means TDA did not upload logs. 2. Log in database and query yesterday's log Some times, TDA did not detect any events, if so, there will be no yesterday's data in DB 3. Check iae_log.txt, check if “Running TDES 2.1 for XXXX (device=50)(customer_id=30) on date: ” existing, this means TMS run IAE for this customer. if NO this content exist, means there is something wrong when process IAE.

Copyright 2009 Trend Micro Inc. FAQ How to re-generate report manually? Login TDES machine, change dir to "/opt/TrendMicro/tdss/tdes/"; –1. Daily Report #php gendailydata.php user_id YEAR MONTH DAY example: "php gendailydata.php trend " generate daily report of for customer "trend“ –2. Executive Report (Weekly / Monthly) #php genexecdata.php user_id START_DATE START_DATE yes m/w START_DATE: report start data with format “YYYY-MM-DD” START_DATE: report end data with format “YYYY-MM-DD” yes: if this report will be imported into DB m/w: monthly or weekly example: "php genexecdata.php trend yes m" to generate monthly report for customer "trend" of –3. Upsell Report # php genupselldata.php user_id START_DATE END_DATE no example: " php genupselldata.php trend no" to generate upsell report for customer "trend" from to Note: Upsell report will not be imported into DB and cannot download from admin console After execute these command, reports will be re-generated and imported into database for downloading

Copyright 2009 Trend Micro Inc. FAQ What’s the steps to deploy new report php file? Sometimes, reports generation related php need to be modified and deploy to TDES, here are the steps to do this: 1.back up old php file 2.replace with new php file 3.remove cache_*.php under TDES installation folder 4.Re-generate report to verify new php files if necessary

Copyright 2009 Trend Micro Inc. Q&A

Copyright 2009 Trend Micro Inc. Classification 9/23/ THANK YOU!