Logging Microservice Deep Dive Tech Talks – Session 9

Agenda What and Why a Logging Microservice? Logging Service Technology Spring/Java core Logback Wrapped Persistence options Basics of how it works Adding Logging to your microservice Configuration Logging objects REST API Adding log entries Query the logs Upcoming Tech Talks Q&A

Jim White EdgeX Core/Supporting Services WG Chair Dell Distinguished Engineer & Senior Software Architect From Dell End User Computing (EUC) CTO Chief Architect and lead developer of Fuse I wrote code line #1  Fuse => EdgeX Foundry james_white2@dell.com

Big Picture and Logging

Logging’s Role Provides centralized management of microservice log entries. Offering long term persistence of log messages for query/analysis/bug diagnosis Consolidated picture of participating microservice log events – assisting with broader issues Well defined API allows for improved/value add logging in the future (security auditing, push of logs to remote server,…) At a high level, the logging microservice Provides a RESTful API for other microservices to request log entries in a non-blocking manner with minimal impact to system performance Provides RESTful APIs to query, clear, or prune log entries based on any combination timestamp, log level, tag, origination, etc. Allows log entries to be associated to its originating service (to faciliate easy search/filtering). Supports multiple logging levels, for example trace, debug, info, warn, error, fatal, and so forth. Supports log entry tagging; tags can be anything dictated by the calling services. Persist log entries (in either file or database), and the persistence storage should be managed at configurable levels Take advantage of an existing logging framework internally and provide the “wrapper” for use by EdgeX Foundry Follow applicable standards for logging where possible Handle a mundane activity with little or now dev effort - allow developers the ability to stay focused on value add capability Use of the Logging microservice is optional! Microservices can (and do) log independently as needed This is often the case during development

Logging Implementation Details Created with Java/Spring Framework/Spring Boot Uses Spring MVC for REST communications Internally, the logging microservice utilizes LOGBack as its underneath logging framework https://logback.qos.ch/ “Successor” to popular log4j I.e. – the logging microservice is a wrapper around LOGBack Log entries can be persisted in (based on configuration) MongoDB (recommended) File system logs (used primarily for development) Client services can… Use the REST API to add/remove/query log entries Use a Java client library to incorporate the logging services

Logging Microservice Architecture

Logging Microservice Implementation MVC Controller (provides REST interface)

Configuring the Logging Microservice Property Default Value(s) Description logging.persistence file "file" to save logging in file; "mongodb" to save logging in MongoDB Following config only take effect when logging.persistence=file logging.persistence.file File path to save logging entries; ex: c:/users/public/edgex/logs logging.persistence.file.maxsize 10MB Threshold to roll and archive the logging file. It can be specified in bytes, kilobytes, megabytes or gigabytes by suffixing a numeric value with KB, MB and respectively GB. For example, 5000000, 5000KB, 5MB and 2GB are all valid values, with the first three being equivalent. Following config only take effect when logging.persistence=mongodb spring.data.mongodb.username logging MongoDB user name spring.data.mongodb.password password MongoDB password spring.data.mongodb.host localhost MongoDB host name spring.data.mongodb.port 27017 MongoDB port number spring.data.mongodb.database MongoDB database name

Using the logging services 2 choices Use the REST APIs directly If using Java, use the Logging Client Convenience classes/methods

Logging API See service API RAML for more details Make a log entry (POST) http://localhost:48061/api/v1/logs {"logLevel": "WARN", "originService": “core-data", "message": "Hello, Logging Service!", "labels": [“greeting", “non-problem"]} Fetch log entries (GET) http://localhost:48061/api/v1/logs/10 10 is the number of log entries you want returned Many query and delete log entry APIs. By… Time/time range Label(s) Originating service(s) Keyword(s) - text in entry Log level(s) [ERROR, WARN, INFO, DEBUG, TRACE] Many combinations of above Example: http://localhost:48061/api/v1/logs/keywords/rocks/0/1504809979476/10 Get log entries with keyword “rocks” between 0 & 1504809979476 timestamps, and limit 10 entry return See service API RAML for more details https://wiki.edgexfoundry.org/display/FA/APIs--Supporting+Services--Logging NOTE: TRACE entries are not persisted in MongoDB!!

Logging Client Add support-logging-client to your pom.xml See any existing Java service as an example In classes that need to log… Add the following static Add your log statement requests Add the appropriate configuration private static final org.edgexfoundry.support.logging.client.EdgeXLogger logger = org.edgexfoundry.support.logging.client.EdgeXLoggerFactory.getEdgeXLogger(YourClassHere.class); logger.error("Error getting command: " + e.getMessage()); logging.remote.enable=true logging.remote.url=http://localhost:48061/api/v1/logs #default origin service name is determined by app name spring.application.name=core-metadata

Upcoming Tech Talks Using the SDK to generate a device service Next week – presented by Tyler Cox (Dell) – 9am EST, Thursday Suspending Tech Talks after next week Focusing on Barcelona release Restart Tech Talks after Barcelona (mid – October) Suggestions welcome on other topics/direction of tech talks Email me suggestions james_white2@dell.com

Questions and Answer Time