Unified Agent - Everything You Need to Know About Logging Configuration
As a customer it's almost certain that you've been asked by your friendly Mend Technical Support Engineer at least once during your relationship with us to increase logging levels to try and resolve an issue and that perhaps this lead to questions about how logging works. The purpose of this article is to explain to you everything you need to know about Unified Agent (UA) logging including capabilities, configuration, execution and other key pieces of information that will help you manage logging so that your configuration suits your needs.
Logging Levels:
If you've ever worked with a Java application that uses log4J logging, our logging options should look at least a bit familiar to you. They are (in descending order of verbosity):
TRACE
DEBUG
INFO
WARN
ERROR
The back-end configuration for each of these options is very complex, but at a high level each logging level includes the previous levels plus additional logging messages based on our server configuration. The two you will most likely use are INFO and DEBUG. INFO is standard logging and DEBUG is more robust and includes more detail about each step. At the INFO level, the UA will let you know each step it's starting and completing, as well as potentially provide very high level detail about the steps in between. For example, here is the step where the UA scans your repository looking for files that match your include pattern with INFO logging enabled:
------------------------------------------------------------------------
-------------------- Start: Scan Files Matching Includes Pattern -------
------------------------------------------------------------------------
[INFO] [2019-09-06 11:13:27,087 -0400] - Attempting to resolve dependencies
[INFO] [2019-09-06 11:13:27,087 -0400] -
In DEBUG mode, however, the UA will return detailed information about each step. Here's an example of the same step from a DEBUG log:
------------------------------------------------------------------------
-------------------- Start: Scan Files Matching Includes Pattern -------
------------------------------------------------------------------------
[DEBUG] [2019-09-06 11:34:29,493 -0400[ - Scanning C:\Tools\Dev\guava-master
[DEBUG] [2019-09-06 11:34:29,509 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\android\guava\pom.xml
[DEBUG] [2019-09-06 11:34:29,563 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\android\guava-bom\pom.xml
[DEBUG] [2019-09-06 11:34:29,564 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\android\guava-testlib\pom.xml
[DEBUG] [2019-09-06 11:34:29,613 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\android\guava-tests\pom.xml
[DEBUG] [2019-09-06 11:34:29,641 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\android\pom.xml
[DEBUG] [2019-09-06 11:34:29,642 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\futures\failureaccess\pom.xml
[DEBUG] [2019-09-06 11:34:29,648 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\futures\listenablefuture1\pom.xml
[DEBUG] [2019-09-06 11:34:29,652 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\futures\listenablefuture9999\pom.xml
[DEBUG] [2019-09-06 11:34:29,653 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\guava\pom.xml
[DEBUG] [2019-09-06 11:34:29,984 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\guava-bom\pom.xml
[DEBUG] [2019-09-06 11:34:29,985 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\guava-gwt\pom.xml
[DEBUG] [2019-09-06 11:34:30,123 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\guava-testlib\pom.xml
[DEBUG] [2019-09-06 11:34:30,176 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\guava-tests\pom.xml
[DEBUG] [2019-09-06 11:34:30,219 -0400[ - Dependency file found location: C:\Tools\Dev\guava-master\pom.xml
[INFO] [2019-09-06 11:34:30,221 -0400[ - Attempting to resolve dependencies
Logging Configuration
There are two types of logging that can be configured with the UA:
1. Console logging (the logging you see when executing the UA from the command line), including redirecting console output to a file
2. Log files (in UA versions 19.7.2 and higher)
Let's start with the former.
Console Logging:
There are two ways to configure console logging levels:
1. From the command line by adding -logLevel <level>
2. Within your UA config file by adding log.level=<level>
Important things to note:
1. The parameter names are case sensitive but the log level is not
2. The command line logging level will supersede the config file log level - this is helpful in cases where you might want to have lower level logging by default in a build pipeline but might want to grab a DEBUG log for a problematic repo scan to provide to Support. You can set the default config file to INFO and then run with -logLevel debug from the command line to get a single DEBUG run.
Log Files:
In UA versions 19.7.2 and higher you are also able to configure log files to be automatically generated in the background. This is configured solely in the configuration file and the console parameters above do not affect these settings.
There are three parameters you can add to your config file to control automatic log file configuration:
1. log.files.level - the verbosity of the stored log files, this accepts the same values as logLevel but also includes Off which deactivates this logging (default, DEBUG1)
2. log.files.maxFileSize - the maximum size of each log file generated (default, 10 MB)
3. log.files.maxFilesCount - the maximum number of log files to store (default, 3 files)
Important things to note:
1. Log files are automatically generated and created in the following locations:
a. In versions 19.7.2 to 19.7.3.3:
a1. C:\Users\<WindowsUser>\AppData\Roaming\UA on Windows systems
a2. /var/log/UA on Unix systems
b. In versions 19.8.1 and later:
b1. The same top level folder structures will be created in 19.8.1 and later
b2. A subfolder will then be created for each execution of the UA with the following structure: Day-Month-DayOfMonth-Year-Time
b3. A separate log file will be created based on the log.files.maxFileSize and log.files.maxFilesCount parameters - whitesource_<n>.log (the maximum n value = log.files.maxFilesCount parameter value
2. By default this log file generation (DEBUG level) is enabled in UA versions 19.7.2 and above even if the log generation parameters are not expressly set in the UA config file. Should you wish to completely disable this automatic log generation feature, add the log.files.level= parameter to your config file and set the value to "Off"
3. When log.files.level is set to Off you will see a 0KB log file generate in your log folder (see 1 above), but it will not be written to and it will be cleaned up (deleted) by the UA once the scan is complete.
4. When the log.files.maxFileSize or log.files.maxFilesCount have been reached, the respective files will roll over (meaning, the oldest will be deleted and a new one created)
5. log.files.maxFileSIze and log.files.maxFilesCount are on a per UA execution basis - not global
Hopefully this article has helped to demystify some of the expected behaviors, options and configuration around logging for the UA.
Happy scanning!
________________________________________________________________________________________________________________________________
1 The reason the default value for the automatic log file generation is set to DEBUG is that these logs are auto generated
for troubleshooting purposes. In the event that you need this log, INFO level logging would not be verbose enough to
diagnose issues. This troubleshooting log is intended to be generated automatically for you so that you would not need
to rerun your UA scan.