Part 1: Leveraging Logs
Many of the issues we troubleshoot with our customers involve errors that vary in how and when they show up within an application. When something is not working the way it should, log files are a great place to look for clues. Here are a few pointers we share with customers:
- Logs are your friend – Spend a bit of time with your logs. When all is good, logs provide a great baseline and can also let you tune out some of the no/low impact errors that can show up in log entries. Environments change over time and so can the baselines, so keep in touch.
- Keep logs current – There’s little use keeping old logs unless your business processes require it. Creating tasks or routines to purge old log files is a good practice, especially if these files are large. Another way to keep logs current is to have them save more frequently, creating smaller files. It is easier to pinpoint an error in a log that is created daily rather than sifting through a log containing a week’s worth of data.
- Use tools to help read logs – Our log files are text and can be easily read in Notepad. Using smarter text readers like CMTrace or Notepad++ allows for better presentation of the logged data. CMTrace will color-code line entries that contain the word ‘error’ or ‘warning’. Notepad++ shows the text with line numbers for easier referencing.
- Cireson’s SMP Helper tool – Our support team developed a very useful tool for bringing common troubleshooting tasks together. Check it out and download the app for free on our Community site.
A couple of examples of logging in our Console apps are shown below in Notify Analyst Settings and Asset Management workflow settings. In these cases, a log path is created to capture events triggered by the application. These can be very helpful in pinpointing when a problem occurs and reporting back errors.
As log files are created for each event, a large number of files can build up.
In the Cireson Portal, the two key log files used for troubleshooting are the webconsole log and cachebuilder log. Logging to these files can be configured as required. In normal use, the logging level is set to ‘ERROR’ to just capture error events. When troubleshooting an issue, the level should be set to ‘ALL’ to record all events in the log. As these can get quite large, it is best to return the levels back to ‘ERROR’ after a complete log is captured. More information on how to change configuration settings can be found in our support KB articles.
The webconsole.log file will capture any activity that takes place in the web portal – click, updates, navigation, etc. This is a helpful log for issues that show up as webpage errors.
The Cachebuilder.log file is tracking activity related to the cachebuilder service, which is the service that syncs Service Manager data to the Cireson (ServiceManagement) database. Issues with permissions, scoped access, work items and configuration items are usually tracked here. The log is written to periodically and whenever the cachebuilder service is restarted.
As mentioned earlier, SMP helper simplifies setting log configurations and a whole lot more.
We took a quick look at some examples of logging in our Console and Portal apps and how important they are to keep the system in good working order. System log monitoring goes far beyond those found in our applications. Many issues that show up in a Cireson app often stem from the environment. Event viewer, Server, networking and other logs can be helpful in getting to the root cause of an issue.
Whenever you can, always try to send the most relevant and complete log when submitting an incident to Cireson Support.