Clean up Log directory in IBM Cognos TM1

As part of development, over a period of time we often see plenty of files created in the logs directory.  Typically these files consists of :

1. Errors encountered in execution of Turbo Integrator (TI) code
2. Saved files of tm1s.log (after SaveDataAll has been run or service is recycled)
3. AsciiOutput / TextOutput files processed as part of TI code for debugging
4. Files generated as required for upstream/downstream processing

Some sort of a method is required to clean these files periodically.  There is also a possibility of lot of disk space wasted to store these files on the server.

To configure the logs directory, you’d need to set LoggingDirectory parameter in the tm1s.cfg file.  While this parameter is optional, it is recommended to set it to a folder of your choice.  Typically this folder is a sibling to the data directory and my recommendation is that, path mentioned to this logs folder in the tm1s.cfg be relative, instead of fully qualified!

If the parameter is not specified then:

• By default it will write to the data directory, there by cluttering the directory
  
• If there is more than one DataBaseDirectory mentioned in tm1s.cfg, then it is written to the 1st one

So let’s see how a sample log directory looks after few weeks/months.  In our case, there has been not much development activity, since the minor errors are far less in number :)

 

We built a TI process that is scheduled to run every morning/week before we walk-in that clears up the log file.  Let’s look at the code.  There are 2 parameters for the process:

 ps_SearchStr – You’d need to specify the beginning part of the file name, which needs to be deleted.  Code does NOT use wildcards

pi_Days – Specify number of days.  Any file older than this, and with the above search pattern will be deleted.  Files lesser than this number will be retained.  If you want to delete all the files then enter pi_Days = -1, when running the process

Code here looks assumes that date of the file is embedded in the file name.  It is mainly written to cater deletion of time stamped tm1s logs as well as TM1ProcessError files.  Both of these have date as part of the file name.

Prolog

In this tab, we will delete those files that match the pattern of the 1st parameter (SearchStr) and whose date is greater than 2nd parameter (Days).

vi_StartIdx  = Long (ps_SearchStr) + 1;
vs_FileName  = ”;
vs_FilePath  = GetProcessErrorFileDirectory;
# Search for presence of given file
vs_FileName  = WildcardFileSearch (vs_FilePath | ps_SearchStr | ‘*.log’, ”);
WHILE (vs_FileName @<> ”);
# Get its date and subtract from today’s date.  If more than nn days – specified in the parameter, then delete it
vLogFileDate = DAYNO ( SUBST (vs_FileName, vi_StartIdx, 4) | ‘-‘ | SUBST (vs_FileName, vi_StartIdx + 4, 2) | ‘-‘ | SUBST (vs_FileName, vi_StartIdx + 4 + 2, 2)) ;
vTodayDate   = DAYNO(TODAY(1));
vDifference  = vTodayDate – vLogFileDate;
IF (vDifference > pi_Days);
AsciiDelete (vs_FilePath | vs_FileName);
vs_FileName = ”;
ENDIF;
# Search for the next file
vs_FileName = WildcardFileSearch (vs_FilePath | ps_SearchStr | ‘*.log’, vs_FileName);
END;

Epilog

Code written is mainly to delete other types of that get created.  As an example, all the .txt files created by AsciiOuput / TextOutput that developers put for debugging; perhaps few .csv files, .log files etc.  You’d need to alter the code to suit your needs

# Delete other .log files here, which users/developers would have created that is not required
vs_LogFile1  = ‘TM1ProcessError_’;
vs_LogFile2  = ‘tm1s’;
vs_FileName  = ”;
vs_FilePath  = GetProcessErrorFileDirectory;
vs_FileName  = WildcardFileSearch (vs_FilePath | ‘*.log’, ”);
WHILE (vs_FileName @<> ”);
# Ensure it is not one of the TI errors or the timestamped tm1s.log file
IF (Scan (vs_LogFile1, vs_FileName) = 0 & Scan (vs_LogFile2, vs_FileName) = 0);
AsciiDelete (vs_FilePath | vs_FileName);
vs_FileName  = ”;
ENDIF;
vs_FileName = WildcardFileSearch (vs_FilePath | ‘*.log’, vs_FileName);
END;
# Delete miscelaneous files like .txt, .csv, .cma files from log directory
# Below is an example for .txt alone
vs_FileName  = WildcardFileSearch (vs_FilePath | ‘*.txt’, ”);
WHILE (vs_FileName @<> ”);
AsciiDelete (vs_FilePath | vs_FileName);
vs_FileName = WildcardFileSearch (vs_FilePath | ‘*.txt’, ”);
END;