5,838
edits
Monthly log files: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
→File Locking restricting access by other processes
mNo edit summary |
|||
| Line 19: | Line 19: | ||
For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”. | For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”. | ||
= | = Accessing the monthly log file = | ||
As the log file for the current month will be updated frequently by Cumulus (when it is running), Cumulus applies an exclusive lock, and (as explained further below) conflicts can happen if another process seeks to access this file. Consequently don't let your antivirus scans access this file, nor try to edit it outside Cumulus while Cumulus is running. | == File Locking restricting access by other processes == | ||
As the log file for the current month will be updated frequently by Cumulus (when it is running), Cumulus applies an exclusive lock, and (as explained further below) conflicts can happen if another process seeks to access this file. Consequently don't let your antivirus scans access this file, nor try to edit it outside Cumulus while Cumulus is running. | |||
Also note these log files do not include a header line, | Also note these log files do not include a header line, and should not be edited to include it. | ||
The appropriate header file for the monthly data logs can be found for any installed version of Cumulus by looking for "monthlyfileheader.txt", in the directory above the 'data' directory containing the monthly log. The viewing methods for the two flavours, as described next, automatically includes the header line. | |||
A full discussion of the problems with conflicts of access to the monthly log file can be found [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=12721 in this support forum topic]. | |||
== Cumulus 1 == | == Cumulus 1 == | ||
| Line 40: | Line 46: | ||
== Problems == | == Problems == | ||
If no logs seen, see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. Also note, this '''Data logs''' viewer is only for viewing monthly log files (not any other files in the same ''data'' directory), the column headings for standard monthly log files will always be shown at the top of the screen. | If no logs seen, see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. Also note, this '''Data logs''' viewer is only for viewing monthly log files (not any other files in the same ''data'' directory), the column headings for standard monthly log files will always be shown at the top of the screen. | ||
== | If you try to look at the current log, and it does not display in the MX admin interface, it will be because you are trying to view it at the same time as the MX engine is writing the next line to it. The exclusive locking stops the admin interface and the engine being able to both access the same file at the same time. So if your logging interval is every ten minutes, don't try to view it within 2 minutes of the hour and within 2 minutes of every ten minutes thereafter. That still gives you 6 minutes between each 10 minute update for viewing! Of course there is no conflict for past monthly logs. | ||
===Uploading current log to your | |||
The current monthly log can be updated at your usual | == Additional ideas for how Monthly log files can be used within Cumulus == | ||
=== Uploading current log to your web site === | |||
The current monthly log can be updated at your usual web site update interval using ''<currentlogfile>'' as the local file name. If your web site upload interval is the same as the logging interval, there might be a conflict because of the file locking, so this might not work every time. However, extra web files are not processed all at once, so by listing this file against a higher number the chance of conflict is reduced. | |||
For Cumulus 1, list this log file as the local file by specifying the parameter 'ExtraLocal[0-99]=<currentlogfile>' in [[Cumulus.ini#Section:_FTP_Site]]). For Cumulus MX, use the '''Extra web files''' option within the "Settings" tab, and enter "<currentlogfile>" in the local column. | |||
You can also use <currentlogfile> as part of the remote file path, as explained in the Cumulus '''Help'''. You will also need to specify a path in the remote column for MX or the 'ExtraRemote[0 to 99]=...' for Cumulus 1. You might of course want to specify a local path, and leave the FTP column unticked (or set to 0 for Cumulus 1). You can then use a separate FTP process to upload this copy. It is not recommended that you do not try to FTP the actual log file (only a copy) because the FTP process is relatively slow, particularly as the file size increases as the month progresses, so there is an increasing chance of conflict with logging - see [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=12721&p=99793#p9979 support forum here] for more information. An alternative approach is to upload the monthly file just once a day, at a quiet period such as an odd number of seconds after quarter-past midnight using a process scheduled outside Cumulus. | |||
=== Cumulus 1: transferring the contents of the monthly log file into a database table === | |||
A script can | A script can be used to read the your copy of the monthly log file into either a database table (see [[ImportCumulusFile]]), or into an array, for further processing. It is, as advised above, best not to work with the file that Cumulus is actually updating, because there can be a conflict of the locks, Cumulus needs to have exclusive use when it does its normal update. But this script can safely be used with older monthly log files, or a copy of the latest monthly log file. | ||
Although the '''ImportCumulusFile''' script can be used with MX too, MX provides 2 other ways to transfer the contents of a monthly log file into a database table. The first, uploads past data, using '''ExportMySQL.exe''' found in the top level folder where ''CumulusMX.exe'' is found. It can be run using the same way as you run the main program, but you need to specify "monthly" as the first parameter. The second, which only uploads the line most recently added to the monthly log file is found in the '''My SQL''' settings section of the admin interface. Find more information at [[Cumulus_MX#MySQL_settings|MySQL settings]]. | |||
== Editing the log file == | == Editing the log file == | ||
* You can't edit the current log file while Cumulus is running, because Cumulus needs exclusive write access. You can edit the log file for a past month, unless you are looking at past months using Cumulus editors. | * You can't edit the current log file while Cumulus is running, because Cumulus needs exclusive write access. You can edit the log file for a past month, unless you are looking at past months using Cumulus editors. | ||
* You can't edit any log file with a word processor, as they add control characters and other information that Cumulus cannot understand. | * You can't edit any log file with a word processor, as they add control characters and other information that Cumulus cannot understand. | ||
