Monthly log files: Difference between revisions

From Cumulus Wiki
Jump to navigationJump to search
m (→‎Manipulation outside Cumulus: noting humidity is integer)
m (→‎Potential Name Confusion: rewrite of this section)
 
(95 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Category:Cumulus Files]]Please see [[Standard log files]], [[Extra Sensor Files]], [[Air_Link_Log.txt]], [[Month.ini]], or [[Monthlyalltime.ini]], as relevant.
Files are automatically created with names in the form <Month><Year>log.txt (for example, the file for August 2009 is called <tt>Aug09log.txt</tt>); an entry is made to the current month's file every ''ten'' (default value; you can change this on the station settings screen) minutes, recording the current sensor values. The file is in comma-separated format with one line per entry.


Please be aware that all of the above links lead to documentation about files that may be described as ''monthly log files'' when reading individual Cumulus forum and wiki references.
Times in these files are in the form hh:mm using the 24 hour clock and local time (system time). All data is logged in the units which which have been selected by the user on the station configuration screen. Note that 'comma-separated' does not necessarily mean that a comma is used to separate the values! If your PC uses a semicolon for the list separator, that will be used in these files.


==Potential Name Confusion==
For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”.


Those who used early versions of Cumulus will remember that the term '''Log File''' was originally used for [[Log.xml]], but Steve Loft later accepted that all the files with a [[:Category:Ini Files|.ini]] file extension could be described as log files, so he introduced the term [[Weather Diary]]. A bit later, [[Standard log files| the detailed data file]], began to be also called '''log files''', and the confusion grew. When [[Extra Sensor Files]] were added, these also became '''log files'''. The Wiki page now called "Standard Log Files" was first called "Log files", then became "Monthly Log Files" because "Log files" became a category. Then "Extra Sensor Files" were added and "standard log files" replaced "Monthly Log Files" to make it clearer which file held the standard values, and which file held values for extra sensors.
==Viewing monthly log files==


In the '''View''' menu, select ''Data logs'', click '''Load...'''. The standard Windows File Select dialogue is displayed for the 'data' subdirectory of the Cumulus installation directory. Select the required monthly log, click '''Open''' and you will see the contents in a neat table with column headings, rows striped, and the date/time on a lighter background. This is a text viewer, and works best when at full screen, but even then you are likely to need to scroll both horizontally and vertically to look at all the figures. If you click on an individual figure the cell will be highlighted, but you cannot edit the figures on a view screen.


Steve Loft wrote the Cumulus software mainly for his own use, although after a while he let others use it and added features that others requested. Cumulus was developed in his spare time outside a full-time employment, and consistency and terminology were not issues he worried about.
If no logs seen, see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. Also note, this '''Data logs''' viewer is only for viewing monthly log files (not any other files in the same ''data'' directory), the column headings for standard monthly log files will always be shown at the top of the screen.


During the development of the original Cumulus software (Cumulus 1 and 2) by Steve Loft, the name "monthly log" was most frequently used as a collective term for the last two (.ini) files linked above because those files tracked the monthly extreme records.
==Viewing summary figures for a month==
In the '''View''' menu, select ''This month'', and a summary for the current calendar month will be calculated from the monthly log. Note that many parameters are different to those available as web tags.


When Steve Loft was working on his [[Cumulus_3_(MX)_beta_documentation|MX Beta]], he decided to implement within MX functionality the database schema that Mark Crossley had designed on [[ImportCumulusFile#Monthly_table_.28old.29|this]] page, where Mark selected the name '''Monthly''' for the table. David A Jamieson created versions 1.0 and 1.1 of ImportCumulusFile for creating a single table called '''dayfile''', but Mark extended it to cover ''realtime'' and what he called ''monthly log files''. That means the standard log can in MX optionally feed data into a database table called '''Monthly''' by default in MX.
To view for a different month (or if you use 9am/10am rollover, if '''current local time on the 1st day of a month''' is before rollover time so the log for the current calendar month has not been started), select the month and year required using the options at bottom left, click ''Update Display'' and the revised summary will be calculated. (Alternatively you can select ''This period'' and select a start date and end date then click ''Update Display'' to get the equivalent calculation for part of a month or any other period).


This database table terminology can be misleading as ''that database table is not organised by month in any way'', a single table contains every logged line that has been inserted into it by Steve's '''ExportMySQL.exe''', Mark's '''ExportToMySQL.exe''', or by '''CumulusMX.exe''' itself. In fact MX allows you to give this table another name, which it will then use in all SQL.
==How can Monthly log files be used within Cumulus?==


Mark Crossley has taken over development of MX, the terminology "monthly log" is now being used for the first linked file (standard logs) instead of for '''month.ini''' the file that is truly logging monthly extremes. The name has been adopted simply because a new log file is started for each month.
'''After you have edited (or created any missing) Monthly log files, you can:'''
* update the highs and lows in [[Alltime.ini]] by choosing ''all time records'' from the '''Edit''' menu. See [[Alltimelog.txt]] for current and previous values.
* update (if the created/edited monthly log is within the current year) the highs and lows in [[year.ini]] by choosing ''This year's records'' from the '''Edit''' menu.
* update (if the created/edited monthly log is for the current month) the highs and lows in [[month.ini]] by choosing ''This month's records'' from the '''Edit''' menu. See [[Diags]] for current and previous values.
* update (from version 1.9.3) the highs and lows in [[monthlyalltime.ini]] by choosing ''Monthly records'' from the '''Edit''' menu. Click the ''Help'' button for specific instructions on using ''Reset'' and the two ''Copy'' column header buttons in this ''Monthly Records (Highs and Lows) Editor'' to action all rows.
'' '''Note''' in each of above 4 editing screens you can:
# see the currently stored extremes, and optionally ''Reset'' (row by row) to pre-editing value and timestamp.
# load the monthly log(s) and dayfile.txt to view extremes calculated from those figures (as available) and
# optionally ''Copy'' (row by row) the logged values (and associated date/time information) into the relevant .ini file.
# click the ''Help'' button for detailed instructions on using ''The Records (Highs and Lows) Editors''.
# store your revised figures by clicking ''OK'' (or abandon all your edits by clicking ''Cancel'').
(Each of these screens is a text editor, and works best when at full screen).''
* see '''[[Average_temperature | Mean temperatures]]''', '''[[Heat/cold_degree_days_and_Chill_hours | Air frosts]]''', '''[[Gale_Days |Gale days]]''', '''[[Rain_measurement#Precipitation_measurement_and_Cumulus | Rain days, Dry days]]''', and '''Total [[Windrun |Wind run]]''' using '''View''' menu displays for ''This month'', ''This period'', and ''This year'' (These figures are generated by reading the appropriate monthly log(s) and doing the necessary calculation, they are not stored anywhere nor available as webtags).
* create the relevant monthly and/or annual NOAA style report by choosing ''NOAA Monthly Report'' or ''NOAA Annual Report'' from the '''View''' menu, then select the required period using the selectors, click the ''Update Display'' button to see various statistics (including [[Average_temperature|mean temperature]]) calculated from processing the monthly log files, and finally press ''Save'' button to store the new or amended report. (Note: You can generate a NOAA report for a period without a monthly log, if data for that period is present on [[dayfile.txt]]. Generation of complete NOAA reports requires information in dayfile.txt, and the monthly logs for period in question).
* obtain (by [[Monthly_log_files#Viewing_monthly_log_files | viewing the created/edited log file]] just before rollover time) figures needed for correcting rogue figures in [[dayfile.txt]]. See for example [[FAQ#My_monthly_and.2For_annual_rainfall_totals_are_wrong | FAQ re correcting rainfall]].
* use '''create missing''' button in the ''[[dayfile.txt]]'' selection in the '''Edit''' menu; This is a text editor, and works best when at full screen. Cumulus will then look through all monthly log files and create approximate records for any ''missing dates'' in the dayfile if those dates have observations stored in the relevant monthly log. (It creates a file in the Cumulus folder called dayfileeditlog.txt which contains the entries it created). Click the ''Help'' button for more information. In past and current versions (including 1.9.3), ''create missing'' will not affect any records that are incomplete or contain some rogue values (see [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] below).


For novices, there can be a feeling of confusion:
==Correcting any logged data problems==
* The new practice leaves "month.ini" without a descriptive name
Some Weather stations may occasionally supply corrupted or rogue values to Cumulus. Cumulus provides via '''Configuration''' menu ''Calibration'' screen the ability to screen out spikes in data picked up from your weather station. See Cumulus help screen if you decide to use that to cope with future spikes. Some rogue values may be obviously invalid (e.g. a zero pressure), others (e.g. wind shake on a tipping bucket rain gauge on a fine day) may be within the accepted all-time range (so cannot be prevented by spike settings), but you know the value is not right for that particular time.
* Each "dayfile" line represents a new day so the filename matches the content, but a line in "Oct16log.txt" does not represent a month, and the file name does not contain the phrase "month"
* The names "Month.ini" and "Monthlyalltime.ini" do contain the phrase "month", they do contain monthly data in each line, and do log extreme records that clearly identify with a particular month


A future Wiki editor may decide on a different strategy, but at the moment the Cumulus Wiki has returned to the original Steve Loft terminology for each of the log files, for consistency with older articles in this Wiki, and the oldest posts on the forum.
Frequently, the graphical views ('''View''' menu, ''Graphs'' or ''Select-A-Graph'' screens) are the best way to spot many rogue values and to help you guess by interpolation a more reasonable value. Another possiblity is to examine the relevant values in previous versions of the log files stored in the ''backup'' or ''backup/daily'' subdirectories of where your cumulus executable is stored. You may choose to just edit the rogue value into a reasonable value, or (if you are unable to decide on a better value) to delete that particular entire row in the monthly log. As explained above once the log is corrected, new highs and lows in other logs are easily rectified. The screen ''within Cumulus'' for viewing the monthly log does not allow editing of values.


=Enhancement never implemented=
=== Manipulation in Cumulus Toolbox ===


Steve Loft was several times asked if he could change the design of his software and what it stored in [[Standard log files]], [[Extra Sensor Files]] and the other detailed logging files.
[[File:ToolBoxFileEditorScreen.jpg‎ | right]]
The add-on [[Toolbox#Current_feature_list| Cumulus Toolbox]] contains a '''file editor''' that can manipulate monthly log files using a display rather like this screen (in this figure it is actually dayfile.txt that is being edited).


All these files currently periodically store spot readings. Some of the competition in weather software, periodically store the highest and lowest since the previous line was stored.
===Manipulation outside Cumulus===
Any log files for ''previous'' months can be edited (outside Cumulus) with Cumulus running, and after editing them, you can do further fetch edits within Cumulus as described above. The current month log file can also be edited outside Cumulus, but you must stop Cumulus first.


One advantage of the alternative approach is that you never lose any extremes. Another advantage is the alternative allows people to report extremes for parts of days. For example, some people wanted 9 am to 9 pm reported separately to night time extremes, in other nations, the dividing time was 3 pm not 9 pm.
'''Tips''' -- take a copy of the original log file before you work on it outside Cumulus (perhaps give the ''not to be touched copy'' a filename of "<Month><Year>log.csv").
Edit the original file using an editor that treats all fields as text [either any text editor, a Comma Separated Value editor, or a spreadsheet program that can be instructed ''not'' to recognise special field (like date) types].


Steve Loft would respond he had made many design decisions that were not ideal, but the software was based on his own needs. If people wanted drastic changes, they should do what he had done, and write software that did what suited them.
'''Rules''' -- Each line must contain the fields in correct sequence (since new versions/builds can add to number of fields, Cumulus will accept lines of various lengths without the more recent fields at the end). Nulls (',,' if your field separator is a comma, ';;' if your field separator is a semicolon) are not allowed in the row, so if you do not know the value for a particular field within the row, then type in a zero, -999, or -999.9 (check the format and range of numbers allowed for particular fields).


Yet it appears he did plan to implement this suggestion in MX, see [[Cumulusmx.db#Releases 3.0.0 to 3.11.4 inclusive]].
'''Important''':
* The date format uses two digits for the year and should be treated as text. Do not edit this file using software (e.g. Excel) with default of recognising formats, as it is likely to change the dates to have four figure years, and then Cumulus will no longer be able to use the log file.
* Times in these files are in the form ''hh:mm'' using the 24 hour clock and local time (system time).
* Note that the fields related to solar data will only contain valid data if your station has the appropriate sensor(s), but if they apply make sure you note which are stored as integers.
* Apart from bearings and humidity values (these are stored as integers), most value fields are in format x.y using your system decimal notation. See [[Monthly_log_files#Example_lines_from_the_file|Monthly log files: Example lines from the file]].

===Importing pre-Cumulus data===
See [[FAQ#I.E2.80.99ve_just_installed_Cumulus.2C_and_it_didn.E2.80.99t_download_all_the_old_data_from_my_weather_station|FAQ: I've just installed Cumulus and it didn't download all the old data from my weather station]]

Given that monthly log files are used as input for '''updating the all-time, month-by-month alltime (from version 1.9.3), and this year record extremes''', for '''creating missing dayfile.txt entries''', and for '''creating NOAA style reports''', you may have some observations from your weather station recorded manually or electronically for a period before you first starting using Cumulus and want to create new monthly logs to feed into Cumulus.

'''Here are some issues to consider.''' There are some postings in the Support Forum about importing past data. Essentially match the fields listed in [[Monthly_log_files#List_of_fields_in_the_file|Monthly log files: List of fields in the file]] below with the fields you have available in your source.
* Wind directions might be reported as cardinal points (north, south etc) and need converting to bearings in (integer) degrees (spreadsheet packages provide ''look up'' functionality to do such conversions).
* Dates and times might need some pre-processing (spreadsheet packages usually have ability to select part of a text string and to cocatenate a number of strings) to convert them to text in the formats mentioned above. (Note - it is crucial in a spreadsheet to treat the date as text; Do not let the spreadsheet recognise it as a date as it might change the format or store as a 'days since X' number).
* Wind speed fields may not directly match to weather station outputs. See [[#Cumulus_Wind_Speed_Terminology]]
* Temperature fields may not directly match to weather station outputs. Copy across the fields you can and set the others to -999.99 so they are obvious. See the [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] described below for a way of getting Cumulus to generate (from your newly created monthly logs) apparent temperature, wind chill and heat index to replace any -999.99 figures.
* Rainfall fields may not directly match to weather station outputs. [[FAQ#How_is_my_rain_rate_calculated.3F | Rain rate]] in field 8 may be generated from the [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] if unavailable from your station. The daily total in field 9 is since rollover time, and available weather station outputs may need some processing in your spreadsheet to derive this. Some stations output a count of total rainfall, others the total annual rainfall, either of these may be used for field 11. Rainfall processing issues are frequently discussed in the Support Forum, so you may find the solution there or ask there for assistance.

'''The date that Cumulus first started tracking all-time records''' (<#recordsbegandate> see [[Webtags#Records]]), does ''not'' need to be updated for Cumulus to recognise earlier monthly log files in its ''data'' subfolder and to use any monthly logs found in the various '''View''' and '''Edit''' options available from the Cumulus main screen. As <#recordsbegandate> is on the default Cumulus ''recordsT.htm'' template with the label "Records began on", inserting pre-Cumulus data implies you might want to amend that label wording on that template to indicate it is the date that Cumulus was first operational, rather than the date of the earliest tracked highs and lows.

If you prefer to edit the start date and keep the ''recordsT.htm'' template label "Records began on" unchanged, the method is to stop Cumulus, and amend the ''StartDate='' line in [[cumulus.ini]] within the main Cumulus folder before re-starting Cumulus, but (in case you make a mistake) back up everything first!
=== Cumulus Wind Speed Terminology ===
How wind speeds relate to various weather stations is explained in FAQ.
[[FAQ#What_do_the_various_wind_speeds_in_Cumulus_correspond_to.2C_with_Davis_stations.3F|FAQ: Wind speeds in Cumulus with Davis stations]], the ''EasyWeather'' differences in [[FAQ#What_do_the_various_wind_speeds_in_Cumulus_correspond_to.2C_with_Fine_Offset_stations.3F|FAQ: Wind speeds in Cumulus with Fine Offset stations]] and [[FAQ#How_do_I_set_up_recording_wind_speeds_in_Cumulus.2C_with_Oregon_Scientific_and_La_Crosse_stations| FAQ: Wind speeds with Oregon Scientific and La Crosse stations]]

===Using Monthly logs to deal with shorter (or incomplete) dayfile.txt records for particular dates===
'''WORKAROUND if dates are present in monthly log and dayfile.txt but not all fields for that date exist in dayfile.txt'''

Example: records created by earlier version of Cumulus [to help you, the versions (not builds) at which fields were added are indicated below].
Second example: records imported prior to Cumulus processing them, so some calculated parameters (apparent temperature, heat index, rain rate, wind run) may not have been available from your weather station to insert in the log files.

''One method is (not near rollover time) to (1) take a copy of dayfile.txt original as backup, (2) in original file delete any days with partial information (e.g. from Cumulus versions that created fewer fields), (3) use ''create missing'' in the [[dayfile.txt]] selection in the '''Edit''' menu option of Cumulus (note whilst datafile.txt normally calculates all parameters like minimums and maximums from very frequent samples of your weather station, the resolution of create missing is limited to the interval between logging records in the monthly log), (4) rename the amended dayfile.txt as dayfile(generated).txt, (5) create a new dayfile.txt and (6) use a text editor to merge the required fields from the new dayfile(generated).txt with all other fields from (with reading access only) the backup copy of the original file. (This method preserves the original as a backup so you can experiment with different merges and do some cross-checking).''

==List of fields in the file==
The field number (starting from zero to be consistent with index used for arrays in programming languages like JavaScript) and the equivalent letter that would be seen in a spreadsheet like Libre Office, Open Office, Excel etc are listed here.
(Note shows version for all changes since version 1.8.5)
* 00(A): Date in the form dd/mm/yy or dd-mm-yy
* 01(B): Current time
* 02(C): Current temperature
* 03(D): Current humidity
* 04(E): Current dewpoint
* 05(F): Cumulus 'Average' wind speed (See [[#Cumulus_Wind_Speed_Terminology]])
* 06(G): Cumulus 'Gust' wind speed
* 07(H): Average wind bearing (integer degrees)
* 08(I): [[FAQ#How_is_my_rain_rate_calculated.3F | Current rainfall rate]]
* 09(J): Total rainfall today so far (i.e. resets to zero at rollover)
* 10(K): Current sea level pressure
* 11(L): Total rainfall counter (content depends on weather station type, might be [[FAQ#It.27s_January_1st_and_my_.27rain_since_midnight.27_and.2For_.27last_24_hours_rain.27_are_wrong.2Fnegative | rainfall total so far this year]])
* 12(M): Inside temperature
* 13(N): Inside humidity
* 14(O): Cumulus 'Latest' gust (See [[#Cumulus_Wind_Speed_Terminology]])
* 15(P): [[wind chill]] (content depends on weather station type and Cumulus station settings)
* 16(Q): [[Heat index]]
* 17(R): UV Index (only valid if output by weather station)
* 18(S): Solar Radiation (only valid if solar sensor on weather station)
Added in 1.9.1:
* 19(T): Evapotranspiration (only valid if output by weather station)
* 20(U): Annual Evapotranspiration (only valid if ET sensor on weather station)
* 21(V): [[Apparent temperature]]
* 22(W): Current theoretical max solar radiation (only valid if solar sensor on weather station)
* 23(X): Hours of sunshine so far today (only valid if solar sensor on weather station)
Added in 1.9.2:
* 24(Y): Current Wind bearing (See 7)
Added in 1.9.3:
* 25(Z): RG-11 rain today (only valid if output by sensor)
Added in 1.9.4 beta:
* 26(AA): Total Rainfall since midnight

==Example lines from the file==

Note that your field delimiters may be different, and your date delimiter too.


An extract of a few lines of the file (v.1.9.0)

Here logging is quarter hourly, the field delimiter is a semicolon, the decimal separator is a comma, and the date separator a dash.
30-09-10;19:00;16,4;94;15,4;5,2;13,3;17;3,6;21,0;995,3;47,7;25,6;62;6,1;16,4;16,4;0,0;0
30-09-10;19:15;16,4;94;15,4;5,6;11,2;12;18,0;24,0;995,0;50,7;25,6;62;7,2;16,4;16,4;0,0;0
30-09-10;19:30;16,2;94;15,2;7,9;15,8;355;7,2;25,8;994,3;52,5;25,7;62;12,2;16,2;16,2;0,0;0
30-09-10;19:45;16,0;94;15,0;9,9;19,4;7;7,2;27,9;993,3;54,6;25,7;62;14,8;15,8;16,0;0,0;0
30-09-10;20:00;15,9;94;14,9;12,4;20,9;354;7,2;30,0;993,0;56,7;25,7;62;19,4;15,4;15,9;0,0;0
30-09-10;20:15;15,8;94;14,8;8,4;15,8;349;14,4;32,7;993,4;59,4;25,8;61;12,2;15,8;15,8;0,0;0
30-09-10;20:30;15,4;94;14,4;13,8;33,1;317;28,8;40,5;993,7;67,2;25,8;61;23,4;14,7;15,4;0,0;0
30-09-10;20:45;15,1;94;14,1;20,3;34,2;356;7,2;43,8;992,3;70,5;25,8;60;29,5;13,8;15,1;0,0;0
30-09-10;21:00;15,3;94;14,3;20,2;35,6;358;10,8;46,8;991,0;73,5;25,8;60;28,1;14,0;15,3;0,0;0
30-09-10;21:15;15,3;95;14,5;16,6;31,7;358;10,8;49,5;991,4;76,2;25,8;60;20,9;14,3;15,3;0,0;0
30-09-10;21:30;15,3;94;14,3;14,0;27,0;324;18,0;54,3;992,3;81,0;25,8;60;15,8;14,5;15,3;0,0;0
30-09-10;21:45;15,2;94;14,2;13,0;25,6;323;10,8;57,9;992,3;84,6;25,8;59;24,5;14,5;15,2;0,0;0
30-09-10;22:00;15,0;94;14,0;16,7;31,7;312;10,8;60,6;993,0;87,3;25,8;59;23,4;13,9;15,0;0,0;0
30-09-10;22:15;14,9;94;13,9;16,0;30,6;357;10,8;63,0;991,6;89,7;25,8;59;20,9;13,9;14,9;0,0;0
30-09-10;22:30;14,9;94;13,9;17,6;31,7;3;3,6;63,3;990,6;90,0;25,8;59;19,4;13,7;14,9;0,0;0

this is an example of a file from 1.9.1, with solar data:

Here logging is increased to every five minutes, and using UK settings, the field delimiter is a comma, the decimal separator is a full stop, and the date separator a slash.

22/04/11,10:25,8.1,96,7.5,13,20,138,0.0,0.0,1013.24,215.2,20.3,53,17,4.8,8.1,0.0,197,0.08,171.88,3.4,663,0.0
22/04/11,10:30,8.1,96,7.5,13,20,142,0.0,0.0,1013.28,215.2,20.2,53,11,4.8,8.1,0.0,216,0.08,171.88,3.3,673,0.0
22/04/11,10:35,8.1,96,7.5,12,18,142,0.0,0.0,1013.31,215.2,20.2,53,11,5.1,8.1,0.0,227,0.08,171.88,3.8,682,0.0

and from version 1.9.2:

Here logging is at the default interval, and the delimiters have their default settings for UK systems.

01/10/11,04:40,18.0,75,13.5,5.0,6.7,169,0.0,0.0,1021.4,885.3,24.4,61,5.6,18.0,18.0,0.0,0,0.00,0.00,17.5,0,0.0,158
01/10/11,04:50,17.9,75,13.4,4.5,9.5,169,0.0,0.0,1021.4,885.3,24.4,61,2.0,17.9,17.9,0.0,0,0.00,0.00,17.5,0,0.0,180
01/10/11,05:00,17.8,75,13.3,5.1,9.5,177,0.0,0.0,1021.2,885.3,24.4,61,7.5,17.8,17.8,0.0,0,0.00,0.00,17.2,0,0.0,180


[[Category:Log Files]]

Latest revision as of 23:17, 31 December 2021

Please see Standard log files, Extra Sensor Files, Air_Link_Log.txt, Month.ini, or Monthlyalltime.ini, as relevant.

Please be aware that all of the above links lead to documentation about files that may be described as monthly log files when reading individual Cumulus forum and wiki references.

Potential Name Confusion

Those who used early versions of Cumulus will remember that the term Log File was originally used for Log.xml, but Steve Loft later accepted that all the files with a .ini file extension could be described as log files, so he introduced the term Weather Diary. A bit later, the detailed data file, began to be also called log files, and the confusion grew. When Extra Sensor Files were added, these also became log files. The Wiki page now called "Standard Log Files" was first called "Log files", then became "Monthly Log Files" because "Log files" became a category. Then "Extra Sensor Files" were added and "standard log files" replaced "Monthly Log Files" to make it clearer which file held the standard values, and which file held values for extra sensors.


Steve Loft wrote the Cumulus software mainly for his own use, although after a while he let others use it and added features that others requested. Cumulus was developed in his spare time outside a full-time employment, and consistency and terminology were not issues he worried about.

During the development of the original Cumulus software (Cumulus 1 and 2) by Steve Loft, the name "monthly log" was most frequently used as a collective term for the last two (.ini) files linked above because those files tracked the monthly extreme records.

When Steve Loft was working on his MX Beta, he decided to implement within MX functionality the database schema that Mark Crossley had designed on this page, where Mark selected the name Monthly for the table. David A Jamieson created versions 1.0 and 1.1 of ImportCumulusFile for creating a single table called dayfile, but Mark extended it to cover realtime and what he called monthly log files. That means the standard log can in MX optionally feed data into a database table called Monthly by default in MX.

This database table terminology can be misleading as that database table is not organised by month in any way, a single table contains every logged line that has been inserted into it by Steve's ExportMySQL.exe, Mark's ExportToMySQL.exe, or by CumulusMX.exe itself. In fact MX allows you to give this table another name, which it will then use in all SQL.

Mark Crossley has taken over development of MX, the terminology "monthly log" is now being used for the first linked file (standard logs) instead of for month.ini the file that is truly logging monthly extremes. The name has been adopted simply because a new log file is started for each month.

For novices, there can be a feeling of confusion:

  • The new practice leaves "month.ini" without a descriptive name
  • Each "dayfile" line represents a new day so the filename matches the content, but a line in "Oct16log.txt" does not represent a month, and the file name does not contain the phrase "month"
  • The names "Month.ini" and "Monthlyalltime.ini" do contain the phrase "month", they do contain monthly data in each line, and do log extreme records that clearly identify with a particular month

A future Wiki editor may decide on a different strategy, but at the moment the Cumulus Wiki has returned to the original Steve Loft terminology for each of the log files, for consistency with older articles in this Wiki, and the oldest posts on the forum.

Enhancement never implemented

Steve Loft was several times asked if he could change the design of his software and what it stored in Standard log files, Extra Sensor Files and the other detailed logging files.

All these files currently periodically store spot readings. Some of the competition in weather software, periodically store the highest and lowest since the previous line was stored.

One advantage of the alternative approach is that you never lose any extremes. Another advantage is the alternative allows people to report extremes for parts of days. For example, some people wanted 9 am to 9 pm reported separately to night time extremes, in other nations, the dividing time was 3 pm not 9 pm.

Steve Loft would respond he had made many design decisions that were not ideal, but the software was based on his own needs. If people wanted drastic changes, they should do what he had done, and write software that did what suited them.

Yet it appears he did plan to implement this suggestion in MX, see Cumulusmx.db#Releases 3.0.0 to 3.11.4 inclusive.

Retrieved from "https://www.cumuluswiki.org/index.php?title=Monthly_log_files&oldid=10125"