5,838
edits
Standard log files: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
Release 3.9.0 field 30 section corrected as change delayed
m (Text replacement - "Category:MX txt Files" to "Category:Files_with_Comma_Separated_Values") |
m (Release 3.9.0 field 30 section corrected as change delayed) |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 17: | Line 17: | ||
=How often is data stored in the standard file?= | =How often is data stored in the standard file?= | ||
These store the full set of current (read and derived) values at whatever interval you have set for Cumulus to keep detailed logging. The default time interval between adding data logging lines is every 10 minutes, but you can change this: | These store the full set of current (read and derived) values at whatever interval you have set for Cumulus to keep detailed logging. | ||
*Cumulus 1: On main screen, select '''Configuration''' menu, select the [[Cumulus_Screenshots#Station|'''station settings''' screen]], look for '''Data log interval''' box on extreme RH side. | * The interval has to be a fraction of an hour, so the least often storing is every half hour. | ||
*Cumulus MX: From [[MX Administrative Interface|Interface]], find settings menu, select the '''Station settings''' page, select '''General Settings''', and find '''Log interval''' where there is a drop down that offers the various timings that are permitted. | * For some weather station types, you MUST match the interval at which the weather station logs data internally | ||
* The default time interval between adding data logging lines is every 10 minutes, but you can change this: | |||
** Legacy Cumulus 1: On main screen, select '''Configuration''' menu, select the [[Cumulus_Screenshots#Station|'''station settings''' screen]], look for '''Data log interval''' box on extreme RH side. | |||
** Cumulus MX: From [[MX Administrative Interface|Interface]], find settings menu, select the '''Station settings''' page, select '''General Settings''', and find '''Log interval''' where there is a drop down that offers the various timings that are permitted. | |||
=What name is used for the standard file?= | =What name is used for the standard file?= | ||
| Line 49: | Line 52: | ||
For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”. This is installed with Cumulus 1, but can be read with Windows after downloading from [[Software#Resources]] page in this Wiki. | For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”. This is installed with Cumulus 1, but can be read with Windows after downloading from [[Software#Resources]] page in this Wiki. | ||
=List of fields in the file= | |||
The table is split by Cumulus version for all changes since version 1.7.x (Steve Loft did not retain his list of changes in 1.4, his '''changes.txt''' is rather sparse in details for many releases, and the support forum excludes any posts before 2 Jun 2008, so earlier changes are not now documented). | The table is split by Cumulus version for all changes since version 1.7.x (Steve Loft did not retain his list of changes in 1.4, his '''changes.txt''' is rather sparse in details for many releases, and the support forum excludes any posts before 2 Jun 2008, so earlier changes are not now documented). | ||
| Line 75: | Line 78: | ||
#* MX since version 3.7.0 is more fussy, but less consistent, certain fields contain a single space when not used, others contain zero, and others can be left empty | #* MX since version 3.7.0 is more fussy, but less consistent, certain fields contain a single space when not used, others contain zero, and others can be left empty | ||
#The final column describes the observation often with a link to where there is more information. | #The final column describes the observation often with a link to where there is more information. | ||
==Number of fields per line varies by release== | |||
The fields that are available in the release you are running are listed in [[Monthlyfileheader.txt]] (found in folder level above the data sub-folder). | |||
If you have some early files with fewer fields than the latest files, there are limited possibilities for calculating correct values for those missing fields | |||
[[File:Badge v1.png]] For the legacy Cumulus software, the number of fields in the file increased as the software was developed. The Cumulus author, Steve Loft, did not provide any utility for populating fields missing from earlier versions of his software. A third party did produce a [[Calculate_Missing_Values#Using_a_PHP_script_on_your_web_server|PHP script]] that could: | |||
* add Heat Index to lines created prior to 1.7.11 | |||
* add Apparent Temperature (not other fields, see below) to lines created before 1.9.1 | |||
* add Feels Like temperature (not other fields, see below) to lines created before 3.6.0 | |||
The fields that ''cannot'' be calculated by this script are (set them all to zero when editing file manually): | |||
* UV Index | |||
* Solar Radiation | |||
* Evapotranspiration (spot value and annual cumulative total) | |||
* Current theoretical max solar radiation | |||
* Hours of sunshine so far today | |||
* Current Wind bearing | |||
* RG-11 rain today | |||
* Total Rainfall since midnight (this is for 0900/1000 'rollover' users; normally same as rain today for 'midnight rollover' users) | |||
[[File:Badge vMx.png]] As MX has developed, further fields have been added to the file at 3.6.0 and 3.6.12: | |||
* Feels Like temperature | |||
* Canadian Humidity Index (Humidex) | |||
The main MX developer, Mark Crossley, has written a [[Calculate_Missing_Values#CreateMissing.exe|Create Missing utility]], this is dependent on having a significantly recent MX release, but it can calculate the following derived values (coping with cases where any of these are missing: | |||
* [[Wind chill|Wind Chill]] | |||
* [[Apparent_temperature|Australian Apparent Temperature]] | |||
* [[Feels Like|Feels Like temperature]] | |||
* [[Heat index|North American Heat Index]] | |||
* [[Temperature_(and_humidity)_measurement#Dry_and_Wet_Bulb|Wet Bulb temperature]] | |||
* [[Temperature_(and_humidity)_measurement#How_Cumulus_software_handles_Temperature_and_Humidity|Dew Point]] | |||
* Rain since midnight) | |||
''Please note the developer does not mention any of this at [https://github.com/cumulusmx/CreateMissing/blob/master/README.md his utility details explanation] so the author of this Wiki update cannot guarantee the above list is correct'' | |||
You will need to check in MX release announcements for any update, at time of typing this, the developer has written a routine within CumulusMX.exe to calculate Evapotranspiration, but that is not in the CreateMissing.exe Utility. | |||
===Before Version 1.7.10:=== | ===Before Version 1.7.10:=== | ||
| Line 250: | Line 289: | ||
|Unsigned integer | |Unsigned integer | ||
|158 | |158 | ||
| Current Wind bearing (See | | Current Wind bearing (See average wind bearing in field 8) | ||
|} | |} | ||
''For Cumulus MX editing a file created by Cumulus 1'', any of the preceding fields that are not present will be '''defaulted to an empty string''', but there will always be 50 fields (unless you are using MX version before 3.6.0, it which case MX cannot read the file. | ''For Cumulus MX editing a file created by Cumulus 1'', any of the preceding fields that are not present will be '''defaulted to an empty string''', but there will always be 50 fields (unless you are using MX version before 3.6.0, it which case MX cannot read the file. | ||
| Line 321: | Line 360: | ||
| [[Humidex]] | | [[Humidex]] | ||
|} | |} | ||
=== Added in 3.nn?.0 === | |||
This new field makes it easier to work out a time in UTC, and identifies uniquely those lines created during a switch from Daylight Saving Time to Winter time, that would otherwise have duplicate times for an hour at time-zone changeover. | |||
The following change was initially planned for release of MX 3.9.0 (b3095), but this section has been amended as the introduction of this new field has been postponed, because of implications of other changes planned at same time (see below table). | |||
The developer has said it is still his plan to add this field during a major release in 2022, but he cannot yet say when it will actually be available to public. | |||
{| border="1" class="wikitable" | |||
|- | |||
! style="width:60px" | Field # | |||
! style="width:180px" | Rule | |||
! style="width:80px" | Example | |||
! style="width:450px" | Description | |||
|- | |||
| 30(AD) | |||
|datetime integer | |||
| 1579564800000 | |||
| UNIX time-stamp | |||
|} | |||
''The proposed "Updates.txt" entry read <pre> | |||
- Major changes to how Cumulus MX handles it's data files | |||
- Dayfile.txt | |||
- The date field format is now fixed at "dd-MM-yy" (old order is retained for third party compatibility) | |||
- The decimal point is now fixed at "." | |||
- The list separator is now fixed at "," | |||
- Monthly log files | |||
- The file names are fixed at "yyyyMMlog.txt" | |||
- The date field format is now fixed at "dd-MM-yy" (old order is retained for third party compatibility), any lines using "dd/MM/yy" are converted to new separator | |||
- The time field format is now fixed at "HH:mm" | |||
- An extra field is added that is the UTC date-time integer | |||
- The decimal point is now fixed at "." | |||
- The list separator is now fixed at "," | |||
- Extra Data log files | |||
- No change in file names | |||
- The date field format is now fixed at "dd-MM-yy" (old order is retained for third party compatibility), any lines using "dd/MM/yy" are converted to new separator | |||
- The time field format is now fixed at "HH:mm" | |||
- An extra field is added that is the UTC date-time integer | |||
- The decimal point is now fixed at "." | |||
- The list separator is now fixed at "," | |||
</pre>'' | |||
= Accessing the Standard Log File = | = Accessing the Standard Log File = | ||
