5,838
edits
Dayfile.txt: Difference between revisions
m
Minor resequencing of page
m (→Specific issues for MX: Time-stamp issue added) |
m (Minor resequencing of page) |
||
| Line 1: | Line 1: | ||
{{Template:Version badge Mx}}{{Version badge 1}}This Wiki page applies to both Cumulus flavours. | {{Template:Version badge Mx}}{{Version badge 1}}This Wiki page applies to both Cumulus flavours. [[Category:Files_with_Comma_Separated_Values]] | ||
The support forum may have links to content that is no longer on this Wiki page, so if the link you followed in the forum takes you to here at the start of this page, rather than to a particular section, the information you seek may now be on another page. | |||
= This Wiki page = | |||
This Wiki page describes one of the [[:Category:Cumulus Files|files]] not included in any release download. This daily summary file (its title is frequently shortened to "day" plus "file") uses a [[:Category:Files_with_Comma_Separated_Values|".txt" extension]] and is created in [[Data folder|data sub-folder]] of your Cumulus installation when Cumulus needs to store its first line in this file. | |||
This page is full of links that take you to content previously on this page, the links in the previous paragraph take you to new pages that cover details once repeated on several pages including this page. The names of [[Amending dayfile]] page, and [[Calculate Missing Values]] page, should give you a clue about what has been moved from this page to those new pages. | |||
You might also find it useful to read [[Correcting Extremes]] page which also includes some information that once was on this page about fixing other files where the highest/lowest/total recorded for day has been corrupted by rogue values. | |||
The content that remains on this page is summarised by the table of contents that follows.{{TOCright}} | |||
=About this file= | =About this file= | ||
| Line 15: | Line 22: | ||
==How Cumulus Creates and Updates this file== | ==How Cumulus Creates and Updates this file== | ||
Cumulus reads values supplied by your weather station, converts them to the units you prefer, applies any calibration (multiplier and offset) you have set, and then sees if the resulting value implies [[:Category:Ini Files|any extreme records file]] needs to be updated (these hold derived values like totals, highest, and lowest). Two of those files are [[Today.ini#When_Cumulus_is_running|today.ini]] and [[yesterday.ini]] and values from those two files are | Cumulus reads values supplied by your weather station, converts them to the units you prefer, applies any calibration (multiplier and offset) you have set, and then sees if the resulting value implies [[:Category:Ini Files|any extreme records file]] needs to be updated (these hold derived values like totals, highest, and lowest). Two of those files are [[Today.ini#When_Cumulus_is_running|today.ini]] and [[yesterday.ini]] and values from those two files are extracted (see linked Wiki pages for how entries in those files map across to the daily summary file) at the end of a [[meteorological day]] to calculate the necessary [[#List_of_fields|fields]] for a new line to be added to this file. | ||
As explained on the "meteorological day" page linked above, if your weather station stores weather data, then Cumulus on restarting can (optionally) read the '''historic data''' and catch-up by processing those past readings (as per description in start of previous paragraph), including activating any end of day actions (see [[Today.ini#Restart_and_Catch-up|today.ini]]), for the period since it was last running. | As explained on the "meteorological day" page linked above, if your weather station stores weather data, then Cumulus on restarting can (optionally) read the '''historic data''' and catch-up by processing those past readings (as per description in start of previous paragraph), including activating any end of day actions (see [[Today.ini#Restart_and_Catch-up|today.ini]]), for the period since it was last running. | ||
| Line 21: | Line 28: | ||
==How to view or edit this file== | ==How to view or edit this file== | ||
An editor has been included within Cumulus | An editor has been included within Cumulus: | ||
* [[File:Badge vMx.png]] From release 3.4.5 (13 Mar 2020) onwards: In [[MX_Administrative_Interface#The_Data_Log_Viewing_and_Editing_interface|the interface]] go to '''Data logs''' menu and select ''Dayfile'' | * [[File:Badge vMx.png]] From release 3.4.5 (13 Mar 2020) onwards: In [[MX_Administrative_Interface#The_Data_Log_Viewing_and_Editing_interface|the interface]] go to '''Data logs''' menu and select ''Dayfile'' | ||
* Cumulus 2 | * Cumulus 2 had no viewing option | ||
* [[File:Badge v1.png]] From version 1.9.2 (5th October 2011) to final legacy release: On Main Screen from [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu|'''Edit''' menu]] select ''Dayfile.txt'' | * [[File:Badge v1.png]] From version 1.9.2 (5th October 2011) to final legacy release: On Main Screen from [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu|'''Edit''' menu]] select ''Dayfile.txt'' | ||
**'''Note for obsolete version 1.9.0 to 1.9.3:''' There is a bug in these versions in that 'Create missing' inserts 'heating and cooling degree day' values the wrong way round. | **'''Note for obsolete version 1.9.0 to 1.9.3:''' There is a bug in these versions in that 'Create missing' inserts 'heating and cooling degree day' values the wrong way round. | ||
| Line 30: | Line 37: | ||
For detailed information, please see [[Amending dayfile|viewing/editing '''dayfile.txt''']]. | For detailed information, please see [[Amending dayfile|viewing/editing '''dayfile.txt''']]. | ||
=Reading the file= | =Reading the file= | ||
| Line 45: | Line 50: | ||
== Specific issues for MX == | == Specific issues for MX == | ||
WARNING: It is important to note that MX | WARNING: It is important to note that some releases of MX are very fussy about consistency in all lines of '''dayfile.txt'''. Earlier releases tended to have better compatibility with the legacy software. Subsequent releases expected every line to be expressed exactly as specified in the locale. MX in its latest release is trying to cope better with inconsistent date separators, and at the time of writing this there was a proposal for forcing this file to use decimal points (full stops not commas) regardless of locale. Therefore, information here might be incorrect for the release that you are using. | ||
Date field: Cumulus 1 (C1) did not care what character (or characters) separated the day, month, and year elements of the date field. MX insists that the same character (or characters), as defined in the locale, is used for all lines in file. | Date field: Cumulus 1 (C1) did not care what character (or characters) separated the day, month, and year elements of the date field. MX insists that the same character (or characters), as defined in the locale, is used for all lines in file. | ||
| Line 63: | Line 68: | ||
Thus it is worth backing up this file, to another device than that running Cumulus, on a regular basis, probably a few times a week, to ensure this precious data is not lost if your device running Cumulus has a malfunction (or is damaged), or an electrical blip (or mistake by you) causes corruption to the original file, | Thus it is worth backing up this file, to another device than that running Cumulus, on a regular basis, probably a few times a week, to ensure this precious data is not lost if your device running Cumulus has a malfunction (or is damaged), or an electrical blip (or mistake by you) causes corruption to the original file, | ||
Cumulus does periodically copy this file within the installation, but the copies are only kept for a limited time, and so your main file is corrupted you might not be able to go far back enough for an uncorrupted file: | Cumulus does periodically copy this file within the installation, but the copies are only kept for a limited time, and are on the same physical storage device, and so your main file is corrupted you might not be able to access the Cumulus back up or might not be able to go far back enough for an uncorrupted file: | ||
* The file is backed up when MX is restarted into a date/time stamped sub-folder of the [[Backup folder|'''backup''' folder]] | * The file is backed up when MX is restarted into a date/time stamped sub-folder of the [[Backup folder|'''backup''' folder]] | ||
*The file is also backed up (to a date/time stamped sub-folder within '''daily''' sub-folder of that backup sub-folder) during the end of day process, depending on the release you are running the back up copy may, or may not, include the line that is appended in the same end of day | *The file is also backed up (to a date/time stamped sub-folder within '''daily''' sub-folder of that backup sub-folder) during the end of day process, depending on the release you are running the back-up copy may, or may not, include the line that is appended in the same end of day | ||
Retention of these back ups: | Retention of these back ups: | ||
* | * [[File:Badge vMx.png]] MX only keeps the last 9 of the date/time stamped subfolders. | ||
* [[File:Badge v1.png]] Legacy Cumulus 1 only keeps up to 8 of the date/time stamped subfolders. | * [[File:Badge v1.png]] Legacy Cumulus 1 only keeps up to 8 of the date/time stamped subfolders. | ||
| Line 82: | Line 87: | ||
*The fields are now numbered starting from 1 to fit in with Cumulus MX where when the log file is read, the processing code adds a line number in front of the date field on each line it holds. | *The fields are now numbered starting from 1 to fit in with Cumulus MX where when the log file is read, the processing code adds a line number in front of the date field on each line it holds. | ||
** The Cumulus MX user may not be aware of this happening as it is within the internal workings, where data from the file is transferred to an array, or from the array is written back | ** The Cumulus MX user may not be aware of this happening as it is within the internal workings, where data from the file is transferred to an array, or data from the array is written back into the file. | ||
**By using line numbers, MX is able to identify which line has been deleted or edited | **By using line numbers, MX is able to identify which line has been deleted or edited, MX coding treats the date as a fixed bit of text (MX does not allow you to change a date) | ||
**Cumulus 1 does not actually number lines, it | ** Please note that the editor in Cumulus 1 allows you to change the date, as well as all the other fields, although the lines must be kept in ascending date order to avoid errors when subsequently reading the file | ||
**Cumulus 1 does not actually number lines, however it does count lines as it reads them, so if there is an error when it reads the file, the original Cumulus will report the line number where it first found an error (this uniquely identifies the line even if a date is duplicated or a line feed has been deleted so two lines are merged). | |||
*The original table below was for Cumulus 1 and then field number '''was''' starting from zero. So in some forum posts you might see references to old numbering, in others to new numbering. The old numbering from zero had two advantages: | *The original table below was for Cumulus 1 and then field number '''was''' starting from zero. So in some forum posts you might see references to old numbering, in others to new numbering. The old numbering from zero had two advantages: | ||
*#Cumulus 1 stressed that the date field was different to the rest, as it was used as identifier. The date '''must''' be a unique identifier, the same date should not be repeated in another line, however Cumulus 1's editor allows you to change that date field. | *#Cumulus 1 stressed that the date field was different to the rest, as it was used as identifier. The date '''must''' be a unique identifier, the same date should not be repeated in another line, however Cumulus 1's editor allows you to change that date field. | ||
*#The remaining fields were all either numerical values, or a time paired with preceding numerical value. Cumulus 1 actually enforces this pairing (i.e. it validates that a time is present where it is needed). | *#The remaining fields were all either numerical values, or a time paired with (except for first wind field) preceding numerical value. Cumulus 1 actually enforces this pairing (i.e. it validates that a time is present where it is needed). | ||
**Numbering starting from zero is consistent with standard indexing used for arrays in programming languages (like JavaScript), so the number shown '''was''' the number to quote in any scripts where a line was converted to an array, and you needed to address a single field. | **Numbering starting from zero is consistent with standard indexing used for arrays in programming languages (like JavaScript), so the number shown '''was''' the number to quote in any scripts where a line was converted to an array, and you needed to address a single field. | ||
*The alphabetic column identifiers used by many spreadsheets are shown, please remember to select options in your spreadsheet to ensure '''all columns are set to "text" format''', you will corrupt this file if you let your spreadsheet recognise content as dates or time, or change the number of decimal places. | *The alphabetic column identifiers used by many spreadsheets are shown, please remember to select options in your spreadsheet to ensure '''all columns are set to "text" format''', you will corrupt this file if you let your spreadsheet: | ||
*The type of field is shown in the table, you must not include a sign for an unsigned field, you can not specify a decimal point in an integer field, all time fields must use HH:mm format | ** recognise content as dates or time, | ||
** or change the number of decimal places. | |||
*The type of field is shown in the table, you must not include a sign for an unsigned field, you can not specify a decimal point in an integer field, all time fields must use 5 character "HH:mm" format | |||
*The field description is shown, together with references to where that terminology is explained | *The field description is shown, together with references to where that terminology is explained | ||