5,838
edits
Dayfile.txt: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
→List of Fields
m (→List of Fields) |
m (→List of Fields) |
||
| (11 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
<big>'''This article is about the Daily Summary logging file'''</big> | |||
= Introduction = | = Introduction = | ||
| Line 78: | Line 80: | ||
* For Cumulus MX, there are backups of 10 files, the extra ones are the weather diary and Cumulus.ini, that are copied to start-up folders in \CumulusMX\backup\, again there are only 9 kept, unless you back these up somewhere else. | * For Cumulus MX, there are backups of 10 files, the extra ones are the weather diary and Cumulus.ini, that are copied to start-up folders in \CumulusMX\backup\, again there are only 9 kept, unless you back these up somewhere else. | ||
== How you can use the daily log == | == How you can use the daily summary log file == | ||
* If you want to run scripts that use the daily log, it is best if you take a copy first, you can ask Cumulus 1 to take a copy after each update by using the '''Daily''' box in the bottom left of the ''Sites/Options'' frame within the ''Internet'' options screen from the '''Configuration''' menu; that will safely take a copy of 'dayfile.txt' after it is updated. This has the advantage it happens even if Cumulus has been stopped and restarted and rollover is happening during catch-up and so not at usual rollover time according to the computer clock. See Cumulus 1 '''Help''' for information on using this feature, I add a redirection ">daily_batch.log" in the parameter box alongside so that any output from running the command file I specify in the main box is sent to a log file overwritten in each run; this enables me to see the reason for any failure. | * If you want to run scripts that use the daily summary log file, it is best if you take a copy first, you can ask Cumulus 1 to take a copy after each update by using the '''Daily''' box in the bottom left of the ''Sites/Options'' frame within the ''Internet'' options screen from the '''Configuration''' menu; that will safely take a copy of 'dayfile.txt' after it is updated. This has the advantage it happens even if Cumulus has been stopped and restarted and rollover is happening during catch-up and so not at usual rollover time according to the computer clock. See Cumulus 1 '''Help''' for information on using this feature, I add a redirection ">daily_batch.log" in the parameter box alongside so that any output from running the command file I specify in the main box is sent to a log file overwritten in each run; this enables me to see the reason for any failure. | ||
* Cumulus MX has option to list files to be transferred once a day as part of rollover, so you can use that to generate your extra copy. This has the advantage it happens even if Cumulus has been stopped and restarted and rollover is happening during catch-up. | * Cumulus MX has option to list files to be transferred once a day as part of rollover, so you can use that to generate your extra copy. This has the advantage it happens even if Cumulus has been stopped and restarted and rollover is happening during catch-up. | ||
* A third party tool "Cumulus Toolbox" can also be used to copy/transfer files at a particular time. Note this cannot tell whether Cumulus has done its rollover at the normal time, or during catch-up. | * A third party tool "Cumulus Toolbox" can also be used to copy/transfer files at a particular time. Note this cannot tell whether Cumulus has done its rollover at the normal time, or during catch-up. | ||
| Line 88: | Line 90: | ||
*Some third-party tools have to be told what separator you use for dates, before they can read your dayfile.txt. | *Some third-party tools have to be told what separator you use for dates, before they can read your dayfile.txt. | ||
= | = Safely editing the daily summary log = | ||
Both Cumulus 1 and 3 (2 did not) now provide editors where you can see what is in your dayfile.txt log, and if you click on a particular line you can edit the fields in that line, or indeed delete that line. | Both Cumulus 1 and 3 (2 did not) now provide editors where you can see what is in your dayfile.txt log, and if you click on a particular line you can edit the fields in that line, or indeed delete that line. | ||
If you use these provided editors, then you don't need to worry about the editing rules in the next section. The in-built editors always insert a full line including all fields available in the version of Cumulus you are currently using, so you cannot accidentally insert blank lines. The Cumulus 1 editor also does some validation to ensure the right entry format is used for each field. There is no validation in the MX editor (it was set up relatively quickly in version 3.4.5 as the first of 3 log file editors to plug a gap in MX functionality in earlier versions), but all lines are passed back via an application programme interface to the MX engine which might raise an error when it can't replace the line in the logging file. | |||
== Important Rules when editing dayfile.txt == | == Important Rules when editing dayfile.txt == | ||
| Line 148: | Line 152: | ||
#*Problems with terminating characters are intercepted by operating system, before it reaches the software, but may still stop the software understanding the resulting file, so be careful if you edit the file on a different device to that running Cumulus. | #*Problems with terminating characters are intercepted by operating system, before it reaches the software, but may still stop the software understanding the resulting file, so be careful if you edit the file on a different device to that running Cumulus. | ||
=== Dealing with errors identified by the software === | === Dealing with errors identified by the software === | ||
| Line 168: | Line 164: | ||
See earlier section on this page for the rules and later section for advice on dealing with rogue values. | See earlier section on this page for the rules and later section for advice on dealing with rogue values. | ||
=== Create Missing === | === Using the Cumulus 1 editing feature === | ||
{{Version badge 1}} | |||
'''This section applies to Cumulus 1.x.y only'''. The last command in '''Edit''' [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu |menu]] is ''dayfile.txt''. ''This is how you view'' the dayfile.txt from within Cumulus. This is a text editor, so you can type new values over those currently displayed, insert and delete rows, and it works best when at full screen. Click the ''Help'' button for detailed instructions. Cumulus Help is comprehensive. | |||
You can use this Cumulus 1 editor as follows: | |||
*use '''insert''' key to add one or more missing rows (complete days) manually, | |||
*correct individual values by over-typing, | |||
*use '''delete''' key to remove an entire day (e.g. if you get a 'duplicate' error message), | |||
*use '''Create missing''' button to insert missing rows (complete days) by reading from [[Monthly_log_files]] and automatically calculating the best approximations for each field for those missing days. | |||
==== Create Missing ==== | |||
The Cumulus 1 editor provides a "Create Missing" option where it will, for any dates for which a line does not exist, create a line if it can from reading the [[Monthly log files|detailed log file]] to extract all values relevant to that day and do the necessary minimum/maximum/total/average calculation for each dayfile.txt field, storing the time from the relevant other log file in any time-stamp field in dayfile.txt. If a particular day does not exist as a row on the daily summary log, then 'create missing' can search the observations in the relevant monthly log, and calculate approximate highs, lows and totals to insert as an extra row in the daily summary log. These are approximate because the actual highs and lows for that day are quite likely to have occurred at moments in-between those that were logged. For ''Create missing'' a list of inserted records is produced in [[dayfileeditlog.txt]]. If just some fields are wrong in a particular row (meteorological day) on day file, then there is a [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | work around]] as at all current versions (up to 1.9.4) you can only use 'Create missing' to read from the [[Monthly_log_files]] if the whole day (a line starting with that date) is missing in ''dayfile.txt''. Although Cumulus does not recognise the concept of a sensor not being available, it will write solar information even if you don't have a solar sensor; it does have to cope with reading a monthly log file that might have fewer derivatives than it wants (when using Create Missing) and therefore it may not know what to write into dayfile.txt as the calculated value. Cumulus 1 can't write a null value, so it writes zero for values, and "00:00" for time stamps. If you are using a 9am or 10am rollover time, be aware that create missing in Cumulus 1 always inserts 00:00 for null time-stamps, but in normal running Cumulus uses the rollover time for null time-stamps. | The Cumulus 1 editor provides a "Create Missing" option where it will, for any dates for which a line does not exist, create a line if it can from reading the [[Monthly log files|detailed log file]] to extract all values relevant to that day and do the necessary minimum/maximum/total/average calculation for each dayfile.txt field, storing the time from the relevant other log file in any time-stamp field in dayfile.txt. If a particular day does not exist as a row on the daily summary log, then 'create missing' can search the observations in the relevant monthly log, and calculate approximate highs, lows and totals to insert as an extra row in the daily summary log. These are approximate because the actual highs and lows for that day are quite likely to have occurred at moments in-between those that were logged. For ''Create missing'' a list of inserted records is produced in [[dayfileeditlog.txt]]. If just some fields are wrong in a particular row (meteorological day) on day file, then there is a [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | work around]] as at all current versions (up to 1.9.4) you can only use 'Create missing' to read from the [[Monthly_log_files]] if the whole day (a line starting with that date) is missing in ''dayfile.txt''. Although Cumulus does not recognise the concept of a sensor not being available, it will write solar information even if you don't have a solar sensor; it does have to cope with reading a monthly log file that might have fewer derivatives than it wants (when using Create Missing) and therefore it may not know what to write into dayfile.txt as the calculated value. Cumulus 1 can't write a null value, so it writes zero for values, and "00:00" for time stamps. If you are using a 9am or 10am rollover time, be aware that create missing in Cumulus 1 always inserts 00:00 for null time-stamps, but in normal running Cumulus uses the rollover time for null time-stamps. | ||
| Line 186: | Line 192: | ||
=== Editing inside MX === | === Editing inside MX === | ||
Pick Edit, click that and an editing | Pick Edit, click that and an editing dialog pops up, that does not let you change the line number nor the date, but all other fields show their current contents and you can overtype as necessary. Scroll down to see 2 buttons (how they are labelled depends on which version you are using), the left button ignores any edits you have made (it is labelled 'Close' or "Cancel" and simply does same effect as clicking the "x" in the top right corner), and the right hand button saves your changes (even if it is labelled 'Edit' rather than "Save" in the version you are using). | ||
=== Editing externally === | === Editing externally === | ||
| Line 236: | Line 242: | ||
==Viewing summary figures for a month or period== | ==Viewing summary figures for a month or period== | ||
'''These notes apply to Cumulus 1 only''' | |||
To view a summary of dayfile.txt for a month, calendar year or selected period, use ''This month'' (choose any month, default is month from your computer system date), ''This year'' (choose any year, default is year from your computer system date), or ''This period'' (choose any start and end dates, default is yesterday calculated from your computer system date), within the '''View''' [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu|menu]]. | To view a summary of dayfile.txt for a month, calendar year or selected period, use ''This month'' (choose any month, default is month from your computer system date), ''This year'' (choose any year, default is year from your computer system date), or ''This period'' (choose any start and end dates, default is yesterday calculated from your computer system date), within the '''View''' [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu|menu]]. | ||
*Remember the daily summary log has its records based on rollover to rollover days. | *Remember the daily summary log has its records based on rollover to rollover days. | ||
| Line 256: | Line 265: | ||
**The web tags report longest dry or wet period in the month. | **The web tags report longest dry or wet period in the month. | ||
= List of fields in dayfile.txt = | |||
== Variation by Cumulus version == | |||
The dayfile.txt has grown as Cumulus's functionality has been extended, the table below shows fields grouped by the Cumulus version when those fields were added. | The dayfile.txt has grown as Cumulus's functionality has been extended, the table below shows fields grouped by the Cumulus version when those fields were added. | ||
| Line 269: | Line 277: | ||
'''For your installed build please see ''dayfileheader.txt'' (stored within the folder that contains your Cumulus executable), as that will list which fields are available for you.''' | '''For your installed build please see ''dayfileheader.txt'' (stored within the folder that contains your Cumulus executable), as that will list which fields are available for you.''' | ||
== Information shown in the table == | |||
*The field number is shown starting from zero, this has two advantages: | *The field number is shown starting from zero, this has two advantages: | ||
