5,838
edits
Calculate Missing Values: Difference between revisions
m
→CreateMissing.exe: updated for Create Missing 1.1.0
mNo edit summary |
m (→CreateMissing.exe: updated for Create Missing 1.1.0) |
||
| Line 141: | Line 141: | ||
This is only available if you already have MX installed and have been using it. | This is only available if you already have MX installed and have been using it. | ||
These notes relate to "Create Missing Version 1.1.0" which is for "Cumulus MX release 3.12.0" and above. At the time of amending this section Create Missing version 1.0.2 had been tested and was found to have a bug. | |||
Download link at [[Software#Create_Missing]] page, unzip, and install in same folder as CumulusMX.exe. The read me for this utility program is at https://github.com/cumulusmx/CreateMissing/blob/master/README.md. | Download link at [[Software#Create_Missing]] page, unzip, and install in same folder as CumulusMX.exe. The read me for this utility program is at https://github.com/cumulusmx/CreateMissing/blob/master/README.md. | ||
| Line 150: | Line 152: | ||
* On Microsoft Windows operating systems, type <code>CreateMissing</code> in a command window, Powershell window, or Terminal window (whichever is available when you right click in the folder or on the "Start" icon. | * On Microsoft Windows operating systems, type <code>CreateMissing</code> in a command window, Powershell window, or Terminal window (whichever is available when you right click in the folder or on the "Start" icon. | ||
The utility program can be run while MX is left running (except at rollover time when MX writes to dayfile.txt, this includes any time while MX is doing "catch-up" and therefore can do rollovers for past days), but as MX only reads the dayfile.txt as MX starts up, any changes this utility makes will not be picked up by MX until MX is stopped and restarted. | The utility program, CreateMissing.exe, can be run while MX is left running (except at rollover time when MX writes to dayfile.txt, this includes any time while MX is doing "catch-up" and therefore can do rollovers for past days), but as MX only reads the dayfile.txt as MX starts up, any changes this utility makes will not be picked up by MX until MX is stopped and restarted. | ||
This utility program looks in [[Cumulus.ini]] for: | This utility program, CreateMissing.exe, looks in [[Cumulus.ini]] for: | ||
# The Cumulus start date in "StartDate=" parameter, which defaults to the date you first ran Cumulus (although it can be edited to another date, such as when you imported earlier data or moved to a new home after you first used Cumulus). That will be the earliest date the utility program processes. However, if a dayfile.txt file exists and has an earlier date, then a prompt will ask you if you want to use the earlier date or the "StartDate=" date. | # The Cumulus start date in "StartDate=" parameter, which defaults to the date you first ran Cumulus (although it can be edited to another date, such as when you imported earlier data or moved to a new home after you first used Cumulus). That will be the earliest date the utility program processes. | ||
#* However, if a dayfile.txt file exists and has an earlier date, then a prompt will ask you if you want to use the earlier date or the "StartDate=" date. If you answer "Y" to use earlier date, the utility program will continue, starting at that earlier date. If you answer with anything else, the utility program will exit. | |||
#* The utility program will also check the start date against the current date, and will exit unless you have more than one day completed since you started using MX. | |||
# The meteorological day start time in "RolloverHour=" and "Use10amInSummer=" parameters. This identifies which standard log lines belong to each day by checking against date and time of that line. | # The meteorological day start time in "RolloverHour=" and "Use10amInSummer=" parameters. This identifies which standard log lines belong to each day by checking against date and time of that line. | ||
# Units and Number of decimal places associated with temperature, wind, wind average, rainfall, pressure, Ultra Violet, sunshine, evapotranspiration, wind run, and temperature trend. | |||
# Thresholds for Heating Degree Days, Cooling Degree Days, and chill hours. | |||
# station type (needed to determine which source values are available) | |||
# month when Chill hours season starts | |||
# options (use zero bearing, use 10 minute wind average, use speed or gust for wind average calculation, fix maximum humidity, read or calculate dew point, read or calculate wind chill, synchronisation time, , set clock option, read or calculate pressure trends, log extra sensors or not, ignore station clock or not, round wind speeds or not, air quality sensor or not, check that minimum number of sensors are updating or not, time for averaging wind bearings, time for averaging wind speed, and time for peak gust) | |||
The utility program, CreateMissing.exe, uses the same module to calculate derived values (like dew point, wind chill, apparent temperature, feels like, and Humidity Index) from source values as CumulusMX.exe. This is how it is able to insert missing fields in the standard log files. As it processes each line in any standard log file, if a particular field is missing, it will calculate it from other fields in the line if it can. The utility program, CreateMissing.exe, can update the various [[:Category:Ini Files]] with any new extremes found while it is calculating new fields for any standard log file. | |||
The utility program,CreateMissing.exe, looks in the [[Data Folder|data sub-folder]] to see if dayfile.txt exists (if it does, its file name has a '''.sav''' suffix added after the ''.txt''). If a file with .sav suffix already exists, the utility program will stop, as it cannot create a file if it already exists. | |||
As the utility program, CreateMissing.exe, reads each line in any standard log file it is processing, it works out from the date and time in that line which meteorological date to assign that line to, using the rollover time and use 10am in summer settings. Any day with less than 5 standard log file lines is ignored for dayfile.txt, as is current day. | |||
#At the start of each day, any maximum field in the output file (that is created as dayfile.txt) is set to an extreme negative, any minimum field in the same file is set to an extreme maximum, any daily total field is set to zero, and the cumulative chill hours field is set (either to the figure for the day before, or reset to zero for first day of new season). | |||
# As each standard file line in that meteorological day is processed, any maximum field in the output file is updated if the read figure is higher than previous figure, any minimum field in the same file is updated if the read figure is lower than the previous figure, the total rainfall calculation changed between versions 1.0.0 and 1.0.1 (basically rain total is taken from the final line per day in the standard log file field 9) | |||
# For fields like Heating Degree Days and Cooling Degree Days, CreateMissing.exe is tracking the temperature field 2 against the threshold it has read from Cumulus.ini, so if the degree day figure is missing, the utility is able to insert a figure based on the difference between temperature and threshold and the time since previous log (for heating degree days it is heating threshold minus temperature times minutes since last previous divided by 1440; for cooling it is temperature minus cooling threshold times minutes divided by 1440). | |||
# For total sunshine, day starts just after midnight, and ends next midnight; even if that is not rollover time; the figure stored in dayfile.txt is stored against the meteorological day that applies at 1 minute past midnight. Basically if rollover time is not midnight, the sunshine on a particular calendar day is assigned to previous date in dayfile.txt. | |||
# The average temperature added to dayfile if it is not already in existence, is based on the same approach as used in [[today.ini]], i.e. the utility maintains a cumulative total of time passed for the meteorological day, and a cumulative total incremented as each standard log file line is processed which is an increment of "minutes_since_previous_line multiplied by (latest_temperature + last_temperature) divided by 2". | |||
# Chill hours output is incremented from existing figure by number of minutes since last line divided by 60, if the temperature field is below the threshold. | |||
# Tracking the maximum rainfall in last hour figure for a day is obviously a bit complicated, as the utility has to work on a running hourly rainfall total (total rainfall on line being processed minus total rainfall on line for one hour ago). | |||
The utility program will output to any terminal session open and to a file saved in [[MXdiags_folder|MXdiags sub-folder]]. | The utility program will output diagnostic messages both to any terminal session open, and to a file saved in [[MXdiags_folder|MXdiags sub-folder]]. | ||
Here is a short section of typical output (there will also be messages when each log file is opened or finished with): | Here is a short section of typical output (there will also be messages when each log file is opened or finished with): | ||
| Line 192: | Line 212: | ||
2021-06-08 19:35:45.007 Date: 30/07/2020 : Entry is OK | 2021-06-08 19:35:45.007 Date: 30/07/2020 : Entry is OK | ||
</pre> | </pre> | ||
==Using a PHP script on your web server== | ==Using a PHP script on your web server== | ||