Dayfile.txt and Monthly log files: Difference between pages

(Difference between pages)
m
Text replacement - "http://sandaysoft.com/forum/" to "https://cumulus.hosiene.co.uk/"
m (→‎Manipulation outside Cumulus: Minor change to date and integer rules in line with reply by Steve in forum)
 
m (Text replacement - "http://sandaysoft.com/forum/" to "https://cumulus.hosiene.co.uk/")
 
This article is about the log file that contains the values at whatever interval you have set for Cumulus to keep detailed logging. The main files are automatically created with names in the form <Month><Year>log.txt (for example, the file for August 2009 is called <tt>Aug09log.txt</tt>); an entry is made to the current month's file every ''ten'' (default value; you can change this on the station settings screen) minutes, recording the current sensor values. The file is in comma-separated format with one line per entry.
Cumulus maintains a daily log file that holds the highs and lows of each day, as well as a few other nuggets of information. The figures contained in the file are used for the 'This period' display accessed from the '''View''' menu and to build any graphs based on daily values.
 
See [[speciallog.txt]] for the equivalent log file for detailed logging of internal temperature and humidity.
== Viewing the daily log ==
To view the daily log use the ''dayfile.txt'' command in '''Edit''' menu. This is how you view the dayfile.txt from within Cumulus. If you upload the file to your web site, then see [[AnnualDataSummary]] for information about ways to show values for a particular weather criterion in a calendar like display.
 
Times in these files are in the form hh:mm (Cumulus will not be able to understand the file if you edit the time to a format that includes seconds) using the 24 hour clock and local time (system time). All data is logged in the units which which have been selected by the user on the station configuration screen. Note that 'comma-separated' does not necessarily mean that a comma is used to separate the values! If your PC uses a semicolon for the list separator, that will be used in these files. Similarly, decimals use the Windows decimal separator.
'''Note:''' there is no option in the '''View''' menu. (The ''data log'' option in that menu is for viewing the [[Monthly_log_files]] only).
== Dealing with rogue measurements ==
Cumulus provides via '''Configuration''' menu ''Calibration'' screen the ability to screen out spikes in data picked up from your weather station. See Cumulus help screen if you decide to use that to cope with some spikes. But some rogue values may be within the accepted all-time range, but just not really applicable to that particular day.
 
For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”.
If you discover a rogue measurement (perhaps the wind affected your tiping bucket rain gauge or your weather station just reported a corrupted value), on the day it occured, see [[today.ini]] or [[FAQ]] for further advice.
 
==Viewing monthly log files==
If the rogue measurement is discovered some days after it occured, then in many cases it will have affected your highs and lows for the current month, month-by-month, current year, and/or all-time. As first step you should update the appropriate field in the row for the affected date in dayfile.txt. See [[Alltimelog.txt]] for current and previous values to help you know what rogue value to hunt for and know what the high/low value was before the rogue affected it. Once ''dayfile.txt is correct'' the Cumulus editors will allow you to:
As the log file for the current month will be updated frequently by Cumulus (when it is running), Cumulus applies an exclusive lock, and (as explained further below) conflicts can happen if another process seeks to access this file.
* update the highs and lows in [[Alltime.ini]] by choosing ''all time records'' from the '''Edit''' menu.
* update the highs and lows in [[year.ini]] by choosing ''This year's records'' from the '''Edit''' menu.
Also note these log files do not include a header line, but the appropriate headers can be found for the installed version of Cumulus by looking at "Cumulus\monthlyfileheader.txt", i.e. in the directory above the 'data' directory containing the monthly log. The viewing method described next automatically includes the header line.
* update the highs and lows in [[month.ini]] by choosing ''This month's records'' from the '''Edit''' menu. See [[Diags]] for current and previous values of high or low in the current month or the immediate preceding month if the rogue was recorded less than 10 days ago.
* update the highs and lows month-by-month in [[monthlyalltime.ini]] by choosing ''Monthly records'' from the '''Edit''' menu. Click the ''Help'' button for specific instructions on using ''Reset'' and the two ''Copy'' column header buttons in this ''Monthly Records (Highs and Lows) Editor'' to action all rows.
In the '''View''' menu of Cumulus 1, select ''Data logs'', click '''Load...'''. The standard Windows File Select dialogue is displayed for the 'data' subdirectory of the Cumulus installation directory. Select the required monthly log, click '''Open''' and you will see the contents in a neat table with column headings, rows striped, and the date/time on a lighter background. This is a text viewer, and works best when at full screen, but even then you are likely to need to scroll both horizontally and vertically to look at all the figures. If you click on an individual figure the cell will be highlighted, but you cannot edit the figures on a view screen.
 
If no logs seen, see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. Also note, this '''Data logs''' viewer is only for viewing monthly log files (not any other files in the same ''data'' directory), the column headings for standard monthly log files will always be shown at the top of the screen.
 
==How can Monthly log files be used within Cumulus?==
===Uploading current log to your website===
The current monthly log can be updated at your usual website update interval using ''<currentlogfile>'' on the '''Files''' tab of the ''Internet Settings'' screen off the '''Configuration''' menu (or by specifying the parameter 'ExtraLocal[0-99]=<currentlogfile>' in [[Cumulus.ini#Section:_FTP_Site]]). You can also use <currentlogfile> as part of the remote file path, as explained in the Cumulus '''Help'''. It is recommended that your remote file is local so the updating can use copy and you do not have FTP ticked to avoid conflict with logging - see [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=12721&p=99793#p9979 support forum here] for a suggested method.
 
A script can then read the file into a database (see [[ImportCumulusFile]]), or an array, for further processing.
 
CAUTION: Be aware if you tick FTP to upload this file to a remote server, there can be conflict between the thread that Cumulus uses to run FTP and the independent thread that Cumulus uses to log the latest observations into this file. There is a danger that a FTP in progress still has exclusive access and the logging therefore fails e.g. if the interval between FTP is too small or, it is not near the start of the month so, the file is big; see [[FAQ#I_am_getting_I.2FO_error_32_or_I.2FO_error_103]]. An alterative approach is to upload this file just once a day, at a quiet period such as an odd number of seconds after quarter-past midnight using a process scheduled outside Cumulus.
 
 
== Editing the log file ==
* You can't edit the current log file while Cumulus is running, because Cumulus needs exclusive write access. You can edit the log file for a past month, unless you are looking at past months using Cumulus editors.
* You can't edit any log file with a word processor, as they add control characters and other information that Cumulus cannot understand.
* You can use either a specialised comma separated value file editor or a text editor.
* Text editors designed for programmers will allow you to select the encoding (Cumulus will be confused by any Byte Order Mark, so select the encoding type without BOM).
* If you choose to use a spreadsheet, ensure that all columns are treated as normal text, do not let (don't accept Excel default) the spreadsheet recognise the first field contains a date as it will convert that column into a number (e.g. days since 1900 or days since 1970). For example in Libre Office make sure that "Detect special numbers" is not selected. Many spreadsheets will offer a CSV option for saving the file (in Libre Office tick "Edit Filter Settings" on "save as ...").
* If you amend a field, ensure that replacement is same format as original (same decimal separator if not integer).
* Ensure no blank lines, and if you have added in the column headings then remove that heading line when saving ready for Cumulus.
You can see [[#Manipulation outside Cumulus]] below for more information.
 
 
===After you have edited (or created any missing) Monthly log files, you can:===
* CORRECT HIGHS AND LOWS
** update the highs and lows in [[Alltime.ini]] by choosing ''all time records'' from the '''Edit''' menu. See [[Alltimelog.txt]] for current and previous values.
** update (if the created/edited monthly log is within the current year) the highs and lows in [[year.ini]] by choosing ''This year's records'' from the '''Edit''' menu.
** update (if the created/edited monthly log is for the current month) the highs and lows in [[month.ini]] by choosing ''This month's records'' from the '''Edit''' menu. See [[Diags]] for current and previous values.
** update (from version 1.9.3) the highs and lows in [[monthlyalltime.ini]] by choosing ''Monthly records'' from the '''Edit''' menu. Click the ''Help'' button for specific instructions on using ''Reset'' and the two ''Copy'' column header buttons in this ''Monthly Records (Highs and Lows) Editor'' to action all rows.
'' '''Note''' in each of above 4 editing screens you can:
# see the currently stored extremes, and optionally ''Reset'' (row by row) to pre-editing value and timestamp.
# load the monthly log(s) and dayfile.txt to view extremes derivedcalculated from those figures (after your correction of the rogueas valuesavailable) and
# optionally ''Copy'' (row by row) the logged values (and associated date/time information) into the relevant .ini file.
# click the ''Help'' button for detailed instructions on using ''The Records (Highs and Lows) Editors''.
# store your revised figures by clicking ''OK'' (or abandon all your edits by clicking ''Cancel'').
(Each of these screens is a text editor, and works best when at full screen).''
*USE CORRECTED INFORMATION
** see '''[[Average_temperature | Mean temperatures]]''', '''[[Heat/cold_degree_days_and_Chill_hours | Air frosts]]''', '''[[Gale_Days |Gale days]]''', '''[[Rain_measurement#Precipitation_measurement_and_Cumulus | Rain days, Dry days]]''', and '''Total [[Windrun |Wind run]]''' using '''View''' menu displays for ''This month'', ''This period'', and ''This year'' (These figures are not stored anywhere, nor available as webtags).
** use '''create missing''' button in the ''[[dayfile.txt]]'' selection in the '''Edit''' menu; This is a text editor, and works best when at full screen. Cumulus will then look through all monthly log files and create approximate records for any ''missing dates'' in the dayfile if those dates have observations stored in the relevant monthly log. (It creates a file in the Cumulus folder called ''dayfileeditlog.txt'' which contains the entries it created). Click the ''Help'' button for more information. In past and current versions (including 1.9.3), ''create missing'' will not affect any records that are incomplete or contain some rogue values (see [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] below).
** obtain (by [[Monthly_log_files#Viewing_monthly_log_files | viewing the created/edited log file]]) figures needed for manually correcting rogue figures in [[dayfile.txt]]. See for example [[FAQ#My_monthly_and.2For_annual_rainfall_totals_are_wrong | FAQ re correcting rainfall]].
** create the relevant monthly and/or annual '''NOAA style report''' by choosing ''NOAA Monthly Report'' or ''NOAA Annual Report'' from the '''View''' menu, then select the required period using the selectors. Click the ''Update Display'' button to see various statistics (including [[Average_temperature|mean temperature]]) calculated. Generation of complete NOAA reports takes most information from dayfile.txt (based on rollover to rollover meteorological days), except average wind speed and dominant wind direction (both of these it calculates from the monthly log files) for period in question. Finally press ''Save'' button to store the new or amended report.
 
'''Note for obsolete versions up to 1.9.4 build 1085 only:''' The average wind speed used for NOAA reports and 'This period' type screens was, by a bug, based on midnight to midnight days regardless of rollover time in use. (It is calculated on generation of report or screen from monthly log entries). From build 1086, the calculation is based on the rollover time being used.
Ideally, you will subsequently try to edit the rogue data for the particular time it was logged see [[Monthly_log_files#Correcting_any_logged_data_problems]], but correcting the daily summary in dayfile.txt must always be the priority.
 
==Correcting any logged data problems==
==Editing in Cumulus==
The 'Monthly Data Logs Viewer' screen accessed by selecting ''Data logs'' on the '''View''' menu does not allow editing of values, but you can use it to look and see if you are happy with what has been recorded.
The last command in '''Edit''' menu is ''dayfile.txt''. This is a text editor, and works best when at full screen.
Click the ''Help'' button for detailed instructions. Cumulus Help is comprehensive.
You can use this editor to:
*correct individual values by overtyping,
*use '''delete''' key to remove an entire day (e.g. if you get a 'duplicate' error message),
*use '''insert''' key to add one or more missing rows (complete days) manually,
*use '''Create missing''' button to insert missing rows (complete days) by reading from [[Monthly_log_files]] and automatically calculating the best approximations for each field for those missing days.
If you do have a 'duplicate' error, you need to decide which row to ''delete'', and whether to copy any values from that row into the row you are keeping to ensure the correct extremes are retained.
 
Some Weather stations may occasionally supply corrupted or rogue values to Cumulus. Cumulus provides via '''Configuration''' menu [[Cumulus.ini#Section:_Offsets|''Calibration'']] screen the ability to screen out spikes (i.e. abnormal differences between two successive readings from weather station) in data picked up from your weather station. See Cumulus help screen if you decide to use that to cope with future spikes. Some rogue values may be obviously invalid (e.g. a zero pressure), others (e.g. wind shake on a tipping bucket rain gauge on a fine day) may be within the accepted all-time range (so cannot be prevented by spike settings), but you know the value is not right for that particular time.
For ''Create missing'' a list of inserted records is produced in [[dayfileeditlog.txt]]. If just some fields are wrong in a particular row (meteorological day) on day file, then there is a [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] as at all current versions (up to 1.9.4) you can only use 'Create missing' to read from the [[Monthly_log_files]] if the whole day (a row starting with a date) is missing in ''dayfile.txt''.
 
Frequently, the graphical views ('''View''' menu, ''Graphs'' or ''Select-A-Graph'' screens) are the best way to spot many rogue values and to help you guess by interpolation a more reasonable value. Another possiblity is to examine the relevant values in previous versions of the log files stored in the ''backup'' or ''backup/daily'' subdirectories of where your cumulus executable is stored. You may choose to just edit the rogue value into a reasonable value, or (if you are unable to decide on a better value) to delete that particular entire row in the monthly log. As explained above once the log is corrected, new highs and lows in other logs are easily rectified.
The procedure for importing, and processing, pre-Cumulus observations is explained in [[Monthly_log_files |this Wiki topic]].
 
===Manipulation outside Cumulus===
'''Note for version 1.9.3 only:''' Create missing might in some cases be affected by a bug in 1.9.3 that can cause incorrect date order for records (dayfile.txt uses dd/mm/yy or dd-mm-yy and all records should be in ascending chronological order)
Any log files for ''previous'' months can be edited (outside Cumulus) with Cumulus running, and after editing them, you can do further fetch edits within Cumulus as described above. The current month log file can also be edited outside Cumulus, but as it is being updated frequently you must stop Cumulus first (that explains why there is no edit option within Cumulus!).
 
'''Tips''' -- take a copy of the original log file before you work on it outside Cumulus (perhaps give the ''not to be touched copy'' a filename of "<Month><Year>log.csv").
'''Note for versions 1.9.2 or version 1.9.3''' There is a bug in these versions in that 'Create missing' inserts 'heating and cooling degree day' values the wrong way round.
Edit the original file using an editor that treats all fields as text [use either any text editor, a Comma Separated Value editor, or a spreadsheet program that can be instructed ''not'' to recognise special field (like date and time) types]. [[File:Open office (editing cumulus log files).png | left]] If you wish to use "Calc" in 'Apache Open Office', "Libre Office", or similar, select the field separator you use (in this illustration comma is selected, but your file might use semi-colons between fields, don't select commas if your real numbers use comma between integer and decimal parts) and leave "Detect Special Numbers" unselected (as normal default for this software). Best not to use 'Microsoft Excel' as that normally has opposite default, and may change without any warning the format of all dates so Cumulus can no longer recognise the first two fields:
 
'''Important Rules'''
Any bugs are believed fixed in version 1.9.4.
*Each line must contain the fields in correct sequence (since new versions/builds can add to number of fields, Cumulus will accept different months having various row lengths i.e. older ones without the more recent fields at the end).
* The date format uses two digits for the year and should be treated as text. For software (e.g. Excel) with default of recognising formats, ensure that such recognition is turned off, as it is likely to change the dates to either a number representing days since e.g. 31 Dec 1899, or to have four figure years, and then Cumulus will no longer be able to use the log file. (Also the month must be the middle figure, USA convention cannot apply within ''Log_Files'').
* Times in these files are in the form ''hh:mm'' using the 24 hour clock and local time (system time). See [[FAQ#How_does_Cumulus_handle_Daylight_Saving_Time.3F]].
*See [[#List_of_fields_in_the_file]] to identify if the value you enter for a particular field is an integer (like bearings and humidity values) or a real number (like pressure and rainfall values) in format x.y using your system decimal notation for the decimal point. Fields that accept real number format will also accept integers, but fields expecting integers will not accept decimal points.
*Negative values are allowed for temperatures, all others must be zero or positive. Humidity has a maximum of 99, bearing has a maximum of 360.
*Nulls (',,' if your field separator is a comma, ';;' if your field separator is a semicolon) are not allowed in the row, so if you do not know the value for a particular field within the row, then type in a zero, -999, or 9999 (check the format in the middle column of [[#List_of_fields_in_the_file]] and think about the range of numbers allowed for particular fields as per examples in previous bullet). In other words put in an obviously strange number as Cumulus does not have a defined null! This strange number may distort graphs and any calculations, but it will remind you to make a more appropriate guess as soon as you can.
* Note that some fields (e.g. those related to evaporation, UV, solar data) will only contain valid data if your station has the appropriate sensor(s) (without the sensor they contain the default value of zero), but if they apply make sure you note which are stored as integers.
 
==Manipulation outside=Importing pre-Cumulus data===
See [[FAQ#I.E2.80.99ve_just_installed_Cumulus.2C_and_it_didn.E2.80.99t_download_all_the_old_data_from_my_weather_station|FAQ: I've just installed Cumulus and it didn't download all the old data from my weather station]]
The file is called <tt>dayfile.txt</tt> which can be viewed in a text editor or imported into various database systems or spreadsheets to manipulate as you wish. Just remember that Cumulus updates it as part of the rollover process, so never attempt to work on it either when Cumulus has just been restarted and is checking/updating the logs, or around the midnight/9am/10am local rollover time when Cumulus is writing a new row.
 
Given that monthly log files are used as input for '''updating the all-time, month-by-month alltime (from version 1.9.3), and this year record extremes''', for '''creating missing dayfile.txt entries''', and for '''creating NOAA style reports''', you may have some observations from your weather station recorded manually or electronically for a period before you first starting using Cumulus and want to create new monthly logs to feed into Cumulus.
'''Tip''': Take a copy of the file before you work on it outside Cumulus (perhaps call the copy 'dayfile.csv' so Windows can recognise its structure).
 
'''Here are some issues to consider.''' There are some postings in the Support Forum about importing past data. Essentially match the fields listed in [[Monthly_log_files#List_of_fields_in_the_file|Monthly log files: List of fields in the file]] below with the fields you have available in your source.
'''Note''': Since new versions/builds can add to number of fields, Cumulus will accept lines of various lengths without the more recent fields at the end. (Additions for the last few builds only are indicated below, you can explore details of earlier versions via the official [http://sandaysoft.com/products/cumulus product] page).
* Dates and times might need some pre-processing (spreadsheet packages usually have ability to select part of a text string and to cocatenate a number of strings) to convert them to text in the formats mentioned above. (Note - it is crucial in a spreadsheet to treat the date as text; Do not let the spreadsheet recognise it as a date as it might change the format or store as a 'days since X' number).
* Remember that null values are not allowed within a row, so ensure that you enter an obviously wrong value (-99 or zero might be do) between your operating system defined field separator (normally ',' or ';') and make the row length the minimum shown in the list of fields below. If you use a spreadsheet, treat the file as Comma Separated Value type, but on saving choose the appropriate field separator character (e.g in Libre Office or Open Office 'Save as CSV' select 'Edit Filter Settings').
* Wind directions might be reported as cardinal points (north, south etc) and need converting to bearings in (integer) degrees (spreadsheet packages provide ''look up'' functionality to do such conversions).
* Wind speed fields may not directly match to weather station outputs. See [[#Cumulus_Wind_Speed_Terminology]]
* Temperature fields may not directly match to weather station outputs. Copy across the fields you can and set the others to -999.99 so they are obvious. See the [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] described below for a way of getting Cumulus to generate (from your newly created monthly logs) apparent temperature, wind chill and heat index to replace any -999.99 figures.
* Rainfall fields may not directly match to weather station outputs. [[FAQ#How_is_my_rain_rate_calculated.3F | Rain rate]] in field 8 may be generated from the [[Monthly_log_files#Using_Monthly_logs_to_deal_with_shorter_.28or_incomplete.29_dayfile.txt_records_for_particular_dates | workaround]] if unavailable from your station. The daily total in field 9 is since rollover time, and available weather station outputs may need some processing in your spreadsheet to derive this. Some stations output a count of total rainfall, others the total annual rainfall, either of these may be used for field 11. Rainfall processing issues are frequently discussed in the Support Forum, so you may find the solution there or ask there for assistance.
 
'''The date that Cumulus first started tracking all-time records''' (<#recordsbegandate> see [[Webtags#Records]]), does ''not'' need to be updated for Cumulus to recognise earlier monthly log files in its ''data'' subfolder and to use any monthly logs found in the various '''View''' and '''Edit''' options available from the Cumulus main screen. As <#recordsbegandate> is on the default Cumulus ''recordsT.htm'' template with the label "Records began on", inserting pre-Cumulus data implies you might want to amend that label wording on that template to indicate it is the date that Cumulus was first operational, rather than the date of the earliest tracked highs and lows.
'''Important Rules''':
 
* The (meteorological) date format uses ''two digits for the year''. Edit this file using an editor that treats all fields as text (a text editor or a spreadsheet program that can be instructed ''not'' to recognise special field types). For software (e.g. Excel) with default of recognising formats, ensure that such recognition is turned off, as it is likely to change the dates to either a number representing days since e.g. 31 Dec 1899, or to have four figure years, and then Cumulus will no longer be able to use the log file. (Also the month must be the middle figure, USA convention cannot apply within logfiles).
If you prefer to edit the start date and keep the ''recordsT.htm'' template label "Records began on" unchanged, the method is to stop Cumulus, and amend the ''StartDate='' line in [[cumulus.ini]] within the main Cumulus folder before re-starting Cumulus, but (in case you make a mistake) back up everything first!
* Times are in ''format hh:mm''
 
* Most value fields are in ''real number format x.y'' using your system decimal notation, a few (e.g. bearings, solar, humidity) are ''integers'' (see list of fields in file). Whilst an integer can be used for a real number field, decimals are not allowed in an integer field.
=== Cumulus Wind Speed Terminology ===
* If you insert a ''lowest or highest value'' into a day, where there was none before, insert a ''timestamp'' too, as a dayfile.txt row is only accepted by the Cumulus editor if a value and any related timestamp are either both present or both absent. (Use a timestamp of your rollovertime 00:00, 09:00 or 10:00 if you have not looked up the precise time).
How wind speeds relate to various weather stations is explained in FAQ. Basically there are 3 values processed by Cumulus:
* Nulls ',,' are not allowed in the line, so if you do not know the value for a particular field within the line, then type in a zero for integer values and an extreme with opposite value (e.g. -999.9 for a signed decimal maximum, and 999.9 for a decimal minimum) for decimal format (replace the full stops with your decimal separator).
#Gust - the highest wind speed over a particular period (this period defaults to 10 minutes),
* Make sure that any editing does not create any ''blank lines'' in the file.
#Average - the moving average of wind speed measurements over a particular period (this period can be set to what suits you, but defaults to 10 minutes),
* All rows must ''start with date'' and include some of the parameters listed ''in correct sequence''.
#Latest - the most recent (wind speed or gust speed) measurement.
* Rows can vary in length but only by missing off ''fields at the end''.
[[FAQ#What_do_the_various_wind_speeds_in_Cumulus_correspond_to.2C_with_Davis_stations.3F|FAQ: Wind speeds in Cumulus with Davis stations]], the ''EasyWeather'' differences in [[FAQ#What_do_the_various_wind_speeds_in_Cumulus_correspond_to.2C_with_Fine_Offset_stations.3F|FAQ: Wind speeds in Cumulus with Fine Offset stations]] and [[FAQ#How_do_I_set_up_recording_wind_speeds_in_Cumulus.2C_with_Oregon_Scientific_and_La_Crosse_stations| FAQ: Wind speeds with Oregon Scientific and La Crosse stations]]
* The row terminator normally expected is ''CRLF'', ensure any external editor does not change the terminator.
 
When Cumulus is running normally, it actually stores (in the fifth item ''after'' the date in each entry, column F in a spreadsheet) the highest (moving) average in the period since the previous logger record. If Cumulus is not running, it stores the average from the station logger, that is the latest reported moving average. For this reason the most accurate wind speed records are achieved if Cumulus is running all day every day.
Cumulus calculates the monthly/annual/period ''average wind speed'' (as used for 'This ...' screens in '''View''' menu and the NOAA report) by trawling through the monthly logs, summing the values for those individually logged average wind speed, and dividing by the number of entries for the required calendar days. It therefore assumes that each entry has equal 'weight', i.e. each one covers the same period. If your monthly log(s) is/are a mixture of values stored while Cumulus is running and values copied from the station logger during periods when Cumulus was not running, then it is possible the two logging intervals have not been set to the same time, so the output value could be skewed for that month/year/period because of the difference in what was stored and the difference in the logging interval.
 
[[Windrun]] is also calculated from average wind speed measurements, in this case every minute if Cumulus is running, or from the values that are logged (at station logging interval) as average wind speed when Cumulus is restarted and catches up from the station logger.
 
===Using Monthly logs to deal with shorter (or incomplete) dayfile.txt records for particular dates===
'''WORKAROUND if dates are present in monthly log and dayfile.txt but not all fields for that date exist in dayfile.txt'''
 
Example: records created by earlier version of Cumulus [to help you, the versions (not builds) at which fields were added are indicated below].
Second example: records imported prior to Cumulus processing them, so some calculated parameters (apparent temperature, heat index, rain rate, wind run) may not have been available from your weather station to insert in the log files.
 
''One method is (not near rollover time) to (1) take a copy of dayfile.txt original as backup, (2) in original file delete any days with partial information (e.g. from Cumulus versions that created fewer fields), (3) use ''create missing'' in the [[dayfile.txt]] selection in the '''Edit''' menu option of Cumulus (note whilst datafile.txt normally calculates all parameters like minimums and maximums from very frequent samples of your weather station, the resolution of create missing is limited to the interval between logging records in the monthly log), (4) rename the amended dayfile.txt as dayfile(generated).txt, (5) create a new dayfile.txt and (6) use a text editor to merge the required fields from the new dayfile(generated).txt with all other fields from (with reading access only) the backup copy of the original file. (This method preserves the original as a backup so you can experiment with different merges and do some cross-checking).''
 
==List of fields in the file==
The table is split by Cumulus version for all changes since version 1.8.5; it shows:
*Alternative lists:
*The field number (starting from zero to be consistent with index used for arrays in programming languages like JavaScript) and the equivalent letter that would be seen in a spreadsheet like Libre Office, Open Office, Excel etc are listed in first column.
**A version of this list with spreadsheet column letters can be downloaded from the support forum [http://sandaysoft.com/forum/viewtopic.php?f=18&t=8690 here],
*The second column shows an example, and the values shown are sometimes integers and sometimes floating point numbers, and this depends on a number of factors, including your station type. If you are editing the values manually, you should use integers for humidity, wind bearings, and the two solar radiation figures, and floating point for the others (add a ".0" if necessary). Note that the figure varies depending on the units you select, and your weather station may not have all the sensors needed, in that case for you some figures will always be zero.
**the appropriate list of fields for your installed build is stored as dayfileheader.txt within the folder that contains your Cumulus executable.
*The third column describes the observation often with a link to where there is more information.
===Version 1.8.5:===
{| border="1" class="wikitable"
|-
! style="width:60px" | Field #
! style="width:80px" | Example
! style="width:450px" | Description
|-
| 00(A)
|22/04/11
| Date as ''2 figure day [separator] 2 figure month [separator] 2 figure year'' - the separator is that set in the windows system short date format (see [[setup]])
|-
| 01(B)
|10:25
| Current time
|-
| 02(C)
|8.1
| Current temperature
|-
| 03(D)
|96
| Current [http://en.wikipedia.org/wiki/Relative_humidity relative humidity]
|-
| 04(E)
|7.5
| Current [http://en.wikipedia.org/wiki/Dewpoint dewpoint]
|-
| 05(F)
|13.5
| Cumulus 'Average' wind speed (See [[#Cumulus_Wind_Speed_Terminology]])
|-
| 06(G)
|20.3
| Cumulus 'Gust' wind speed
|-
| 07(H)
|138
| Average wind bearing (integer degrees)
|-
| 08(I)
|7.2
| [[FAQ#How_is_my_rain_rate_calculated.3F | Current rainfall rate]]
|-
| 09(J)
|5.4
| Total rainfall today so far (i.e. resets to zero at daily rollover)
|-
| 10(K)
|30.10
| Current sea level pressure
|-
| 11(L)
|215.2
| Total rainfall counter (content depends on weather station type, might be [[FAQ#It.27s_January_1st_and_my_.27rain_since_midnight.27_and.2For_.27last_24_hours_rain.27_are_wrong.2Fnegative | rainfall total so far this year]]. This figure is not intended to be useful in itself, it may vary for no apparent reason. It is used internally by Cumulus.)
|-
| 12(M)
|20.3
| Inside temperature
|-
| 13(N)
|53
| Inside humidity
|-
| 14(O)
|17.6
| Cumulus 'Latest' gust (See [[#Cumulus_Wind_Speed_Terminology]])
|-
| 15(P)
|4.8
| [[wind chill]] (content depends on weather station type and Cumulus station settings)
|-
| 16(Q)
|8.1
| [[Heat index]]
|-
| 17(R)
|0.5
| UV Index (only valid if output by weather station)
|-
| 18(S)
|197
| Solar Radiation (only valid if solar sensor on weather station)
|}
 
===Added in 1.9.1===
{| border="1" class="wikitable"
|-
! style="width:60px" | Field #
! style="width:80px" | Example
! style="width:450px" | Description
|-
| 19(T)
|0.08
| Evapotranspiration (only valid if output by weather station)
|-
| 20(U)
|171.88
| Annual Evapotranspiration (only valid if ET sensor on weather station)
|-
| 21(V)
|3.4
| [[Apparent temperature]]
|-
| 22(W)
|663
| Current theoretical max solar radiation (see [[FAQ#What_is_this_.22Solar_Max.22_value_which_Cumulus_is_displaying.3F | the FAQ]])
|-
| 23(X)
|3.1
| Hours of sunshine so far today (only valid if solar sensor on weather station)
|}
 
===Added in 1.9.2===
{| border="1" class="wikitable"
|-
! style="width:60px" | Field #
! style="width:80px" | Example
! style="width:450px" | Description
|-
| 24(Y)
|158
| Current Wind bearing (See 7)
|}
===Added in 1.9.3===
{| border="1" class="wikitable"
|-
! style="width:60px" | Field #
! style="width:80px" | Example
! style="width:450px" | Description
|-
| 25(Z)
|6.0
| RG-11 rain today (only valid if output by sensor)
|}
===Added in 1.9.4===
{| border="1" class="wikitable"
|-
! style="width:60px" | Field #
! style="width:80px" | Example
! style="width:450px" | Description
|-
| 26(AA)
|5.7
| Total Rainfall since midnight (this is for 0900/1000 'rollover' users; normally same as rain today for 'midnight rollover' users)
|}
 
==Example lines from the file==
The field number (starting from zero to be consistent with index used for arrays in programming languages like JavaScript) with description and cross-references to further explanation:
* 00: Date as ''2 figure day [separator] 2 figure month [separator] 2 figure year'' - the separator is that set in the windows system short date format (see [[setup]])
* 01: Highest wind gust
* 02: [[Wind_measurement#Wind_Direction | Bearing]] of highest wind gust (integer)
* 03: Time of highest wind gust
* 04: Minimum [[Temperature_(and_humidity)_measurement#Cumulus_Calculated_Parameters | temperature]]
* 05: Time of minimum temperature
* 06: Maximum temperature
* 07: Time of maximum temperature
* 08: Minimum [[Pressure_Measurement | sea level pressure]]
* 09: Time of minimum pressure
* 10: Maximum sea level pressure
* 11: Time of maximum pressure
* 12: Maximum [[Rain_measurement#Rain_Rate | rainfall rate]]
* 13: Time of maximum rainfall rate
* 14: Total rainfall for the day
* 15: [[Average temperature]] for the day
* 16: Total [[Windrun | wind run]]
* 17: Highest Average Wind Speed
* 18: Time of Highest Avg. Wind speed
* 19: Lowest [[Temperature_(and_humidity)_measurement | humidity]]
* 20: Time of lowest humidity
* 21: Highest humidity
* 22: Time of highest humidity
* 23: Total evapotranspiration
* 24: Total hours of sunshine
* 25: High [[Heat index]]
* 26: Time of high heat index
* 27: High [[Apparent temperature]]
* 28: Time of high apparent temperature
* 29: Low apparent temperature
* 30: Time of low apparent temperature
* 31: High hourly rain
* 32: Time of high hourly rain
* 33: Low [[wind chill]]
* 34: Time of low wind chill
* 35: High [[Temperature_(and_humidity)_measurement#Cumulus_Calculated_Parameters | dew point]]
* 36: Time of high dew point
* 37: Low dew point
* 38: Time of low dew point
 
Note that your field delimiters may be different, and your date delimiter too.
The next 3 entries were added in version 1.9.2 Build 1004
* 39: Today's dominant/average wind direction (integer)
* 40: [[Heat/cold degree days and Chill hours | Heating degree days]]
* 41: [[Heat/cold degree days and Chill hours | Cooling degree days]]
 
Added in version 1.9.3 build 1036
* 42: High solar radiation
* 43: Time of high solar radiation
* 44: High UV Index
* 45: Time of high UV Index
 
==ExampleAn extract of a few lines of the file== (v.1.9.0)
 
Here logging is quarter hourly, the field delimiter is a semicolon, the decimal separator is a comma, and the date separator a dash.
An extract of a few lines of the dayfile.txt
30-09-10;19:00;16,4;94;15,4;5,2;13,3;17;3,6;21,0;995,3;47,7;25,6;62;6,1;16,4;16,4;0,0;0
30-09-10;19:15;16,4;94;15,4;5,6;11,2;12;18,0;24,0;995,0;50,7;25,6;62;7,2;16,4;16,4;0,0;0
30-09-10;19:30;16,2;94;15,2;7,9;15,8;355;7,2;25,8;994,3;52,5;25,7;62;12,2;16,2;16,2;0,0;0
30-09-10;19:45;16,0;94;15,0;9,9;19,4;7;7,2;27,9;993,3;54,6;25,7;62;14,8;15,8;16,0;0,0;0
30-09-10;20:00;15,9;94;14,9;12,4;20,9;354;7,2;30,0;993,0;56,7;25,7;62;19,4;15,4;15,9;0,0;0
30-09-10;20:15;15,8;94;14,8;8,4;15,8;349;14,4;32,7;993,4;59,4;25,8;61;12,2;15,8;15,8;0,0;0
30-09-10;20:30;15,4;94;14,4;13,8;33,1;317;28,8;40,5;993,7;67,2;25,8;61;23,4;14,7;15,4;0,0;0
30-09-10;20:45;15,1;94;14,1;20,3;34,2;356;7,2;43,8;992,3;70,5;25,8;60;29,5;13,8;15,1;0,0;0
30-09-10;21:00;15,3;94;14,3;20,2;35,6;358;10,8;46,8;991,0;73,5;25,8;60;28,1;14,0;15,3;0,0;0
30-09-10;21:15;15,3;95;14,5;16,6;31,7;358;10,8;49,5;991,4;76,2;25,8;60;20,9;14,3;15,3;0,0;0
30-09-10;21:30;15,3;94;14,3;14,0;27,0;324;18,0;54,3;992,3;81,0;25,8;60;15,8;14,5;15,3;0,0;0
30-09-10;21:45;15,2;94;14,2;13,0;25,6;323;10,8;57,9;992,3;84,6;25,8;59;24,5;14,5;15,2;0,0;0
30-09-10;22:00;15,0;94;14,0;16,7;31,7;312;10,8;60,6;993,0;87,3;25,8;59;23,4;13,9;15,0;0,0;0
30-09-10;22:15;14,9;94;13,9;16,0;30,6;357;10,8;63,0;991,6;89,7;25,8;59;20,9;13,9;14,9;0,0;0
30-09-10;22:30;14,9;94;13,9;17,6;31,7;3;3,6;63,3;990,6;90,0;25,8;59;19,4;13,7;14,9;0,0;0
 
this is an example of a file from 1.9.1, with solar data:
<pre>
01/08/11,19.3,61,10:22,12.5,06:58,23.8,14:49,1014.26,20:46,1018.83,09:28,0.0,00:00,0.0,17.8,21.6,4.6,10:44,36,14:14,86,01:56,3.56,8.9,23.8,14:49,23.1,14:50,12.3,06:59,0.0,00:00,12.5,06:58,11.3,00:16,6.9,14:34,354,2.0,1.5
 
Here logging is increased to every five minutes, the field delimiter is a comma, the decimal separator is a full stop, and the date separator is also a full stop.
02/08/11,16.1,20,16:55,14.7,06:45,24.2,13:54,1013.79,19:13,1015.65,11:14,0.0,00:00,0.0,18.9,13.7,8.0,15:55,42,20:42,85,06:50,2.79,4.9,24.2,13:54,24.3,13:55,15.1,06:40,0.0,00:00,14.7,06:45,14.8,11:59,7.0,21:09,57,1.0,1.7
 
03/08/11,14 22.5,36,17:23,1404.911,0510:5025,248.61,14:4696,10127.705,18:4413,1015.9920,08:34138,0.0,00:00,0.0,191013.424,17215.2,420.83,16:0453,5017,14:38,79,07:04,34.058,5.8,24.61,14:46,25.4,14:47,150.0,05:50197,0.0,00:0008,14171.988,05:50,143.24,20:01663,80.9,00:16,32,0.8,1.9
22.04.11,10:30,8.1,96,7.5,13,20,142,0.0,0.0,1013.28,215.2,20.2,53,11,4.8,8.1,0.0,216,0.08,171.88,3.3,673,0.0
22.04.11,10:35,8.1,96,7.5,12,18,142,0.0,0.0,1013.31,215.2,20.2,53,11,5.1,8.1,0.0,227,0.08,171.88,3.8,682,0.0
 
and from version 1.9.2:
04/08/11,17.7,16,15:43,14.1,06:20,25.3,15:06,1013.08,18:42,1015.31,08:28,0.0,00:00,0.0,20.2,19.4,8.1,14:12,52,18:20,92,06:55,3.30,9.1,25.3,15:06,26.8,14:55,14.9,06:20,0.0,00:00,14.1,06:20,15.8,14:55,12.5,06:25,36,1.0,2.9
 
Here logging is at the default interval, and the delimiters have their default settings for UK systems.
05/08/11,16.1,32,12:52,14.2,06:12,22.2,14:07,1013.89,00:01,1016.36,09:43,0.0,00:00,0.0,18.6,21.6,5.2,13:00,62,15:57,87,06:11,3.30,8.4,22.2,14:07,23.5,14:10,14.8,07:19,0.0,00:00,14.2,06:12,15.4,10:33,12.0,06:03,34,0.9,1.3
 
06 01/0810/11,16.1,309,1104:1540,1418.30,05:2975,2213.45,17:12,10145.460,20:02,10166.977,10:38169,0.0,00:00,0.0,181021.4,19885.23,524.54,16:2161,55,13:33,92,05:20,25.796,718.9,22.4,17:12,23.30,18:17,15.1,06:090,0.0,00:000,140.300,05:29,140.200,18:12,1017.95,10:380,32,10.10,1.3158
01/10/11,04:50,17.9,75,13.4,4.5,9.5,169,0.0,0.0,1021.4,885.3,24.4,61,2.0,17.9,17.9,0.0,0,0.00,0.00,17.5,0,0.0,180
01/10/11,05:00,17.8,75,13.3,5.1,9.5,177,0.0,0.0,1021.2,885.3,24.4,61,7.5,17.8,17.8,0.0,0,0.00,0.00,17.2,0,0.0,180
 
07/08/11,17.7,342,13:24,12.9,05:47,24.1,14:53,1013.92,19:49,1016.43,09:36,0.0,00:00,0.0,18.4,19.1,6.3,14:06,48,12:45,89,05:36,3.30,9.0,24.1,14:53,24.6,15:48,13.3,05:47,0.0,00:00,12.9,05:47,14.6,15:52,10.7,11:33,11,1.6,1.7
</pre>
 
[[Category:Log Files]]