Standard log files - Revision history

Sfws: Release 3.9.0 field 30 section corrected as change delayed

2022-04-10T12:15:35Z

Release 3.9.0 field 30 section corrected as change delayed

@@ Line 360: / Line 360: @@
 | [[Humidex]]
 |}
 = Accessing the Standard Log File =

Sfws: /* Number of fields per line varies by release */ Missing bracket

2022-04-09T17:42:49Z

Number of fields per line varies by release: Missing bracket

← Older revision		Revision as of 17:42, 9 April 2022
Line 109:		Line 109:
	* [[Temperature_(and_humidity)_measurement#Dry_and_Wet_Bulb\|Wet Bulb temperature]]		* [[Temperature_(and_humidity)_measurement#Dry_and_Wet_Bulb\|Wet Bulb temperature]]
	* [[Temperature_(and_humidity)_measurement#How_Cumulus_software_handles_Temperature_and_Humidity\|Dew Point]]		* [[Temperature_(and_humidity)_measurement#How_Cumulus_software_handles_Temperature_and_Humidity\|Dew Point]]
	* Rain since midnight		* Rain since midnight)
	''Please note the developer does not mention any of this at [https://github.com/cumulusmx/CreateMissing/blob/master/README.md his utility details explanation] so the author of this Wiki update cannot guarantee the above list is correct''		''Please note the developer does not mention any of this at [https://github.com/cumulusmx/CreateMissing/blob/master/README.md his utility details explanation] so the author of this Wiki update cannot guarantee the above list is correct''

Sfws: add some cross-references

2022-04-09T17:41:15Z

add some cross-references

@@ Line 52: / Line 52: @@
 For ''more information'' on these files see in the '''Cumulus help''' file, in the section “The Data log file”. This is installed with Cumulus 1, but can be read with Windows after downloading from [[Software#Resources]] page in this Wiki.
-==List of fields in the file==
+=List of fields in the file=
 The table is split by Cumulus version for all changes since version 1.7.x  (Steve Loft did not retain his list of changes in 1.4, his '''changes.txt''' is rather sparse in details for many releases, and the support forum excludes any posts before 2 Jun 2008, so earlier changes are not now documented).
@@ Line 78: / Line 78: @@
 #* MX since version 3.7.0 is more fussy, but less consistent, certain fields contain a single space when not used, others contain zero, and others can be left empty
 #The final column describes the observation often with a link to where there is more information.
 ===Before Version 1.7.10:===
@@ Line 253: / Line 289: @@
 |Unsigned integer
 |158
-| Current Wind bearing (See 7)
+| Current Wind bearing (See average wind bearing in field 8)
 |}
 ''For Cumulus MX editing a file created by Cumulus 1'', any of the preceding fields that are not present will be '''defaulted to an empty string''', but there will always be 50 fields (unless you are using MX version before 3.6.0, it which case MX cannot read the file.

Sfws: /* How often is data stored in the standard file? */

2022-04-09T16:41:12Z

How often is data stored in the standard file?

← Older revision		Revision as of 16:41, 9 April 2022
Line 17:		Line 17:
	=How often is data stored in the standard file?=		=How often is data stored in the standard file?=

	These store the full set of current (read and derived) values at whatever interval you have set for Cumulus to keep detailed logging. ~~The default time interval between adding data logging lines is every 10 minutes, but you can change this:~~		These store the full set of current (read and derived) values at whatever interval you have set for Cumulus to keep detailed logging.
			* The interval has to be a fraction of an hour, so the least often storing is every half hour.
⚫	*Cumulus 1: On main screen, select '''Configuration''' menu, select the [[Cumulus_Screenshots#Station\|'''station settings''' screen]], look for '''Data log interval''' box on extreme RH side.
			* For some weather station types, you MUST match the interval at which the weather station logs data internally
⚫	*Cumulus MX: From [[MX Administrative Interface\|Interface]], find settings menu, select the '''Station settings''' page, select '''General Settings''', and find '''Log interval''' where there is a drop down that offers the various timings that are permitted.
			* The default time interval between adding data logging lines is every 10 minutes, but you can change this:
		⚫	** Legacy Cumulus 1: On main screen, select '''Configuration''' menu, select the [[Cumulus_Screenshots#Station\|'''station settings''' screen]], look for '''Data log interval''' box on extreme RH side.
		⚫	** Cumulus MX: From [[MX Administrative Interface\|Interface]], find settings menu, select the '''Station settings''' page, select '''General Settings''', and find '''Log interval''' where there is a drop down that offers the various timings that are permitted.

	=What name is used for the standard file?=		=What name is used for the standard file?=

Sfws: Text replacement - "Category:MX txt Files" to "Category:Files_with_Comma_Separated_Values"

2021-12-31T21:30:43Z

Text replacement - "Category:MX txt Files" to "Category:Files_with_Comma_Separated_Values"

← Older revision		Revision as of 21:30, 31 December 2021
Line 1:		Line 1:
	{{Template:Version badge Mx}}{{Version badge 1}}[[Category:Cumulus Files]][[Category:~~MX txt Files~~]]		{{Template:Version badge Mx}}{{Version badge 1}}[[Category:Cumulus Files]][[Category:Files_with_Comma_Separated_Values]]
	This page applies to all flavours.		This page applies to all flavours.

Sfws: /* What name is used for the standard file? */ typo

2021-08-10T20:09:43Z

What name is used for the standard file?: typo

← Older revision		Revision as of 20:09, 10 August 2021
Line 25:		Line 25:
	These files are automatically created with names in the form <tt><Month><Year>log.txt</tt>.		These files are automatically created with names in the form <tt><Month><Year>log.txt</tt>.

	In UK English locale, the file for August 2009 is called <code>Aug09log.txt</code>).		In UK English locale, the file for August 2009 is called <code>Aug09log.txt</code>.

	The month can be in another language, or a different abbreviation, depending on your locale.		The month can be in another language, or a different abbreviation, depending on your locale.

Sfws: Correct categories, slight resequence of page

2021-08-10T20:07:22Z

Correct categories, slight resequence of page

Show changes

Sfws: /* Added in 3.6.0 */

2021-08-10T19:38:23Z

Added in 3.6.0

← Older revision		Revision as of 19:38, 10 August 2021
Line 404:		Line 404:
	\|-		\|-
	\| 28(AB)		\| 28(AB)
	\|signed number		\|signed number (or space)
	\|13.7		\|13.7
	\| [[Feels Like]] temperature		\| [[Feels Like]] temperature

			NOTE: MX adds a space here, if a line (created before 3.6.0 without a valid value in this field) is edited.
	\|}		\|}

Sfws: /* Importing pre-Cumulus data */ add links

2021-06-10T07:34:53Z

Importing pre-Cumulus data: add links

← Older revision		Revision as of 07:34, 10 June 2021
Line 116:		Line 116:
	If you were recording weather data manually, or with different weather station software, you might create files, using the standard data log files format, holding that pre-Cumulus data. Cumulus can treat these files just like any where it created the data log entries, and therefore you can apply any of the techniques explained in [[Calculate Missing Values]] and [[Correcting Extremes]].		If you were recording weather data manually, or with different weather station software, you might create files, using the standard data log files format, holding that pre-Cumulus data. Cumulus can treat these files just like any where it created the data log entries, and therefore you can apply any of the techniques explained in [[Calculate Missing Values]] and [[Correcting Extremes]].

			In the configuration file, Cumulus.ini, there is a parameter '''StartDate=''', this is assigned a value when you first start using Cumulus of the current date, in the format specified by the '''local''' you are using.
		⚫	* The original Cumulus 1 software takes no notice of this date, other than to use it to populate the <#recordsbegandate> see [[Webtags#Records]], so it is quite happy to process standard data logs with lines with earlier dates.
			* The '''ExportMySQL.exe''' which was written by Steve Loft and included with the beta releases of MX, and subsequent early MX release distributions from Mark Crossley, also ignores the date in the configuration file.
			* All releases containing '''CumulusMX.exe''', will ignore any data log lines dated before the date in the configuration file.
			* The other Mark Crossley utility packages linked from [[Software#Other_utilities\|Software page utilities section]] will also generally ignore any data log lines dated before the date in the configuration file.
			** [[Calculate_Missing_Values#CreateMissing.exe\|CreateMissing.exe]] will use the earliest date in [[dayfile.txt]] if that is earlier than the date in the configuration file.
			** '''ExportToMySQL.exe''' (not currently documented in this Wiki) will normally ignore any data log lines dated before the date in the configuration file.

			Don't worry if you are using MX and your pre-cumulus data does not have all the fields listed at [[#List_of_fields_in_the_file]]. [[Calculate_Missing_Values#CreateMissing.exe\|CreateMissing.exe]] can be used to calculate (at lease some of, maybe all of) those missing fields. That utility will also create [[dayfile.txt]] entries for the days in the pre-Cumulus data.
⚫	The original Cumulus 1 software takes no notice of the <#recordsbegandate> see [[Webtags#Records]], so it is quite happy to process standard data logs with lines with earlier dates. The same applies to '''ExportMySQL.exe'''. However, by default, '''CumulusMX.exe''' will ignore any data log lines dated before the [[Cumulus.ini_(Cumulus_1)#Section:_Station\|start date]] it recorded in the configuration file.

			If you are not using MX, you may still be able to populate missing fields (see [[Calculate_Missing_Values\|calculate missing values]] page), and create lines in dayfile.txt (see [[Amending dayfile]] page).
⚫	===Potential issues when trying to match formats===

		⚫	===Potential issues when trying to match formats===

	'''Here are some issues to consider.''' There are some postings in the Support Forum about importing past data. Essentially match the fields listed in [[~~Standard_log_files~~#List_of_fields_in_the_file\|List of fields in the file]] below with the fields you have available in your source.		'''Here are some issues to consider.''' There are some postings in the Support Forum about importing past data. Essentially match the fields listed in [[#List_of_fields_in_the_file\|List of fields in the file]] below with the fields you have available in your source.
	* Dates and times might need some pre-processing (spreadsheet packages usually have ability to select part of a text string and to concatenate a number of strings) to convert them to text in the formats mentioned above. (Note - it is crucial in a spreadsheet to treat the date as text; Do not let the spreadsheet recognise it as a date as it might change the format or store as a 'days since X' number).		* Dates and times might need some pre-processing (spreadsheet packages usually have ability to select part of a text string and to concatenate a number of strings) to convert them to text in the formats mentioned above. (Note - it is crucial in a spreadsheet to treat the date as text; Do not let the spreadsheet recognise it as a date as it might change the format or store as a 'days since X' number).
	* Remember that null values are not allowed within a row, so ensure that you enter an obviously wrong value (-99 or zero might be do) between your operating system defined field separator (normally ',' or ';') and make the row length the minimum shown in the list of fields below. If you use a spreadsheet, treat the file as Comma Separated Value type, but on saving choose the appropriate field separator character (e.g in Libre Office or Open Office 'Save as CSV' select 'Edit Filter Settings').		* Remember that null values are not allowed within a row, so ensure that you enter an obviously wrong value (-99 or zero might be do) between your operating system defined field separator (normally ',' or ';') and make the row length the minimum shown in the list of fields below. If you use a spreadsheet, treat the file as Comma Separated Value type, but on saving choose the appropriate field separator character (e.g in Libre Office or Open Office 'Save as CSV' select 'Edit Filter Settings').

Sfws: /* Introduction */

2021-06-10T07:10:35Z

Introduction

← Older revision		Revision as of 07:10, 10 June 2021
Line 3:		Line 3:
	= Introduction =		= Introduction =

	Cumulus has ability to create a number of data logs, most ('''[[Speciallog.txt]]''' , [[Extra_Sensor_Files]], and [[Air_Link_Log.txt]]) are optional, in that settings determine whether they are created or not. However, this page documents the data logs that are not optional (hence termed "standard"). These store the full set of current (read and derived) values at whatever interval you have set for Cumulus to keep detailed logging. The default time interval between adding data logging lines is every 10 minutes, but you can change this:		Cumulus has ability to create a number of data logs, most ('''[[Speciallog.txt]]''' , [[Extra_Sensor_Files]], and [[Air_Link_Log.txt]]) are optional, in that settings determine whether they are created or not.

			However, this page documents the data logs that are not optional (hence termed "standard"). These store the full set of current (read and derived) values at whatever interval you have set for Cumulus to keep detailed logging. The default time interval between adding data logging lines is every 10 minutes, but you can change this:
	*Cumulus 1: From Configuration menu, select the [[Cumulus_Screenshots#Station\|station settings screen]], '''Data log interval''' box on extreme RH side.		*Cumulus 1: From Configuration menu, select the [[Cumulus_Screenshots#Station\|station settings screen]], '''Data log interval''' box on extreme RH side.
	*Cumulus MX: From settings tab, select the '''Station settings''' screen, and find '''Log interval''' after the ~~''Ecowitt~~ ~~GW1000~~ ~~Settings''~~ ~~section~~.		*Cumulus MX: From settings tab, select the '''Station settings''' screen, and find '''Log interval''' after the weather station specific sections.

	These files are automatically created with names in the form <tt><Month><Year>log.txt</tt> (for example, the file for August 2009 is often called <tt>Aug09log.txt</tt>), but the month can be in another language or a different abbreviation depending on your locale. There is no setting to change this format.		These files are automatically created with names in the form <tt><Month><Year>log.txt</tt> (for example, the file for August 2009 is often called <tt>Aug09log.txt</tt>), but the month can be in another language or a different abbreviation depending on your locale. There is no setting to change this format.