Correcting Extremes - Revision history

Sfws: /* Correcting an error in today's total rainfall */ extend list of where might send rainfall data

2022-09-09T09:21:04Z

Correcting an error in today's total rainfall: extend list of where might send rainfall data

← Older revision		Revision as of 09:21, 9 September 2022
Line 352:		Line 352:
	=== Correcting an error in today's total rainfall ===		=== Correcting an error in today's total rainfall ===

			You can correct rainfall total (for current [[meteorological day]] since rollover).
⚫	~~Easy~~ ~~- correct today's total using~~ the [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus \| 'today's rain']] editor on the edit menu.

⚫	* [[File:Badge v1.png]]select 'Today's rain in the [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu\|edit menu]] accessed from main screen in Cumulus 1,
		⚫	Just use the [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus \| 'today's rain']] editor on the edit menu.
		⚫	* [[File:Badge v1.png]]select ''Today's rain'' in the [[Cumulus_Screenshots#File.2FEdit.2FHelp_Menu\|edit menu]] accessed from main screen in Cumulus 1,
	* [[File:Badge vMx.png]]select '''Today's rain''' in the [[MX_Administrative_Interface#Today.27s_rain.27\|edit menu]] of MX interface.		* [[File:Badge vMx.png]]select '''Today's rain''' in the [[MX_Administrative_Interface#Today.27s_rain.27\|edit menu]] of MX interface.

	This edit will actually alter the start of day rainfall counter figure:		This edit will actually alter '''the start of day rainfall counter figure held by Cumulus internally''' (RAM):
	*If you want today's rain to seem less (perhaps you or the wind knocked the rain gauge), Cumulus will increase the start of day counter		* If you want today's rain to seem less (perhaps you, an animal, or the wind, knocked the rain gauge), Cumulus will increase the start of day counter
	*If you want today's rain to seem greater (perhaps the rain gauge got blocked by a leaf), Cumulus will decrease the start of day counter		* If you want today's rain to seem greater (perhaps the rain gauge got blocked by a leaf, or a bird), Cumulus will decrease the start of day counter
			* Be aware that if you stop Cumulus, and restart it, before end of meteorological day, internally held variables are lost, so the start of day figure will revert

⚫	Please note that this edit does not affect "rain rate", "maximum hourly rain", "maximum 24-hour rain", or "rain since midnight", nor does it update ~~every~~ data log line so it has correct rainfall counter reading. ~~Also,~~ if ~~you~~ ~~ask~~ MX to ~~automatically~~ ~~insert~~ a ~~new~~ ~~row~~ ~~into~~ a ~~monthly~~ ~~table~~ ~~on your database server whenever a new line~~ is ~~stored~~ ~~in the [[Standard_log_files]], your database will retain incorrect values, as these~~ are ~~not updated by this correction~~.

	Please see [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus]] for details of how to edit those related fields.


		⚫	Please note that this edit ''does not affect'' "rain rate", "maximum hourly rain", "maximum 24-hour rain", or "rain since midnight", ''nor does it update'' ANY EXISTING data log line so it has correct rainfall counter reading. Please see [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus]] for details of how to edit those related fields so when Cumulus is restarted all are corrected.

			If you send rainfall data to:
			* any third-party external web site
			* MQTT or other local system
			* any database tables (e.g. if you ask MX to automatically insert a new row into a monthly table on your database server whenever a new line is stored in the [[Standard_log_files]])
			''then be aware that'' that this correction does '''not send any update''' for any messages/values '''already sent'''.

	=Some definitions=		=Some definitions=

Sfws: Some spelling corrections, and layout sequence improvements

2022-09-07T17:17:31Z

Some spelling corrections, and layout sequence improvements

Show changes

Sfws: minor resequencing

2022-08-05T13:37:13Z

minor resequencing

← Older revision		Revision as of 13:37, 5 August 2022
Line 287:		Line 287:
	Although the release did not automatically initialise monthly-all-time extreme records, the new monthly records editor provided in that release had a "fetch dayfile" button. By clicking just one '''Copy''' button, the one ''in the header row'', all the relevant daily records were copied into the monthly-all-time records for the month of the selected tab. Therefore by doing that again for every other tab (except any tab for a month when you had never used the original Cumulus), and then clicking '''OK''' button, you manually initialised all the parameters (assuming your dayfile had all the parameters - see [[Calculate Missing Values]]).		Although the release did not automatically initialise monthly-all-time extreme records, the new monthly records editor provided in that release had a "fetch dayfile" button. By clicking just one '''Copy''' button, the one ''in the header row'', all the relevant daily records were copied into the monthly-all-time records for the month of the selected tab. Therefore by doing that again for every other tab (except any tab for a month when you had never used the original Cumulus), and then clicking '''OK''' button, you manually initialised all the parameters (assuming your dayfile had all the parameters - see [[Calculate Missing Values]]).


			=Correction of extremes for past days=

			Since '''today.ini''' is used to create lines in '''dayfile.txt''', you can follow instructions in [[Amending_dayfile]] to make any necessary corrections for past days.

	=Correction of extremes for today=		=Correction of extremes for today=

	~~The~~ Cumulus '''Edit' menu includes a '''Today's rain''' option where you can adjust the [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus\|total rainfall for today]] (e.g. if you or the wind have knocked your rain gauge) as ~~described~~ ~~below.~~ ~~There~~ is no ~~facility~~ ~~provided~~ ~~to edit any other content of [[today.ini]]~~, ~~but~~ since ~~'''today.ini'''~~ is ~~used~~ to ~~create lines in '''dayfile.txt'''~~, ~~you~~ ~~can~~ ~~follow~~ ~~instructions~~ in ~~[[Amending_dayfile]]~~ to ~~make~~ ~~any~~ ~~necessary~~ ~~corrections~~ ~~for~~ ~~past days~~.		As described below, the Cumulus '''Edit' menu includes a '''Today's rain''' option where you can adjust the [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus\|total rainfall for today]] (e.g. if you or the wind have knocked your rain gauge), but note this does not correct "rain this hour", "rain since midnight", "maximum daily rain rate", "maximum 24 hour rain", or any extremes (for this month/year or all time).

			There is no facility provided to edit any other content of [[today.ini]]. Manual correction may be possible, but do read advice on [[today.ini]] page.

	== There is an error in today's total rainfall ==		== There is an error in today's total rainfall ==
Line 305:		Line 310:
	Please note that this edit does not affect "rain rate", "maximum hourly rain", "maximum 24-hour rain", or "rain since midnight", nor does it update every data log line so it has correct rainfall counter reading. Also, if you ask MX to automatically insert a new row into a monthly table on your database server whenever a new line is stored in the [[Standard_log_files]], your database will retain incorrect values, as these are not updated by this correction.		Please note that this edit does not affect "rain rate", "maximum hourly rain", "maximum 24-hour rain", or "rain since midnight", nor does it update every data log line so it has correct rainfall counter reading. Also, if you ask MX to automatically insert a new row into a monthly table on your database server whenever a new line is stored in the [[Standard_log_files]], your database will retain incorrect values, as these are not updated by this correction.

	Please see [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus]] for details of how to edit related fields.		Please see [[Today.ini#Editing_rainfall_in_today.ini_within_Cumulus]] for details of how to edit those related fields.


	=Correction of extremes for past month=		=Correction of extremes for past month=

Sfws: Rewrite of out of date page for MX 3.18.0 changes

2022-06-16T11:27:59Z

Rewrite of out of date page for MX 3.18.0 changes

Show changes

Sfws: /* Introduction */ restored accidentally deleted cell

2021-07-26T10:37:40Z

Introduction: restored accidentally deleted cell

← Older revision		Revision as of 10:37, 26 July 2021
Line 61:		Line 61:
	\| Editor for "Monthly Records"		\| Editor for "Monthly Records"
	\|[[Webtags#Monthly_All_Time_Records\|monthlyrecord.htm]]		\|[[Webtags#Monthly_All_Time_Records\|monthlyrecord.htm]]
			\| Similar to previous row, but different start date, and separate extremes maintained for each month (regardless of the year)
	\|}		\|}
	Explaining columns in above table:		Explaining columns in above table:

Sfws: /* Introduction */ typos

2021-07-26T10:35:33Z

Introduction: typos

← Older revision		Revision as of 10:35, 26 July 2021
Line 41:		Line 41:
	\| Please see [[#Accuracy Note]]		\| Please see [[#Accuracy Note]]
	\|-		\|-
	\|colspan="3" style="background:pink;"\|monthyyyyMM.ini are archived copies for past months		\|colspan="5" style="background:pink;"\|monthyyyyMM.ini are archived copies for past months
	\|-		\|-
	! scope="row"\|For current year-to-date		! scope="row"\|For current year-to-date
Line 49:		Line 49:
	\| Please see [[#Accuracy Note]]		\| Please see [[#Accuracy Note]]
	\|-		\|-
	\|colspan="3" style="background:pink;"\|yearyyyy.ini are archived copies for past years		\|colspan="5" style="background:pink;"\|yearyyyy.ini are archived copies for past years
	\|-		\|-
	! scope="row"\|For all readings since a '''start date''		! scope="row"\|For all readings since a '''start date''

Sfws at 10:34, 26 July 2021

2021-07-26T10:34:26Z

← Older revision		Revision as of 10:34, 26 July 2021
Line 40:		Line 40:
	\|[[Webtags#Monthly\|thismonth.htm]]		\|[[Webtags#Monthly\|thismonth.htm]]
	\| Please see [[#Accuracy Note]]		\| Please see [[#Accuracy Note]]
		⚫	\|-
			\|colspan="3" style="background:pink;"\|monthyyyyMM.ini are archived copies for past months
	\|-		\|-
	! scope="row"\|For current year-to-date		! scope="row"\|For current year-to-date
Line 45:		Line 47:
	\| Editor for "This year's records"		\| Editor for "This year's records"
	\|[[Webtags#Yearly\|thisyear.htm]]		\|[[Webtags#Yearly\|thisyear.htm]]
	\| Please see [[#Accuracy Note]]		\| Please see [[#Accuracy Note]]
			\|-
			\|colspan="3" style="background:pink;"\|yearyyyy.ini are archived copies for past years
	\|-		\|-
	! scope="row"\|For all readings since a '''start date''		! scope="row"\|For all readings since a '''start date''
Line 57:		Line 61:
	\| Editor for "Monthly Records"		\| Editor for "Monthly Records"
	\|[[Webtags#Monthly_All_Time_Records\|monthlyrecord.htm]]		\|[[Webtags#Monthly_All_Time_Records\|monthlyrecord.htm]]
⚫	\|
	\|}		\|}
	Explaining columns in above table:		Explaining columns in above table:
Line 68:		Line 71:

	The editors built into Cumulus, for long term extremes (over a period of a month or more), give you the ability to display, for each extreme record:		The editors built into Cumulus, for long term extremes (over a period of a month or more), give you the ability to display, for each extreme record:
	# The figure taken from a search for that extreme by examining all entries in the [[dayfile.txt]] for that period		# The (hopefully more accurate) figure taken from a search for that extreme by examining all entries in the [[dayfile.txt]] for that period
	# The figure taken from a search for that extreme by examining all entries in the [[Standard log files]] for that period		# The (usually less accurate) figure taken from a search for that extreme by examining all entries in the [[Standard log files]] for that period

	~~Normally~~ the first returns the more accurate result (unless the '''dayfile.txt''' line, either was created with a rogue value, or has been corrupted)~~. Let me explain why~~:		Let me explain why the first returns the more accurate result (unless the '''dayfile.txt''' line, either was created with a rogue value, or has been corrupted):
	* Using [[Standard log files]] as source for recalculating past extremes:		* Using [[Standard log files]] as source for recalculating past extremes:
	** Let us assume you are using the default logging interval of 10 minutes		** Let us assume you are using the default logging interval of 10 minutes
Line 98:		Line 101:
	==Why might your extremes need to be corrected?==		==Why might your extremes need to be corrected?==

			Sometimes a mistake is made in setting up or calibrating a sensor, or (despite the warnings within both flavours of Cumulus about getting your choice of units correct from the start), you decide to change your units. In both cases, you will identify a constant/multiplier adjustment to be applied to adjust all values (luckily times and dates of extremes are not affected) over the past period. In both cases, you need to correct past entries in any [[Extra Sensor Files]], any [[Standard log files]], in [[dayfile.txt]], plus the multiple [[Category:Ini Files\|extreme record files]].
⚫	If there is a [[Correcting_Extremes#Rogue_value\|rogue value]] read from your weather station (this could be due to noise affecting communications, or because a sensor has been knocked), it can get into any of ~~those~~ extreme record files and ~~it might also make~~ related derived value extremes wrong as well.

			'''The instructions on this page will help you correct the extreme record files''' as there are relatively few edits to do for a single source value changing because of units, or a calibration multiplier. There is no change to when a maximum/minimum occured, nor does it affect any derived values. For example a change in units, or calibration multiplier, for temperature or wind speed, does not affect calculations for wind chill and feels like temperature. It is a bit more complex if you change a calibration offset, as that might affect threshold for calculation of some derivatives (e.g. changing an offset for temperature or wind speed also affects wind chill). It is slightly more complicated changing units for temperature (as you may have many extra temperatures).

			The easiest way to change entries in any Extra Sensor Files, any Standard log files, and in dayfile.txt, is to either write a batch editing script (see [https://cumulus.hosiene.co.uk/viewtopic.php?p=155539#p155539 changed rainfall units example]), or to use a spreadsheet (be careful not to affects dates or times) like '''Libre Office Calc''' where you can paste special a multiplier to all cells in a particular column.

		⚫	If there is a [[Correcting_Extremes#Rogue_value\|rogue value]] read from your weather station (this could be due to noise affecting communications, or because a sensor has been knocked), it can get into the daily summary file, any of the extreme record files, and any related derived value extremes may become wrong as well.
	It is also possible that you have discovered that you made a mistake in setting up or calibrating a sensor, and this leads you to identifying a constant/multiplier adjustment
	to be applied to adjust all values over the past period.

	A sensor might fail, and Cumulus does not recognise that "Null" (this might mean the weather station sends all bits set to zero or all bits set to one) values should be ignored when comparing against existing extreme records, and so set the extreme record to zero or maximum number that the number of bits can convey.		A sensor might fail, and Cumulus does not recognise that "Null" (this might mean the weather station sends all bits set to zero or all bits set to one) values should be ignored when comparing against existing extreme records, and so set the extreme record to zero or maximum number that the number of bits can convey.

Sfws: /* All-time extreme functionality */ accessibility

2021-07-24T06:59:33Z

All-time extreme functionality: accessibility

← Older revision		Revision as of 06:59, 24 July 2021
Line 123:		Line 123:
	!style="width:100px"\|First in Build		!style="width:100px"\|First in Build
	\|-		\|-
	\|highest/lowest apparent temperature		! scope="row"\|highest/lowest apparent temperature
	\|26 Oct 2010		\|26 Oct 2010
	\|1.9.1 beta		\|1.9.1 beta
	\|957		\|957
	\|-		\|-
	\|highest/lowest feels like temperature		! scope="row"\|highest/lowest feels like temperature
	\|24 June 2020		\|24 June 2020
	\|3.6.10		\|3.6.10
	\|3086		\|3086
	\|-		\|-
	\|highest Canadian Humidity Index (humidex)		! scope="row"\|highest Canadian Humidity Index (humidex)
	\|28 Jul 2020		\|28 Jul 2020
	\|3.7.0		\|3.7.0
	\|3089		\|3089
	\|-		\|-
	\|highest minimum temperature		! scope="row"\|highest minimum temperature
	\|15 April 2004		\|15 April 2004
	\|1.2.2		\|1.2.2
	\|(lost)		\|(lost)
	\|-		\|-
	\|highest USA heat index		! scope="row"\|highest USA heat index
	\|29 Aug 2010		\|29 Aug 2010
	\|1.9.0 beta		\|1.9.0 beta
	\|955		\|955
	\|-		\|-
	\|wettest month		! scope="row"\|wettest month
	\|5 April 2004		\|5 April 2004
	\|1.2		\|1.2
	\|(lost)		\|(lost)
	\|-		\|-
	\|highest daily wind run		! scope="row"\|highest daily wind run
	\|3 Jul 2011		\|3 Jul 2011
	\|1.9.2 beta		\|1.9.2 beta

Sfws: /* Introduction */ accessibility

2021-07-24T06:58:08Z

Introduction: accessibility

← Older revision		Revision as of 06:58, 24 July 2021
Line 23:		Line 23:
	!style="width:300px"\|Notes		!style="width:300px"\|Notes
	\|-		\|-
	\|For current day so far		! scope="row"\|For current day so far
	\|[[today.ini]]		\|[[today.ini]]
	\| Editor for "Today's rain" (no editor for other derivatives)		\| Editor for "Today's rain" (no editor for other derivatives)
Line 29:		Line 29:
	\|Many entries in this file (for non-midnight rollover, use is made of [[yesterday.ini]] too) get transferred to [[dayfile.txt]] at end of day.		\|Many entries in this file (for non-midnight rollover, use is made of [[yesterday.ini]] too) get transferred to [[dayfile.txt]] at end of day.
	\|-		\|-
	\| For past days		! scope="row"\| For past days
	\| [[dayfile.txt]]		\| [[dayfile.txt]]
	\| See [[Amending dayfile]]		\| See [[Amending dayfile]]
Line 35:		Line 35:
	\| Often used as source for corrections - see [[#Accuracy Note]]		\| Often used as source for corrections - see [[#Accuracy Note]]
	\|-		\|-
	\|For current month-to-date		! scope="row"\|For current month-to-date
	\|[[month.ini]]		\|[[month.ini]]
	\| Editor for "This month's records"		\| Editor for "This month's records"
Line 41:		Line 41:
	\| Please see [[#Accuracy Note]]		\| Please see [[#Accuracy Note]]
	\|-		\|-
	\|For current year-to-date		! scope="row"\|For current year-to-date
	\|[[year.ini]]		\|[[year.ini]]
	\| Editor for "This year's records"		\| Editor for "This year's records"
Line 47:		Line 47:
	\| Please see [[#Accuracy Note]]		\| Please see [[#Accuracy Note]]
	\|-		\|-
	\|For all readings since a '''start date''		! scope="row"\|For all readings since a '''start date''
	\|[[alltime.ini]]		\|[[alltime.ini]]
	\| Editor for "All Time Records"		\| Editor for "All Time Records"
Line 53:		Line 53:
	\|See table below for start date		\|See table below for start date
	\|-		\|-
	\|For a particular month in all years		! scope="row"\|For a particular month in all years
	\|[[monthlyalltime.ini]]		\|[[monthlyalltime.ini]]
	\| Editor for "Monthly Records"		\| Editor for "Monthly Records"
Line 60:		Line 60:
	\|}		\|}
	Explaining columns in above table:		Explaining columns in above table:
	# The first column is self-explanatory		# The first (label) column is self-explanatory
	# The second column contains a link to the page that explains more about the file named there, which is where the extreme records are stored for that period		# The second column contains a link to the page that explains more about the file named there, which is where the extreme records are stored for that period
	# The third column gives the name for the '''Edit''' menu item to choose to edit these extreme records		# The third column gives the name for the '''Edit''' menu item to choose to edit these extreme records

Sfws: /* How editing accuracy depends on source selected */ update links

2021-07-15T09:10:56Z

How editing accuracy depends on source selected: update links

← Older revision		Revision as of 09:10, 15 July 2021
Line 77:		Line 77:
	** That means the [[Monthly log files]] do not capture any extremes recorded in the time (by default 599 seconds) between logs		** That means the [[Monthly log files]] do not capture any extremes recorded in the time (by default 599 seconds) between logs
	** Therefore the detailed log files are not normally the most accurate source		** Therefore the detailed log files are not normally the most accurate source
	** Please note, this less accurate way of deducing daily extremes/totals (to update dayfile.txt) is used by ~~the~~ ~~[[Calculate_Missing_Values#CreateMissing.exe\|CreateMissing.exe]] utility (that is only available for MX).~~		** Please note, this less accurate way of deducing daily extremes/totals (to update dayfile.txt) is used by Cumulus software:
			*** For Legacy Cumulus 1: [[Amending_dayfile#Create_Missing_on_legacy_dayfile_editor\|Create Missing in legacy dayfile editor]].
			*** For MX: [[Calculate_Missing_Values#CreateMissing.exe\|CreateMissing.exe]] utility.
	* Using [[dayfile.txt]] as source for recalculating past extremes		* Using [[dayfile.txt]] as source for recalculating past extremes
	** MX typically processes data from your weather station every second (even if you use a weather station type that only reads its sensors every 40 or 60 seconds). Cumulus 1 processes data from your weather station at intervals that vary for the different station types, but we can assume it is at least every 60 seconds.		** MX typically processes data from your weather station every second (even if you use a weather station type that only reads its sensors every 40 or 60 seconds). Cumulus 1 processes data from your weather station at intervals that vary for the different station types, but we can assume it is at least every 60 seconds.