5,838
edits
Migrating from Cumulus 1 to MX: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
corrected cross reference that was linking to text that has moved to different page
m (corrected cross reference that was linking to text that has moved to different page) |
|||
| (One intermediate revision by the same user not shown) | |||
| Line 58: | Line 58: | ||
When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment). (You can also run MX as a service, that has different ways to start and stop, discussed in the links given in previous section). | When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment). (You can also run MX as a service, that has different ways to start and stop, discussed in the links given in previous section). | ||
The separate [[MX Administrative Interface|admin interface]](unfortunately this is called various names in the support forum, including "user interface", "dashboard | The separate [[MX Administrative Interface|admin interface]](unfortunately this is called various names in the support forum, including "user interface", "dashboard", and "interface") is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network. | ||
If you install MX on Microsoft Windows, then a few extra one-off steps are needed to allow this web server functionality: | If you install MX on Microsoft Windows, then a few extra one-off steps are needed to allow this web server functionality: | ||
| Line 67: | Line 67: | ||
The default URL if the browser is on the same machine as MX is http://localhost:8998/. Newer releases of MX will tell you an IPv4 address to use, such as by typing "http://192.168.1.64:8998/". | The default URL if the browser is on the same machine as MX is http://localhost:8998/. Newer releases of MX will tell you an IPv4 address to use, such as by typing "http://192.168.1.64:8998/". | ||
For security reasons, the admin interface should not be accessible via the public internet. | For security reasons, the admin interface should not be accessible via the public internet. | ||
| Line 76: | Line 74: | ||
If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and there is quite a bit to read, taking up most of the rest of this Wiki page. The Cumulus configuration files for Cumulus 1 and for MX share the same name, but their content is very different, so these are discussed at length in subsections below. | If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and there is quite a bit to read, taking up most of the rest of this Wiki page. The Cumulus configuration files for Cumulus 1 and for MX share the same name, but their content is very different, so these are discussed at length in subsections below. | ||
Obviously, you will want to copy ALL (except weather diary which uses a different file) the files in the [[#"data" folder]], from any old installation into the new installation, but migrating these is not always easy, here is a quick summary of the potential issues (all discussed further in various subsections later): | Obviously, you will want to copy ALL (''except weather diary which uses a different file'') the files in the [[#"data" folder]], from any old installation into the new installation, but migrating these is not always easy, here is a quick summary of the potential issues (all discussed further in various subsections later): | ||
* '''When you first run MX, the [[#Start date|Start Date]] is recorded in the configuration file''' | * '''When you first run MX, the [[#Start date|Start Date]] is recorded in the configuration file''' | ||
** In the legacy software, this date is purely used as something to appear on one of the [[Customised_templates#The_Standard_Templates_for_Cumulus_1|sample web templates]] | ** In the legacy software, this date is purely used as something to appear on one of the [[Customised_templates#The_Standard_Templates_for_Cumulus_1|sample web templates]] | ||
| Line 82: | Line 80: | ||
** You can edit this [[Cumulus.ini#Data_Logging|'''StartDate=xxxxx''']] parameter, in the admin interface select ''Station Settings → Common Options → Advanced Options → Records Began date'' | ** You can edit this [[Cumulus.ini#Data_Logging|'''StartDate=xxxxx''']] parameter, in the admin interface select ''Station Settings → Common Options → Advanced Options → Records Began date'' | ||
* Be aware your new installation has to use the same "locale" as the old installation, or MX will struggle as the locale affects how new lines are stored, and how MX expects old lines to have been stored the same way. | * Be aware your new installation has to use the same "locale" as the old installation, or MX will struggle as the locale affects how new lines are stored, and how MX expects old lines to have been stored the same way. | ||
* | ** The legacy software was fairly tolerant about date formats, and use of decimal commas | ||
* Cumulus MX is much more fussy than Cumulus 1 about every line in any data file using exactly the same locale formatting: | * Cumulus MX is much more fussy than Cumulus 1 about every line in any data file using exactly the same locale formatting: | ||
** Cumulus 1 is able to accept any character (other than the list separator, and space) being used to separate hour and minutes in time-stamps, MX | ** Cumulus 1 is able to accept any character (other than the list separator, and space) being used to separate hour and minutes in time-stamps, MX only accepts a colon ":" | ||
** [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]] | ** [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]] | ||
** [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files, and how MX prefers decimal points to decimal commas | ** [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files, and how MX prefers decimal points to decimal commas | ||
* If your old installation, is on a different operating system to the new installation, remember that Microsoft Windows uses different line terminators to all other operating systems, although MX should cope with mixed line terminators, any third party routines reading your data files will probably not accept a line terminator change. | |||
==Simplest migration== | ==Simplest migration== | ||
| Line 93: | Line 92: | ||
# Copy the configuration files ('''Cumulus.ini''' and the optional '''strings.ini''') from the directory with Cumulus.exe in it to the directory with CumulusMX.exe in it | # Copy the configuration files ('''Cumulus.ini''' and the optional '''strings.ini''') from the directory with Cumulus.exe in it to the directory with CumulusMX.exe in it | ||
# Copy all the [[:Category:Ini Files|extreme tracking files]] and the [[:Category:Files with Comma Separated Values|files with | # If you use the [[Weather Diary]], see that cross reference, because MX cannot read the old file, and there is no conversion utility. | ||
# If you have files in the [[Reports folder]], copy them across to the MX folder of same name | # Copy all the [[:Category:Ini Files|extreme tracking files ending in .ini]] and the [[:Category:Files with Comma Separated Values|files with your data ending in .txt]] in the old [[Data folder]] across to MX folder with same name | ||
# If you have files in the [[Reports folder]], copy them across to the MX folder of same name, see that cross reference for information about different defaults | |||
==File names== | ==File names== | ||
| Line 112: | Line 112: | ||
For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed. | For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed. | ||
To change end of line characters, run each file through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its '''Edit''' menu, choose '''EOL conversion'''. On a Linux system like a Raspberry Pi computer, please see [[ | To change end of line characters, run each file through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its '''Edit''' menu, choose '''EOL conversion'''. On a Linux system like a Raspberry Pi computer, please see [[Preparing_your_Linux_computer_for_MX#Changing_line_terminators|Changing line terminators]]. | ||
If you use PHP Hypertext Pre-processor anywhere on your installation, normally that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" function to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on! | If you use PHP Hypertext Pre-processor (php) anywhere on your installation, normally that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" function to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on! | ||
==Configuration file== | ==Configuration file== | ||
| Line 120: | Line 120: | ||
MX's [[Cumulus.ini]] has different content to the legacy [[Cumulus.ini_(Cumulus_1)|'''cumulus.ini''']]. From release 3.12.0,many more new settings were added, more settings were removed, and some parameters in the file were renamed. The first cross reference, in listing all the parameters, makes it clear which settings were introduced in the legacy software, and which MX release added new settings, this should help you to grasp how many new settings exist! | MX's [[Cumulus.ini]] has different content to the legacy [[Cumulus.ini_(Cumulus_1)|'''cumulus.ini''']]. From release 3.12.0,many more new settings were added, more settings were removed, and some parameters in the file were renamed. The first cross reference, in listing all the parameters, makes it clear which settings were introduced in the legacy software, and which MX release added new settings, this should help you to grasp how many new settings exist! | ||
===Differences=== | |||
The original Cumulus MX beta by Steve Loft was partly written by machine code translation from his Cumulus 1 code, and therefore MX initially worked fairly similarly to the legacy software. | |||
The development of MX since then, has replaced significant coding and it is safest to assume that MX works differently to the legacy software. | |||
This does mean that some settings do have to be changed when moving from the legacy software to MX: | |||
* Some settings actually have a different parameter name, e.g. '''Port=n''' is replaced by '''Comport=tttttttt''', these differences are explained later | |||
* Some settings work in a different way e.g. the real-time interval in the legacy software is the number of seconds after real-time actions end to when they starts again, so two real-time cycles cannot overlap. For Cumulus MX, a real-time interval kicks off a cycle after the specified number of seconds and therefore you probably need to increase the time in the setting to ensure all actions in one cycle finish before the next attempt starts | |||
Consequently, if you are migrating from the legacy software to MX now, it is best to rename your old '''cumulus.ini'' file so it is is not seen by MX. Let MX create a new configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]]. That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly. | Consequently, if you are migrating from the legacy software to MX now, it is best to rename your old '''cumulus.ini'' file so it is is not seen by MX. Let MX create a new configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]]. That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly. | ||
| Line 153: | Line 161: | ||
===Settings=== | ===How to make new Settings take effect=== | ||
The settings pages in | '''The [[MX_Administrative_Interface#Changing_Settings|settings pages]] in MX, work differently to the [[Cumulus_Screenshots#Configuration_Menu_Screens|settings screens]] in Cumulus 1: | ||
* for Cumulus 1 you choose to save changes by clicking OK, | * for Cumulus 1 you choose to save changes by clicking OK, | ||
* for MX changes are only saved when you click a '''Save''' button if one is provided. | * for MX changes are only saved when you click a '''Save''' button if one is provided. | ||
* The alarms page works slightly differently, with an "Update alarms" button. | * The alarms page works slightly differently, with an "Update alarms" button. | ||
* If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved to configuration file when you move to next field/line, and acted on when you next restart MX. | * If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved to configuration file when you move to next field/line, and acted on when you next restart MX. | ||
Be aware that you | Be aware that you do have to restart MX after changing certain settings: | ||
* You certainly need to restart after any changes that relate to weather station input reading | |||
* You may need to restart after changes that relate to processing of outputs | |||
* At time of writing there is no list of which settings require a restart | |||
* Early releases of MX tended to write new settings to configuration file, but not change how the internal code behaved | |||
* Such settings only took effect on restart, when MX reads the configuration file | |||
* As MX is rewritten for later releases, the handling of settings is changing, and more and more take effect immediately | |||
===Start date=== | ===Start date=== | ||
| Line 186: | Line 200: | ||
A typical parameter value for other serial connecting devices might be "/dev/ttyUSB3" where the final digit will change depending on the new connection. You can search for posts on the support forum that talk about how to find out what connection is being used, depending on what hardware you are using. | A typical parameter value for other serial connecting devices might be "/dev/ttyUSB3" where the final digit will change depending on the new connection. You can search for posts on the support forum that talk about how to find out what connection is being used, depending on what hardware you are using. | ||
===Fine Offset Read Avoid Setting=== | |||
The "read-avoid" timer setting for traditional Fine Offset Stations, was intended to reduce the lock-up problem. | |||
This type of weather station consists of some remote sensors that transmit a short burst of readings every 40 seconds (those models with solar sensors do a second transmission every 60 seconds). | |||
The legacy Cumulus 1 software would read data from these weather stations every 30 seconds, i.e. slightly more frequently than the updates, so sometimes it would read the same data twice. The way the "read-avoid" was implemented in the legacy software was that the software attempted to see which subsequent reads were same, and which were different, as the latter indicated a transmission had taken place. Thereafter, a (configurable) delay of a few seconds would ensure thereafter the 30 second interval reads would never clash with the update following transmission. | |||
MX reads the same station type every 10 seconds, with extra processing once a minute, both timings are fixed, based on computer time. Therefore if read-avoid is enabled, then some of the fixed 10 second interval actions will be skipped. The current advice is not to enable the read avoid as the result can be that almost all reads are skipped. | |||
===RG11 Rain gauge=== | ===RG11 Rain gauge=== | ||
| Line 205: | Line 229: | ||
If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini". | If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini". | ||
If you have not created a [[Strings.ini|strings.ini]] file in your ( | If you have not created a [[Strings.ini|strings.ini]] file in your (legacy) Cumulus top level folder, then you have no file to move to your MX installation, and you should skip the rest of this sub-section. | ||
The contents of the [[Samplestring.ini|samplestring.ini]] file you get in your MX release distribution varies depending on the release you have downloaded. | The contents of the [[Samplestring.ini|samplestring.ini]] file you get in your MX release distribution varies depending on the release you have downloaded. | ||
| Line 291: | Line 315: | ||
When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, so he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=15167 backwards compatible]. | When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, so he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=15167 backwards compatible]. | ||
==dayfile.txt== | |||
MX (except early releases) reads the whole of this file as it starts running, so all lines must use exactly the same format. | MX (except early releases) reads the whole of this file as it starts running, so all lines must use exactly the same format. | ||
===date format issue=== | |||
For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file. Please see [[Amending_dayfile#Date_Separator_in_MX]] for more detail of what MX expects. | For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file. Please see [[Amending_dayfile#Date_Separator_in_MX]] for more detail of what MX expects. | ||
| Line 313: | Line 337: | ||
It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select '''Regular expression''' and then set '''Find what''' to "^([\d]{2})-([\d]{2})-" and '''Replace with''' to "$1/$2/". Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it. | It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select '''Regular expression''' and then set '''Find what''' to "^([\d]{2})-([\d]{2})-" and '''Replace with''' to "$1/$2/". Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it. | ||
===time-stamp format issue=== | |||
The dayfile.txt contains time-stamps following any value that represents a highest or lowest in the day. | The dayfile.txt contains time-stamps following any value that represents a highest or lowest in the day. | ||
| Line 323: | Line 347: | ||
MX will only accept colon ":" separator, all dayfile.txt time-stamps must be in "HH:mm" format. You will need to edit your old lines if any use a different separator. | MX will only accept colon ":" separator, all dayfile.txt time-stamps must be in "HH:mm" format. You will need to edit your old lines if any use a different separator. | ||
===value format issues=== | |||
If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ","). Again, these must be consistent in all dayfile.txt lines for MX, and must match what is defined in the locale used. | If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ","). Again, these must be consistent in all dayfile.txt lines for MX, and must match what is defined in the locale used. | ||
| Line 337: | Line 361: | ||
=Library software= | =Library software= | ||
Any non-technical person can skip this sub-section! | Any non-technical person can skip this final sub-section! | ||
MX uses a number of library software utilities like Highstock, jQuery, boot strap, and others, see [[MX_Basic_info#Library_software_for_your_web_server]]. Be aware that MX determines the versions of these it seeks, and they may not match the versions needed for anything on your web server that is not supplied in the MX release distribution. A browser tries to reuse components that are already loaded, so there is a possibility of the wrong version being loaded. | MX uses a number of library software utilities like Highstock, jQuery, boot-strap, and others, see [[MX_Basic_info#Library_software_for_your_web_server]]. Be aware that MX determines the versions of these it seeks, and they may not match the versions needed for anything on your web server that is not supplied in the MX release distribution. A browser tries to reuse components that are already loaded, so there is a possibility of the wrong version being loaded. | ||
