Software: Difference between revisions

From Cumulus Wiki
Jump to navigationJump to search
3,340 bytes added ,  20 November 2022
(23 intermediate revisions by 4 users not shown)
[[Category:Cumulus MX]] This Wiki page has been created for those who use Cumulus MX. It lists all standard utilities as well as MX itself.
 
= What is Cumulus MX? =
 
: a cross platform version of Cumulus software that runs on Microsoft Windows, all varieties of UNIX and Linux (including the Raspberry Pi Operating System), and Mac OS X.
 
'''NOTE relating to MX Release v3.9.1 or later: On Linux/RPi/macOS, Mono version 5, or 6, is needed for CreateMissing.Exe, ExportToMySQL.exe, and CumulusMX.exe. Mono v4 is subject to random crashes.'''
 
==CumulusMX.exe Usage Restriction==
 
==MX Usage Restriction==
 
Both the mandatory [[MX Administrative Interface|MX Interface]], and the optional [[New Default Web Site Information|default MX web site]], include the [https://www.highcharts.com HighStock] product.
If you need any help with Cumulus MX, please first consult, in this Wiki, [[What_to_do_when_I_have_a_problem_with_MX]].
 
It is recommended you check you have complied with any instructions in the announcement for the latest MX release in [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 the forum] before installingseeking ithelp (to see what changes in any release affect you, and what actions you may need to take).
 
If you wish to research the documentation on MX to try to resolve your problem yourself, use the entry point of the [[:Category:Cumulus MX]] page for pointers to other Wiki pages connected with MX that may help.
 
If you cannot answer your questions from information in this Wiki (and MX changes so quickly any information here can be very out of date), then use the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 correct MX support forum] toand seethe ifdeveloper othersor haveCumulus answerscommunity will offer assistance.
 
==Upgrading==
 
== Current Release ==
We strongly advise [[Updating_MX_to_new_version#Introduction_to_upgrading_MX|upgrading]] to the latest release:
* You may wish to wait a few days after a new release comes out, just in case a new release has some bugs
** MX is extremely complex, the developers work on MX edits in their spare time, and do not have the full set of sensors that MX can support, so pre-release testing is never perfect
* By switching to latest release, you gain any extra functionality it may offer (although it may not be useful to you), but more importantly you gain fixes of bugs your existing release may have
* Also the developer can more easily offer support if you are using the build he most recently released.
 
There are interdependencies''inter-dependencies between different files'' in the MX release distribution:
* ''Therefore you must always install into your MX folders, every file included in the download zip''.
* If you customise any file that is in the distribution, either give the new file a new name (perhaps add a prefix of "_" in front of files you edit) or place your tailored version in a different folder
** You can take a back-up of your existing installation before you copy in any new files, that will provide you with the ability to regress to the earlier release, if the new release does cause you an issue
** Either approach will ensure your tailored files are not overwritten by files from the download, and let you compare your tailored file with the new file in any future download to see if you need to tweak your tailored file
* If you customise any file that is in the distribution, either give the new file a new name (I add a prefix of "_" in front of files I edit) or place your tailored version in a different folder
* When upgrading, it is strongly recommended that you install all files in the download for a new release of MX directly over your existing installation.
** Either approach will ensure your tailored files are not overwritten by files from the download, and let you compare your tailored file with the new file in the download to see if you need to tweak your tailored file
** You can take a back-up of any existing installation before you copy in any new files, that will provide you with the ability to regress to the earlier release, if the new release does cause you an issue
* It is strongly recommended that you install all files in the download for a new release of MX directly over your existing installation.
** You may end up with some files no longer needed by the new release, left over from older release, but this is better than risking missing some vital files
** You need a good understanding of the use of each file, before considering deleting any file that you think might be obsolete, because not all files that Cumulus uses are contained within the release zip, some are created as it runs.
 
: There is an [[Updating_MX_to_new_version#Installer_Option|installer utility]] available to help with upgrading.
 
===Latest build distribution download===
*Important, please read all of the release notes (in the [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 appropriate forum announcement] threads) for the builds since the version you are upgrading from, especially noting when there are references to necessary once-off actions.
*If you have never upgraded before, or you are currently running a fairly old release of MX, you are advised to read the [[Updating MX to new version]] page in this Wiki,
** overwriting an existing installation will add any new files, but it won't remove obsolete files, and sometimes [[MX_Basic_info#Library_software|replacement files]] have different names to their predecessors
** (a few builds will include one-off actions, e.g. rewriting [[Cumulus.ini]] or changes to columns in database tables, in these cases you probably need to upgrade in multiple steps, to ensure these special actions are done at the particular release versions where the actions are supposed to be done).
*If you are upgrading from a previous version of the legacy Cumulus 1, then you might want to read [[Migrating from Cumulus 1 to MX]] page; and if you need more justification read [[Compare_C1_and_MX|Compare legacy Cumulus and MX]] page
** However, it is likely that information on those pages was written for earlier MX releases, and might be out of date, so like many other people you might want to ask in the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 correct MX support forum] to see if others have more up-to-date advice.
 
~ Download [https://github.com/cumulusmx/CumulusMX/releases/download/b3215/CumulusMXDist3215.zip Cumulus MX v3.22.4 build 3215, 20-November-2022]
Note by Steve Loft: you may find that anti-virus software flag a new release of Cumulus as a ‘threat’, for a few days or weeks after a new version has been released, even though it does not actually contain a virus of any kind. This is due to a system they call ‘reputation’ where they automatically block any versions of software that have recently been released. You may want to consider complaining to the provider of your anti-virus software about this ridiculous system, or better still, use some better anti-virus software.
 
===Latest CurrentMX ReleaseSource Code===
 
~ View/download [https://github.com/cumulusmx/CumulusMX Cumulus MX Current Source Code]
===Latest build distribution download===
 
This source code base is updated for all releases resulting from development by M Crossley, it includes some contributions from other developers.
~ Download [https://github.com/cumulusmx/CumulusMX/releases/download/b3194/CumulusMXDist3194.zip Cumulus MX v3.19.1 build 3194, 14-July-2022]
 
===Raspberry Pi Image===
 
~ Download [//{{SERVERNAME}}/Downloads/rpi-CumulusMX-lite-3.1922.0.zip Raspberry Pi Image for Cumulus MX 3.1922.0]
 
As at April 2022, the image includes Raspberry Pi. O. S. 11 "Bullseye" version of Linux.
: Please note, this image may not always contain latest MX release, but once you have installed a release of MX on your Raspberry Pi, [[Updating MX to new version|upgrading]] is done using main software download link, as none of the other components in the image need to be upgraded (unless the release announcement states that the new release needs a specific version of Mono).
 
==Upgrading==
==Latest MX Source Code==
 
: There is an [[Updating_MX_to_new_version#Installer_Option|installer utility]] available to help with upgrading.
~ View/download [https://github.com/cumulusmx/CumulusMX Cumulus MX Current Source Code]
 
We strongly advise [[Updating_MX_to_new_version#Introduction_to_upgrading_MX|upgrading]] to the latest release:
This code base is updated for all releases resulting from development by M Crossley with some contributions from other developers.
* The main developer can most easily offer support if you are using the most familiar recently released build.
* You may wish to wait a few days after a new release comes out, just in case a new release has some bugs
** MX is extremely complex, the developers work on MX edits in their spare time, have settings that suit them, and do not have the full set of sensors that MX can support, so pre-release testing is never perfect
* By switching to latest release, you gain any extra functionality it may offer (although it may not be useful to you), but more importantly you gain fixes of bugs your existing release may have
 
 
*Important, please read all of the release notes (in the [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 appropriate forum announcement] threads) for the builds since the version you are upgrading from, especially noting when there are references to necessary once-off actions.
*If you have never upgraded before, or you are currently running a fairly old release of MX, you are advised to read the [[Updating MX to new version]] page in this Wiki,
** you may be recommended to upgrade in stages, as certain releases do essential one off actions that are missed if you skip that release; e.g. rewriting [[Cumulus.ini]] or changes to columns in database tables
** overwriting an existing installation will add any new files, but it won't remove obsolete files, and sometimes [[MX_Basic_info#Library_software|replacement files]] have different names to their predecessors so don't overwrite the old files
*If you are upgrading from a previous version of the legacy Cumulus 1, then you might want to read [[Migrating from Cumulus 1 to MX]] page; and if you need more justification read [[Compare_C1_and_MX|Compare legacy Cumulus and MX]] page
** However, it is likely that information on those pages was written for earlier MX releases, and might be out of date, so like many other people you might want to ask in the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 correct MX support forum] to see if others have more up-to-date advice.
 
Note by Steve Loft: you may find that anti-virus software flag a new release of Cumulus as a ‘threat’, for a few days or weeks after a new version has been released, even though it does not actually contain a virus of any kind. This is due to a system they call ‘reputation’ where they automatically block any versions of software that have recently been released. You may want to consider complaining to the provider of your anti-virus software about this ridiculous system, or better still, use some better anti-virus software.
 
==Older releases==
==By Mark Crossley==
 
The followingutilities listed here are only for use with MX, they may be included in the [[#Latest build distribution download|MX zipsrelease zip]], but links to dolatest notreleases includewithout the ''*.dll'' files ("software libraries") needed to run the respective utility, these are provided in the MX release zip that is stated alongside download linkbelow.
 
IfThe you"software libraries" are usingprovided anin earlierthe MX release thanzip that is stated, upgradealongside MXdownload tolink. bring in theThe newdifferent "*.dll"utilities filesvary needed,with becauseregard theto utilitywhich willsoftware notlibraries workthey withrequire, theand "softwarethose libraries"with includedtechnical withunderstanding earliercan MXcheck releasesthe code sources.
 
If you are using an earlier MX release than that stated beside a utility download below, upgrade MX to bring in the new "*.dll" files needed, because the utility will not work with the versions of "software libraries" included with earlier MX releases.
If you are using a later MX release than that stated, check [https://cumulus.hosiene.co.uk/viewtopic.php?t=17887 the MX release announcements] for all releases since that stated to see if the "software libraries" have been upgraded, if they have then the respective utility because it has been compiled with older versions of the respective .dll files will probably not work. (Sometimes the names of new .dll files are different to the names of the old .dll files, so an upgrade of MX done by copying files over an existing installation will leave the old files there, in that case the utility might still work).
 
If you are using a later MX release than that stated, check [https://cumulus.hosiene.co.uk/viewtopic.php?t=17887 the MX release announcements] for all releases since that stated below to see if the "software libraries" have been upgraded, if they have then the respective utility because it has been compiled with older versions of the respective .dll files will probably not work. (Sometimes the names of new .dll files are different to the names of the old .dll files, so an upgrade of MX done by copying files over an existing installation will leave the old files there, in that case the utility might still work by picking up old names).
 
=== Create Missing ===
 
~ The CreateMissing.exe ''v1.3.0'' download dated 18 August 2022 (works with MX release of 3.20.0 onwards) [https://github.com/cumulusmx/CreateMissing/releases/download/v1.3.0/CreateMissing-v1.3.0.zip is here].
This is a MX utility, by Mark Crossley, to create a new [[dayfile.txt|daily summary (dayfile.txt)]]. If you already have a "dayfile.txt", that is renamed, the new file should include individual fields, and lines, missing from your old file.
 
Check compatibility with the MX release you have running as per notes above. (Although earlier releases of "Create Missing" exist on "github.com/cumulusmx/CreateMissing/releases", their software bugs imply they should not be used). Install into same directory as "CumulusMX.exe".
The utility reads the [[Calculate_Missing_Values#Source_value|source values]] in your [[Standard log files|month by month (MMMyylog.txt)]] files as input files. This utility has the ability to calculate many of the [[Calculate_Missing_Values#Derived_value|derived fields]], so will add those derived values to your month by month files as it proceeds. Therefore the utility should be used to add the [[Standard_log_files#Number_of_fields_per_line_varies_by_release|extra fields]] for those months where the input file was produced by Cumulus 1 or an earlier release of MX (which might have populated fewer fields or used a [[Feels_Like#How_to_express_Feels_Like_for_highest_temperatures|different calculation]]).
 
This MX utility, affects [[Category:Files with Comma Separated Values|Cumulus CSV files]]:
~ The CreateMissing.exe ''v1.2.1'' download dated 20 May 2022 (works with release of MX 3.14.0 onwards) [https://github.com/cumulusmx/CreateMissing/releases/download/v1.2.1/CreateMissing-v1.2.1.zip is here].
# It will create a new [[dayfile.txt|daily summary (dayfile.txt)]]. If you already have a "dayfile.txt", that is renamed, the new file should include individual fields, and lines, missing from your old file.
# It will read, and where appropriate update, your [[Standard log files|month by month (MMMyylog.txt)]] files.
#* The utility reads the [[Calculate_Missing_Values#Source_value|source values]] in as input values. This utility has the ability to calculate many of the [[Calculate_Missing_Values#Derived_value|derived fields]], so will add any missing derived values to your month by month files as it proceeds.
#* Therefore the utility should be used to add the [[Standard_log_files#Number_of_fields_per_line_varies_by_release|extra fields]] for those months where the input file was produced by Cumulus 1 or an earlier release of MX (which might have populated fewer fields or used a [[Feels_Like#How_to_express_Feels_Like_for_highest_temperatures|different calculation]]).
 
Check compatibility with the MX release you have running as per notes above. (Although earlier releases of "Create Missing" exist on "github.com/cumulusmx/CreateMissing/releases", their software bugs imply they should not be used).
 
Usage information by the author of the utility can be [https://github.com/cumulusmx/CreateMissing/blob/master/README.md found here.]
* Learn about this utility by reading the [[Calculate_Missing_Values#CreateMissing.exe|calculate missing]] page in this Wiki.
* Learn more about this utility by reading the [[Calculate_Missing_Values#CreateMissing.exe|calculate missing]] page in this Wiki.
** Elsewhere on that same page learn when this utility might be useful, and what else you can try.
* Check in the support forum for information about status of bugs in this utility, at time of last updating this paragraph there was an outstanding issue concerning storing the line in dayfile.txt for the first of some months if your rollover time is not midnight.
* For full information about solving problems with your '''dayfile.txt''' file read the [[Amending dayfile|amending daily summary file]] page.
 
=== Create Records ===
Usage information by the author of the utility can be [https://github.com/cumulusmx/CreateMissing/blob/master/README.md found here.]
 
This utility is not yet available.
 
A utility to create, or recreate, missing [[Correcting_Extremes|extreme records]] from Cumulus MX by parsing your [[dayfile.txt|daily summary (dayfile.txt)]].
 
=== Export ''To'' MySQL ===
 
~ The ExportToMySQL.exe ''v1.6.1'' download dated 02 September 2022 (works with release of MX 3.21.0) [https://github.com/cumulusmx/ExportToMySQL/releases/download/v1.6.1/ExportToMySQL-v1.6.1.zip is here.]
A utility, by Mark Crossley, to export content of Cumulus MX [[Standard log files|month by month (MMMyylog.txt)]] and [[dayfile.txt|daily summary (dayfile.txt)]] files to a [[Your Own Server|MySQL, or Maria, database server]].
 
Check compatibility with the MX release you have running as per notes above. Note that compatibility is needed both with the .dll files and with the columns present in the database table to be updated.
 
~If Theyou ExportToMySQL.exeare ''v1.4.1''not downloadusing datedthe 21MX Junerelease 2022named (worksabove, withthen other release ofzips MX(and 3.18.0source code) for this utility are available at [https://github.com/cumulusmx/ExportToMySQL/releases/download/v1.4.1/ ExportToMySQL-v1.4.1.zip/releases] and each release there is herelabelled as to which MX release creates default tables with correct columns.]
 
This utility reads [[MySqlConnect|Settings]] that are also used by "CumulusMX.exe". Parameters supplied to this utility enable it to do ''one of the following actions'' affecting rows not already present on a [[Your Own Server|MySQL, or MariaDB, database server]]:
Check compatibility with the MX release you have running as per notes above. Earlier release zips (and source code) for this utility are available at [https://github.com/cumulusmx/ExportToMySQL/releases ExportToMySQL/releases].
# The parameter used is "dayfile", will insert missing rows into a database table that is called "dayfile" (by default, the user can change the name of that table) that (as minimum) has columns named as per the MX default for that table (at CumulusMX.exe release quoted for which the download applies) from the contents of [[dayfile.txt|daily summary (dayfile.txt)]] file
# Insert missing rows into a database table that is called "Monthly" (by default, the user can change the name of that table) that (as minimum) has columns named as per the MX default for that table (at CumulusMX.exe release quoted for which the download applies) from the contents of the single file of [[Standard log files|month by month (MMMyylog.txt)]] whose relative path "data/..." is named in the parameter
# The parameter used is "monthly", will insert missing rows into a database table that is called "Monthly" (by default, the user can change the name of that table) that (as minimum) has columns named as per the MX default for that table (at CumulusMX.exe release quoted for which the download applies) reading data from the contents of all files of type [[Standard log files|month by month (MMMyylog.txt)]] found in the [[Data folder|data sub-folder]].
 
UsageInstallation and usage information by the author of the utility can be [https://github.com/cumulusmx/ExportToMySQL/blob/main/README.md found here.]
 
''This new utility replaces the "ExportMySQL.exe" (no "To" in that name) utility included by Steve Loft in his MX beta distributions.'' That is not compatible with any recent MX releases.
=== Weather Display Converter ===
 
This utility was provided by Steve to convert '''Weather Display''' log files to legacy Cumulus 1 format, download it [https://cumuluswiki.org/a/File:WDconverter.zip here].
 
Please carefully read the '''readme''' file in the zip. There is a [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=10118 forum thread about using this converter with the legacy Cumulus here].
 
Remember this utility will not support all fields used by MX. It is advised you use [[#Create Missing]] (see above) after using this utility to populate the other fields. If you do find this utility does not work with your MX release, you may need to manually recreate records, etc. using Cumulus 1 formats, and then [[Migrating from Cumulus 1 to MX|Migrate all your files from Cumulus 1 to MX]].
There is a [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=10118 forum thread about using this converter with the legacy Cumulus here].
 
=== WeatherLink Converter ===
<br>
 
This utility was provided by Steve to convert Davis '''WeatherLink''' log files to Cumulus format, download it [https://cumuluswiki.org/a/File:WLconverter.zip here]. Please carefully read the '''readme''' file in the zip to understand how to use this utility.
At time of creating this utility, the functionality offered by MX was limited, and this utility was not useful with MX.
 
Since then, MX has developed and has changed so much, so this utility can be used with MX, but will not support all fields used by MX. It is advised you use [[#Create Missing]] (see above) after using this utility to populate the other fields.
 
If you do find this utility does not work with MX, you may need to manually recreate records, etc. using Cumulus 1, and then [[Migrating from Cumulus 1 to MX|Migrate all your files from Cumulus 1 to MX]].
 
=== WeatherLink Converter ===
 
NOTE: this has nothing to do with Davis ''Weather Link Live'' despite confusing similarity in name.
This utility was provided by Steve to convert WeatherLink log files to Cumulus format, download [https://cumuluswiki.org/a/File:WLconverter.zip here].
 
Basically, this converter utility reads '''.wlk''' files, it creates a folder called '''converted''' into which it places [[Standard log files]] and [[dayfile.txt]] files as used by Cumulus software. You need to be able to manually merge (using a text file editor) the newly created files with any files of same name in your Cumulus [[data folder|'''data''' folder]]. You might also want to read (although not directly related to this converter utility) [[Amending_dayfile#Importing_data_not_recorded_by_Cumulus|Importing_data_not_recorded_by_Cumulus]].
Please read the '''readme''' file in the zip.
 
A search on the support forum will reveal various posts on use of this converter. The key topics are:
* There is a [https://cumulus.hosiene.co.uk/viewtopic.php?f=6&t=20017 recent forum thread about using this converter with MX here]
* There is an older [https://cumulus.hosiene.co.uk/viewtopic.php?f=6&t=1134920017 forum threadtopic about using this converter with the legacy CumulusMX here].
* [https://cumulus.hosiene.co.uk/viewtopic.php?t=11349 forum topic about using this converter with the legacy Cumulus here].
 
== Third Party Contributions by other Cumulus Users ==
 
For files, and add-on's, contributed by Cumulus users, see these Categories (where third party authors may have mentioned what they offer):
# [[:Category:3rd Party Apps]] - contributions for various devices,
# [[:Category:AddOns]] - adding extra functionality,
# [[:Category:User_Contributions]] - key contributions from the user community
 
[[Category:Cumulus MX]]
 
=Current MX Developed from =
== Cumulus 1 ==
 
Cumulus 1 software was popular, when it went public in 20082004, and its [[Cumulus_Users|usage]] grew very rapidly in the subsequent decade until final 2014 release. Even now (in mid-2022) some people are still using the final release version (and a quite substantial number of public viewable weather sites are still driven by earlier builds).
 
Be aware, the longer you wait before swapping from legacy to Cumulus MX software; the more that MX will have developed away from the legacy. MX's development has effectively abandoned its former focus on compatibility, so you might encounter more difficulty with [[Migrating from Cumulus 1 to MX|transferring your data files when you migrate]].

Navigation menu