MX on Linux: Difference between revisions

2,686 bytes added ,  27 August 2021
m
(20 intermediate revisions by the same user not shown)
===sudo===
 
By default, a Linux user will log in as a default user with limited rights. For example, on the Raspberry Pi Operating System, there is a single default user, initially called "Pi", with their home folder that can be referenced as "~".
 
For example, the Raspberry Pi Operating System has a single default user "Pi", with their home folder that can be referenced as "~".
The initial "sudo" part of many commands gives us super-user (root) rights when executing the instruction that follows.
 
In the '''apt''' context here, those limited rights allow use of actions that just read (such as '''search''', ''show'').
If we are working away from the folder(s) owned by the default user, we can consider whether we want to change the ownership of these parts of the file system (how to do this for a MX installation is explained later) so that we can access the files with our default user without using "sudo" for file/folder commands.
 
However, for any action involving writing (such as '''install''', ''full-upgrade'', '''update''', ''autoremove''), the package manager needs additional rights, and we prefix the "apt" with '''sudo'''.
In some limited cases, it might make sense to just change "read" rights,so that the default user can read a file/folder, but other actions are not possible without using "sudo" prefix. This limits our ability to accidentally delete files that other processes need.
 
: Elsewhere on this page you might notice [[#Running any MX executable with a terminal session left open|'''cd''']] can move round the file structure (without a "sudo"), but "sudo" is used with [[# Installing/Configuring the MX service and the -service parameter|'''cp''']] as that writes a file.
 
Novice readers should exercise caution, because using "sudo", changing folder/file ownership, adding write rights, and even using "-R" or "-y" flags, can all make it very easy to inadvertently do the wrong action, even perhaps delete vital folders/files.
 
: '''For technical readers only''', "sudo" is further explained later [[#su_and_sudo]]). Briefly, "sudo" gives super-user (root) rights when executing the instruction that follows for actions on files that are either not in the current user's ownership, or for which the current user lacks necessary (read, write, execute) permissions.
 
===apt===
The second part of our installation commands is “apt” meaning “Advance Package Tool”. In simple terms, it runs the “package manager” used in Linux.
 
If you are looking online for a tutorial on how to install packages, you might see “apt-get” or "apt-cache" used in examples they quote. It should be safe to replace these older package managers with "apt". I have included [[#Package Manager – a brief technical aside|a little technical explanation]] at the end of this page if this really worries you.
 
===install===
 
MX uses this library for weather stations (like Fine Offset and USB connected Oregon Scientific models) that appear as a HID via a USB connection. To check if your weather station appears as a HID, download this [https://cumulus.hosiene.co.uk/download/file.php?id=11414 USB HID test package] and run it.
 
 
This HidSharp library calls a package ''libudev shared library'' file called ''libudev.so.1''. Most recent Linux distributions will already include the libudev shared library. On my Raspberry Pi, [[#install|typing]] <code>apt search libudev</code> showed ''libudev.so.1'' was already installed. If it is missing, then [[#install]] it.
: <code>sudo apt install -y mono-complete</code>.
 
* The “sudo”, “apt”, “install”, and “-y” have [[#The various components to commands for installation|already been explained]].
* The "mono-complete" is the package we want.
 
The "mono-complete" is the package we want.
 
:Please note that if you just specify “mono”, you will get ‘’’mono-devel’’’ (the developer edition) that does not include all the components required by each of the Cumulus executables.
 
Please note that a particular MX build might specify it needs a particular version of Mono. Hence, although normally you can upgrade aany (CumulusMX.exe, ExportToMySQL.exe, CreateMissing.exe, proposed CreateRecords) cumulus package without upgrading Mono, sometimes you will need to install Mono again.
 
The latest release of Mono can always be downloaded from [https://www.mono-project.com/download/stable/#download-lin], followfor stepa 1variety there,of butLinux in step 2 replace ‘’’mono-devel‘’’ by ‘’’mono-complete’’’distributions:
# follow step 1 there,
# but in step 2 replace ‘’’mono-devel‘’’ by ‘’’mono-complete’’’
 
The complete mono package includes a component (mono-xsp4) that creates a simple web server to run ASP.NET 4.0 applications. MX does not need this, so type <code>sudo update-rc.d mono-xsp4 disable</code>.
 
=Cumulus packages=
 
Note use of plural in section name above, the following sub-sections will install various packages produced by Developer Mark Crossley. If you are using the [[Software#Cumulus_MX|pre-built disc image]], then (unless the MX release version included in your image is an old one) you should skip the instructions for installing "CumulusMX" itself.
 
==Handling zip files==
When your browser saves the zip it might be into a folder called “downloads” on your computer, or you may be able to save into another folder that you prefer (perhaps on a different partition). Your browser might even remember the folder you used before for files of type zip.
 
When the download has completed, whatever computer type this is on, mouse click (it might need a right click or a double click depending on settings) on the download file and it should unzip (it may create a folder whose name is taken from the zip file name in the same folder by default, or it may ask you where you want to unzip to). If you are unable to use a mouse, there should be a keyboard code to use. If you are using a file manager, with a graphical interface, there may be a differentmenu wayitem (or two menu items) to select the file and unzip it.
 
 
==Where to install all packages?==
*# The developer suggests you use ‘’’/opt’’’ for EXISTING PATH (which should be available on any Linux computer).
*#* By default, the code Mark provides for installing Cumulus as a service, will run that service as a root user, and the root user has full permissions in /opt (and everywhere else)
*#* (Novices: Skip this step) If you do choose a EXISTING PATH outside your home folder, youthen mighta wantmore totechnical user can change the ownership of the "CumulusMX" sub-folder, to the default user (Pi), if so, typewith <code>sudo chown -R pi: EXISTING PATH/CumulusMX</code>., and Thereduce advantagesthe ofneed thatto command is you no longer needuse "sudo" toon accessmany theactions. files (however, if you are running MX as a service, you also need to edit the user in the script provided to create the service, so MX does not create files with root ownership - this will be covered later)
 
==Packages to install==
 
<big>We shall install the Cumulus software listed on [[Software]] page</big>:
# '''CumulusMX’’’CumulusMX''':
#* '''CumulusMX.exe’’’exe''' is written in C#, that has been developed by Mark Crossley, but still contains some code by Steve Loft
#* Download '''CumulusMX zip file’’’ from the link at[[Software#Latest_build_distribution_download]]
# [[Software#Create_Missing|'''Create Missing''']]:
#* '''CreateMissing.exe''' is anotheralso written in C#, created, and developed, by Mark packageCrossley
#* Download '''Create Missing zip file''' from the link at [[Software#Create_Missing]]
#* Using '''CreateMissing.exe''' is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki (it will populate missing fields in [[standard log files]] and/or missing lines in [[dayfile.txt]]),
#** This takes you to a github page with a "ReadMe" providing minimal instructions
#* Simple Instructions for using this executable are on the github page where they are found, you can find the link for that at [[Software#Create_Missing]] in this Wiki.
#* Using '''CreateMissing.exe''' is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki
#* (it will populate missing fields in [[standard log files]] and/or missing lines in [[dayfile.txt]]).
# '''ExportToMySQL'''
#* '''ExportToMySQL.exe''' is also written in C# by Mark Crossley
#* Download '''ExportToMySQL.exeExport To My SQL zip file''' package from the link at [[Software#ExportToMySQL]]
#** This takes you to a github page with a "ReadMe" providing minimal instructions
#* '''ExportToMySQL.exe''' is not (at the time this was written) documented in this Wiki although [[MX_Administrative_Interface#MySQL_settings]] does describe a similar utility (written by Steve Loft) that was actually included in early MX release downloads.
#* '''ExportToMySQL.exe''' is not (at the time this was written) documented in this Wiki although [[MX_Administrative_Interface#MySQL_settings]] does describe a similar utility (written by Steve Loft) that was actually included within early CumulusMX zip downloads.
 
As at 9 March 2020, there is another utility, '''CreateRecord''', has been initialised in the Github areas managed by the developer where Cumulus is worked on, although it appears to be just a concept (any code on the developer's computer has not yet been included) on github. This will, if my understanding is correct, read [[dayfile.txt]] and use that to update the various [[:Category:Ini Files|extreme record files]]. However, at the time of writing this, it is nothing more than a concept that needs to be coded, and (as far as I know) there has been no progress on that utility for at least 4 months.
 
===Alternative download link for older MX releases===
All the (optional) files in the [[Reports folder]] can be copied across from a previous installation. If your previous installation was not on Linux, see [[#Line terminators in .txt files]]
 
<br>
All the files in the [[data folder]] can also be copied across. If your previous installation was not on Linux, see [[#Line terminators in .txt files]]
 
Whilst you should copy ALL the files in the [[data folder]], from any old installation into the new installation, there are several extra considerations:
 
* '''Your new MX installation will ignore any entries prior to the MX Start Date''' in [[Monthly log files]], such as [[Standard log files]], and [[Extra Sensor Files]]:
If your previous Cumulus installation was version 1.9.4, or earlier, then you need to do a lot of reading:
** See [[Cumulus.ini#Data_Logging|'''StartDate=xxxxx''']] parameter, edit using ''Station Settings → Common Options → Advanced Options → Records Began date''
* [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]]
* Be aware your new installation has to use the same "locale" as the old installation, or MX will struggle as the locale affects how new lines are stored, and how MX expects old lines to have been stored.
* [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files
* Microsoft Windows uses different line terminators, see [[#Line terminators in .txt files]], although MX should cope, any third party routines reading your data files will probably not accept a line terminator change.
* [[Migrating from Cumulus 1 to MX]] gives some advice about differences in settings, but be aware that the way MX handles settings varies by release, and information on the linked page may be out of date
* If your previous Cumulus installation was of the legacy software, version 1.9.4, or earlier, then you need to do a lot of reading:
** [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]]
** [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files
** [[Migrating from Cumulus 1 to MX]] gives some advice about differences in settings, but be aware that the way MX handles settings varies by release, and information on the linked page may be out of date
 
==Configuration Files to copy across from any previous Cumulus installation==
*[[Cumulus.ini]] (note initial capital, then lower case) – main configuration file
 
Just copy the existing files from old to new installation, if
If your old installation was for a relatively recent release, then just copy these files to new installation and optionally skip the next 2 sub-sections.
# Your locale is still the same
# All files on your new install are in same paths as on your old install (some settings involve specifying paths)
# Your old installation has a relatively recent MX release (compare the "y" in 3.y.z,between old and new installation, a difference of more than 1 means you do not have a recent release)
# Your old installation was on a Linux computer (not a computer running Microsoft Windows Operating System)
 
If you are upgrading from an older release, please read the next 2 sub-sections for advice.
 
If you are upgrading from an older release, please read the table for advice.
===strings.ini===
 
{| class="wikitable"
'''This is an optional file'''. Its [[strings.ini|purpose]] is to allow customisation of some of the outputs from Cumulus. You might want to use customisation to abbreviate (or extend) some outputs, or to change those outputs into another language.
|-
! scope="col" style="width:450px; color:blue" | Cumulus.ini !! scope="col" style="width:450px; color:navy" | strings.ini
|-
| Your old installation will have this file. In general, ''if your old installation was any release before 3.8.0'', the advice is give the old file a different name when you copy it across to the new installation, and let MX create the file as you work through all the settings.
| '''This is an optional file'''. Its [[strings.ini|purpose]] is to allow customisation of some of the outputs from Cumulus. You might want to use customisation to abbreviate (or extend) some outputs, or to change those outputs into another language.
|-
| When you work through the Settings pages, MX will create this file if it does not exist.
* See [[#Moving from Microsoft Windows to Linux]] if your old installation is on a Microsoft operating system, as several changes will be needed for extra web file settings on your Linux computer
* If your old installation was of the legacy software then also see [[Migrating from Cumulus 1 to MX]]
* As MX evolves, the former "read-only" settings in this file are becoming "advanced" settings in the interface.
| You create a “strings.ini” file by '''selecting some of the parameter'''s from the [[Samplestring.ini]] file that is included in each MX release, and ''modifying the value for the listed attributes'' as you type just those you selected (under the same group titles - these are enclosed in [ ] as before).
 
The sections that appear in '''samplestring.ini''', and the parameters that appear within a section, depend upon which release you are using. So be cautious if you try to reuse a "strings.ini" file originally created by the legacy software, you may find you need to specify your customisation using different parameters in the latest "samplestring.ini".
You create a “strings.ini” file by selecting some of the parameters from the [[samplestrings.ini]] file that is included in each MX release, and modifying the value for the listed attributes as you type them (under the same group titles - these are enclosed in [ ] as before).
|-
| The content of "Cumulus.ini" is changing as MX is developed, the [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 Release Announcements] normally list any new parameters as they appear in the file, without always mentioning those that have become redundant. The announcements tend to avoid any detail, so you have to guess ''from the attribute'' what values it might take, and generally have no idea of where to make any change.
 
To remove any parameters no longer used in this file, see [[Cumulus.ini#How_to_Remove_Redundant_parameters_from_file|remove redundant parameters]]
The sections that appear in '''samplestrings.ini''', and the parameters that appear within a section, changed drastically between Cumulus 1.9.4 and MX. So be cautious if you try to use a "strings.ini" file originally created by the legacy software, check whether the parameters you used before are still available in the latest "samplestrings.ini".
 
If your old file contains any [[Cumulus.ini (Cumulus 1)|legacy read-only]] parameters not yet converted into advanced settings, or any [[Cumulus.ini (MX 3.0.0 to 3.7.0)|MX read-only parameters not yet converted into advanced settings]], you may need to manually add such missing parameters back into new file by stopping MX (after finishing all the settings you can configure in the interface), doing an external file edit, and then restarting MX
The content of "samplestrings.ini" is changing as MX is developed. Therefore, your existing “strings.ini” might need to be modified. There is no automatic way to check your “strings.ini” file, if MX does not understand any parameter in this file, it ignores it. Instead, you need to manually check each parameter you have in your “strings.ini” file to see if that parameter is in “samplestring.ini” included in the release you have installed. You may also find new parameters in “samplestring.ini” that you wish to add to your “strings.ini” file to tailor new functionality to your preferences.
| The content of "samplestring.ini" is changing as MX is developed:
 
* Therefore, your existing “strings.ini” might need to be modified.
===Cumulus.ini===
* There is no automatic way to check your “strings.ini” file, if MX does not understand any parameter in this file, it ignores it.
 
* Instead, you need to manually check each parameter you have in your “strings.ini” file to see if that parameter is still in the “samplestring.ini” included in the release you have installed.
If your old release was 3.8.0 or later, then copy the [[Cumulus.ini]] file used in that old installation into your new one. The file is growing with lots of new parameters, and the tables on the page I have just linked indicated when parameters were introduced, and where those settings may be changed to make it simple for you to ensure all settings are right for your new release.
* You may also find new parameters in “samplestring.ini” that you wish to add to your “strings.ini” file to tailor new functionality to your preferences.
 
|}
If your old release was older that 3.8.0, then you will see [[:Category:Configuration Files|the Configuration Files page]] shows different links for documentation on older "cumulus.ini" files. Because of those differences, many people prefer to rename their existing “Cumulus.ini” file when upgrading from such an old release, or making an install on a new device. To create a new file, they work through all the settings manually, so that MX can create a fresh file, with them having confidence every setting reflects their preference. However, you do need to be aware that such older releases included a lot of "read-only" parameters that had to be set manually in the file, you may find some of those parameters are still needed by you, so you will want to edit the new file, with MX stopped, and copy any of those old "read-only" parameters that might not yet be settings you configure within the MX interface (although the number of these is diminishing with each new release).
 
If you previously used an older release of Cumulus, but in this new installation will be using the latest release (latest is what is normally best, unless it has bugs), you may want to read up on all the changes between your old release and the current release, not just changes that affect the configuration file.
# as a service and
# interactively with terminal window left open.
 
As a side-note, some weather station types require MX to be run with '''root''' rights, this is why ''sudo'' is included in each example command, but if you have technical knowledge and know your weather station can be run with default user, you can omit the ''sudo'' provided your user has ownership of all folders and files used by MX.
 
== Optional parameters to add to the instruction to run the MX engine ==
 
This list of parameters covers every parameter, if you are using the latest release skip the historic 3.0.0 sub-sections.
 
IMPORTANT: The "sudo" prefix shown in code examples for the various parameters can be omitted if the user that is running MX owns the folder "CumulusMX" and all its contents. In the [[#Where to install all packages?|installation notes earlier]] possible locations and ownership issues were mentioned. That earlier section also defined EXISTING PATH, if that is not mentioned in the code examples, it is assumed you have already issued a <code>cd EXISTING PATH</code> to be in the right folder for issuing the command.
 
Note that you ''may'' need to supply your root password after typing any 'sudo ...' command line if the instruction changes something set by another package (e.g. changing the locale for MX from the locale set by mono). The system will prompt you for this if it is needed.
=== -port parameter for changing Port ===
 
When Cumulus starts, it will display the URL of the user interface. It runs on port 8998 by default; if this is not suitable for some reason you can over-ridechange it using the '-port' parameter on the command line, e.g. to use port 9999 instead:
<pre>sudo mono CumulusMX.exe -port 9999</pre>
 
=== -lang parameter for changing Locale ===
 
Individual computers may have different default for their locale. The forum reported an issue with OS X with its default US locale even if that is not your locale.
On Linux and (in particular) OS X, Cumulus MX may not be given the correct locale to use, and you may get the default US locale even if that is not your locale. It will output the local it is using when it starts; if it is not correct, close it down and start it again, this time specifying your locale on the command line, using the -lang parameter . For example, in the UK, on a non-Windows device type:
 
As MX starts its output (to diagnostic log and to any interactive terminal window) says which locale is being used, '''Current culture: English (United States)'''. That default locale will relate to the one your computer had when mono is installed, as it is mono that tells MX all the locale settings. If you notice MX starting with wrong locale, issue the correct instruction to close MX.
 
You can use a '''-lang''' parameter to change the locale as you restart MX. For example, in the UK, type:
:<code>sudo mono CumulusMX.exe -lang en-GB</code>
 
Other locale examples: '''CumulusMX.exe Current culture: English (United States)''', '''CumulusMX.exe -lang de-DE''', '''CumulusMX.exe -lang el-GR''' (this is one of the locales that reads numbers with '''integer,decimal''' format), '''CumulusMX.exe -lang nl-NL'''.
 
If you are not sure what value you need to supply for the -lang parameter, there is a list here - http://msdn.microsoft.com/en-gb/library/ee825488%28v=cs.20%29.aspx. You need to supply the code in the first column ("Language Culture Name") in that list.
 
=== Installing/Configuring the MX service and the -service parameter===
 
Parameter for running as service a used within file for installing/configuring the service ===
 
The parameter syntax is <code>sudo mono CumulusMX.exe -service</code>
You don't use this parameter in a terminal or interactive instruction for running MX. Instead it appears within a file that we use to set up for running MX as a service.
For therecent latest releasereleases, (see [https://cumulus.hosiene.co.uk/viewtopic.php?p=146473#p146473 this lateroriginal release announcement]):
#There is a task to do just once to configure the service
#Find the '''EXISTING PATH/CumulusMX/MXutils/linux/''' sub-folder, that might be in home directory and therefore found using "~/CumulusMX/MXutils/linux" as explained elsewhere on this page
 
There are some edits needed to that section:
# ReplaceTechnical users might want to edit the '''User=root''' ifline youso wantthat the service towill run as a different user, novice users are best to leave this alone
# EditEveryone must edit the line that begins with '''ExecStart='''
#* The mandatory change is to replace '''/home/install''' by what you have selected for EXISTING PATH
#* An optional change is to add additional parameters after the '''-service''' (select from '''-debug''', -locale, -port) as described in sub-sections above
# Next save file with a new name '''cumulusmx_minecumulusmx_edited.service''' (using a new name stops it being overwritten when we upgrade MX
#now Now copy file to where it is needed to run the service: <code>sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx_edited.service /etc/systemd/system/cumulusmx.service</code>
# Finally, optionally create a symbolic link to that file using <code>sudo systemctl enable cumulusmx</code> if you want the service to start after a reboot
:<code>sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx_mine.service /etc/systemd/system/cumulusmx.service</code>
 
Here is how the file might look after the changes:
<pre>[Unit]
Description=CumulusMX service
Documentation=https://cumuluswiki.org/a/Main_Page
After=network-online.target
 
[Service]
User=pi
Group=pi
ExecStart=/usr/bin/mono-service -d:/opt/CumulusMX CumulusMX.exe -service -debug
Type=forking
ExecStopPost=/bin/rm /tmp/CumulusMX.exe.lock
 
[Install]
WantedBy=multi-user.target</pre>
 
====Setting up as a service in release 3.8.0 ====
 
The previous section covered how to set up a service in all recent releases, but if you are still using the [https://cumulus.hosiene.co.uk/viewtopic.php?p=145048#p145048 3.8.0 release] then different instructions were supplied:
Skip this sub-section for latest release.
 
For historic reasons, note the original instructions in the 3.8.0 [https://cumulus.hosiene.co.uk/viewtopic.php?p=145048#p145048 release announcement]:
# Ensure you are in the folder containing CumulusMX.exe
# Type <code> mono-service -l:/var/run/cmx.pid CumulusMX.exe -service</code>
 
=== -debug parameter for adding debugging ===
 
This is only available for [https://cumulus.hosiene.co.uk/viewtopic.php?p=138839#p138839 release 3.4.4 - Build 3068] onwards. This switches on debug and data logging from the start-up of Cumulus MX by adding a parameter:
:<code>sudo mono CumulusMX.exe -debug</code>
 
MX has a default level of logging that stores in the [[MXdiags_folder]] folder a log file that shows some of the interaction with the weather station and some of the output actions done as MX runs. A new log is started each time MX is restarted.
 
If there is a problem, then there is a great benefit in actually increasing the level of detail in these logs; and that is done either within the settings (on recent MX releases this is on '''Program Settings''' page of admin interface - please see [[MXdiags_folder:Category:Diagnostics]] page for details) or by adding a parameter:the above command.
:<code>sudo mono CumulusMX.exe -debug</code>
 
=== Parameters only applicable to Version 3.0.0 Beta builds of MX ===
Beta builds in MX version 3.0.0 had an optional parameter <code>sudo mono EXISTING PATH/CumulusMX/CumulusMX.exe -wsport nnnn</code> that determined which port (represented by a 4 digit number ''nnnn'') was used for '''Web Sockets'''.
 
ThatNote use of this parameter is now deprecated, as it has been incorporated into '''-port''' parameter described earlier. The reason is that [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887&p=138815&hilit=sockets#p138815 is now deprecated as Web Sockets in all builds since 3045] use the same port for web sockets as for the HTTP port of the [[MX_Administrative_Interface#The_API_interface|Admin Interface]],. see -port parameter described earlier.
 
==== -logging parameter for debugging of data flow between station and MX====
 
UseBeta '''builds in MX version 3.0.0 had an optional parameter <code>sudo mono CumulusMX.exe -Logging=1'''</code> (for the station to MX transfers to have increased debugging logging).
 
Note use of this parameter is now deprecated.
 
Note use of this parameter is now deprecated, as it has been incorporated into '''-debug''', see above.
Although this is not mentioned in any release announcements, it appears that on all recent MX releases this effect is incorporated into the '''-debug''' parameter described earlier. Perhaps someone could confirm whether this is true.
 
==Running as a service==
5,838

edits