MX on Linux: Difference between revisions

'''This page focuses on aspects of MX that are specific to the Linux operating systems.’’’

This page is intended to cover all Linux-based devices, but the content here is based on experience of those who have contributed to this page.  

Linux is available in a multitude of different kernels, on a multitude of devices, but this Wiki page will largely ignore any technical variations (some included at end of page) and focus on giving some background content to support the basics.

 

The Raspberry Pi Operating System is based on Debian, one of the Linux kernels.

 

If you use a different kernel, or feel that this page inadequately covers what you want to know, can you add sub-sections to this page to ensure it covers the Linux device you use?

Microsoft has had a deliberate policy of being different, so it is not UNIX-like. The [[MX_on_Windows_OS|Running Cumulus MX on Microsoft Windows]] page covers those aspects of MX that are specific to Personal Computers running Microsoft's Windows Operating System.  

= Do you have a Kernel? =

Is your Linux computer already working? Or does it need a Kernel to be installed?

Operating system is alternative name for the kernel, this is the software that is loaded onto your computer to provide Linux commands and everything else that males a computer work.

Please see [[Raspberry Pi computer page]] if you want guidance on choosing which model to buy and how to install an operating system, so you are ready to install MX.

=Ready to install MX?=

 

Assuming our Linux computer has a kernel, how do we add packages to our system? We need a short technical digression to explain the command, skip this next section if you already know how to install packages.

There is a longer technical explanation towards the end of this page, all I will say here is that you might see “apt-get” used in some on-line examples. I suggest you replace “apt” where “apt-get” is seen as generally “apt” is more friendly. Equally, “apt” can replace “apt-cache”, which I won’t explain.

|To make sure your computer has up to date information about repositories installed and to report if these contain packages that can be upgraded

|To see which of your packages have newer versions now available in repositories, and to replace those packages with the newer versions

|To check all components in the packages you have installed and remove those components that are not needed by the dependencies of the packages you use.

As an example, when you install mono-complete, there might be components that are never used and “autoremove” can be used to tidy up when you finish all your installations

|If you want to remove just one component manually, use “remove” followed by the name of component you no longer want

|To remove any installed package and delete all related configuration files

After the “install” part mentioned above, we can add “-y” to signify that we want the install to continue where without the “y” the package manager will ask periodically if we want it to continue.

The final part of the installation command is the name of the package or component that we want to install.

Before we do an install of a new package, we typically use some of these commands to ensure our computer is in the best state to work out dependencies of what we are about to install:

If you read all the previous section, it explained Linux has a ‘’’source list’’’ of ‘’’repositories’’’ from which it can load software packages.

#Our first task will be to install the appropriate mono repository, if that is not already in our source list

# Our second task will install a package called ‘’’mono-complete’’’ from the repository we just installed, (this is needed to run software written in the C# languages)

# Then to install ‘’’CumulusMX’’’, this is written in C# and we download it from [[Software#Latest_build_distribution_download]]

# Next to install ‘’’ Create Missing’’’ another C# package (it will missing fields or missing lines in log files) package downloading from [[Software#Create_Missing]]

#Finally, install ‘’’Export (old data) To a Maria (or other MySQL) database server’’’ package downloaded from [[Software#ExportToMySQL]]

If you type <code>sudo apt search mono-complete</code>, you will find out whether the package is available from one of the repositories already in our source list.  Each of the parts of that command was explained earlier.

If mono-complete is not available (or only available in an older version incompatible with MX), then we have to add a new repository, and the one to add depends on which kernel we have, so choose the right sub-section below.

The two Mono repositories listed here are specific to the 2017 and 2019 releases (respectively) of the operating system for a Raspberry Pi computer.  These are taken from  [https://www.mono-project.com/download/stable/#download-lin-raspbian].

{| class="wikitable" border="1"

!style="width:300px" | Raspberry Operating System 9 (stretch)

!style="width:300px" | Raspberry Operating System 10 (buster)

echo "deb https://download.mono-project.com/repo/debian stable-raspbianstretch main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list

echo "deb https://download.mono-project.com/repo/debian stable-raspbianbuster main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list

==Add the Mono repository to Ubuntu, Debian, Fedora==

At time of writing, [https://www.mono-project.com/download/stable/#download-lin-ubuntu], shows the instructions for versions 16, 18, and 20 of Ubuntu.

Equally, [https://www.mono-project.com/download/stable/#download-lin-debian], gives details for debian, and [https://www.mono-project.com/download/stable/#download-lin-fedora] for Fedora.

Note use of plural in section name above, the following sub-sections will install MX and the other packages already mentioned, all by Developer Mark Crossley.

==Where to install all packages?==

For simplicity on this page EXISTING PATH is used to represent any location in the Linux file structure where you decide to install Cumulus:

*Many with a Raspberry Pi add an external drive to reduce wear on the internal micro-SD card, and so if they have to reload the kernal  (sometimes called “operating system”), they don’t lose their Cumulus packages and data.

All the Cumulus packages, should be put into a folder called “CumulusMX” (note where capital letters must be used).

You can create that folder as you unzip a MX release, or you can type <code>sudo mkdir EXISTING PATH/CumulusMX</code> first (note that EXISTING PATH is explained above and always starts with a slash “/”.

==Alternative download link for older MX releases==

For completeness, you may have discovered (from posts in the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 Cumulus Support Forum] that the current release of MX has a bug that affects an aspect of MX that you intend to use. Remember, it is impossible for the developer to check all the ways in which versatile MX can be used (different weather station types, different computer types, plus a whole host of features, and different external upload sites, that are only used by a sub-set of people).

In such a case, download any earlier build, without the bug, from [https://github.com/cumulusmx/CumulusMX/releases CumulusMX/releases].

Each release is presented as a zip.

The download and unzip procedure is exactly same on your Linux computer, as it would be on a Windows PC.  So if you have two devices available, you can download on either device, and if it is not the computer you want to install on, you can use a file transfer package to move the files between devices, or use a drive (or even a memory card) with a partition formatted so that you can read it on both devices.  Windows and Linux partitions are formatted in different ways, and it is likely that Linux can read a Microsoft partition, but unlikely that Windows can read a Linux partition.

When your browser saves the zip it might be into a folder called “downloads” on your computer, or you amy be able to save into another folder that you prefer (perhaps on a different partition).  Your browser might even remember the folder you used before for files of type zip.

When the download has completed, whatever computer type this is on, mouse click (it might need a right click or a double click depending on settings) on the download file and it should unzip (it may create a folder whose name is taken from the zip file name in the same folder by default, or it may ask you where you want to unzip to).  If you are unable to use a mouse, there should be a keyboard code to use. If you are using a file manager, with a graphical interface, there may be a different way to select the file and unzip it.

==Where to install MX==

As mentioned earlier, you can choose where you install your three Cumulus downloads.  It is important to minimise the length of the path name, because this has to be passed between various different software languages (and longer paths risk truncation). Here I use “EXISTING PATH” (which will start with a slash “/”, but not end with a slash) to represent whatever path you have selected.

It was also mentioned before that you can create the folder to hold the packages in advance using <code>sudo mkdir EXISTING PATH/CumulusMX</code>.

==Upgrading a Cumulus package==

Upgrading to a new MX release is explained [[Updating_MX_to_new_version|here]], but basically follow instructions above, and install over your existing files.  The alternative is to install in a new folder (or rename the old one), and copy across files not in the release.

==Folders to copy across from previous Cumulus installations==

'''If you have used Cumulus before''', you will be seeking to keep your existing (data and extremes in) log files. This means you must transfer the whole [[Data folder|/data]] sub-folder.

If you are using the optional “NOAA report” functionality, you must also transfer the whole [[Reports folder|Reports]] sub-folder.

===Line terminators in these files===

 

If you are moving from Microsoft Windows to Linux, you need to be aware that Microsoft ends each line with two characters (Carriage Return and Line Feed) while Unix/Linux ends each line with a single character (Line Feed).

 

The kernel in your Linux computer might be able to detect that it is getting Microsoft files, and discard the extra end of line character.

 

However, if you are reading that file in a script, it might not detect the end of line encoding and so if that script uses “LF” to terminate the line, when the script is reading the final field of each line has an unwanted “CR” in it. Equally, if the script uses “CR” as line terminator, then the “LF” is added to the first field of the next line.

 

In each case, any checking for numerical input might fail, and any attempt to check/extract characters from these fields might fail because their position relative to start/end is changed.

 

===Changing line terminators===

Many editors designed for programmers (they might be described as providing a programming development environment) can change the line ending of every line (either while file is being edited or when file is saved).  

‘’’Geany’’’ is a programming development editor provided on some Linux systems including Raspberry Pi, that can do this.

Both have capabilities to make such changes on either the single file that has focus or all loaded files.

==Files to copy across from a previous MX installation==

This is an optional file based on [[samplestrings.ini]].  As the latter can change dramatically sometimes when a new release comes out, your existing “strings.ini” might need to be modified.

There is no automatic way to check your “strings.ini” file, if MX does not understand any parameter in this file, it ignores it.

Therefore, you need to manually check each parameter you have in your “strings.ini” file against parameters in “samplestring.ini” of your new install.

Some people prefer to omit their existing “Cumulus.ini” file from their new install. Instead, they work through all the settings manually, so that MX can create a fresh file, with them having confidence every setting reflects their preference.

If you previously used an older release of Cumulus, but in this new installation will be using the latest release, you may want to read up on all the changes.

The documentation for the latest release can be found at [[Cumulus.ini]], note that this does not explain the changes from 3.9.7 to 3.10.1, but it does include advice on (while MX is running) it can automatically update the file. This automatic approach is easy, but you might want to work through settings to change preferences for any new parameters.

#directly with terminal window left open.

Beta builds in MX version 3.0.0 had an optional parameter <code>-wsport nnnn</code> that determined which port (represented by a 4 digit number ''nnnn'') was used for '''WebSockets'''. That parameter is now deprecated as WebSockets in all builds since 3045 uses the same port as the rest of the [[MX_Administrative_Interface#The_API_interface|Admin Interface]]. The remaining parameters that are still available are described in subsequent sub-sections.

=== Parameter for adding debugging ===

MX has a default level of logging that stores in  the [[MXdiags_folder]] folder a log file that shows some of the interaction with the weather station and some of the output actions done as MX runs. A new log is started each time MX is restarted.

If there is a problem, then there is a great benefit in actually increasing the level of detail in these logs; and that is done either within the settings ('''options''' section of ''station settings'') in admin interface  while MX is running, or by adding 1 or 2 parameters when you start MX.  Obviously this log file continues to grow, the longer MX is left running, and if debugging is switched on the file will grow in size must faster. Consequently, the default is not to add the extra debugging information and the settings can be used to switch it off again if you do have it switched on. Whether you start it with a parameter or enable it within settings, stopping MX will end the extra debugging, and on restart it will default back to no debugging unless turned on again with parameter or setting.

You can also add  '''CumulusMX.exe -debug''' (to have full debugging of actions by MX turned on as MX starts), and/or '''CumulusMX.exe -Logging=1''' (for the station to MX transfers to have increased debugging logging).  

<pre>sudo mono CumulusMX.exe -debug -Logging=1</pre>

Since this parameter is applied when you start MX, it applies while MX continues to run. Obviously, it must be applied every time you start MX if you want this increased level of logging to continue every time you restart MX.

The comments in the MX source suggests -debug turns on both debug and data logging (see [[MX_Administrative_Interface#Options|Station_Settings#Options]] in admin interface settings), but I believe that is wrong as per example above, there are 2 separate parameters.

=== Parameter for changing Locale ===

On Linux and (in particular) OS X, Cumulus MX may not be given the correct locale to use, and you may get the default US locale even if that is not your locale. It will output the local it is using when it starts; if it is not correct, close it down and start it again, this time specifying your locale on the command line, using the -lang parameter . For example, in the UK, on a non-Windows device type:

Other local examples:  '''CumulusMX.exe Current culture: English (United States)''', '''CumulusMX.exe -lang de-DE''',  '''CumulusMX.exe -lang el-GR''' (this is one of the locales that reads numbers with '''integer,decimal''' format), '''CumulusMX.exe -lang nl-NL'''.

If you are not sure what value you need to supply for the -lang parameter, there is a list here - http://msdn.microsoft.com/en-gb/library/ee825488%28v=cs.20%29.aspx. You need to supply the code in the first column ("Language Culture Name") in that list.

Note that this does not affect the language used by Cumulus MX (although it may in the future), it affects the decimal separator and the list separator.

Note that you ''may'' need to supply your administrator password after typing the 'sudo ...' command line. The system will prompt you for this if it is needed.

=== Parameter for running as service ===

Use of this is explained next. The parameter syntax is <code>sudo mono CumulusMX.exe -service</code>

=== Setting up as a service ===

If you have installed MX release 3.8.0 or later, you can set up MX to run as a service.

#Find the '''EXISTING PATH/CumulusMX/MXutils/linux/''' sub-folder, that might be in home directory and therefore found using "~/CumulusMX/MXutils/linux" as explained elsewhere on this page

# As described later there is a choice of editors, but you can use <code>sudo nano cumulusmx.service</code> to edit the service configuration file

# Look for '''ExecStart=/usr/bin/mono-service -d:'''

#* Replace the path that follows the above text with the path to your CumulusMX.exe, add the '''-service''' and optionally add any other parameter (e.g. '''-debug''', -locale, -port) as described in sub-sections above.

:<code>sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx.service /etc/systemd/system/</code>

If you upgrade to a new release, the file in EXISTING PATH will be overwritten, but the critical file in "/etc/sytemd/system" will not be affected.

To manually start MX as a service, (such as after any MX upgrade to a new release, or when you want to first run it), simply type <code>sudo systemctl start cumulusmx</code>

Use <code>systemctl status cumulusmx.service</code> in a terminal session to see status of Cumulus service

=== Running with a terminal session left open ===

Whichever operating system you are using, to run MX requires an instruction that changes to the directory (EXISTING PATH) where it is installed (the instruction below is assuming it is in the standard Pi user home directory, the change directory command will be different if you have installed it elsewhere), and then starting the executable (using mono in the instruction below that applies to any non-Windows operating system). You may wish to add [[Cumulus_MX#Optional_parameters_to_add_to_the_instruction_to_run_the_MX_engine|Optional_parameters]].

The simplest instruction to run Cumulus MX  is <code>cd EXISTING PATH/CumulusMX && sudo mono CumulusMX.exe</code>.  Just in case it is not obvious .... if you start MX using this command in a terminal window on your Pi, you must leave that session running, then MX will continue to run.  

=That is all folks=

If you have read up to hear, you now know the basics for using MX on Linux.