5,838
edits
m (→Downloading and Unzipping MX Distribution: split) |
m (→Add the Mono repository for a Raspberry Pi: corrected link display) |
||
(39 intermediate revisions by 3 users not shown) | |||
{{Template:WorkInProgressBanner}}{{Template:Version badge Mx}}
If you are running MX on on any computer running the Microsoft Windows Operating System, then you should be reading the [[MX on Windows OS]] page instead.
{{TOCright}}
=Page Content=
This page:
* describes the options available for installing MX, and the other Cumulus packages
* describes the pre-requisite '''MONO''' software needed to run the various Cumulus executables, (for Raspberry Pi only), how to add the Mono repository to your system, and how to upgrade MONO
* explains the few key Linux commands it uses
* describes the administrative interface and instructions for configuring MX
* it tries to be useful to anyone who has never used MX, and anyone who knows Cumulus, but has not run MX on Linux before
* describes the various options available to run MX
* describes the optional parameters you can add when starting MX
* describes the other executables
There are various related pages to get more information:
*Go to [[:Category:Terminology]] for links to pages that explain terminology used by Cumulus (some of these need updating for MX)
*Go to [[:Category:Cumulus MX]] for links to all pages in this Cumulus Wiki that relate specifically to MX
*[[MX Administrative Interface|Admin interface]] provides information on configuration and web pages for viewing your weather data locally
*Go to [[:Category:Cumulus Files]] for links to all pages describing the sub-folders and files used by MX
*If you encounter a problem when running MX, see [[What to do when I have a problem with MX]]
*The [[Cumulus MX FAQ]] page was created with snippets from the forum, but nobody has yet sorted this out into a useful page or updated it for recent releases
*If you were using the original (now legacy) Cumulus software, please read [[Migrating_from_Cumulus_1_to_MX]], although that is mostly directed at those using MX on the same Windows PC as they used for Cumulus 1, and was written for an old MX release, it will help you understand configuration differences.
*If you want to use a script language, you might want to read [[PHP|PHP Hypertext Pre-processor and JavaScript]] page
=Using MX on UNIX-derived Operating Systems=
MX runs on any UNIX-derived operating systems (OS):
* including those found on Apple Mac computers,
* and those found on a multitude of devices running Linux.
UNIX is a long established operating system, and both UNIX and its derivatives have good long term compatibility. This means that commands are generally easy to learn and use. Most devices also have a graphical user interface that can do the more straightforward tasks without needing to know all the commands.
==Device Coverage==
Linux is available based on a multitude of different kernels (the building block for the operating system), on a multitude of devices.
It is hoped that contributions to this page will be made by Cumulus users with a range of such devices. The page has been originated by a contributor using the Raspberry Pi Operating System (this is based on Debian, one of the Linux kernels). Where appropriate, this page gives instructions specific to a [[Raspberry Pi computer page|Raspberry Pi computer]].
:Until somebody creates a separate page for Apple Mac computers (that would be a good idea), this page is the closest.
==What do people think about MX on Linux?==
Contributions to the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 Cumulus Support Forum] suggest that:
*Use on a Raspberry Pi (RPi) computer is very popular
*In general, people find installing, and running, MX on Linux is easy
*The few people who do have difficulties are those who have good knowledge of Microsoft systems and feel scared to swap to something different.
Microsoft has had a deliberate policy of being different to traditional computers mostly based on UNIX, and the [[MX_on_Windows_OS|Running Cumulus MX on Microsoft Windows]] page covers those aspects of MX that are specific to Personal Computers running Microsoft's Windows Operating System.
In the Cumulus support forum, there are many posts from people who are struggling with MX on PCs, as it seems people often find “installing”, and using, MX is more difficult when using Microsoft Windows.
=For those using Raspberry Pi computers=
You have two choices:
CHOICE ONE: ‘’’Create a micro-SD card that has everything on it to load a kernel onto your computer and run MX’’’
The developer has created [[Software#Raspberry_Pi_Image|an image you can download]].
That image contains:
#Raspberry Pi Lite Operating System as kernel (no graphical user interface, designed for a RPi without keyboard or monitor)
#Mono-complete package
#Cumulus MX package
* If you are new to MX, after booting from image, you will need to use the [[MX_Administrative_Interface#Station_Settings|admin interface]] to define station type, your choice of units, and some other settings, before MX can start recording data from the connected weather station.
* If you are migrating from another computer, after booting from image, you need to add (using an external memory stick or file transfer from your other device to the RPi), the following:
**and any (optional) files from old Reports sub-folder.
If you want to pursue that approach, please read [[Raspberry_Pi_Image]] page, instead of continuing to read this page. Obviously, you can return to this page if you want to learn more.
CHOICE TWO: ‘’’Load the software packages individually’’’
Please read on, this page will tell you all you need to know.
= Do you have a Operating System? =
New hardware might come preloaded with an operating system, or might allow you to choose which operating system to install on it.
Please see [[Raspberry Pi computer page]] if you want guidance on choosing which model to buy and how to install an operating system on those computers, so you are ready to install MX.
=Do you know how to install packages?=
If you already know how to install packages, your computer has the pre-installed image, or you have installed MX before, you will want to skip the sub-sections that follow on here, and continue reading at [[#Cumulus packages]].
Otherwise, here is a short technical digression to explain the commands.
==The various components to commands for installation==
Linux computers have a “source list” which references the repositories from which software packages can be installed.
If a particular package can not be found in repositories already in the source list, then another repository can be added to the source list.
===sudo===
By default, a Linux user will log in as a default user with limited rights. For example, on the Raspberry Pi Operating System, there is a single default user, initially called "Pi", with their home folder that can be referenced as "~".
The initial "sudo" part of many commands gives us super-user (root) rights when executing the instruction that follows.
If we are working away from the folder(s) owned by the default user, we can consider whether we want to change the ownership of these parts of the file system (how to do this for a MX installation is explained later) so that we can access the files with our default user without using "sudo" for file/folder commands.
In some limited cases, it might make sense to just change "read" rights,so that the default user can read a file/folder, but other actions are not possible without using "sudo" prefix. This limits our ability to accidentally delete files that other processes need.
===apt===
The second part of our installation commands is “apt” meaning “Advance Package Tool”. In simple terms, it runs the “package manager” used in Linux.
If you are looking online for a tutorial on how to install packages, you might see “apt-get” or "apt-cache" used in examples they quote. It should be safe to replace these older package managers with "apt". I have included a little technical explanation at the end of this page if this really worries you.
===install===
The third part of our installation instruction is “install”, which tells our package manager what we are trying to do.
For the record only, here is full list of what can follow “apt”:
{| class="wikitable" border="1"
|-
!style="width:30px" | Instruction following “apt”
!style="width:300px" | Description
|-
|install
|To install a package and its dependencies
|-
|update
| The "source list" [[#The various components to commands for installation|mentioned earlier]] that references the repositories from which software packages can be installed, needs to be updated periodically so it reflects any changes within those repositories. The instruction "update" is included after "apt" to make sure your computer has up to date information about repositories installed, and to report if these contain packages that can be upgraded
|-
|upgrade
| Once your "source list" is up to date, the instruction "upgrade" can be included after "apt" to download any newer versions now available in repositories, and to replace those packages that are already installed on your computer with those newer versions
|-
|autoremove
| The instruction "autoremove" can be included after "apt" to check all components in the packages you have installed onto your computer, and remove any components that are not needed by the dependencies of the packages you are using. A download for software frequently includes some components specifically for their software to work with particular other optional packages.
When we install mono-complete later, the other packages we install do not need every component that has installed, and “autoremove” can be used to tidy up when all our installations are finished.
|-
|remove
|If you want to remove just one component manually, after "apt", use “remove” followed by the name of component you no longer want
|-
|purge
| To remove any installed package and delete all related configuration files, after "apt" type "purge" then the name of the package we no longer want
|-
|search
|To search in repositories in source list for a package you specify after “search”
|-
|show
|To show any information available about a package that you name after the “show”
|}
=== Flags ===
The basic syntax is either one or two hyphens, followed by one or two letters (each letter has to be a specific case). Various examples will be seen on this page, but here just one is explained here.
If we selected "install" or "upgrade" to follow "apt", we can add a “-y” flag to signify that we want the install to continue. Without this flag, the package manager will ask periodically if we want it to continue, and we have to then respond with a “y” each time. For example, when we ask to install a package, "apt" will do a search, it will list what components it has found, and output how big their demands are on storage, without "-y" flag, it will then ask (at least once) if it is okay to continue to installing.
=== Name ===
The final part of the package command is the name of the package or component that we want to install/upgrade/purge/show/remove or search for.
=Install starts here=
If your computer has the pre-installed image, or you have installed MX before, you can skip the sub-sections that follow on here and continue reading at [[#Cumulus packages]].
== Preparing for an install==
Before we do an install of a new package, we typically use this series of commands to ensure our computer is in the best state to work out dependencies of what we are about to install:
<code>
sudo apt update
sudo apt -y upgrade
sudo apt autoremove
</code>
Each of those can be understood from information in previous section.
If you are installing onto a Pi zero, or similar slow computer, please ensure the size of the swapfile is as big as possible, as the mono-complete we will install is large. In linux, we type <code>free -m</code> to see our RAM size and our swapfile size.
To change swapfile size on the Raspberry Pi, you need to edit a file <code>sudo nano /etc/dphys-swapfile</code>. Move the cursor down line by line until it reaches '''CONF_SWAPSIZE=100'''. That is showing that the swapfile is only 99 mb by default in the Raspberry Pi Operating System. Now move the cursor to the 100, and change it to "512" which is enough for mono even on a Pi zero. Next, stop, and restart the relevant service using <code>sudo /etc/init.d/dphys-swapfile stop && sudo /etc/init.d/dphys-swapfile start</code>. That should complete quickly, and we can type <code>free -m</code> again to see the 99 we saw previously has been replaced by 511.
=== USB HID ===
There is one more prerequisite package for MX with some weather station types.
The cross-platform [https://www.nuget.org/packages/HidSharp/ Universal Serial Bus (USB) Human Interface Device (HID) library] used by MX to connect to weather stations that appear as a HID connecting via USB (like Fine Offset and USB connected Oregon Scientific models) calls a package file called ''libudev.so.1'', so you may need to [[#install|search]] your Linux computer for this file, and [[#install]] it if it is missing. Alternatively, you might just need a symbolic link where MX looks to where the file is.
To check your USB devices, type <code>sudo lsusb -t</code).
To check HID, download this [https://cumulus.hosiene.co.uk/download/file.php?id=11414 USB HID test package] and run it.
Optionally read about [https://cumulus.hosiene.co.uk/viewtopic.php?f=39&t=14310&p=111593&hilit=libudev.so.1#p111593 Oregon Scientific] issues.
To read about [https://cumulus.hosiene.co.uk/viewtopic.php?p=107913#p107913 Fine Offset issues and the (file may need to be linked to where MX wants it) file link instructions]
==Changing the Source List==
The "source list" [[#The various components to commands for installation|mentioned earlier]] may not contain all the repositories we need for our installations.
Consequently, type <code>sudo apt search mono-complete</code>, to find out whether the mono package is available from one of the repositories already in our source list, and if the version available is compatible with MX (Release announcements for MX should specify which versions of Mono will work). Each of the parts of that command was explained earlier.
If mono-complete is not available (or only available in an older version incompatible with MX), then we have to add a new repository, and the one to add depends on which Linux kernel is used by our Operating System.
There are a number of sub-sections below, please check which applies to you, and only read that one.
===Add the Mono repository for a Raspberry Pi===
The two Mono repositories listed here are specific to the 2017 and 2019 releases (respectively) of the operating system for a Raspberry Pi computer. These are taken from [https://www.mono-project.com/download/stable/#download-lin-raspbian download-lin-raspbian].
# the first line (in each case) installs a certificate
# the echo line defines a repository to add to the sources list.
{| class="wikitable" border="0"
|-
!style="width:450px" | Raspberry Operating System 9 (stretch)
!style="width:450px" | Raspberry Operating System 10 (buster)
|-
| sudo apt install apt-transport-https dirmngr gnupg ca-certificates<
| sudo apt install apt-transport-https dirmngr gnupg ca-certificates
|-
| <code>sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF</code>
| <code>sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF</code>
|-
| <nowiki>echo "deb https://download.mono-project.com/repo/debian stable-raspbianstretch main"</nowiki>
| <nowiki>echo "deb https://download.mono-project.com/repo/debian stable-raspbianbuster main"</nowiki>
|-
| sudo tee /etc/apt/sources.list.d/mono-official-stable.list
| sudo tee /etc/apt/sources.list.d/mono-official-stable.list
|}
===Add the Mono repository to Ubuntu, Debian, Fedora===
At time of writing, [https://www.mono-project.com/download/stable/#download-lin-ubuntu download-lin-ubuntu], shows the instructions for versions 16, 18, and 20 of Ubuntu.
Equally, [https://www.mono-project.com/download/stable/#download-lin-debian download-lin-debian], gives details for debian, and [https://www.mono-project.com/download/stable/#download-lin-fedora] for Fedora.
Others can be found by choosing other tabs on any of those links.
== Installing Mono instruction ==
Now we have the certificate needed, and the repository for mono-complete is added to our source list, we can do the actual install:
: <code>sudo apt install -y mono-complete</code>.
The “sudo”, “apt”, “install”, and “-y” have already been explained.
The "mono-complete" is the package we want.
:Please note that if you just specify “mono”, you will get ‘’’mono-devel’’’ (the developer edition) that does not include all the components required by each of the Cumulus executables.
Please note that a particular MX build might specify it needs a particular version of Mono. Hence, although normally you can upgrade a cumulus package without upgrading Mono, sometimes you will need to install Mono again.
The latest release of Mono can always be downloaded from [https://www.mono-project.com/download/stable/#download-lin], follow step 1 there, but in step 2 replace ‘’’mono-devel‘’’ by ‘’’mono-complete’’’
=Cumulus packages=
Note use of plural in section name above, the following sub-sections will install various packages produced by Developer Mark Crossley. If you are using the [[Software#Cumulus_MX|pre-built disc image]], then (unless the MX release version included in your image is an old one) you should skip the instructions for "CumulusMX".
==Handling zip files==
Each release is presented as a zip.
The download and unzip procedure is exactly same on your Linux computer, and on a Windows PC. So if you have two devices available, you can download on either device, and if it is not the computer you want to install on, you can use a file transfer package to move the files between devices, or use a drive (or even a memory card) with a partition formatted so that you can read it on both devices. Windows and Linux partitions are formatted in different ways. While it is likely that Linux can read a Microsoft formatted partition, Microsoft Windows can never read a Linux formatted partition.
When your browser saves the zip it might be into a folder called “downloads” on your computer, or you may be able to save into another folder that you prefer (perhaps on a different partition). Your browser might even remember the folder you used before for files of type zip.
When the download has completed, whatever computer type this is on, mouse click (it might need a right click or a double click depending on settings) on the download file and it should unzip (it may create a folder whose name is taken from the zip file name in the same folder by default, or it may ask you where you want to unzip to). If you are unable to use a mouse, there should be a keyboard code to use. If you are using a file manager, with a graphical interface, there may be a different way to select the file and unzip it.
==Where to install all packages?==
For simplicity on this page EXISTING PATH (the contents of this will start with a slash “/”, but not end with a slash) is used to represent any location in the Linux file structure where you decide to install all the Cumulus packages.
The phrase “EXISTING PATH” is used, because it is most likely you want to create the sub-folder called “/CumulusMX” (note where capital letters must be used) in a part of the Linux file structure that already exists.
It is important to minimise the length of the path name, because this path name has to be passed between various different software languages (and longer paths risk truncation).
* You can create sub-folder called “/CumulusMX” as you unzip a MX release, or you can type <code>sudo mkdir EXISTING PATH/CumulusMX</code> first (note that EXISTING PATH is explained above and always starts with a slash “/”).
* By using the phrase EXISTING PATH this advice avoids telling you to install Cumulus where you do not want it:
*# Many people with a Raspberry Pi, and a little technical understanding, add an external drive to reduce wear on the internal micro-SD card, and keep their Cumulus files away from the drive that holds the operating system.
*#* This page is not going to get technical by telling you how to create, or mount, Linux partitions on your external drive. If your drive was bought from a Raspberry Pi reseller, they might help you.
*# Other people using a Raspberry Pi without that technical expertise, might use ‘’’/home/pi’’’ for EXISTING PATH as that is the default folder for the default user (Pi) and can be referenced as "~" in file path instructions they issue (although Cumulus will not understand that shorthand)
*#* Within that ‘’’/home/pi’’’ folder, the default user has full permissions automatically.
*# The developer suggests you use ‘’’/opt’’’ for EXISTING PATH (which should be available on any Linux computer).
*#* By default, the code Mark provides for installing Cumulus as a service, will run that service as a root user, and the root user has full permissions in /opt (and everywhere else)
*#* If you do choose a EXISTING PATH outside your home folder, you might want to change the ownership of the "CumulusMX" sub-folder, to the default user (Pi), if so, type <code>sudo chown -R pi: EXISTING PATH/CumulusMX</code>. The advantages of that command is you no longer need "sudo" to access the files (however, if you are running MX as a service, you also need to edit the user in the script provided to create the service, so MX does not create files with root ownership - this will be covered later)
==Packages to install==
<big>We shall install the Cumulus software listed on [[Software]] page</big>:
# '''CumulusMX’’’:
#* '''CumulusMX.exe’’’ is written in C#
#* Download '''CumulusMX zip file’’’ from [[Software#Latest_build_distribution_download]]
# [[Software#Create_Missing|'''Create Missing''']]:
#* '''CreateMissing.exe''' is another C# package
#* Using '''CreateMissing.exe''' is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki (it will populate missing fields in [[standard log files]] and/or missing lines in [[dayfile.txt]]),
#* Simple Instructions for using this executable are on the github page where they are found, you can find the link for that at [[Software#Create_Missing]] in this Wiki.
# '''ExportToMySQL'''
#* '''ExportToMySQL.exe''' is also written in C#
#* Download '''ExportToMySQL.exe''' package from [[Software#ExportToMySQL]]
#* '''ExportToMySQL.exe''' is not (at the time this was written) documented in this Wiki although [[MX_Administrative_Interface#MySQL_settings]] does describe a similar utility (written by Steve Loft) that was actually included in early MX release downloads.
As at 9 March 2020, there is another utility, '''CreateRecord''', initialised in the Github areas managed by the developer where Cumulus is worked on. This will, if my understanding is correct, read [[dayfile.txt]] and use that to update the various [[:Category:Ini Files|extreme record files]]. However, at the time of writing this, it is nothing more than a concept that needs to be coded, and (as far as I know) there has been no progress on that utility for at least 4 months.
===Alternative download link for older MX releases===
Skip this sub-section if either you have installed the "pre-built disc image", or the current MX release is stable (it has been available for a while and nobody has reported any bugs).
Check if posts in the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 Cumulus Support Forum] tell you that the current release of MX has one or more bug(s) that affects one or more aspect(s) of MX that you intend to use.
Remember, it is impossible for the developer to check all the ways in which versatile MX can be used:
* Different weather station types (the developer only has a Davis),
* Different computer types (development is mostly on Microsoft Windows),
* Plus a whole host of optional features, and different external upload sites, (typically each of these optional features are only used by a sub-set of Cumulus users).
Anyway, '''you can download any earlier build, without the bug''', from [https://github.com/cumulusmx/CumulusMX/releases CumulusMX/releases].
You need to ensure that you use the right version of "CreateMissing.exe" and "ExportToMySQL.exe" utilities for the MX release you are running, so if you are using an old MX release, you will need to go directly to the [https://github.com/cumulusmx Cumulus MX github] page, and navigate to the utility of interest, to download an older version of these utilities that matches your older MX.
==Upgrading a Cumulus package==
Upgrading to a new MX release is explained [[Updating_MX_to_new_version|here]], but basically follow instructions above, and install over your existing files. The alternative is to install in a new folder (or rename the old one), and copy across files not in the release from old location to new location.
== Report and data files to copy across from any previous Cumulus location ==
All the (optional) files in the [[Reports folder]] can be copied across from a previous installation. If your previous installation was not on Linux, see [[#Line terminators in .txt files]]
All the files in the [[data folder]] can also be copied across. If your previous installation was not on Linux, see [[#Line terminators in .txt files]]
If your previous Cumulus installation was version 1.9.4, or earlier, then you need to do a lot of reading:
* [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]]
* [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files
* [[Migrating from Cumulus 1 to MX]] gives some advice about differences in settings, but be aware that the way MX handles settings varies by release, and information on the linked page may be out of date
==Configuration Files to copy across from any previous Cumulus installation==
There are two configuration files that are not included in any MX release:
*[[strings.ini]] (note all lower case) – optional file to customise output
*[[Cumulus.ini]] (note initial capital, then lower case) – main configuration file
If your old installation was for a relatively recent release, then just copy these files to new installation and optionally skip the next 2 sub-sections.
If you are upgrading from an older release, please read the next 2 sub-sections for advice.
===strings.ini===
'''This is an optional file'''. Its [[strings.ini|purpose]] is to allow customisation of some of the outputs from Cumulus. You might want to use customisation to abbreviate (or extend) some outputs, or to change those outputs into another language.
You create a “strings.ini” file by selecting some of the parameters from the [[samplestrings.ini]] file that is included in each MX release, and modifying the value for the listed attributes as you type them (under the same group titles - these are enclosed in [ ] as before).
The sections that appear in '''samplestrings.ini''', and the parameters that appear within a section, changed drastically between Cumulus 1.9.4 and MX. So be cautious if you try to use a "strings.ini" file originally created by the legacy software, check whether the parameters you used before are still available in the latest "samplestrings.ini".
The content of "samplestrings.ini" is changing as MX is developed. Therefore, your existing “strings.ini” might need to be modified. There is no automatic way to check your “strings.ini” file, if MX does not understand any parameter in this file, it ignores it. Instead, you need to manually check each parameter you have in your “strings.ini” file to see if that parameter is in “samplestring.ini” included in the release you have installed. You may also find new parameters in “samplestring.ini” that you wish to add to your “strings.ini” file to tailor new functionality to your preferences.
===Cumulus.ini===
If your old release was 3.8.0 or later, then copy the [[Cumulus.ini]] file used in that old installation into your new one. The file is growing with lots of new parameters, and the tables on the page I have just linked indicated when parameters were introduced, and where those settings may be changed to make it simple for you to ensure all settings are right for your new release.
If your old release was older that 3.8.0, then you will see [[:Category:Configuration Files|the Configuration Files page]] shows different links for documentation on older "cumulus.ini" files. Because of those differences, many people prefer to rename their existing “Cumulus.ini” file when upgrading from such an old release, or making an install on a new device. To create a new file, they work through all the settings manually, so that MX can create a fresh file, with them having confidence every setting reflects their preference. However, you do need to be aware that such older releases included a lot of "read-only" parameters that had to be set manually in the file, you may find some of those parameters are still needed by you, so you will want to edit the new file, with MX stopped, and copy any of those old "read-only" parameters that might not yet be settings you configure within the MX interface (although the number of these is diminishing with each new release).
If you previously used an older release of Cumulus, but in this new installation will be using the latest release (latest is what is normally best, unless it has bugs), you may want to read up on all the changes between your old release and the current release, not just changes that affect the configuration file.
==Moving from Microsoft Windows to Linux==
===Line terminators in .txt files===
If you are a novice to computers, skip this sub-section and the next, go directly to [[#File Names & Paths]].
You need to have some technical understanding to do an action that might encounter the issues discussed here. For normal MX usage, line terminators do not matter.
If you are moving from Microsoft Windows to Linux, be aware that Microsoft ends each line with two characters (Carriage Return and Line Feed) while Unix/Linux ends each line with a single character (Line Feed). Cumulus can cope with both approaches for existing files, but will create new files correctly for Linux.
If you run your Linux computer in a headless mode, accessing its files by a remote terminal session, be aware that the line terminator used by the remote computer may be applied to files affected by whatever command you do remotely. Equally, running a Cumulus executable (MX or one of the utilities) may create new files with the wrong end of line terminator. The latest releases have been amended to match existing files, when they create a new file to replace an old file (or because that file is missing/corrupted).
If you try to edit a file outside MX, the tool you choose to use might be able to detect that it is getting Microsoft files, and discard the extra end of line character.
However, if you are reading a comma separated value file (such as MX uses for various files) in a script, your script might not detect the end of line encoding:
* If the script expects the Linus LF, and finds CR LF, then the final field of each CSV line has an invalid character in it, so your script will not be able to understand that field as a numerical value (or time-stamp).
* If the script expects CR only, and finds CR LF, then the first field of the each line presented to the script (except the first line) starts with a “LF” and the script will not recognise it as a date.
* If the script expects CR only, but just finds LF, the script will believe the whole file is just one line, and the fields before and after the LF will be treated as a single field so your script will not be able to understand that field as a numerical value (or time-stamp).
* If the script was written for a Microsoft Windows environment, it will expect CR LF, but might be confused if the end of line is different
===Changing line terminators===
Many editors designed for programmers (they might be described as providing a programming development environment) can change the line ending of every line (either while file is being edited or when file is saved).
‘’’Geany’’’ is a programming development editor provided on some Linux systems including Raspberry Pi, that can do this ('''Document''' menu, -->> '''Select Line Endings''').
Notepad++ is another editor available for multiple operating systems ('''Edit''' menu -->> '''EOL conversion''').
Both have capabilities to make such changes on either the single file that has focus, or all loaded files.
===File Names & Paths===
Another issue that may be encountered when moving from Windows to Linux is the difference in File Names & Paths.
Linux file names are Case Sensitive, and the separator is "/" Vs "\".
This could be applicable if you are using Extra Web files.
Here is an example changing the case and the separators: ''Web\extrapageT.htm'' in Windows, changed to read '''web/extrapageT.htm''' in Linux.
=Running MX on Linux=
This section explores the optional parameters, and then covers 2 ways to run MX:
# as a service and
# interactively with terminal window left open.
== Optional parameters to add to the instruction to run the MX engine ==
This list of parameters covers every parameter, if you are using the latest release skip the historic 3.0.0 sub-sections.
IMPORTANT: The "sudo" prefix shown in code examples for the various parameters can be omitted if the user that is running MX owns the folder "CumulusMX" and all its contents. In the [[#Where to install all packages?|installation notes earlier]] possible locations and ownership issues were mentioned. That earlier section also defined EXISTING PATH, if that is not mentioned in the code examples, it is assumed you have already issued a <code>cd EXISTING PATH</code> to be in the right folder for issuing the command.
Note that you ''may'' need to supply your root password after typing any 'sudo ...' command line if the instruction changes something set by another package (e.g. changing the locale for MX from the locale set by mono). The system will prompt you for this if it is needed.
=== Parameter for changing Port ===
When Cumulus starts, it will display the URL of the user interface. It runs on port 8998 by default; if this is not suitable for some reason you can over-ride it using the '-port' parameter on the command line, e.g. to use port 9999 instead:
<pre>sudo mono CumulusMX.exe -port 9999</pre>
=== Parameter for changing Locale ===
On Linux and (in particular) OS X, Cumulus MX may not be given the correct locale to use, and you may get the default US locale even if that is not your locale. It will output the local it is using when it starts; if it is not correct, close it down and start it again, this time specifying your locale on the command line, using the -lang parameter . For example, in the UK, on a non-Windows device type:
:<code>sudo mono CumulusMX.exe -lang en-GB</code>
Other locale examples: '''CumulusMX.exe Current culture: English (United States)''', '''CumulusMX.exe -lang de-DE''', '''CumulusMX.exe -lang el-GR''' (this is one of the locales that reads numbers with '''integer,decimal''' format), '''CumulusMX.exe -lang nl-NL'''.
If you are not sure what value you need to supply for the -lang parameter, there is a list here - http://msdn.microsoft.com/en-gb/library/ee825488%28v=cs.20%29.aspx. You need to supply the code in the first column ("Language Culture Name") in that list.
Note that this does not affect the language used by Cumulus MX (although it may in the future), it affects the decimal separator and the list separator.
=== Parameter for running as service ===
The parameter syntax is <code>sudo mono CumulusMX.exe -service</code>
You don't use this parameter in a terminal or interactive instruction for running MX. Instead it appears within a file that we use to set up for running MX as a service.
For the latest release, (see [https://cumulus.hosiene.co.uk/viewtopic.php?p=146473#p146473 this later release announcement]:
#There is a task to do just once to configure the service
#Find the '''EXISTING PATH/CumulusMX/MXutils/linux/''' sub-folder, that might be in home directory and therefore found using "~/CumulusMX/MXutils/linux" as explained elsewhere on this page
#* At time of typing this, the sub-folder only contains one file, the one we need to edit
# As described later there is a choice of editors, but you can use <code>sudo nano cumulusmx.service</code> to edit the service configuration file
# Within the provided file you should find a [Service] section:
<pre>[Service]
User=root
Group=root
ExecStart=/usr/bin/mono-service -d:/home/install/CumulusMX CumulusMX.exe -service
Type=forking
ExecStopPost=/bin/rm /tmp/CumulusMX.exe.lock</pre>
There are some edits needed to that section:
# Replace '''User=root''' if you want the service to run as a different user
# Edit the line that begins with '''ExecStart='''
#* The mandatory change is to replace '''/home/install''' by what you have selected for EXISTING PATH
#* An optional change is to add additional parameters after the '''-service''' (select from '''-debug''', -locale, -port) as described in sub-sections above
#save file with a new name '''cumulusmx_mine.service''' (using a new name stops it being overwritten when we upgrade MX
#now copy file to where it is needed to run the service:
:<code>sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx_mine.service /etc/systemd/system/cumulusmx.service</code>
Here is how the file might look after the changes:
<pre>[Unit]
Description=CumulusMX service
Documentation=https://cumuluswiki.org/a/Main_Page
After=network-online.target
[Service]
User=pi
Group=pi
ExecStart=/usr/bin/mono-service -d:/opt/CumulusMX CumulusMX.exe -service -debug
Type=forking
ExecStopPost=/bin/rm /tmp/CumulusMX.exe.lock
[Install]
WantedBy=multi-user.target</pre>
== Setting up as a service in release 3.8.0 ==
Skip this sub-section for latest release.
For historic reasons, note the original instructions in the 3.8.0 [https://cumulus.hosiene.co.uk/viewtopic.php?p=145048#p145048 release announcement]:
# Ensure you are in the folder containing CumulusMX.exe
# Type <code> mono-service -l:/var/run/cmx.pid CumulusMX.exe -service</code>
# (to verify) note this does not allow you to add -port, -debug, -locale parameters
=== Parameter for adding debugging ===
MX has a default level of logging that stores in the [[MXdiags_folder]] folder a log file that shows some of the interaction with the weather station and some of the output actions done as MX runs. A new log is started each time MX is restarted.
If there is a problem, then there is a great benefit in actually increasing the level of detail in these logs; and that is done either within the settings (on recent MX releases this is on '''Program Settings''' page of admin interface - please see [[MXdiags_folder]] page for details) or by adding a parameter:
:<code>sudo mono CumulusMX.exe -debug</code>
=== Parameters only applicable to Version 3.0.0 Beta builds of MX ===
The following two parameters cannot be used with MX since it came out of 3.0.0 beta.
====
Beta builds in MX version 3.0.0 had an optional parameter <code>sudo mono EXISTING PATH/CumulusMX/CumulusMX.exe -wsport nnnn</code> that determined which port (represented by a 4 digit number ''nnnn'') was used for '''Web Sockets'''.
That parameter [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887&p=138815&hilit=sockets#p138815 is now deprecated as Web Sockets in all builds since 3045] use the same port for web sockets as for the HTTP port of the [[MX_Administrative_Interface#The_API_interface|Admin Interface]], see -port parameter described earlier.
==== Debugging of data flow between station and MX====
Note use of this parameter is now deprecated.
Although this is not mentioned in any release announcements, it appears that on all recent MX releases this effect is incorporated into the '''-debug''' parameter described earlier. Perhaps someone could confirm whether this is true.
==Running as a service==
The instructions to set up the ability to run MX as a service [[#Parameter for running as service| were given earlier]]. If you want MX to automatically start when your Linux computer is booted, just type <code>sudo systemctl enable cumulusmx</code> once, and it will be activated on each reboot.
The full set of commands to use with this service are at [[Raspberry_Pi_Image#systemctl_commands|systemctl_commands]],they are not duplicated here, so there is only one place to do any update.
In the [[#Parameter for running as service|earlier instructions]] it was suggested you change the user for the service, if you have done that you can omit the "sudo" in <code>sudo systemctl start cumulusmx</code> which manually restarts the service. Equally, you can omit "sudo" from any command that accesses Cumulus files created by the service.
== Running any MX executable with a terminal session left open ==
This is alternative to the running as service as described above.
The simplest instruction to run Cumulus MX is <code>cd EXISTING PATH/CumulusMX && sudo mono CumulusMX.exe</code>.
* This is two commands issued together, the first changes the working folder, the second actually starts the main executable
** To run "Create Missing" or '''ExportToMySQL''', just edit what appears after "mono" to the correct executable name
* If you have followed advice at [[#Where to install all packages?]], the user you are using will own the "EXISTING PATH/CumulusMX" folder and you can omit the "sudo".
* You can add parameters after "CumulusMX.exe" (select from '''-debug''', -locale, -port) as described in sub-sections above
Just in case it is not obvious .... if you start any executable using this command in a terminal window on your Pi, you must leave that session running, or that executable will stop running.
You can start it off directly on your Pi, and then
*optionally disconnect the keyboard,
*switch off monitor or TV attached to your Pi,
*Just ensure you leave Pi on (with that window minimised) so that terminal session continues running.
Use <code>'''ps -ef | grep -i cumulus | grep -v grep'''</code> to see if Cumulus is running or not.
= Operating a web site with uploads from MX engine =
== The standard web pages ==
=== From release 3.10.1 ===
The web pages are a one-off upload from '''CumulusMX/webfiles'''. The data to be shown on these web pages are uploaded from [[:Category:JSON_Files|.json]] files in the [[web_folder]].
Please read [[New_Default_Web_Site_Information|this page]] for more information about styling options and other details.
=== Until release 3.9.7 ===
The styling and library files were a one-off upload from '''CumulusMX/webfiles'''. These release use [[Cumulus_template_file|template files]], these are [[Customised_templates#What_is_meant_by_.27Cumulus_processes_templates.27|processed by MX to add the variable data]], and this will create web pages that are uploaded to your web site.
Please read [[Customised_templates]] for further information about the various pages originally provided, and how you could customise them to suit you.
=== Comparison with legacy Cumulus 1 web pages ===
* Note that any web files, designed for Cumulus 1, cannot be used with MX, for multiple reasons
** so if moving from Cumulus 1 to MX, delete all your Cumulus 1 files from the "web" and "webfiles" sub-folders, and all files from your web server; then upload files from the new "webfiles" folder.
* The standard gauges are rather like the stand alone [[SteelSeries Gauges|SteelSeries gauges]] you could optionally add to Cumulus 1, but the MX ones are different.
** The new default gauges page does not display a graph when you hover over a gauge (Cumulus 1 generated some images that the stand alone Steel Series gauges page could use)
* The various charts pages for MX rely on MX to generate files with time and value pairs, these are stored in json format, the various web pages use a library package (Highstocks) to draw graphs from those data pairs.
** (The Trends page provided as standard in Cumulus 1 simply displayed images of graphs uploaded to the web server)
== Alternative ways to obtain web pages ==
You can choose to use some of the alternative web pages available from third parties and described [[:Category:User Contributions|on User Contributions page]].
== Using your own web pages ==
* Of course you can use your own web pages, instead of the standard ones. Assuming they need to include figures that are available as web tags, there are three alternative ways to implement this:
*# MX can process template files with a HTML structure and those web tags in the structure where values are required just as it does with the standard templates, and MX can upload the resulting web pages at either the real-time interval, the standard interval, or after end of day. All of this is covered on the [[Customised_templates|Customised templates]] page in this Wiki.
*# MX can process a file with a string of web tags mirroring the realtime.txt option in MX, and upload the resulting file so your web pages can use JavaScript for a one-off insert of the values or an Ajax routine to update the web page at a fixed interval.
*# Alternatively, you can use template scripts processed locally by MX that don't create web pages, but are uploaded by MX at either the real-time interval, the standard interval, or after end of day. These scripts simply initialise script variables with values obtained from web tags. You then independently have a set of web pages resident only on your web server (they don't exist where you run MX) using a combination of HTML and script content that bring in the script(s) with the variables by the appropriate syntax. All of this is covered on the [[Php_webtags|PHP web tags]] page in this wiki. As it suggests there, you might therefore have several files processed by Cumulus MX at these different intervals, converting the web tags into script variables, and then use AJAX (JavaScript that may use json format to bring in the variables) or PHP (using <tt>'require_once 'filename';</tt> syntax) to put those variables into a web page.
You may find [[PHP|this wiki page]] useful for understanding more about the different script languages.
=That is enough folks=
If you have read up to here, you now know the basics for using MX on Linux.
The remaining sections are more technical and so you can skip them.
=Technical Extra=
PLEASE SKIP ALL SUBSEQUENT SUB-SECTIONS IF YOU WANT TO AVOID TECHNICAL EXPLANATIONS.
== A very quick introduction to Linux ==
This article is not the place to teach you Linux, you can find books and on-line articles for yourself, but I list here enough for you to understand the instructions used elsewhere in this article.
If you have a Raspberry Pi with a monitor attached, you will see a raspberry icon that you can click to get Graphical User Interface access to many features, including shutdown options.
=== su and sudo ===
There is a command <code>su</code> that allows a terminal session to become a super user session with root privileges. If you use that command, without a '''sudo''' command in front, you need to type in the password (we changed earlier) when prompted. if you type <code>sudo su</code>, then you get root privileges without being asked to quote password. All subsequent lines in this terminal session will have a prompt that reminds you that you have root access and do not need to prefix subsequent commands with "sudo".
Normally, all terminal sessions will use the default "pi" user, and for individual commands, you will use a "sudo" prefix each time that command needs administrative rights, as this allows a standard Pi user to do tasks that otherwise only work for the root user.
*'''sudo reboot''' (or "sudo reboot -verbose" for diagnostic output during shutdown and reboot) = makes your Pi close down and then reboot; used when you change settings, and after you install new software, to ensure Pi starts with all applications running using the latest settings and latest already installed software
=== ~ and / ===
The tilde symbol '''~''' denotes the home directory for the current user. Sub-directories within the current user's folder can be identified by '''~/documents''' or similar notation.
If you are using the RPi OS GUI, it provides a file manager that displays folders and files, and if you have a mouse you can click on an object to see what actions are available. The file manager has "Home" and "Root" as bookmarks by default, you can bookmark others. Typically, any new partitions created can also be accessed from bookmarks. Depending on options you select, there may be icons on the GUI desktop to link to particular folders and clicking on these offers various options including opening them in file manager.
In a terminal environment, to see what files and folders are in the current directory, type <
=== folder commands ===
To make a new folder in the current directory, type <code>sudo mkdir folder_name</code>.
To remove a folder, that has no files in it, type in a particular path, type <code>sudo rmdir /path/directory</code>.
To remove a folder that does have files, and/or sub-folders within it, type tt>sudo rm -r /path/directory</code>, but remember the contents are gone for ever, so be absolutely sure you have specified the right folder!
To copy folders/files from one directory to another use something like <code>cp -R --update --preserve /home/pi/CumulusMX/backup/daily /media/pi/data/CumulusMX/archive</code>
Sometimes, you have a folder or file in just one place in the file system, but want to be able to access it at a different place (because something expects it in the second place), the syntax is <code>ln -s /path/elsewhere path/pointer_location</code>.
An example might be '''ln -s /var/lib/phpliteadmin/diary.db ~/CumulusMX/data/diary.db''' (phplite admin can only update databases in one folder /var/lib/phpliteadmin, or in older releases in /usr/share/phpliteadmin; while MX expects the file to be in its data folder but is happy with a logical pointer to another folder).
=== chmod ===
When you are attempting any of the actions listed in this article that involve reading, creating, editing, exeduting, or moving, files; you might see an error message generally because of a lack of write (or execute) permissions on an existing file or folder. Whilst <
====nano====
The full syntax is <
After typing the nano command you need to specify a filename (it might include a path, see earlier sub-section for use of '''/''' and '''~''') and there are examples later in this article, but if you decide to ''host a web site on your Pi'' then you might want to edit its home page with (.html or .php) name like <
After you have made an alteration to the current contents of the file, various options are shown at the bottom. Here are two key ones:
*Another is '''^X''' which means if you press the control key down and type X you get the exit dialogue. If you have not made any edits, or have already saved the file, this just exits the editor. If you have not used control and O to save the file, it asks whether you want to save the edited file (type '''Y'''), typing just the Y key lets save continue (any other key stroke exits without saving), then it shows the current file name, if you press '''Enter''' then that file will be overwritten.
You might find it useful to type <
====Geany====
== removing an unwanted file ==
You can remove a file with various commands, including <
=== external storage ===
Generally, if you attach USB storage (a disc or a stick), Linux OS distributions will detect any existing partitions (yes a technical term) and allow you to read files stored in them. This applies whether the partition is formatted for Linux or for Microsoft Windows.
However, you may have a brand new, unformatted, drive, or you may want to delete, or add partitions, or to format them as Linux partitions (as these make the input/output significantly more efficient).
You can install software that uses a GUI to make this easy, e.g. '''gparted''' [https://gparted.org/ partition editor].
Alternatively, you can use a terminal session, and lots of commands:
#connect your external storage
#type '''su''' to gain administrative access
#enter your RPi password
#type '''fdisk -l''' (this is only available to root user) to see names for all storage your Linux computer can see
#an external drive will be named something like ''/dev/sd''''a''''' although that "a" might be "b" or a subsequent letter in alphabet depending on what has already been assigned
# if "sda" and "sdb" appear, or any others up to "sdz", the last one will relate to the most recently connected storage
#if your drive has partitions, then you will see further entries like '''/dev/sda1''' and ''/dev/sda2''.
#type '''df''' to see whether your drive is currently mounted (being used by computer system)
#if it is mounted, the command to use next is (type this accurately, there is a temptation to type an English word that adds an extra "n"!) <code>umount /dev/sda</code>, obviously replace the "a" by the appropriate letter seen in the earlier command
#if the drive does not have a partition, create one using <code>fdisk /dev/sda</code>, again changing the "a" into whatever letter was seen in response to the first "fdisk" command
#*"fdisk" is a utility, it will wait for further instructions, follow each with pressing "Enter"
#*type '''n''' as instruction to create a new partition
#*type '''p''' to make this the primary partition on this drive
#* type '''1''' to make this the first partition
#*accept default offered for first cylinder
#*accept default offered for last cylinder, if this is only partition, as that ensures the whole disk (apart from partition table) is available for your data
#*for simplicity, this guidance will not cover the possibility of multiple partitions
#*type '''t''' to say you are specifying the way you want this partition to be specified in partition table
#*optionally type '''L''' to see what file system types are available for the partition table
#*to select a "Linux" partition, type '''83'''
#*type '''w''' to create the partition you have now specified for Linux.
#Now we have a partition table and a partition on our drive, we can repeat '''fdisk -l''' to see the entry now added, it might be '''/dev/sda1''', where again the "a" might be a different letter
#To format this partition for Linux, we specify "ext4" as the way to format it using <code>mkfs.ext4 /dev/sda1</code>, again replacing the "a" as required.
#we need to create a folder within
#*"/media" for Linux in general
#*"/media/pi" for Raspberry PI OS
# As we will learn later, the relevant command (in RPi OS) is '''mkdir /media/pi/my_short_name''', where "my_short_name" is selected by you
#To mount our partition, we type <code>mount /dev/sda1 /media/pi/my_short_name</code>, where "sda1" is replaced by "sdb1" or whatever we saw in '''fdisk -l''', and "pi/my_short_name" is replaced by whatever we used in our make directory command.
#To (optionally) get our partition mounted at boot, we can use an editor (see later) to change the boot instructions, by typing <code>nano /etx/fstab</code>
#*In the editor, use the down arrow on your keyboard to move to last line, and then type '''/dev/sda1''' (change the "a" as necessary), then press "tab", then type '''/media/pi/my_short_name''' (change "pi/my_short_name" to whatever we used in our make directory command), then press "tab", then type '''ext4''' (again matching the format type we selected earlier), then press "tab", then type '''defaults''', then press "tab", then type '''1''', then press "tab", then finally type '''2'''
#*save the file (as described later in nano sub-section), hold down control key and press '''o'' letter key. Press "enter" again to confirm same file name.
#* exit nano by holding control key and pressing "x" key.
== Package Manager – a brief technical aside==
Linux operating systems install software by looking in repositories, and checking a register showing dependencies. When you ask Linux to install a particular package using “apt”, it also checks if all dependencies of the selected package are already present, and installs any that are missing.
If you look up on-line how to install any software in Linux, it may use “apt-get”, that is an earlier package manager, in general you can use “apt” instead now.
The full differences between “apt” and “apt-get” depends on your Linux flavour, so this technical aside now splits further discussion by Linux flavour.
===Debian as used by Raspberry Pi===
“Debian Linux” (and its derivatives such as “Raspberry Pi Operating System”) uses “apt” to mean a ‘’’Package Manager’’’ that can install, update, and remove packages from these computer systems.
For Debian Linux, “apt” is directed at the end-user (it has user friendly features like a staus bar showing progress on a long install or long upgrade, and can produce prompts about what it is doing and can give choices about whether to do individual actions).
There is an alternative “apt-get” which is more powerful, but directed at system level users (those who don’t want to be watching progress and possibly responding to prompts).
As “apt-get” is updated less frequently than “apt”, it may be it will not work with new packages. Put another way “apt-get” may never change what it can do, but “apt” may get modified to do more than it did before.
===Ubuntu===
For Ubuntu only “apt-get” was available up to 2014, when “apt” was added. Both work as described above for Debian. Again “apt” is best to use.
===Mint===
Linux Mint has a different variation. Its “apt” calls its “apt-get” and adds extra features. So both effectively do the same, but (as with previous flavours) “apt” is best to use.
It is likely that other Linux variants will also vary how these alternative commands differ.
|
edits