44
edits
(19 intermediate revisions by 3 users not shown) | |||
=Using MX on UNIX-derived Operating Systems=
[[Category:Cumulus MX]]
MX runs on any UNIX-derived operating systems (OS):
* including those found on Apple Mac computers,
=Preparing your computer for installing the Cumulus MX suite=
Please see [[Preparing your Linux computer for MX]] page if you have not installed MX on Linux before.
That page covers:
** [[Preparing your Linux computer for MX#The various components to commands for installation|Commands]]
* [[Preparing your Linux computer for MX#USB HID|Human Interface Devices]]
* [[Preparing your Linux computer for MX#Installing Mono instruction|'''Installing Mono''']]
* [[Preparing your Linux computer for MX#Moving from Microsoft Windows to Linux|Preparing Microsoft Windows files for Linux]]
* [[Preparing your Linux computer for MX#File Names & Paths|Guide to file names and paths]]
Please be advised some of the above is rather technical reading, but ''Mono is required to run the Cumulus packages'' described next. So do ensure that you installed Mono before continuing.
==Technical aside==
Please note this Wiki page talks about "folders" for compatibility with the [[MX on Windows OS]] page, but Linux prefers to call them directories.
Linux has a well defined filesystem, represented as a hierarchic tree starting at the root "/", that is divided into directories (one of which will be "/boot" and hold the kernel), each of those first level directories can be divided into second level directories, this second level is sometimes referenced to as defining the "scope", an indication that each is meant to be used for a specific purpose. The scope can be sub-divided again at lower levels representing "categories" (categories cover items like binary code, documentation, configuration, hardware, source code, runtime and content), and at a lower level still "applications" (i.e. related to specific programs) with further sub-levels for various options within those applications. Many Linux distributions will use logical links so references to a directory at one level in the hierarchy will actually redirect to files in a different directory, this might be because different programs expect to see files in different places or just to enforce ownership and writing rights.
For the purposes of this Wiki, the terminology "operating system" is used for the whole Linux distribution, you will find that Linux technical people prefer to talk about Linux distributions including:
# a "kernel" for the underlying handling of files, network and so on;
# one or more "shell" components for the handling of commands entered in terminal mode, including those that run programs (whether included in distribution or added later);
# an optional graphical user interface for simpler access to commands and programs.
For simplicity the terminology "terminal" is used for how you access the shell, this term refers to seeing the command prompt if your Linux is running without a graphical user interface, or to a window that you can open within the graphical interface where commands can be typed. Depending on your Linux, that window might be called "Terminal", "Konsole", "xterm", "gnome-terminal", "uxterm", or even something else. If you are accessing your Linux computer over a network from a computer running Microsoft Windows, then again you may encounter a number of terms for how to access the shell on your Linux computer, "Command Window", "Windows Powershell", or "Windows Terminal". Equally you may use software that calls it a teletype mode, e.g. PuTTY software.
=Cumulus packages=
Be aware, you may need to adjust the settings within that application for how it handles the file structure. The preferences may determine whether the unzip process preserves the file structure used when the zip was created (i.e. each file remains in any sub-folder) or it ignores the folder structure. For the Cumulus context, it is essential to preserve the folder structure. You may also be asked where you want the files to be extracted to, or the default settings might always use a particular destination (and that might be a '''tmp''' folder).
For example on the Raspberry Pi operating system, there is a package called '''xarchiver''', in its Graphical User Interface (GUI), there is a menu called "Action", and the final option in that menu is "Preferences". There, in "Archive" section, you can select "zip" as the preferred archive format (using a
the GUI presents a screen of options:
* "Extract to:", use the icon to browse to the required location if it has not been set up in preferences
* "Ensure a
* "Files", select "All files", the advice is to overwrite all of any existing files if you are upgrading, but you definitely need all files if this is a new install
* "Options"
It is important to minimise the length of the path name, because this path name has to be passed between various different software languages (and longer paths risk truncation).
# '''CumulusMX''':
#* '''CumulusMX.exe''' is written in C#, that has been developed by Mark Crossley, but still contains some code by Steve Loft
#* Download '''CumulusMX zip file’’’ from the link at [[Software#Latest_build_distribution_download]]
# [[Software#Create_Missing|'''Create Missing''']]:
#* '''CreateMissing.exe''' is also written in C#, created, and developed, by Mark Crossley
As at 9 March 2020, another utility, '''CreateRecord''', has been initialised in the Github areas managed by the developer where Cumulus is worked on, although it appears to be just a concept on github. This will, if my understanding is correct, read [[dayfile.txt]] and use that to update the various [[:Category:Ini Files|extreme record files]]. The developer is still aiming to make this available, but his work on it (on his computer) has been stalled by the level of pressure being applied for bug-fixes and changes to MX itself.
===Alternative download link for older
Because the developer uses Git Hub to manage releases, the older releases remain available.
====Old Cumulus MX packages====
Skip this subsection if either you have installed the "pre-built disc image", or the current MX release is stable (it has been available for a while and nobody has reported any bugs).
* Plus a whole host of optional features, and different external upload sites, (typically each of these optional features are only used by a sub-set of Cumulus users).
Anyway, '''you can download any earlier MX build, without the bug''', from [https://github.com/cumulusmx/CumulusMX/releases CumulusMX/releases].
====Old utilities====
The zips for "CreateMissing.exe" and "ExportToMySQL.exe" utilities do NOT contain the '''''.dll''''' components that they need when they are running.
This means that each version of "CreateMissing.exe" and "ExportToMySQL.exe" utilities is dependent on it being used with a release of Cumulus MX that does have correct '''''.dll''''' components in its release distribution.
That in turn means you can't use the latest version of the utility with older MX releases, nor can you use an old utility version with latest MX release. This is why [[Software#By_Mark_Crossley|utilities downloads]] make clear which MX release is the minimum for them.
The older versions of these "CreateMissing.exe" and "ExportToMySQL.exe" utilities are available by going directly to the [https://github.com/cumulusmx Cumulus MX github] page, and navigating to the utility of interest. However, to use those older versions, you also need to download the corresponding older MX release, because the MX distributions contain the .dll files that the utilities require, they are not in the utilities zip. Because of this complication, novice users are advised not to attempt to use the older utilities, even if the latest version appears to have a bug. Technical users may be able to work out which .dll files are needed and can be safely added back (if they are not left over from when that past MX release was in use). An alternative is to create a new folder with the old release packages (MX and the utility of interest), a copy of the latest Cumulus.ini file, and a copy of all files from /data sub-folder; then afterwards copy back the changed files into original /data folder.
==Upgrading a Cumulus package==
The alternative is to install in a new folder (or rename the old one), and copy across files not in the release from old location to new location, but in that alternative you might forget some files.
==Changing location of Cumulus MX==
If your install, or upgrade, is creating MX in a different place to where you previously ran Cumulus, then you will want to copy files across that are not in the zip extract distribution.
'''MX interface''' --> Setting menu --> '''Station Settings''' --> Click on ''General Settings'' --> Click on '''Advanced Options''' --> Edit ''Records Began Date'' following instructions below that field
==="data"
Please see [[:Category:Files_with_Comma_Separated_Values]], [[:Category:Ini_Files]], and [[Weather_Diary]], for information if you are moving from Cumulus 1 to MX. Otherwise just copy files from any existing folder to your new one.
===
By default MX now creates monthly and annual reports that are in the style used by NOAA in USA. If you have been using this functionality before (and it is optional) then you need to file transfer, or copy, all the files that were in the old [[Reports folder]] into the new folder of that name. Do look at that cross-reference, and read about the encoding default differences between Cumulus 1 and MX.
=MX can cause problems with storage=
MX now assumes by default that you are going to use its [[New Default Web Site Information|Default Web Site]]. That means that by default MX will re-generate temporary files in its [[Web folder|/web sub-folder]] on a frequent time-scale. That number of files writes will considerably shorten the working life-time of the "high capacity micro-SD" card that is the default storage for the Raspberry Pi. It will also considerably shorten the life of any flash memory (e.g. memory card) that you might install MX on.
The expected life of any storage device, and the extent to which its life is shortened depends on the actual device. The external devices that have the longest life (and therefore can cope most easily with multiple read/write actions) are solid state discs (SSD). Also the larger the capacity of the storage device, the more places on the device where files can be stored and the storing algorithm will try to spread the storing evenly across the entire storage area, so wear at any one location is reduced.
All Linux computers will have some random access memory chips (RAM) and it is worthwhile to define part of that RAM as a drive used for temporary files. For a Raspberry Pi computer, a typical approach would be to edit the fstab file, adding the line ''tmpfs /
==web
All the files in this folder come from the download.
In a terminal session, issue commands in the following format for each file (this example relates to Raspberry Pi and uses "/var/tmp" which was defined in the extra line added to fstab earlier):
<code>sudo ln -s /
Notes:
* Full paths are given both for the file that MX is to be redirected to, and after it the path where it expects to create the file
* "CHOSEN PATH" is defined in [[#Where to install all packages?]], but basically it starts with a "/" and defines the path to get to where "CumulusMX" is a sub-folder.
* The text "
=Running MX=
Once you have got all the files sorted out as described above, you need to run MX.
On the first run of MX, unless you have run a recent release before, you need to work through either the [[First_Run_of_MX|'''Config wizard''']] or all the individual settings pages (or both) as accessed from "Settings" menu. It is suggested you run MX interactively (see below) to do this, as you will then need to close MX, and then start it up again.
Information about settings is on other Wiki pages ([[MX Administrative Interface]] and [[Cumulus.ini]]).
The simplest instruction to run Cumulus MX is <code>cd CHOSEN PATH/CumulusMX && sudo mono CumulusMX.exe [optional parameters]</code>.
* This is two commands issued together, the first changes the working folder, the "&&" means that first command has to succeed before the second command is obeyed and actually starts the main executable
*
When you want MX to stop, you must (for Linux) within your terminal session hold down the "Ctrl" button on your keyboard, and press "c". A word of caution here, if you are accessing your Linux computer over a network from another computer, do be careful about using any control sequences, as it is possible that your "Ctrl" "C" sequence will be applied to an application other than Cumulus MX, if that terminal session has started more than one application. The issue is that all running applications use the same terminator, it should be applied to whatever is regarded as the "foreground" application at the moment the control key sequence is used, which is guaranteed to be MX if that terminal session has only been used for running MX, and MX has not launched any external applications. After that you can choose to close the terminal window.
===Interactive advice===
Open a terminal window, then navigate to the folder where you have installed the 3 Cumulus executables:
To run "Create Missing utility" type <code>cd CHOSEN PATH/CumulusMX && sudo mono CreateMissing.exe</code>. It does not take any parameters, so that is all you need to know, although it is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki.
To run [[Software#Export_To_MySQL|Export To My SQL]], you change the name of the executable above and add the necessary parameters, follow that link for more details.
The MX download includes a file that can be used as a starting point for the service definition. Find this file at ''CHOSEN PATH/CumulusMX/MXutils/linux/cumulusmx.service''. You do have to edit this file, and then you have to copy it to a new location, before you start the MX service for the first time.
# Open the file at ''CHOSEN PATH/CumulusMX/MXutils/linux/cumulusmx.service'' using an editor
#* On a Raspberry Pi with a graphical interface, use the file manager to navigate to this file, right click the file named, and select "Geany's Programmers Editor".
#* If you are accessing from another computer, using a terminal session, then "nano" is a suitable editor (explained at link just mentioned).
# Within the provided file you should find a [Service] section:
<pre>[Service]
ExecStart=/usr/bin/mono-service -d:/home/install/CumulusMX CumulusMX.exe -service
Type=forking
ExecStopPost=/bin/rm -f /tmp/CumulusMX.exe.lock</pre>
*Be aware that what quoted above applies from MX 3.16.0 (b.3182, 30 Apr 2022) onwards, earlier releases did not include the "-f" flag in final line quoted above.
:There is more in the file, but for now focus on the line including "ExecStart=/usr/bin/mono-service -d:". Don't change any of the bit I just quoted.
Almost certainly you will need to change "/home/install/CumulusMX" on that line. Replace that with "CHOSEN PATH/CumulusMX", i.e. the full path to the directory that the executables are being stored in.
The final line, with all possible parameters, could read: <code>'ExecStart=/usr/bin/mono-service -d:CHOSEN PATH/CumulusMX CumulusMX.exe -service -debug -port 999 - lang el-GR</code>
* Note the space between the path (just looked at) and the executable file,
* Note the mandatory parameter "-service" that follows a space after the "CumulusMX.exe", you must leave that untouched,
* Note you can remove/keep the rest of the line after the -service i.e. the other parameters (some with their values) -lang, -port, or -debug, (as defined in table earlier), are all optional.
Technical user may do other edits on the file, these will be described later, for now save the changed file under a new name (so it won't be lost when you do a MX upgrade that replaces original file) within your MX installation:
# Open the "File" menu, and select "Save as" and enter a new name '''cumulusmx_edited.service'''
# Exit out of the editor you are using
# Next, open a terminal session
# Now copy file to where it is needed to run the service <code>sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx_edited.service /etc/systemd/system/cumulusmx.service</code>
# Now type <code>sudo systemctl daemon-reload</code>, this tells "
# Finally, '''optionally''', create a symbolic link to that file using <code>sudo systemctl enable cumulusmx</code> if you want the service to automatically start after a reboot
====Technical users - additional edits====
Novice users, skip this subsection. ''The changes in this subsection have to be made with other changes that are not covered here'' (they depend on your weather station type, and your computer type, so are not appropriate to a Wiki page trying to generalise, and anyway your contributor is not a technical expert).
:Look at the '''[Service]''' part of the file quoted above.
This states Cumulus should use root for both the user it runs under and for the group permissions it uses. ''If you have the technical expertise'', you might choose to run MX in a different user, if your weather station type allows MX to run in a different user. If so, replace the "root" in its two locations. (Please note some weather stations require other changes outside this file before Cumulus can make contact, one example is discussed [https://cumulus.hosiene.co.uk/viewtopic.php?t=20413 on support forum here], but there are other topics that may be relevant).
You may also wish to add an extra line after the "Group" line <code>ExecStartPre=/bin/sleep 5</code>, this [https://cumulus.hosiene.co.uk/viewtopic.php?p=163754#p163754 is to delay the starting of MX by 5 seconds while other services start] (on a reboot of your computer) that might affect MX. (For some users, change 5 into 10, it all depends what else is being started).
:Look at the rest of the file, the '''[Unit]''' part.
For releases 3.8.4 to 3.15.0: you will see one reference to '''network-online.target''' in <code>After=network-online.target</code>.
For release 3.15.1 build 3170 (19 March 2022) onwards: you will see an extra line <code>Wants=network-online.target</code>
If you are a technical user, you might decide to edit
If your computer has online access, then it can look up the correct time online and adjust its clock. However, it might not even try to do that for say 10 minutes after being booted, and so there may be a benefit in making MX wait until after systemd has asked for the time to be synced, and asked that the local file-system is made ready so MX can read/update/store files. To achieve this, you might choose to add a blank line after '''<nowiki>Documentation=https://cumuluswiki.org/a/Main_Page</nowiki>''' and in that blank line, type <code>Requires= time-sync.target local-fs.target</code>. Using "Requires" ensures these requesting events have happened before MX can start, if they fail, MX will not be started, this example has not specified a time that MX should wait for the other services to start!
For that ''time-sync.target'' to work, you need to '''enable''', by creating the symbolic links needed, the appropriate services outside this edit:
<pre>sudo systemctl enable --now systemd-timesyncd.service
sudo systemctl enable --now systemd-time-wait-sync.service
</pre>
#* The terminology ''Requires'' tells "systemd" that the "cumulusmx" service should not be started until the services specified on that line have successfully started
#*# The '''local-fs.target''' specifies that the ''cumulusmx'' service requires the file service to have started, i.e. checks your computer can read files before it attempts to start the ''cumulusmx'' service
#*# The '''time-sync.target''' specifies that the ''cumulusmx'' service requires the computer to have synced with
''Don't forget to save the file under a new name, and copy it as instructed in previous subsection.''
====Technical Notes only relevant to Raspberry Pi====
This Wiki page has tried to avoid being too specific to particular hardware, but to avoid misunderstanding the last subsection, a little does need to be said to justify the claim that only technical users, who understand all the other changes needed, should make changes mentioned there.
A standard Raspberry Pi computer does not include a clock chip. Instead one of the packages it loads as a service on booting is called "fake-hwclock", and that sets the clock to what the date/time was when it was last running, irrespective of how many days/hours it has been off. That counts as a time sync for the purposes of instruction specified above. You can buy and fit a real-time clock chip, and configure your computer to use that, but even that RTC will only keep time when it is kept powered, and even then it will drift off unless periodically able to be corrected by a time from internet.
The issue is your Cumulus MX on restarting will skip the catch-up of historic data (should your weather station settings make that available), because the dummy clock makes MX think the computer was not off for long. Subsequent measurements will then get logged against the wrong time until the correct time is found on the internet (NTP). At that moment, the time will suddenly jump, this is serious if this means the "rollover" time has been skipped over, as it implies the "dayfile.txt" will miss a line, and many measurements will be logged to wrong day. In my experience it can be anything from 2 minutes to 10 minutes after switch on before my RPi does a time sync over the internet.
You might expect <code>sudo systemctl disable fake-hwclock.service</code> (or remove the service, and modify the scripts that call it) could ensure the computer (if online) has to get a time found on the internet (NTP). Nothing is as simple as it might seem!
===Commands to do actions on a service===
You will need to start (or restart) MX after you have defined (or redefined) the service as instructed above. The specific commands to use with MX service are at [[Raspberry_Pi_Image#systemctl_commands|systemctl_commands]], here I simply repeat the basic commands that can be used with any service (status, enable, disable, start, stop, and restart).
Don't forget you may need to type <code>sudo systemctl daemon-reload</code> to tell "systemd" that it needs to reload all service definitions whenever either one has changed, or a new one has been added.
In all these commands, '''just replace [service_name] with ''cumulusmx''''' (or enter the name of another service).
* <code>sudo systemctl status [service_name]</code>
** (displays whether named service has started, whether it has failed, whether it has stopped, also whether enabled, extra information will be added should status change)
** type the single character "q" to quit updating status display and return to prompt
* <code>sudo systemctl enable [service_name]</code>
** (typed just once, and service named will automatically start when your Linux computer is booted)
** the confirmation message says a link has been created
* <code>sudo systemctl disable [service_name]</code>
** (used when you don't want an automatic restart of the named service)
* <code>sudo systemctl start [service_name]</code>
** (will start the named service)
* <code>sudo systemctl stop [service_name]</code>
** (will stop the named service)
** Closing MX with "cumulusmx" as the named service this way does a proper shutdown
* <code>sudo systemctl restart [service_name]</code>
** (issues a stop, then start, command to named service)
** You can upgrade MX by installing new files over the existing ones, while MX is left running, and then use this command to pick up new release with minimum downtime.
|