MX Basic info

This page contains some reference information about MX, that applies whatever operating system you run MX on.

It was correct at release 3.5.0, some updates have been made for later releases, but more work is needed.

This document is 'Work In Progress' so content may not be complete.

Request for help from Wiki Readers

• Do you understand how MX works?
• Do you use hardware, or MX functionality, that is not yet documented? Can you begin that documenting?
• Can you contribute simple text for novice users, examples of what you have done, correction of typing or factual errors, or supply missing details?
• Will you make this page more useful by bringing content up-to-date as new releases change some information written for older releases?
• Does any page need a section for novices, so they don't need to read more technical information further down that page?
• Is there some information on this page, that should be on a separate page? Can you create the new page and move the less relevant information off this page, don't forget this page needs a link to the new page so people who expect to find it here know where it has moved to?

If you plan on contributing to the Wiki, then you will need an account.

• Please use the Request Account form to apply for an account. Note that the Wiki is currently undergoing restructuring and is largely locked for editing, but please apply for an account if you wish to contribute in the future.
• You will find help on how to contribute to this wiki at How to Edit.
• If you need to consult others, please use the Cumulus Wiki suggestions forum.

Please be aware that information on this page may be incorrect.

The provided web pages

THIS SECTION APPLIES FOR MX 3.0.0 TO 3.9.7 ONLY

Setting up a web site was long, long ago,covered in this wiki page and the pages linked from there. I won't repeat that, but will try to explain below the MX context of the various files involved in the beta MX by Steve Loft.

MX will produce web pages locally even if you don't have a remote web site to display them on. You can view the web pages created in the web folder using a browser.

Cumulus MX (MX 3.0.0 TO 3.9.7 ONLY) provided a set of web templates, images, and json files, in \CumulusMX\web.

1. The first of these are called templates (and have a 'T' at the end of the file name before the extension) because they include both text and web tags.
2. MX will process these templates as it creates a web page (file name without a "T" in same web folder), during that processing, any text in the template is copied into new file without change, any web tags found in the template are parsed and the correct value is placed in that position in the output file.
3. If you have a web server, then MX can process these files and upload them for you (by default using File Transfer Process) providing you specify the host, port, protocol, directory, username, and password, for the upload process to use. If you don't understand any of these terms, then this is not the place for explaining them, but generally if your web space is supplied by a provider, they will be able to tell you most of these settings, and you will choose the directory name. If you have set up a web server yourself, then you should know the required settings.

The web templates included (MX 3.0.0 TO 3.9.7 ONLY) were based on designs by Beth Loft used for Cumulus 1, with minimal modifications (apart from trendsT.htm and gaugesT.htm which were new designs). Remember, Steve Loft (who wrote the original Cumulus software) said "They exist because they're our web pages, and they're really only included with Cumulus as examples of how the web tags work. It never occurred to me that most people would simply use the supplied examples instead of creating their own pages!"

The templates (MX 3.0.0 TO 3.9.7 ONLY) were written in fairly simple Hyper-Text Mark-up Language, designed to help people see how to write their own web templates. Here is a list of web templates provided:

• indexT .htm
• todayT.htm
• yesterdayT.htm
• thismonth.htm
• thisyear.htm
• recordT.htm
• monthlyrecordt.htm
• gaugesT.htm
• trendsT.htm

All these templates (except gaugesT.htm and trendsT.htm) included a table for showing values and styling that gave a graded background colour. The tables included a navigation row with links to the other pages in the set. That navigation line fixes the width of the table, and you will realise it was designed in the days when all monitors were a standard shape. Therefore the standard web pages as provided cannot adapt to the range of devices we use for viewing web pages nowadays. There are a selection of alternative web page sets available on the User_Contributions page, and some of these are responsive and adapt to the width of the device they are being viewed on.

The gaugesT.htm is a template similar to SteelSeries Gauges although the latter is designed to work with a range of software and the former is specific to MX. As supplied in MX if you mouse over the provided gauges appearing on your web site you will see a box with figures, not a graph as is seen with the general steel series gauges, but there are some other differences such as how the figures are supplied for the displays. The remaining template trendsT.htm, creates a structure that can display graphs. The data for all the graphs that can be displayed is contained in the various json files in \CumulusMX\web, these files are also processed by Cumulus so latest values are added, and then uploaded so the web page produced by this template can use them.

The image that is provided in \CumulusMX\web is MoonBaseImage.png, MX can be set to use that to generate (on MX start-up and on the hour) "moon.png" which it then can FTP to your web (also on the hour).

To set up your web server for the first time

When you first want to use Cumulus web pages on your web server, you need a number of static (unchanging) files to be put onto your web server. The web pages that MX uploads for you reference that static files and will not look right without them. The files that only have to be uploaded once are found in \CumulusMX\webfiles and its sub-folders. You don't create a folder called webfiles on your web server, but you put the files and sub-folders into position relative to where MX will upload the htm files.

To do this, you will must invoke a FTP process outside of Cumulus, MX does not include any functionality to do this one-off upload for you. The filezilla client is a popular choice as it has probably the most friendly graphical user interface, although other software to do this is also available. You may prefer a tool that lets you do the uploads from a command line without requiring working with a graphical interface.

1. The static files to be uploaded include the standard styling file \CumulusMX\webfiles\weatherstyle.css which you place in the directory specified for the uploads.
2. Next you have three sub-folders, each of those sub-folders need to be replicated within the directory specified for the uploads.
   • For example \CumulusMX\webfiles\images\picture.jpg will be stored in a "images" sub-directory of the upload directory and is used as the background image for web pages.
     • There is nothing to stop you creating your own "picture.jpg" (instead of uploading the supplied one) and then Cumulus web pages will use that for the background image on each page.
   • Similarly \CumulusMX\webfiles\js\cumuluscharts.js needs to be stored in a "js" sub-directory of your upload directory (this is the script that allows you to change the chart shown on the trends page and uses the appropriate json file to populate it with data).
3. The "lib" sub-folder contains further levels of sub-folders all to be replicated on your web site.
4. The trends.htm web page also loads some library software from an internet Content Delivery Network (cdn) to invoke the JavaScript based Highstocks library.

Operating a web site with uploads from MX engine

The standard web pages

• If you want to operate the 'standard' web site, then just the same as with Cumulus 1, you will need to upload the contents of the webfiles folder from the zip file (don't upload the containing webfiles folder itself).
  • Note that the MX web files are not the same as the ones for Cumulus 1, so make sure you upload the MX files if moving from Cumulus 1 to MX.
  • The standard gauges are now the SteelSeries gauges. The default versions do not display a graph when you hover over a gauge as happened when you added the stand-alone Steel Series gauges to Cumulus 1.
  • The trends web page in Cumulus 1 relied on that software generating graphs as images. In MX, the software generates files with time and value pairs, these are stored in json format, the trends page then uses a library package (Highstocks) to draw graphs from those data pairs.

Alternative ways to obtain web pages

You can choose to use some of the alternative web pages available from third parties and described on User Contributions page.

Using your own web pages

• Of course you can use your own web pages, instead of the standard ones. Assuming they need to include figures that are available as web tags, there are three alternative ways to implement this:
  1. MX can process template files with a HTML structure and those web tags in the structure where values are required just as it does with the standard templates, and MX can upload the resulting web pages at either the real-time interval, the standard interval, or after end of day. All of this is covered on the Customised templates page in this Wiki.
  2. MX can process a file with a string of web tags mirroring the realtime.txt option in MX, and upload the resulting file so your web pages can use JavaScript for a one-off insert of the values or an Ajax routine to update the web page at a fixed interval.
  3. Alternatively, you can use template scripts processed locally by MX that don't create web pages, but are uploaded by MX at either the real-time interval, the standard interval, or after end of day. These scripts simply initialise script variables with values obtained from web tags. You then independently have a set of web pages resident only on your web server (they don't exist where you run MX) using a combination of HTML and script content that bring in the script(s) with the variables by the appropriate syntax. All of this is covered on the PHP web tags page in this wiki. As it suggests there, you might therefore have several files processed by Cumulus MX at these different intervals, converting the web tags into script variables, and then use AJAX (JavaScript that may use json format to bring in the variables) or PHP (using 'require_once 'filename'; syntax) to put those variables into a web page.

You may find this wiki page useful for understanding more about the different script languages.

MX End of Day Process

I have added this section, because this process has given me some headaches.

My version

If you write custom SQL, or have a template being processed at end of day, then what I find strange is that web tags related to system date report the new date, but other web tags report weather derivatives from the old day. Put another way, all the date related tags change at start of rollover, but all the weather related web tags change at end of rollover.

However, it is not quite as simple as that, the month and year are reset before the extra web files are processed (so they cannot use monthly web tags at end of month, nor yearly web tags at end of year). The complication is that in the extra web files you can use <currentlogfile> (and from build 3087 <currentextralogfile>) and these pick up the old month/year. Now you see why I found it hard to digest, and why I wanted to write it here to make it easier for others.

If you use Custom SQL and therefore have to quote web tags, the SQL should use monthly and yearly web tags related to previous day, but all the weather tags it uses must be those for current conditions or today. Yes it is confusing.

As part of the so called "end of day" process, MX (just like Cumulus 1) creates a start of day back up in the daily sub-folder. So on first day of a new month (and new year sometimes), the backed standard log files (and extra sensor log files from build 3087) are for the month that has just started, there is no back-up of the old month.

The official version

Mark Crossley says the MX day reset does this (at version 3.5.x)...

   Reset midnight rain
   Entering Day Reset (message about current day of month, at this stage web tag <#metdate> changes to new date)
   Day Reset (message about date ending, time shown as 00:00:00 because time not defined, not because it is midnight, it might be 9am or 10am)
   Run EOD custom SQL
   Save dayfile entry (uses what is still in today.ini that includes old date, i.e. what is now in web tag <#metdateyesterday>)
   Write monthly & yearly file entries
   Write any new daily extreme records
   if day of month = 1 then: copy month.ini to saved file, reset monthly figures
   if day of month = 1 and month = 1 then: copy year.ini to saved file, reset yearly figures
   Copy todays high/lows to yesterdays
   Reset todays high/lows to current
   Write today.ini & yesterday.ini
   Create NOAA reports
   Execute user daily external program
   Process Extra EOD files 

But independent of above EOD thread that occurs on the rollover hour, the normal interval and hourly processes thread is seeking to run at same time, whether that happens at same time depends on processing capability and whether it can process multiple threads.

What actually happens in above list depends on your settings, and if your FTP interval is synchronised with the logging interval.

Library software

For most Cumulus users, the remainder of this page can be skipped, but those few users who have a technical slant might find the remainder of this page useful.

Cumulus MX uses library software (i.e. software written by others and made available by the provider and often also by other content delivery nodes or 'cdn') for a lot of the standard functionality.

The library software for the admin interface and the separate library software for the standard web pages are both mostly included in the distribution zip, although some is used via a link to a cdn.

For many releases of MX, the libraries included were very obsolete. However, Mark Crossley, the current developer, said the following on 30 Sept 2019: "Chasing the latest versions of all the packages for the sake of it is a thankless task, and requires considerable effort to regression test each update. I am only updating packages when required to fix issues or for platform compatibility." Just to mention the other side of this balance. It is difficult to code an addition to MX that works with obsolete versions of libraries. All documentation provided by providers of the libraries relates to current versions of the packages (and what is documented to work now, often will not work with obsolete versions). The documentation for the packages that are no longer supported is only available in archive sites if available at all.

By June 2021, MX development had updated the included library software. Sometimes, trying to incorporate a more recent module, made MX have problems, and so some changes had to be regressed, and some library software incorporated is pinned to a recent version but not the latest. However, multiple changes to library software, does mean this page may not be accurate for the release you have implemented.

Library Software for the MX engine

The distribution zip contains various .dll files and these are the libraries used by MX itself. The exact mix of libraries included has varied at various times, the list below is a snapshot of those included at the version that was investigated when this article was extended to include this section, and may not be right for the current MX version.

Devart

The two files used in earlier MX releases were both related to the database functionality of MX. These have been replaced by "MySqlConnector.dll" from April 2021.

FluentFTP

As the name suggests, this is used by MX for controlling the file transfer functionality. This component was first introduced at version 3.0.0 build 3045 (adding more functionality than that available from System.Net.FtpClien, which was used at earlier MX builds), and FluentFTP.dll has been updated at some subsequent MX releases, see the announcements for details.

Linq

Language INtegrated Query is used to work with sequences of items and pick the ones that are needed, putting them into output format required. MX uses two files in connection with preparing output for Twitter, the same files have been used for all MX releases.

There is a third Linq file for other processing, "System.Reactive.Linq.dll" which is also same in all MX releases.

MQTT

When MX added capability to talk to other devices using the MQTT protocol, it added this component for that optional functionality. The component was updated in February 2021, and may be updated again in future.

Newtonsoft

This component was used for processing JSON strings. It was a very popular choice for developers, and used therefore very widely.

However, SystemText has superseded it,and MX no longer uses it (anybody want to document when MX abandoned it?).

Renci SSH

This component is server connection software, it is what processes the host name, password, and so on. It is still in use in June 2021.

SQLite3

This is used for all interactions with the weather diary. The component used remains same for all MX releases.

HTTP

These files handle the optional HTTP functionality, "System.Net.Http.Primatives.dll" and "System.Net.Http.Extensions.dll" have remained in use since February 2015.

Unosquare

The EmbedIO file is open-source software that handles the web-sockets functionality of MX. The Swan.Lite file is open-source software that handles JSON formatting and threading of tasks in MX.

Library software for admin interface

As the following sections reveal, MX uses external libraries rather than writing its own code whenever possible.

• However, that does not mean MX is good at meeting development standards.
• MX only implements small parts of the functionality of most libraries, the minimum to make a feature work, not all the features available to make it work well
• MX does not use the latest versions of libraries
• MX does not attempt to obey guidance for good user interaction, and although validation is being added, many parts of MX do depend on user understanding what is valid
• MX does not make provision for screen readers and other accessibility aids.

Alpaca

1. Alpaca software is effectively a programming language extension to help people design forms like those MX uses for all its settings, and as a Cumulus user you really don't need to worry about it.
2. It is used for most settings screens. See http://www.alpacajs.org/ for more information. The latest version there is 1.5.27 released on 14 May 2019.
3. MX used Alpaca Release 1.1.3 from https://github.com/gitana/alpaca which was first released on 15 May 2014.
4. It is believed MX updated to newer versions from February 2021, recent releases are trying to improve accessibility, so there might be another update.

Bootstrap

1. Also known by some as Twitter Bootstrap which gives a clue as to its developer and to its origins as an internal tool for those building Twitter, that company still keep making updates as it is now the most popular styling library of all those available widely.

• The simplest way to think about this package is as a standard set of styling promoting easy responsive (means adapts to screen dimensions) web site design.
• To give just a few examples, it defines a standard way to represent buttons, form components, lists, navigation, and breadcrumbs.
• MX uses Bootstrap version 3.3.7, which is very restricted in what it offers. As at June 2021, MX is still using this, as it has for all previous releases
  • Bootstrap version 5 is available (http://getbootstrap.com), so MX is using an obsolete library
  • Bootstrap version 4.5.0 (Bootstrap 4 released as alpha in August 2015, beta in August 2017, and with fully working releases frequently from January 2018) was very widely praised for its improved functionality, and ability to work with latest jQuery and multiple modern devices/browsers.
• MX does not implement key features of Bootstrap like colouring text according to what it represents (primary, secondary, information, warning etc.)

dataTables

1. When MX sends out multiple lines of a log file to view or edit, the application programming interface (api) that transfers the information from the MX engine sends it in dataTables format for display on the web page in the admin interface.
2. Thus dataTables does all the work of providing the ability to present the data in a HTML table, the functionality to move between multiple pages needed (as MX sends only up to 10 lines of a log file at a time to the admin interface).
3. The free version of dataTables used by MX lacks the most useful functionality that needs a subscription licence. For example, its editing functionality requires a subscription.

There have been some updates to how MX implements its data log editors, it is unclear how much of this is due to work by MX developer and how much due to work by library developers.

altEditor

• This is an editing tool that can read what is in dataTables, create what it calls a modal (a pop-up dialog) where rows can be added, edited, or deleted individually.
  • MX when it added editing of log files at version 3.4.5 - Build 3069 (Friday 13 March 2020) adopted this software as it was free (although Mark Crossley said in his release notice: "The main thrust of this release is to add some log file editing capability to Cumulus MX. It works on all three log file types, but it is fairly basic at present. You can edit or delete lines in the files. The editing has to be done via pop-up dialog.
  • I only found two libraries that support JQuery dataTables editing, one is very comprehensive - but costs $$$ - the other is free. The free version does not currently support in-line editing of the table which is a shame.
  • If any web guru out there can come up with a better solution please post about it on the forum, or send a pull request.") (By the way, it is possible to provide in-line editing and make it work with the existing api interface, but making it compatible with the obsolete software used by MX is hard).
• The single line of fields that is result of an edit or deletion done on the modal is sent back via another api to the server (the MX engine in our case) and that then regenerates the dataTables in the state after whatever action was done, sending back again up to 10 lines for the same page as before.
• As it happens there is another JQuery dataTables editing tool, but it has not been maintained since 2012. It is found at https://github.com/NicolasCARPi/jquery_jeditable, but the documentation is now only available in an archive at https://web.archive.org/web/20200615000000*/https://appelsiini.net/projects/jeditable. It is designed for editing table cells, so it does not involve any pop-up dialog.

There was some exploration of alternatives during May 2020 (to deliver in-line editing), but MX is still using AltEditor in June 2021.

datepicker

1. Although modern browsers generally will generate a calendar type interface when they meet an entry field defined as a date, this date picker software ensures all MX users see exactly the same interface for date selection needed for both the standard log and the extra sensors log which are monthly log files (a new one is created each month). It is used for picking which standard (monthly) log or extra (monthly) log is to be viewed by selecting a month and year only.
2. It is also used for selecting individual days in the weather diary editor.

editable grid

• As the name perhaps suggests MX only uses this for the extra web files screens where you can make selections within a grid like interface.
• I suspect it could enhance some other functionality in the future.
• During Spring 2021, there were some accessibility improvements, but no alternative library has been identified.

handlebars

1. Put simply this is a simple HTML generator based on templates.
2. My exploration of the files in Interface failed to discover how "handlebars" is used.

x-editable

• Put simply, this allows in-place editing of web pages using bootstrap.
• In MX it is used for the record editing screens where you adjust the extreme values.

jQuery

1. The admin interface for MX until April 2021, used the obsolete version 1.9.1 of this JavaScript based library.
2. From April 2021, MX is using version 3.6.0.

Note about older version previously used:

1. Of all the old versions of jQuery to choose, MX had picked the only version that the developers withdrew due to an error when they released it.
2. Version 1.9.1, has a serious error in its code, because the developers accidentally combined code from two significantly different versions when they created the release file.
   • This reveals itself in two ways:
     1. The error handling does not work.
     2. It tries to load another script that does not match.
3. Consequently, the developers quickly removed it, but it remained available from Contents Delivery Nodes, which is where MX found it.
4. Not surprisingly, the authors of jQuery strongly advise all 1.9.1 users to move to a later version.
   • Unfortunately, there are interdependencies between all the library code used by MX, so you cannot simply update this component (see next library item).

Jquery Template

• MX continues to use an old version of jQuery template, which was withdrawn many years ago.
• It was originally available from jQuery downloads, but they now offer jsRender.js for this functionality.
• It basically is used to bind the contents of objects (like array elements) into particular locations within HTML.

SteelSeries

MX uses a modified version of the steel series library made available by Mark Crossley for all the gauges (see dashboard and gauges pages of the admin interface) in MX.

Highstocks

Most of the libraries included in MX are directly included in the MX release distribution zip. The odd one out is Highstocks (that includes HighCharts) where for early MX releases the code was included, but from 3.5.0 the code is loaded from a Contents Distribution Node (CDN).

• Inclusion of loading from CDN means that the Charts page in the admin interface will only work when there is an internet connection working to permit download of this software
• If you need to view your admin interface where an internet connection is not available:

Then you need to edit the interface file...

<CMX_Folder>/Interface/charts.html Change lines 20,21 from

<script src="https://code.highcharts.com/stock/8.0/h ... "></script>
<script src="https://code.highcharts.com/themes/grid.js"></script>

to

<script src="webfiles/lib/highstock/js/highstock.js"></script>
<script src="webfiles/lib/highstock/js/themes/grid.js"></script>

1. Each release from 3.5.0 onwards has selected a particular release to load from the CDN, which sometimes is, and sometimes is not, the latest release.
   • As at June 2021 (MX release 3.12.0), Highstocks 9.1.0 is the pinned release, which is not the latest release.
   • (Earlier MX releases, pinned to earlier Highstocks releases, although it was latest release for MX 3.11.x releases)

Library software for your web server

The webfiles/lib folder includes a number of software library items that are needed for the standard web pages included in the MX distribution.

This is fine if your web site is purely for the provided standard web pages, but if on your web server you also have web pages from third parties, or you have written your own web pages, then you may get conflict because all the library items used by MX are obsolete versions, and in one case MX uses a version of library software withdrawn by its originators due to compile error (so only available form some CDN who provide obsolete versions as its originators insist it must not be used)!

1. Highstock
   • Prior to MX release 3.5.0, Highstocks was included in the webfiles/lib folder, and the example web pages included with the MX distribution used this old version of Highstocks included in the webfiles/lib folder.
   • For MX 3.5.0 to MX 3.9.7, Highstocks was still included in the webfiles/lib folder, but not used, instead (like admin interface) an old version was loaded from a CDN.
   • Since 3.10.0, the new default web pages are a one-off upload, and they use the CDN, there is no copy included in the webfiles/lib folder.
2. jQuery
   • Be aware that many MX release distribution were not consistent with different web pages using different obsolete versions of jQuery.
     1. Up to MX 3.9.7, gauges.htm included <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script> to load version 1.8.2 from the Google Content Delivery Node
        • MX included at the end of that web template, if you bother to read it, some instructions about how you can change that CDN load, (but the instruction is neither grammatically correct, nor understandable for me).
     2. Up to MX 3.9.7, trends.htm included <script src="lib/jquery/jquery-latest.min.js"></script> to load version 1.9.1 of jQuery loaded from this badly named file, there is nothing latest about it.
        • Ironically, the old version of jQuery that was previously selected by MX to be included in the webfiles/lib folder was the only old version that jQuery themselves withdrew, and warn you not to use because the error handling content is from a different jQuery version to the correct handling content making the two parts of the code incompatible. In other words, this version only works if everything is set up correctly and no error handling is invoked.
   • The MX 3.10.0 announcement says that the new default web site uses the latest jQuery, although it does not mention a specific version.
3. Steel Series this is treated as library software, as it is copies of that separate product by Mark Crossley.
   • Most of the files are exactly as that was in revision 0.14.13 made 30 January 2015.
   • gauges.js has one tweak for MX, it defaults to not having chart rollovers (for Cumulus 1, this file had default to showing the images of graphs that were uploaded by Cumulus 1).