Cumulus Wiki

CumulusUtils: Difference between revisions

Page Discussion

CumulusUtils (view source)

Revision as of 12:20, 24 February 2026

1,705 bytes removed ,  24 February
m
→‎MAC: Using crontab (advised)
 
(7 intermediate revisions by the same user not shown)
This paragraph describes what is required to use CumuluUtils (To be modified/extended by user experience).
#A working CumulusMX environment on Windows, Linux (RPi) or MacOS
##On Linux (and probably MacOS too) the user must verify whether [[SysInfo#Introduction|the application ''lshw'' has been installed]].
##NOTE: no experience by the author on MacOS exists so be prepared to communicate
#For the ''[[Charts_-_Misc_charts|Miscellaneous Charts]]'' more data - a year starting on jan 1 - is required to make these useful▼
#<s>At least 32 days of data</s> This requirement has been dropped.
#From CMX4 and CMX5 change in the handling of datetime in the logs. Corresponding CUtils versions are available. It is advised to upgrade to the last version of CMX/CUtils as fast as possible because of support.
▲#For the ''[[Charts_-_Misc_charts|Miscellaneous Charts]]'' more data - a year starting on jan 1 - is required
#For longer data series check your locale creates consistent logfile entries and it will be useful to run the [[Software#Create_Missing|CMX supporting application ''CreateMissing.exe'']] beforehand.▼
#For the logfiles:
##<s>A consistent date format, mixed date formats in logfiles are not accepted by CumulusUtils</s> From CMX v4 onwards this is enforced by CMX
##<s>Consistent data and separator use</s> See above
##<s>If a legal date format in a logfile is used which is not implemented, just request it</s> See above
▲#For longer data series check your locale creates consistent logfile entries and it will be useful to run the [[Software#Create_Missing|CMX supporting application ''CreateMissing.exe'']] beforehand.
 
== Goals ==
The goal of ''CumulusUtils'' (abbr: ''Cutils'' or ''CUtils'') is to facilitate website creation for users of CumulusMX without exposure to coding in e.g. PHP or javascript. Having a meteo-website should not be privileged to users with great IT-skills. Making charts should not be demanding for programming skills in SQL or making it otherwise impossible to create charts without diving deep into the technique involved in CumulusMX (which actually is also the case with SQL). Note that this does not mean you can't use IT skills: the ''user configurable menu'' offers an opening to expand as far as you wish.
 
''CumulusUtils'' positions itself as a configurable application for which the understanding of how to run it and how to configure it is the most important thing to know. For configuration ''CumulusUtils'' uses [[Cumulusutils.ini|''cumulusutils.ini'']] file which resides in the CumulusMX directory. If that file does not exist, it will be created. The same holds for the CUstringsXX.ini file (containing your translations for CUtils text strings). All other files related to ''CumulusUtils'' arereside in the utils directory.
 
The charts are created using [https://www.highcharts.com/ HighCharts] using their [https://shop.highsofthighcharts.com/highsoft/form/noncommercialform nonlicense-commercialeula license"Personal Use" License]. If you want to use the charts or the [[Website Generator]] of ''CumulusUtils'', please make yourself acquainted with this license.
 
== Assumptions ==
== Installation ==
 
# ''CumulusUtils'' is available in a distribution which can be downloaded from the [https://cumulus.hosiene.co.uk/viewtopic.php?f=44&t=17998 forum]. In the CumulusMX directory the user must create a directory ''utils'' (case dependent) and must copy the files of the distribution (including the subdirectories) to that directory.
# Rename ''CutilsCharts-default-for-use.def : when used rename'' to ''CutilsCharts.def''▼
# Then, on the domain for the website, the user needs to make two directories in the webroot: 'lib' and 'css'. In addition the distribution directory CUicons must be copied as a whole - with contents - to the webroot.
# Updating ''CumulusUtils'' is done by completely overwriting the contents of the installation directory utils as if it were an installation. If you want to save the old installation, make a copy of the utils directory.
# The language setting of CumulusUtils is for display/language handling only and does not affect the reading of the data in any way apart for the monthly log file name.
# <s>NOTE a seemingly important issue: the datafiles (the naming and the contents) are dependent for their format on the locale / country setting of your machine. If you install and run from scratch and do not bother CMX will take the country setting from the settings of the computer. You may also give the country setting on the commandline when starting CMX. However, it is important to know that CumulusUtils does not handle datafiles with mixed languages and produces lots of errors. When starting with a new install this is not a big deal. But if you have years of old data which you wish to analyse as well you must consider carefully which country setting of CMX is required. Note that the language setting of CumulusUtils is for display/language handling only and does not affect the reading of the data in any way apart for the monthly log file name. If you use a different locale than the one you use for CMX, the please fill in the parameter ''MonthsOfMiracleAndWonder'' (section [General]) with the abbreviated filenames of the locale you use for CMX.</s>
# From CMX version 4, the logfiles are written as invariant and the above striked through note has become invalid.
# From CMX version 4.7 (.NET 10; later rename version 5.0) there are changes in the database which require CMX to run before CUtils 8.2.0 can run.
 
When installed you are ready to run. The first run after an install or an update MUST be without [[Thrifty - Cutils Command Qualifier|Thrifty]].
|--- lib (JavaScript libraries, uploaded by CumulusUtils during a run)
|--- css (css files, uploaded by CumulusUtils during a run)
|--- CUicons (required to be manually copied once)
 
=== Updating CumulusUtils ===
#CUserAbout-example.txt : as an example of the user about file - when used rename to ''CUserAbout.txt''
#CutilsMenu-example-for-use.def : as an example of the menu structure - when used rename to ''CutilsMenu.def''
▲#CutilsCharts-default-for-use.def : when used rename to ''CutilsCharts.def''
#CutilsCharts-examples.def : not intended for direct use, contains some more elaborate examples of the [[ChartsCompiler]] possibilities
 
 
=== Manually installing without FTP ===
NOTE: Not using the automatic upload system is disadvised. Because CumulusUtils is datadriven it requires the JSON files and the ''realtime.txt'' and ''realtimegauges.txt'' from CMX. Besides those, if you are using the ''UserAskedData'' feature of CumulusUtils there will be additional JSON files required for the server and without the upload module youryou will need a way to get them there. There is no copy feature like CMX has as CumulusUtils is designed as a website tool and an upload method is implicitelyimplicitly assumed present. However, manual handling can be done but is not advised.
 
All required installation and generated files are present in the utils directory and copied with ''FTP'' or ''PHP upload'' to the server during a run of CumulusUtils.
 
If you don't use ''FTP'' or ''PHP upload'' you will have to copy manually.
 
The PHP upload is by far preferred.
 
=== Files involved ===
== Running CumulusUtils ==
 
CumulusUtils is - up to v6.x.y - a [https://www.mono-project.com/ ''mono''] executable. This is now deprecated and no longer supported.
 
From v7.0.0 and up it runs under .NET 8 [https://learn.microsoft.com/en-gb/dotnet/core/install/linux-scripted-manual#scripted-install ''Follow this link when on Linux''].
All environments are very different and not interchangeable. However running CumulusUtils is pretty similar on all environments and mono and dotnet (8 and 10) can co-exist on the same machine. See the CMX installation on how to install either mono or dotnet. CumulusUtils assumes the correct installation of either environment.
 
Mono is an open source implementation of Microsoft's .NET Framework based on the ECMA standards for C# and the Common Language Runtime.
''CumulusUtils'' runs on any operating system CumulusMX runs on and it '''MUST''' run in the CumulusMX directory (as ''working directory'').
Running ''CumulusUtils'' is done from the commandline in a command window (under any OS) or from a script.
 
''CumulusUtils'' takes one or more commandline parameters (in short: commands) and must be like (see NOTE 1 below; square brackets means ''optional'' so DON'T TYPE THEM):
 
''dotnet utils/bin/cumulusutils.exedll [command]'' (the ''mono.NET'' syntax - this''dotnet'' assumesis monoobligatory ison alreadythe activecommandline for UNIX. Note the '''dll''')
 
BelowFor is described for ''mono'', when using .NET, please change the command as above.UNIX: Note you will have to carry this on in scripts as well. Using the dotnet command in crontab requires using the explicit path to the dotnet command or defining the PATH environment variable in the crontab script which does not know the environment (a bit awkward).▼
''dotnet utils/bin/cumulusutils.dll [command]'' (the ''.NET'' syntax - ''dotnet'' is obligatory on the commandline. Note the '''dll''')
 
▲Below is described for ''mono'', when using .NET, please change the command as above. Note you will have to carry this on in scripts as well. Using the dotnet command in crontab requires using the explicit path to the dotnet command or defining the PATH environment variable in the crontab script which does not know the environment (a bit awkward).
 
If no commands are given the application responds with:
Please note that CumulusUtils requires live data from CumulusMX to display what you expect. To accomplish that you need to configure CMX as follows:
 
The things it is looking for are realtime.txt and realtimegauges.txt. Those are sent by CMX to the directory you configure in ''Settings=>Internet Settings=>Web/Upload Site''.
 
In ''Settings=>Internet Settings=>Interval Configuration=>Normal Interval Settings'': Tick both options and set the interval time to enable interval Upload
Running CumulusUtils results in output which represents a static view of the data in a table or graphic format for display on your website (it may even create a complete website). The fact that the output is a static view requires the output to be regenerated when new data is available. Rerunning CumulusUtils is also required when you change anything in the configuration and can't wait to see that change reflected on the site. In general: changes in data and configuration require a rerun.
 
Reruns for changes of data are typically once per day, just after rollover time. In that sense it is like running a new query.
 
There are some exceptions to the daily rule: Sysinfo and [[UserAskedData]]
05 1 * * * cd /home/CumulusMX; /usr/share/dotnet/dotnet utils/bin/cumulusutils.dll thrifty website
1-51/10 * * * * cd /home/CumulusMX; /usr/share/dotnet/dotnet utils/bin/cumulusutils.dll sysinfo UserAskedData UserReports
 
===== For mono =====▼
15 1 * * * cd /home/CumulusMX; utils/bin/cumulusutils.exe thrifty website▼
1-51/10 * * * * cd /home/CumulusMX; utils/bin/cumulusutils.exe sysinfo UserAskedData UserReport▼
 
==== Windows: Using the scheduler ====
===== Customisation =====
One can also customise/fine tune the task further by exporting it, default export format is XML. Then can edit in favourite text editor and reimport when done. May be useful in particular for fine tuning the repetition times of the task.
 
==== MAC: Using crontab (advised) ====
Automating the run of CumulusUtils on the MAC is typically done through [https://share.google/aimode/CMq6jMIq3FvXLT8bn ''crontab'']. The link is a Google Gemini shortcut which offers more batch options on the MAC. The advise is to use ''crontab'' to have support available. Below is the crontab on RPi of the author of CumulusUtils to be used as an '''example'''!
 
▲===== For monodotnet =====
▲ 1505 1 * * * cd /home/CumulusMX; /usr/share/dotnet/dotnet utils/bin/cumulusutils.exedll thrifty website
▲ 1-51/10 * * * * cd /home/CumulusMX; /usr/share/dotnet/dotnet utils/bin/cumulusutils.exedll sysinfo UserAskedData UserReportUserReports
 
== Additional remarks ==
 
#(on NOT Windows) When mono is already active (as daemon or from command line in another process) you can start any mono executable without having it prepended by the mono command. As normally (if you did not uninstall it) xsp4 is running as a daemon, mono is always available so you can run mono executables as normal linux executables. You won't notice the difference. However if you don't have the x bit set, you get access refused when trying to execute. Without the x bit set you have to prepend mono as in: ''mono utils/bin/cumulusutils.exe''. Then it executes without the x bit set because then it is an argument to the mono command.
#CumulusUtils is written in C#, HTML and javascript. All HTML and javascript is embedded in the C# code and generated on execution of ''CumulusUtils''. The cumuluscharts.txt charts library was based on the cumuluscharts.js originally distributed with ''CumulusMX'', probably the version a bit before April 2020 and has been modified since. With the appearance of the [[ChartsCompiler]] the name cumuluscharts.txt has been preserved but the technique of creating the charts has been changed.
#As CumulusUtils itself runs in the Cumulus directory, the data directory (data) is one level below. CumulusUtils currently uses - a copy of - dayfile.txt, the monthly logfiles and incidentally the alltime.ini file. It also takes information from Cumulus.ini. All output is written to the utils/ subdirectory. All logs are created in the utils/utilslog/ subdirectory. It also uses the AirLink.log and the ExtraLog files in a similar way when the user has such devices.
#CumulusUtils can output for many languages. See the specific page on [[Language in CumulusUtils|language in CumulusUtils]].
#Development of CumulusUtils started in 9<sup>th</sup> of August 2019 with an initial commit of the Top10 written in C. Development changed rapidly to C# with an initial commit of Top10 on November 19 2019 of version 0.7.0. The rest of the history can be read in the [[Release Notes|release notes]].
#Users canPlease, present themselvesyourselves on the CumulusUtils [[Map - Users of CumulusUtils|usermap]]. Although optional for modules users, the author sees it as payment for the usage of the tool, so please put yourself on the map. For website users it is automatic and obligatory. Thebut the URL needs to be filled in by yourself in the inifile. When not a complete website, please fill in the URL to the page you use CUtils for (e.g. pwsFWI).
#In case of problems there exists [[Errorlogging]].
 
'''NOTE''': comments to this series of articles is welcome. either by using the Talk feature of the Wiki for that page, directDirect edit of the article (request an account) oris switched byoff. contactingContacting the author of CumulusUtils [https://cumulus.hosiene.co.uk/memberlist.php?mode=viewprofile&u=9016 HansR] on the [https://cumulus.hosiene.co.uk/viewtopic.php?f=44 forum] or through PM.
 
[[Category:WebTools]][[Category:User Contributions]][[Category:CumulusUtils]]
Retrieved from "https://www.cumuluswiki.org/a/Special:MobileDiff/12276...12357"