ImportCumulusFile: Difference between revisions

From Cumulus Wiki
Jump to navigationJump to search

ImportCumulusFile (view source)

Revision as of 10:43, 14 January 2022

1,731 bytes added ,  14 January 2022
m
→‎Monthly table (2.0): Minor clarification
m (→‎Monthly table (2.0): Minor clarification)
 
(14 intermediate revisions by 2 users not shown)
{{AddOnBanner|name=ImportCumulusFile|type=PHP|author=David A Jamieson / Mark Crossley|contact='DAJ' / 'mcrossley' via the forum|updated=0501 May 2020September2020|version=4.13}}
 
This is a PHP script designed to run on your web server and will import Cumulus log files into a MySQL database table on your server.
This script was originally written for Cumulus 1, and all versions (except 4.x and later) will work with Cumulus 1. Please only use the latest version (currently 4.x) of the script with Cumulus MX.
 
Please be aware that '''for Cumulus MX the daily and monthly tables in the database can also be updated with rows for past dates by using ExportMySQL.exe'''. For information on how Cumulus MX updates the database tables ''without using that instead of ''the script on this page'' please see [[Cumulus_MXMX_Administrative_Interface#MySQL_settings|seethe admin interface herearticle]].
 
=Requirements=
 
=IMPORTANT NOTE FOR VERSION 4.0+=
 
Designed for MX.
 
Version 4.0 changes the SQL table structures from version 2 & 3. If you are currently using version 2 or 3 and wish to upgrade, then you should DROP the existing tables and let the version 4 script recreate them, or alter the table structures to add the new columns.
 
'HighUV', 'decimal(3,1)'
'THighUV', 'varchar(5)'
'HWindGBearSym', 'varchar(3)'▼
'DomWindDirSym', 'varchar(3)'▼
'MaxFeelsLike', 'decimal(4,1)' /* New in v4.0 */
'TMaxFeelsLike', 'varchar(5)' /* New in v4.0 */
'MinFeelsLike', 'decimal(4,1)' /* New in v4.0 */
'TMinFeelsLike', 'varchar(5)' /* New in v4.0 */
'FeelsLikeMaxHumidex', 'decimal(45,1)' /* New in 4v4.02 */▼
'TMaxHumidex', 'varchar(5)' /* New in v4.2 */
▲'HWindGBearSym', 'varchar(3)'
▲'DomWindDirSym', 'varchar(3)'
</pre>
 
== Monthly table ==
 
It has been pointed out by "water01" in the support forum, that the specifier for the RG-11 field should match the other rainfall fields, and allow 2 decimal places for those using inches as their unit for rainfall, but this script has(checked notat beenversion coded4.1) restricts that field to dojust thatone decimal place.
 
<pre>
'RG11rain', 'decimal(4,1)'
'RainSinceMidnight', 'decimal(4,1)'
'FeelsLike', 'decimal(4,1)' /* New in 4.0 */
'Humidex', 'decimal(4,1)' /* New in 4.2 */
'WindbearingSym', 'varchar(3)'
'CurrWindBearingSym', 'varchar(3)'
▲'FeelsLike', 'decimal(4,1)' /* New in 4.0 */
</pre>
 
 
 
= IMPORTANT NOTE FOR VERSION 2.0+x and 3.x =
 
Designed for Cumulus 1.
 
Version 2.0 changes the SQL table structures from version 1.x. If you are currently using version 1.x and wish to upgrade, then you should DROP the existing tables and let the version 2 script recreate them. If this is unacceptable DO NOT UPGRADE, stay with version 1.x
 
</pre>
 
== Monthly table (old2.0 version) ==
For versions below 2.0, the first combined column as shown belowwith both date and time did not exist, instead there was one column for date and one column for time.
 
As pointed out by Water01 in the support forum, the RG-11 field in the code and in the schema below is wrong, it should have 2 decimal places for those using inches like the other rainfall fields.
 
<pre>
'LogDateTime', 'DATETIME NOT NULL'
=Installation=
 
*Download the PHP script ... [https://cumuluswiki.org/files/ImportCumulusFile_v4-1.txtthere (rightare click,download Savelinks as)]beside andeach thenversion renamein the .txtversion tohistory below.php
*SaveEdit the downloaded file and edit it using a text editor -- there are five entries to be edited
:servername, username, password, database and security key
:these are documented in the file at the top.
:if your decimal separator is a comma this also needs to be changed in the file using the $decimal_separator variable
 
*save the file as ImportCumulusFile.<b>php</b> (or whatever name you prefer) and upload it to your webserverweb server which is where it should be run.
*you will also need to upload any files it reads to your web server.
 
=Using the script=
php importcumulusfilev2-0.php file=realtime.txt type=realtime key=secretKey retainVal=7 retainUnit=day
This allows you to run the script as cron job on a Linux server, or as a scheduled task on a Windows server.
 
Version 4.3 of the script allows you to automate the generation of the monthly log file name rather than passing it as a variable.
You need to edit the following entries to enable this feature...
$auto_month = false; // True: generate from date, false: read from params
$month_file_dir= './data/'; // Directory where monthly log file is stored
 
= Version Control =
Latest MX download:
*4.1 - 5 May 2020 [[https://cumuluswiki.org/files/ImportCumulusFile_v4-1.txt Download]]▼
*4.3 - 1 September 2020 [https://cumuluswiki.org/files/ImportCumulusFile_v4-3.txt Download]
** Steinar Utne: If monthly and $auto_month=true then construct filename from date (i.e. <dir>MonYYlog.txt) rather than read from parameters
 
Earlier MX Downloads:
▲*4.12 - 518 MayAugust 2020 [[https://cumuluswiki.org/files/ImportCumulusFile_v4-12.txt Download]]
** Steinar Utne: Added Humidex in dayfile and monthly
*4.1 - 5 May 2020 [https://cumuluswiki.org/files/ImportCumulusFile_v4-1.txt (Download, right click, Save as)]
**Fix for files that do not have the required number of fields
*4.0 - 4 May 2020 Download removed
**SQL injection protection improved - requires the table names to be defined in this script
*2.8 - 27 April 2015 [[https://cumuluswiki.org/files/ImportCumulusFile_v2-8.txt Download]]
**Added addtionaladditional configuration parameter $rainUnit, default is 'mm'
**Added 'rain since midnight' to the monthly log file table
**This version is REQUIRED to support CumulusMX direct inserts
**To updated existing monthly tables use the following SQL command [if you use 'mm' for rain change (4,2) to (4,1)]:
***ALTER TABLE <<YOUR_MONTHLY_TABLENAME>> ADD COLUMN RainSinceMidnight DECIMAL(4,2) NULL AFTER RG11rain;
 
Legacy Cumulus 1 downloads:
*2.7 - 23 February 2015 [[https://cumuluswiki.org/files/ImportCumulusFile_v2-7.txt Download]]
**Fixed realtimereal-time/monthly/day tables creates, to make rainfall 2dp to allow for inches
**Fixed monthly table create, to make evapotrans 2 dp as it uses same units as rainfall
**Converted from depreciated mysql to mysqli
*2.6 - 02 April 2014, Mark Crossley [[https://cumuluswiki.org/files/ImportCumulusFile_v2-6.txt Download]]
***To alter an existing table...
***ALTER TABLE `dayfile` CHANGE `HighSolarRad` `HighSolarRad` DECIMAL(5,1) NULL DEFAULT NULL
*2.4 - 23 October 2013, Mark crossleyCrossley [[https://cumuluswiki.org/files/ImportCumulusFile_v2-4.txt Download]]
**Changed day file humidity fields from varchar(3) to decimal(4,1)
*2.3 - 05 June 2013, Mark Crossley [[https://cumuluswiki.org/files/ImportCumulusFile_v2-3.txt Download]]
*2.0 - 13 March 2013, Mark Crossley [[https://cumuluswiki.org/files/ImportCumulusFile_v2-0.txt Download]]
**Change monthly and realtime tables to use combined date/time columns rather than separate
**Added optional retention time for realtime table. Records older than the specified age will be deleted. The query-string parameters are '''retainVal=NNN retainUnit=XXXX'''
**Added checking to allow running of the script from a command line as well as via http
**Added 'extra' columns to record wind directions as compass points where they are only provided as bearings:
*1.3 - 27 Nov 2012, Mark Crossley, updated for Cumulus b1050 [[https://cumuluswiki.org/files/ImportCumulusFile_v1-3.txt Download]]
*1.2 - 6 May 2011, Mark Crossley, updated for Cumulus 1.9.1
*1.1c - 11 Mar 2010,David A Jamieson, Small modification to code to handle date delimiter with a dot [[http://wiki.sandsoft.com/files/ImportCumulusFile_v1-1c.txt Download]]
*1.1b - 04 Feb 2010, David A Jamieson, Added a variable to consider decimal separator
*1.1a - David A Jamieson, Added date delimiter
*1.1 - David A Jamieson, Added the facility to identify the delimiter used on the import file
*1.0 - David A Jamieson, First Release, see top of page for details
 
[[Category:WebTools]] [[Category:User Contributions]]
Sfws
5,838

edits

Retrieved from "https://www.cumuluswiki.org/a/Special:MobileDiff/6229...10152"

Navigation menu