Monthly log files and Category:Files with Comma Separated Values: Difference between pages
mNo edit summary |
m (Minor resequencing of page) |
||
| Line 1: | Line 1: | ||
[[Category:Cumulus Files]] |
|||
[[Category:Cumulus Files]]Please see [[Standard log files]], [[Extra Sensor Files]], [[Air_Link_Log.txt]], [[Month.ini]], or [[Monthlyalltime.ini]], as relevant. |
|||
Cumulus software uses a number of files with a '''.txt''' extension that can be described as being in comma separated value format (CSV). |
|||
Cumulus normally simply appends new lines onto the end of these files, although there is some functionality to read them for plotting graphs and for displaying/editing extreme records. |
|||
Please be aware that all of the above links lead to documentation about files that may be described as ''monthly log files'' when reading individual Cumulus forum and wiki references. |
|||
None of the files hold a header line, although the appropriate header labelling the fields can be found in a different file. (Links to those files included in list below). |
|||
==Potential Name Confusion== |
|||
==Field separator== |
|||
Those who used early versions of Cumulus will remember that the term '''Log File''' was originally used for [[Log.xml]], but Steve Loft later accepted that all the files with a [[:Category:Ini Files|.ini]] file extension could be described as log files, so he introduced the term [[Weather Diary]]. A bit later, [[Standard log files| the detailed data file]], began to be also called '''log files''', and the confusion grew. When [[Extra Sensor Files]] were added, these also became '''log files'''. |
|||
'''Although these files are described as CSV format, that does not mean that the character separating one field from the next field has to be ",".''' In some locales, the comma character might separate the integer and decimal parts of a number, so the field separator character might be a semi-colon ";". |
|||
Steve Loft wrote the Cumulus software mainly for his own use, although after a while he let others use it and added features that others requested. Cumulus was developed in his spare time outside a full-time employment, and consistency and terminology were not issues he worried about. |
|||
During the development of the original Cumulus software (Cumulus 1 and 2) by Steve Loft, the name "monthly log" was most frequently used as a collective term for the last two (.ini) files linked above because those files tracked the monthly extreme records. |
|||
==Editing these files== |
|||
When Steve Loft was working on his [[Cumulus_3_(MX)_beta_documentation|MX Beta]], he decided to implement within MX functionality the database schema that Mark Crossley had designed on [[ImportCumulusFile#Monthly_table_.28old.29|this]] page, where Mark selected the name '''Monthly''' for the table. David A Jamieson created versions 1.0 and 1.1 of ImportCumulusFile for creating a single table called '''dayfile''', but Mark extended it to cover ''realtime'' and what he called ''monthly log files''. That means the standard log can in MX optionally feed data into a database table called '''Monthly''' by default in MX. |
|||
The files linked below all use plain text, so that they can be edited outside Cumulus, but there are rules about content, so be very careful about editing outside Cumulus. A "CSV editor", or a simple text editor designed for programmers, is safer to use than a spreadsheet (if you use a spreadsheet, ensure all columns are set to be "text", don't let it recognise dates, times or numbers), and you must never use a word-processor. |
|||
The new terminology can be misleading as that database table is not organised by month in any way, a single table contains every logged line that has been inserted into it by '''ExportMySQL.exe''' or '''CumulusMX.exe'''. |
|||
You may wish to edit into a data file a header line, to make life easier when editing values, but remember to remove that header line before saving the file. |
|||
Mark Crossley has taken over development of MX, the terminology "monthly log" is now being used for the first linked file (standard logs) instead of for '''month.ini''' the file that is truly logging monthly extremes. The name has been adopted simply because a new log file is started for each month. |
|||
The line terminator for file lines depends on the device running MX. In Microsoft Windows Operating System each line is terminated by a pair of characters (CR and LF), a UNIX-derived system (like Linux) uses just LF to end a line, and some versions of Mac Operating System use just CR to terminate each line. |
|||
For novices, there can be a feeling of confusion: |
|||
* The new practice leaves "month.ini" without a descriptive name |
|||
* Each "dayfile" line represents a new day, but a line in "Oct16log.txt" does not represent a month, and the name does not contain the phrase "month" |
|||
* The names "Month.ini" and "Monthlyalltime.ini" do contain the phrase "month", they do contain monthly data in each line, and do log extreme records |
|||
If you move a file between different devices, beware the line endings might get changed (although an editor designed for programmers includes ability to select line ending). |
|||
A future Wiki editor may decide on a different strategy, but at the moment the Cumulus Wiki tries to adopt the original Steve Loft terminology for each of the log files, for consistency with older articles in this Wiki and the oldest posts on the forum. |
|||
Recent releases of MX are coded to recognise which line ending was previously used, and will maintain that even if you run an executable (CumulusMX.exe, CreateMissing.exe, or ExportToMySQL.exe) on another device. |
|||
=Enhancement never implemented= |
|||
==Note for anyone using the Legacy Cumulus 1== |
|||
Steve Loft was several times asked if he could change the design of his software and what it stored in [[Standard log files]], [[Extra Sensor Files]] and the other detailed logging files. |
|||
This Wiki page was created as part of documenting Cumulus MX, and is not therefore written for you. |
|||
All these files currently periodically store spot readings. Some of the competition in weather software, periodically store the highest and lowest since the previous line was stored. |
|||
'''For Legacy Cumulus 1''', only a subset of the files listed here are used. Please be aware that within those files that are used the content differs between MX and the legacy software. |
|||
One advantage of the alternative approach is that you never lose any extremes. Another advantage is the alternative allows people to report extremes for parts of days. For example, some people wanted 9 am to 9 pm reported separately to night time extremes, in other nations, the dividing time was 3 pm not 9 pm. |
|||
Please note that descriptions of those files that are used by the legacy software, and tips, can be found in the Help file within the Cumulus 1 software, it can be read at [https://cumuluswiki.org/Downloads/Cumulus.chm Cumulus.chm] although that help file is formatted to only open if your computer is using a Microsoft Windows Operating System. |
|||
Steve Loft would respond he had made many design decisions that were not ideal, but the software was based on his own needs. If people wanted drastic changes, they should do what he had done, and write software that did what suited them. |
|||
==Cumulus MX users should read this== |
|||
Yet it appears he did plan to implement this suggestion in MX, see [[Cumulusmx.db#Releases 3.0.0 to 3.11.4 inclusive]]. |
|||
This sub-category was created specifically for Cumulus MX. It provides links to pages that give explanations for all of the Log files with a '''.txt''' extension. If you have any files with such an extension that were created by the legacy software: |
|||
# MX software normally ignores any data for dates before the first use of MX, please see [[Migrating_from_Cumulus_1_to_MX#Start_date]] for further advice |
|||
# MX software creates more fields in each line of each of the files, than the fields created by Cumulus 1, please see [[Calculate_Missing_Values#CreateMissing.exe]] for further advice |
|||
Revision as of 05:55, 9 April 2022
Cumulus software uses a number of files with a .txt extension that can be described as being in comma separated value format (CSV).
Cumulus normally simply appends new lines onto the end of these files, although there is some functionality to read them for plotting graphs and for displaying/editing extreme records.
None of the files hold a header line, although the appropriate header labelling the fields can be found in a different file. (Links to those files included in list below).
Field separator
Although these files are described as CSV format, that does not mean that the character separating one field from the next field has to be ",". In some locales, the comma character might separate the integer and decimal parts of a number, so the field separator character might be a semi-colon ";".
Editing these files
The files linked below all use plain text, so that they can be edited outside Cumulus, but there are rules about content, so be very careful about editing outside Cumulus. A "CSV editor", or a simple text editor designed for programmers, is safer to use than a spreadsheet (if you use a spreadsheet, ensure all columns are set to be "text", don't let it recognise dates, times or numbers), and you must never use a word-processor.
You may wish to edit into a data file a header line, to make life easier when editing values, but remember to remove that header line before saving the file.
The line terminator for file lines depends on the device running MX. In Microsoft Windows Operating System each line is terminated by a pair of characters (CR and LF), a UNIX-derived system (like Linux) uses just LF to end a line, and some versions of Mac Operating System use just CR to terminate each line.
If you move a file between different devices, beware the line endings might get changed (although an editor designed for programmers includes ability to select line ending).
Recent releases of MX are coded to recognise which line ending was previously used, and will maintain that even if you run an executable (CumulusMX.exe, CreateMissing.exe, or ExportToMySQL.exe) on another device.
Note for anyone using the Legacy Cumulus 1
This Wiki page was created as part of documenting Cumulus MX, and is not therefore written for you.
For Legacy Cumulus 1, only a subset of the files listed here are used. Please be aware that within those files that are used the content differs between MX and the legacy software.
Please note that descriptions of those files that are used by the legacy software, and tips, can be found in the Help file within the Cumulus 1 software, it can be read at Cumulus.chm although that help file is formatted to only open if your computer is using a Microsoft Windows Operating System.
Cumulus MX users should read this
This sub-category was created specifically for Cumulus MX. It provides links to pages that give explanations for all of the Log files with a .txt extension. If you have any files with such an extension that were created by the legacy software:
- MX software normally ignores any data for dates before the first use of MX, please see Migrating_from_Cumulus_1_to_MX#Start_date for further advice
- MX software creates more fields in each line of each of the files, than the fields created by Cumulus 1, please see Calculate_Missing_Values#CreateMissing.exe for further advice
