Author Topic: Adding missing data  (Read 860 times)

Alan Rowley

  • Strong Breeze
  • ***
  • Posts: 147
  • It is far better to wear out than to rust away.
    • D3696
    • INOTTING15
    • Selston Weather
  • Station Details: Weather station: Davis Vantage Pro2. Software: WeatherCat. Computer: Mac Mini, High Sierra OS
Adding missing data
« on: August 02, 2015, 09:34:29 AM »
I have almost six years of data created with Weather Display. Now I know there isn't an importer available, but could I manually enter data for the times when an important event occurred? I'm thinking mainly of all-time records.

Recently (July 1st) we had out hottest day since I started recording in 2009, but my all-time record doesn't pick this up - I only started using WeatherCat on July 13th. Could I go to the correct time in the data file editor and add the temperature high manually?

What effect would this have on other reports? Would it screw up any averages and means? Am I wasting my time?

ALAN.

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 5006
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011), macOS 10.13.6, WeatherCat 3
Don't give up hope. (Re: Adding missing data)
« Reply #1 on: August 03, 2015, 12:00:18 AM »
Dear Alan, and WeatherCat data collectors,

I have almost six years of data created with Weather Display. Now I know there isn't an importer available, but could I manually enter data for the times when an important event occurred? I'm thinking mainly of all-time records.

Let me suggest that you instead see if you could somehow figure out your own way to import your existing data into WeatherCat.  There is no need to rush into this project and I don't think it is as hard as you might imagine.

Take a look at the files with the title like this: 1_WeatherCatData.cat that you should find in the folder:

~/Library/WeatherCatData/Location1/2015

WeatherCat stores its data as plain text files that look like this:

Code: [Select]
WeatherCat data file ***If you modify this file, be sure to delete the relevent .hrs file***.
t and V are not optional, all other fields are.
t is the day, hour and minute (2 digits each), T is outside temperature, Ti is internal temperature, T1 to T8 is auxiliary temperatures, D is dew point, Pr is barometric pressure, W is wind speed,
Wd is wind direction, Wc is wind chill, Wg is wind gust, Ph is hourly precipitation, P is total precipitation,
H is outside humidity, Hi is internal humidity, H1 to H8 are auxiliary humidity sensors, S is solar,
Sm1 to Sm4 is soil moisture, Lw1 to Lw4 is leaf wetness, St1 to St4 is soil temperature,
Lt1 to Lt4 is leaf temperature, U is UV, Pm is monthly precipitation, Py is annual precipitation, Ed is daily ET, Em is monthly ET, Ey is yearly ET, C is current conditions (delimited by double quotes), V is validation.

VERS:3

    0 t:010000 T:16.44 Ti:23.22 D:13.91 Pr:1013.7453 W:1.61 Wd:142 Wc:16.44 Wg:8.05 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:3.55 Sy2:-0.00 Sy3:24.81 Sy4:303.00 Sy5:-11.00 C:"N/A" V:4
    4 t:010004 T:16.44 Ti:23.22 D:13.91 Pr:1013.8130 W:1.61 Wd:139 Wc:16.44 Wg:8.05 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.81 Sy4:303.00 Sy5:-11.00 C:"N/A" V:4
    5 t:010005 T:16.44 Ti:23.22 D:13.91 Pr:1013.8130 W:1.61 Wd:126 Wc:16.44 Wg:0.00 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.81 Sy4:366.00 Sy5:-11.00 C:"N/A" V:4
    6 t:010006 T:16.44 Ti:23.28 D:13.91 Pr:1013.7453 W:1.61 Wd:119 Wc:16.44 Wg:0.00 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.83 Sy4:366.00 Sy5:-10.94 C:"N/A" V:4
    8 t:010008 T:16.44 Ti:23.39 D:13.91 Pr:1013.6776 W:1.61 Wd:111 Wc:16.44 Wg:0.00 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.89 Sy4:366.00 Sy5:-10.83 C:"N/A" V:4
   10 t:010010 T:16.44 Ti:23.56 D:13.91 Pr:1013.6437 W:0.00 Wd:104 Wc:16.44 Wg:0.00 Ph:0.00 P:0.00 H:85 Hi:60 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:25.01 Sy4:366.00 Sy5:-10.67 C:"N/A" V:4
   11 t:010011 T:16.44 Ti:23.56 D:13.91 Pr:1013.6776 W:0.00 Wd:91 Wc:16.44 Wg:0.00 Ph:0.00 P:0.00 H:85 Hi:60 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:25.01 Sy4:366.00 Sy5:-10.67 C:"N/A" V:4
   12 t:010012 T:16.44 Ti:23.50 D:13.91 Pr:1013.6776 W:0.00 Wd:70 Wc:16.44 Wg:3.22 Ph:0.00 P:0.00 H:85 Hi:60 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.98 Sy4:366.00 Sy5:-10.72 C:"N/A" V:4
   13 t:010013 T:16.39 Ti:23.50 D:13.86 Pr:1013.7114 W:0.00 Wd:75 Wc:16.39 Wg:1.61 Ph:0.00 P:0.00 H:85 Hi:60 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.98 Sy4:366.00 Sy5:-10.67 C:"N/A" V:4
   15 t:010015 T:16.39 Ti:23.44 D:13.86 Pr:1013.7453 W:0.00 Wd:100 Wc:16.39 Wg:1.61 Ph:0.00 P:0.00 H:85 Hi:60 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.96 Sy4:366.00 Sy5:-10.72 C:"N/A" V:4
   19 t:010019 T:16.39 Ti:23.39 D:13.86 Pr:1013.8130 W:0.00 Wd:99 Wc:16.39 Wg:1.61 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.7 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.89 Sy4:366.00 Sy5:-10.78 C:"N/A" V:4
   20 t:010020 T:16.39 Ti:23.39 D:13.86 Pr:1013.7453 W:0.00 Wd:101 Wc:16.39 Wg:4.83 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.1 H1:86.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.89 Sy4:367.00 Sy5:-10.78 C:"N/A" V:4
   25 t:010025 T:16.33 Ti:23.33 D:13.98 Pr:1013.7114 W:1.61 Wd:100 Wc:16.33 Wg:8.05 Ph:0.00 P:0.00 H:86 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.48 Sy2:-0.00 Sy3:24.86 Sy4:365.00 Sy5:-10.83 C:"N/A" V:4
   26 t:010026 T:16.39 Ti:23.33 D:14.04 Pr:1013.6776 W:1.61 Wd:106 Wc:16.39 Wg:3.22 Ph:0.00 P:0.00 H:86 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.49 Sy2:-0.00 Sy3:24.86 Sy4:365.00 Sy5:-10.83 C:"N/A" V:4
   27 t:010027 T:16.33 Ti:23.33 D:13.80 Pr:1013.6437 W:1.61 Wd:109 Wc:16.33 Wg:3.22 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.49 Sy2:-0.00 Sy3:24.86 Sy4:365.00 Sy5:-10.78 C:"N/A" V:4
   28 t:010028 T:16.39 Ti:23.33 D:14.04 Pr:1013.6098 W:1.61 Wd:109 Wc:16.39 Wg:0.00 Ph:0.00 P:0.00 H:86 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.49 Sy2:-0.00 Sy3:24.86 Sy4:365.00 Sy5:-10.83 C:"N/A" V:4
   29 t:010029 T:16.33 Ti:23.33 D:13.98 Pr:1013.6776 W:1.61 Wd:123 Wc:16.33 Wg:12.87 Ph:0.00 P:0.00 H:86 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.49 Sy2:-0.00 Sy3:24.86 Sy4:365.00 Sy5:-10.78 C:"N/A" V:4
   30 t:010030 T:16.33 Ti:23.33 D:13.80 Pr:1013.6098 W:1.61 Wd:127 Wc:16.33 Wg:3.22 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.49 Sy2:-0.00 Sy3:24.86 Sy4:366.00 Sy5:-10.78 C:"N/A" V:4
   31 t:010031 T:16.33 Ti:23.33 D:13.80 Pr:1013.6098 W:1.61 Wd:132 Wc:16.33 Wg:1.61 Ph:0.00 P:0.00 H:85 Hi:61 S:0 U:0.0 T1:16.1 H1:87.0 Lw4:0.0 Pm:0.00 Py:25.15 Ed:0.00 Em:0.00 Ey:341.63 Sy1:2.49 Sy2:-0.00 Sy3:24.86 Sy4:366.00 Sy5:-10.78 C:"N/A" V:4
.
.
.

So you would need to convert Weather Display data into a form like this.  It might not be that hard perhaps using some help like a spreadsheet program.  If you a bit about UNIX text crunching command-line utilities that also might help a lot.  The key would be to export from Weather Display the data in some sort of a text format.  Then you can convert that format into a form that WeatherCat can use.

If that seems too much work, you might just join the chorus of people who would like WeatherCat to have this feature.  There was just such a feature request a few years back:

http://athena.trixology.com/index.php?topic=159.msg1580#msg1580

WeatherCat's evolutionary ancestor had some data import abilities, perhaps you can encourage Stu to bring this capability back.

Cheers, Edouard

Alan Rowley

  • Strong Breeze
  • ***
  • Posts: 147
  • It is far better to wear out than to rust away.
    • D3696
    • INOTTING15
    • Selston Weather
  • Station Details: Weather station: Davis Vantage Pro2. Software: WeatherCat. Computer: Mac Mini, High Sierra OS
Re: Adding missing data
« Reply #2 on: August 03, 2015, 06:41:23 AM »
The original log files are buried deep in a Windows 7 folder within a Parallels Virtual Machine disk on a backup folder on my Mac Mini. The original MacBook Pro that ran the weather station has gone to the great Apple store in the sky (R.I.P.). I could retrieve the data files, but it isn't going to be a quick job.

Then, I would have to figure out a way to change them to WeatherCat format. Not a job that I would relish. Maybe a job for a cold winter's day.

I would certainly add my voice to some form of importer.