Author Topic: Editing Historical Data  (Read 369 times)

dgwilcox

  • Calm
  • *
  • Posts: 2
Editing Historical Data
« on: February 10, 2024, 07:25:14 PM »
With the recent storm we had here in Northern CA, the rain collector funnel of my (circa 2016) Davis Vantage Pro 2 was damaged by wind debris and completely flew off the station. As a result of the wind, the rain collector was exposed and was toggling back and forth all night - to the total of 46 in of rain being recorded - in one night! Now my monthly and annual totals are completely skewed.

Due to the time that my weather station recorded "rain," WeatherCat's Data View/Editor is just too tedious to completely solve this - tried, but not willing to go through almost a thousand entries manually (over multiple days) - plus the Daily Rain column isn't editable anyway. Also, looking at the forums, it appears some data data is still hardwired in the Vantage Pro console itself.

I was able to reset the console (but lost data) and after hours of trying to go through this process, just decided to restore WeatherCat's data from the time before the storm.

So:
1. Vantage Pro console's data has been reset (lost data)
2. WeatherCat was restored from an earlier backup (lost data)

But how can I edit the historical data that WeatherCat keeps, since when that process finishes after restarting, my monthly and yearly totals return to be off-the-scales.

Any insight is appreciated,

- Dean 

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Can be done but not for the timid. (Re: Editing Historical Data)
« Reply #1 on: February 10, 2024, 09:25:12 PM »
Dear Dean and WeatherCat station caregivers,

With the recent storm we had here in Northern CA, the rain collector funnel of my (circa 2016) Davis Vantage Pro 2 was damaged by wind debris and completely flew off the station. As a result of the wind, the rain collector was exposed and was toggling back and forth all night - to the total of 46 in of rain being recorded - in one night! Now my monthly and annual totals are completely skewed.

Sorry to hear about your station being damaged by the recent aggressive storms.  The last California atmospheric river event included some very strong winds indeed.

Due to the time that my weather station recorded "rain," WeatherCat's Data View/Editor is just too tedious to completely solve this - tried, but not willing to go through almost a thousand entries manually (over multiple days) - plus the Daily Rain column isn't editable anyway. Also, looking at the forums, it appears some data data is still hardwired in the Vantage Pro console itself.

I was able to reset the console (but lost data) and after hours of trying to go through this process, just decided to restore WeatherCat's data from the time before the storm.

So:
1. Vantage Pro console's data has been reset (lost data)
2. WeatherCat was restored from an earlier backup (lost data)

But how can I edit the historical data that WeatherCat keeps, since when that process finishes after restarting, my monthly and yearly totals return to be off-the-scales.

It is possible to edit the data files themselves, but it isn't something for someone who isn't too comfortable with software-related tinkering.  You will find the data files at this location on your hard drive:

~/Library/WeatherCatData/Location1/2024

The ~ refers to your home directory and the path is in UNIX format.  There you will find files with names like this:

2_WeatherCatData.cat

and

2_WeatherCatData.cat.hrs

The first file actually contains the weather data and the second in an index file.  You can edit manually the 2_WeatherCatData.cat using any plain text editor.  Here is the header from that file for my installation of WeatherCat:

Code: [Select]
WeatherCat data file ***If you modify this file, be sure to delete the relevent .hrs file***.
t and V are not optional, all other fields are.
t is the day, hour and minute (2 digits each), T is outside temperature, Ti is internal temperature,                               T1 to T8 is auxiliary temperatures, D is dew point,                               Pr is barometric pressure, W is wind speed,
Wd is wind direction, Wc is wind chill, Wg is wind gust, Ph is hourly precipitation, P is total precipitation,
                              H is outside humidity, Hi is internal humidity, H1 to H8 are auxiliary humidity sensors, S is solar, CO21 to CO24 is CO2
Sm1 to Sm4 is soil moisture, Lw1 to Lw4 is leaf wetness, St1 to St4 is soil temperature,
                              Lt1 to Lt4 is leaf temperature, U is UV, Pm is monthly precipitation, Py is annual precipitation, Ed is daily ET, Em is monthly ET, Ey is yearly ET, C is current conditions (delimited by double quotes), V is validation.

VERS:3

    0 t:010000 T:11.39 Ti:19.89 D:9.97 Pr:1001.4189 W:1.61 Wd:134 Wc:11.39 Wg:1.61 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:2.29 Sy2:5.77 Sy3:96.91 Sy4:14.33 Sy5:-9.28 C:"N/A" V:4
    1 t:010001 T:11.39 Ti:19.89 D:9.97 Pr:1001.4528 W:1.61 Wd:126 Wc:11.39 Wg:1.61 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:96.96 Sy4:14.28 Sy5:-9.33 C:"N/A" V:4
    5 t:010005 T:11.44 Ti:19.89 D:10.03 Pr:1001.4528 W:1.61 Wd:127 Wc:11.44 Wg:3.22 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:97.00 Sy4:14.28 Sy5:-9.28 C:"N/A" V:4
    6 t:010006 T:11.39 Ti:19.89 D:9.97 Pr:1001.4189 W:1.61 Wd:128 Wc:11.39 Wg:3.22 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:97.11 Sy4:14.28 Sy5:-9.28 C:"N/A" V:4
    7 t:010007 T:11.39 Ti:19.89 D:9.97 Pr:1001.4189 W:1.61 Wd:141 Wc:11.39 Wg:6.44 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:97.12 Sy4:14.33 Sy5:-9.33 C:"N/A" V:4
    8 t:010008 T:11.39 Ti:19.89 D:9.97 Pr:1001.4189 W:1.61 Wd:142 Wc:11.39 Wg:6.44 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:97.16 Sy4:14.28 Sy5:-9.28 C:"N/A" V:4
    9 t:010009 T:11.44 Ti:19.89 D:10.03 Pr:1001.4189 W:1.61 Wd:134 Wc:11.44 Wg:6.44 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:97.19 Sy4:14.28 Sy5:-9.28 C:"N/A" V:4
   10 t:010010 T:11.39 Ti:19.94 D:9.97 Pr:1001.4528 W:1.61 Wd:140 Wc:11.39 Wg:3.22 Ph:0.00 P:0.00 H:91 Hi:64 S:0 U:0.0 T1:11.1 H1:96.0 Lw4:0.0 Pm:0.00 Py:146.56 Ed:0.00 Em:0.00 Ey:27.43 Sy1:1.92 Sy2:-0.00 Sy3:97.20 Sy4:14.28 Sy5:-9.22 C:"N/A" V:4


You can edit this as described if you delete the index file before launching WeatherCat once more.  If you are bold enough to try this, I recommend using solid editor like BBEdit.  Make a copy of the file you wish to edit and work on that only.  Do not run WeatherCat while you are making the edits.  Your Davis data logger will have the data while WeatherCat isn't running and WeatherCat will download what is missing after you have completed making the changes.

That is the procedure roughly.  By all means, ask any questions you might have to clarify the situation.

Edouard

dgwilcox

  • Calm
  • *
  • Posts: 2
Re: Editing Historical Data
« Reply #2 on: February 11, 2024, 06:26:56 AM »
Thank you Edouard!

I am familiar with BBEdit (from my old HTML days) and may take a stab at it - paying attention to the particulars of editing pref files!

If I'm successful eliminating the bogus data, I may try to use the Daily Rain Editor to "add" in a more reasonable amount of rain for that storm. But I've yet to try that tool. One step at a time.

Thanks again,

- D