Author Topic: Editing weather data  (Read 72 times)

iccb

  • Gentle Breeze
  • **
  • Posts: 52
    • Säätilanne Eurassa
  • Station Details: Mac Mini (late 2012), Davis Vantage Pro2
Editing weather data
« on: January 06, 2019, 05:17:46 PM »
Hi all!

I had last year problems with Davis wind sensor giving way too much wind (over 50m/s). There values are in data and records. What is the best way to correct those values?

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 5061
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011), macOS 10.13.6, WeatherCat 3
Data files can be edited. (Re: Editing weather data)
« Reply #1 on: January 06, 2019, 10:49:11 PM »
Dear iccb and WeatherCat "weather editors," . . .

I had last year problems with Davis wind sensor giving way too much wind (over 50m/s). There values are in data and records. What is the best way to correct those values?

For wind data there is basically no choice but to edit the WeatherCat data files themselves.  Fortunately, they are plain text files.  You'll find them in this folder:

~/Library/WeatherCatData/Location1/2018

The ~ is the UNIX convention for your own home folder.  The files have names like this 1_WeatherCatData.cat and look like this:

Code: [Select]
WeatherCat data file ***If you modify this file, be sure to delete the relevent .hrs file***.
t and V are not optional, all other fields are.
t is the day, hour and minute (2 digits each), T is outside temperature, Ti is internal temperature, T1 to T8 is auxiliary temperatures, D is dew point, Pr is barometric pressure, W is wind speed,
Wd is wind direction, Wc is wind chill, Wg is wind gust, Ph is hourly precipitation, P is total precipitation,
H is outside humidity, Hi is internal humidity, H1 to H8 are auxiliary humidity sensors, S is solar, CO21 to CO24 is CO2
Sm1 to Sm4 is soil moisture, Lw1 to Lw4 is leaf wetness, St1 to St4 is soil temperature,
Lt1 to Lt4 is leaf temperature, U is UV, Pm is monthly precipitation, Py is annual precipitation, Ed is daily ET, Em is monthly ET, Ey is yearly ET, C is current conditions (delimited by double quotes), V is validation.

VERS:3

    0 t:010000 T:8.83 Ti:18.94 D:7.12 Pr:1020.7551 W:0.00 Wd:328 Wc:8.83 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:2.07 Sy2:-3.41 Sy3:1.08 Sy4:739.00 Sy5:-7.61 C:"N/A" V:4
    1 t:010001 T:8.78 Ti:18.94 D:7.06 Pr:1020.7551 W:0.00 Wd:328 Wc:8.78 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.76 Sy2:-0.00 Sy3:1.08 Sy4:739.00 Sy5:-7.61 C:"N/A" V:4
    3 t:010003 T:8.78 Ti:18.94 D:7.06 Pr:1020.8228 W:0.00 Wd:328 Wc:8.78 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.76 Sy2:-0.00 Sy3:1.08 Sy4:739.00 Sy5:-7.61 C:"N/A" V:4
    5 t:010005 T:8.78 Ti:18.94 D:7.06 Pr:1020.8228 W:0.00 Wd:328 Wc:8.78 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.76 Sy2:-0.00 Sy3:1.08 Sy4:739.00 Sy5:-7.61 C:"N/A" V:4
    6 t:010006 T:8.78 Ti:19.00 D:7.06 Pr:1020.7551 W:0.00 Wd:328 Wc:8.78 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.76 Sy2:-0.00 Sy3:1.08 Sy4:622.00 Sy5:-7.56 C:"N/A" V:4
    8 t:010008 T:8.72 Ti:18.94 D:7.01 Pr:1020.7551 W:0.00 Wd:328 Wc:8.72 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.76 Sy2:-0.00 Sy3:1.08 Sy4:622.00 Sy5:-7.61 C:"N/A" V:4
    9 t:010009 T:8.67 Ti:18.94 D:6.96 Pr:1020.7212 W:0.00 Wd:328 Wc:8.67 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.77 Sy2:-0.00 Sy3:1.08 Sy4:622.00 Sy5:-7.50 C:"N/A" V:4
   10 t:010010 T:8.61 Ti:18.94 D:6.90 Pr:1020.7551 W:0.00 Wd:328 Wc:8.61 Wg:0.00 Ph:0.00 P:0.00 H:89 Hi:59 S:0 U:0.0 T1:8.9 H1:90.0 Lw4:0.0 Pm:0.00 Py:0.00 Ed:0.00 Em:0.00 Ey:0.00 Sy1:1.77 Sy2:-0.00 Sy3:1.08 Sy4:622.00 Sy5:-7.44 C:"N/A" V:4
. . . .

At the top are comments that should guide you in finding the errant data.  As noted in the comments, you must delete the associated  .hrs file to force WeatherCat to recreate the index file for the data file.

Caution is advised for this sort of exercise.  My recommendation is that first:

  • Quit WeatherCat
  • Make copies of the data files that you suspect have the errant data.
  • Start WeatherCat back up so you don't lose too much real-time collection.

You can then go through the copies of the data at your leisure.  You can use Apple's TextEdit to edit the files.  However, you might be better served to use a dedicated editor like Text Wrangler.  When you are satisfied that you corrected all the errors, repeat the general procedure:

  • Quit WeatherCat
  • Copy your edited files to replace the ones that have bad data in them
  • Delete the corresponding .hrs files
  • Now you can restart WeatherCat with all data corrected

Let us know if this procedure does the trick for you.

Cheers, Edouard

iccb

  • Gentle Breeze
  • **
  • Posts: 52
    • Säätilanne Eurassa
  • Station Details: Mac Mini (late 2012), Davis Vantage Pro2
Re: Editing weather data
« Reply #2 on: January 07, 2019, 06:00:35 PM »
Thanks for your helpfull post again Edouard!  [snow]

I will try to correct them that way!  ThU32:-)

Blicj11

  • Storm
  • *****
  • Posts: 3083
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP™ Data Logger | iMac (late 2013), 3.5 GHz Intel Core i7, 24 GB RAM, macOS Mojave | Sharx SCNC2900 Webcam | Supportive Wife
Re: Editing weather data
« Reply #3 on: January 07, 2019, 06:26:50 PM »
Iccb

I have done exactly as Edouard outlines and it work perfectly. But, please be careful and only edit what you intend. I used BBedit because TextWrangler is no longer supported. They are from the same developer, and you can get a free 30-day eval of BBEdit, after which, if you do not want to pay for it, its editing functions will still work just fine. BBEdit is much easier for me to make sure I don't make an editing mistake than Apple's TextEdit, although either will work. BBEdit will let you open up your original file in one pane and your file you are editing in a second pane, so you can compare what you are doing.

Let us know how this project turns out.
Blick