Author Topic: Imported Data  (Read 1305 times)

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Imported Data
« on: January 22, 2023, 07:41:41 AM »
I have recently bought and installed WeatherCat, having been a long term Davis Weatherlink user.

I have converted and imported 27 years of data using the WlinkToWcat program.  The data conversion went well and the data imported to WeatherCat without problems.  The data looks fine when I view it in the WeatherCat Data Viewer, but when I report on the historical data using the inbuilt NOAA and other reports, there are significant errors in the figures.  This is particularly so with respect to historical rainfall which seems to be consistently under reporting.

I am having no problems connecting to my Davis VantageVue weather station and the reporting on data dowloaded from the station is correct.

Is anyone able to suggest how I might be able to overcome the issue with reporting on historical imported data?

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6499
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Errors actually in data or just reports? (Re: Imported Data)
« Reply #1 on: January 22, 2023, 10:50:24 PM »
Dear IanW and WeatherCat faithful,

I have recently bought and installed WeatherCat, having been a long term Davis Weatherlink user.

Glad to have another WeatherCat user and welcome to the WeatherCat forum!

I have converted and imported 27 years of data using the WlinkToWcat program.  The data conversion went well and the data imported to WeatherCat without problems.  The data looks fine when I view it in the WeatherCat Data Viewer, but when I report on the historical data using the inbuilt NOAA and other reports, there are significant errors in the figures.  This is particularly so with respect to historical rainfall which seems to be consistently under reporting.

Sorry to hear that you are observing this problem.  Can you figure out if the data actually wasn't imported correctly or if there is something wrong in the WeatherCat reports?  You can have a look at the actual data with a text editor like TextEdit.  WeatherCat stores it data in files that are located in this directory:

~/Library/WeatherCatData/Location1

The file path is in UNIX convention and the ~ stands for your home directory.  In that folder you will find subfolders that are named for the year of the data.  In each of those year folders you will find 12 pairs of files that have names like this: 1_WeatherCatData.cat  These are text files that you can look at.  There is a header that describes the format of the data.

Be careful not to modify any of these files.  The safest thing to do is to make copies and only use the editor on the copies.  You should be able to compare the data in these files to what your station console is reporting.  To make it easy, try to find a instance where the data you are looking for is obvious.  For example, perhaps pick a day where there was only a brief rain shower.

It would be of great help to ascertain if the error is a result of the data importing or something else.

Let us know what you discover.

Cheers, Edouard

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #2 on: January 23, 2023, 12:05:53 AM »
Hi Edouard

Thanks for your reply

The data in the WeatherCatData.cat files appears to be correct when I view it in TextEdit.  It is also correct when I look at it in the WeatherCat Data Viewer/Editor in Tools.  It is only the reporting that is giving incorrect figures.

It is very strange as the reporting from data downloaded direct from my weather station since I installed WeatherCat is correct.

Any ideas?

Ian

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #3 on: January 23, 2023, 10:42:08 PM »
Hi Edouard

Further to my post yesterday, I have been examining the data further.

Nearly every day with large rainfall totals (>20mm) are under reporting. Most days with small rainfall totals are reporting correctly, but I have found the odd day with small rainfall that is reporting incorrectly.

The reporting for the only day with a large rainfall total which has been downloaded directly from my weather station is correct.

Regards

Ian

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6499
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Could be an error in the report generators only. (Re: Imported Data)
« Reply #4 on: January 23, 2023, 11:07:31 PM »
Dear Ian and WeatherCat faithful,

The data in the WeatherCatData.cat files appears to be correct when I view it in TextEdit.  It is also correct when I look at it in the WeatherCat Data Viewer/Editor in Tools.  It is only the reporting that is giving incorrect figures.

It is very strange as the reporting from data downloaded direct from my weather station since I installed WeatherCat is correct.

It is possible that you have successfully imported your data, but there is some sort of issue in the WeatherCat report generators.  There were some reports of errors in the reports when the generators were first included.  I don't know what is the status of those errors. 

Nearly every day with large rainfall totals (>20mm) are under reporting. Most days with small rainfall totals are reporting correctly, but I have found the odd day with small rainfall that is reporting incorrectly.

I will look at my own WeatherCat installation to see if I can reproduce this error.  Hopefully other WeatherCat users will do the same.  If you can spot any other anomalies please let us know.  If we can reproduce these errors, then we can document them as a bug in need of correction.

In the meantime, you can use the other functionality of WeatherCat knowing that you have successfully imported your many years of collected data.

Cheers, Edouard

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #5 on: February 16, 2023, 04:23:10 AM »
Hi Edouard

I am now having the same problem with rainfall data downloaded from my weather station.  Two days ago we had some very intense rainfall over a one hour period.  The data is fine in the Data Viewer window and I can plot a custom graph which shows the correct figures.  However, the NOAA reports for the day in question show a rainfall figure less than half of the actual total.

Have you had any success replicating the issue?

Regards

Ian W

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6499
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Anyone else seeing this? (Re: Imported Data)
« Reply #6 on: February 16, 2023, 10:37:50 PM »
Dear IanW and WeatherCat troubleshooters,

I am now having the same problem with rainfall data downloaded from my weather station.  Two days ago we had some very intense rainfall over a one hour period.  The data is fine in the Data Viewer window and I can plot a custom graph which shows the correct figures.  However, the NOAA reports for the day in question show a rainfall figure less than half of the actual total.

Have you had any success replicating the issue?

I did try to replicate this but was unsuccessful.  Has anyone else observed this?  When the NOAA reports feature was first introduced, I call that there some reporting bugs identified, but I don't know if they were resolved or not.  You might try searching this forum to see if can find more details about it.

You might try contacting WeatherCat technical support directly.  The contact information is on this page of the main trixology website:

https://trixology.com/contact-us/

Unfortunately, Stuart Ball, who is the WeatherCat developer, has been overwhelmed by commitments associated with his day-job for some time now.   If you choose to contact him directly, you will need to be patient.

Sorry . . . unfortunately, such are the conditions that prevail.

Edouard

Blicj11

  • Storm
  • *****
  • Posts: 3951
    • EW3808
    • KUTHEBER6
    • Timber Lakes Weather
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP Data Logger | iMac (2019), 3.6 GHz Intel Core i9, 40 GB RAM, macOS Ventura 13.6 | Sharx SCNC2900 Webcam | WeatherCat 3.3 | Supportive Wife
Re: Imported Data
« Reply #7 on: February 17, 2023, 05:46:21 PM »
I have noticed some anomalies in the reporting of my data with regards to rainfall amounts. One example is when it is raining during the last hour of the day, continuing into the first hour of the next day, the precipitation reports for the first day show different amounts depending on which report I am viewing. I have reported this to Stu.
Blick


IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #8 on: February 19, 2023, 05:48:33 AM »
Thanks Edouard

This is a real deal breaker for me. 

Is there any other way to report on historical data other than the NOAA reports?  I can't find anything in Tools.

I tried to contact Weather Cat support in early January but have never had a reply.

Blicj11 - I have noticed the same thing as well where rain after, say, 11.30 pm gets reported in the following day.  This is a bit annoying but at least the totals are correct.

Ian W

xairbusdriver

  • Storm
  • *****
  • Posts: 3128
Re: Imported Data
« Reply #9 on: February 19, 2023, 09:04:12 PM »
How much historical data do you want to report and in what format? I've never used the NOAA reports for anything, but I don't enjoy looking at spreadsheets. I do use WC to record and report (Current and Previous) data for Daily, Weekly, Monthly, Yearly and data from the date my system was installed.
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #10 on: February 20, 2023, 04:41:53 AM »
How do you do your reporting?  Do you export the data?

Regards

Ian W

xairbusdriver

  • Storm
  • *****
  • Posts: 3128
Re: Imported Data
« Reply #11 on: February 20, 2023, 02:47:11 PM »
Actually, WC does all the "work". All I did was use the built-in Web-Tags on a web page. Descriptions start on page 196 of the Manual and runs through ~18 pages. I'll gladly send you a couple of html files and you can compare them with what shows in a web browser. The layout/style/frequency of updating/etc. is completely up to you. ThU32:-)
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #12 on: February 26, 2023, 11:17:41 PM »
Thanks airbusdriver.

I will give it a go.  I will let you know if I need the html files.

Regards

Ian W

IanW

  • Calm
  • *
  • Posts: 8
  • Station Details: Davis VantageVue
Re: Imported Data
« Reply #13 on: March 25, 2023, 07:40:10 AM »
Hi xairbusdriver

The stuff in the manual on page 196 is all gobbledygook to me.  I haven't got a clue where to start.

All I want to do is produce accurate historical rainfall reports  :'(

Ian W