Data problems

[David Lewis]

Hi all


Just noticed that for the last few weeks i've been getting data anomalies. Sudden rises in temperature or rainfall, then reverting back to correct values for a while. This has resulted in weird graphs on my webpage. http://dtwrlewis.com/
Yet the data exported to Wunderground doesn't seem to have the same problem. http://www.wunderground.com/personal-weather-station/dashboard?ID=INEWBRUN36#history


any ideas?


I tried updating to latest weather cat, disconnecting usb drivers then reconnecting. My Vantage view battery is fine, zero comms errors, zero sensor errors.


I've attached todays data export.


I'm running OS X Yosemite


Thanks for any help.


David

[LesCimes]

I looked at your graphs. Couldn't come up with any idea on the anomaly. Odd that the graphs are correct on one site but incorrect on another. But, if WU isn't showing anomalies, then surely your ISS sensors are fine and the problem is associated with the web side of things.

[elagache]

Dear David and WeatherCat troubleshooters,

Just noticed that for the last few weeks i've been getting data anomalies. Sudden rises in temperature or rainfall, then reverting back to correct values for a while. This has resulted in weird graphs on my webpage. http://dtwrlewis.com/
Yet the data exported to Wunderground doesn't seem to have the same problem. http://www.wunderground.com/personal-weather-station/dashboard?ID=INEWBRUN36#history

I just took a look at your data, and I am noticing "blips" that look like unreasonably high temperatures in between the cold actual data.  The numbers seem consistent with that I see when my Weather Envoy loses contact with my Davis console.  When that happens, WeatherCat reports sensor errors and temperature values end up at something resembling the values you are reporting.

Have you looked at the WeatherCat log or the console log to see if there are any messages reporting problem?  Do you have WeatherCat setup to send you diagnostics via email?

Let us know if you find any addition information that might help us diagnose your problem.

Edouard

[David Lewis]

Thanks Les and Eduaord


Yes it seems that Weathercat is generating these anomalies, i also suspect that its  a comms error, but cant seem to see it reported. I will check the log. How do I set up diagnostics via email?


Thanks


David

[David Lewis]

Well this is strange. At midnight last night, it seemed to reset itself and is now recording the correct data values. I have done nothing too it since my original email. So whether the weathercat upgrade fixed it at the change of the day, i have no idea?


That now just leaves me the problem of cleaning the historical data - about 2 weeks worth. Any suggestions?


David

[WCDev]

Sounds like it might have been a corrupted data file - depending on how much data you have in your logger, try restoring your WeatherCatData  folder from a backup - either from Within WeatherCat itself or from TimeMachine. When WeatherCat starts up, it'll download the missing data from your logger (assuming the data there is good).

Be sure to take a copy of your current WeatherCatData folder first, just in case.

[David Lewis]

Thanks Stu


Unfortunately I had my data logger set to 1 min sampling, so only had a day of data there.


Is there a way to access the data in a spreadsheet or some other way to be able to delete whole lines of data (temp, rain, etc) for any given time?


Thanks


David

[WCDev]

Hi David,
You can open the data file in a (real) text editor, such as the free TextWrangler* and hand edit it - each entry is a single line so it's pretty easy to delete stuff (see the section WeatherCat and data in the manual), but if the file is corrupt you may have problems, particularly if there's cross linked sectors etc.

Before doing anything run Disk Utility across the disk, then take a copy of your WeatherCatData folder.

*http://www.barebones.com/products/textwrangler/

[Blicj11]

David:

I agree with Stu: Text Wrangler is free and quite good. I have used it for a couple of years and really like it.

[wurzelmac]

David:

I agree with Stu: Text Wrangler is free and quite good. I have used it for a couple of years and really like it.

Me, too!
I do my whole website with TextWrangler. 

Cheers,
Reinhard

[David Lewis]

This is getting more and more curious.


I followed Stu's advice. But could not find any of the anomalies in the data. See link to novemeber's data:
https://www.dropbox.com/s/tlkpl4mtnt09dvh/11_WeatherCatData.cat?dl=0


This is the current datafile from library/weathercatdata, after I had restored from a backup from 1st December.


Just look at the graphs that this data is generating !


It doesn't seem to match ?


Thanks


David

[elagache]

Dear David and WeatherCat troubleshooters,

I followed Stu's advice. But could not find any of the anomalies in the data. See link to novemeber's data:

I just made a quick cruise through that data and I don't see anything either, but I didn't have time to check it remotely carefully.

This is the current datafile from library/weathercatdata, after I had restored from a backup from 1st December.

Could you post the December data file just in case we see something in that?

Sorry, I don't have any more constructive suggestions at this moment.  Let us know if you spot anything.

Edouard

[David Lewis]

Here is current December data file


https://www.dropbox.com/s/3w80f0qhelypmrz/12_WeatherCatData.cat?dl=0

[David Lewis]

The December data looks good too.


These anomalies have affected my stats for November. http://dtwrlewis.com/


The problem seems to start on November 8th. The interpretation of the data corrects itself at midnight on 30th November.


So if the data itself is fine, why is weathercat interpreting it wrongly for that period?


David

[WCDev]

Hello David,
Thanks for the file - I'll take a look at it and get back to you.