Author Topic: Newbie: Sampling Interval?  (Read 3358 times)

atnbirdie

  • Calm
  • *
  • Posts: 6
    • DW8300
    • KMDMIDDL9
  • Station Details: Davis Wireless Vantage Pro2 w/Envoy & Data Logger; TRENDnet TV-IP110W Camera; iMac OS 10.6.8; WC v1.0.2
Newbie: Sampling Interval?
« on: July 10, 2012, 08:06:44 PM »
I'll be converting from WeatherLink on a PC to WeatherCat on a Mac shortly.  As I peruse the WC manual, I am curious as to how often folks are sampling the data (Preferences/Misc2)?  I intend to set sampling to 'adaptive' so I can sample less frequently to conserve disc space, but am not sure how long to go.  If I have adaptive set, does it make sense to stretch sampling to the max since the program will adjust anyway?

Thanks for any insights you may have on this.
Steve

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4805
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Data files aren't so big. (Re: Sampling Interval?)
« Reply #1 on: July 10, 2012, 09:36:41 PM »
Dear Steve and WeatherCat fans,

I'll be converting from WeatherLink on a PC to WeatherCat on a Mac shortly.  As I peruse the WC manual, I am curious as to how often folks are sampling the data (Preferences/Misc2)?

Unfortunately, I don't know the answer to your second question because of how I responded to the first.  At the time, I was told that the WeatherCat data files weren't that large so why not spurge and collect relatively frequently.  I have my sampling interval set at 5 minutes.  My "WeatherCat" database for 2010 takes up a bit less than 69 Mb.  2011 is more hefty at 92 Mb.  Still since disk drive space is relatively inexpensive, I don't see any reason to be miserly with respect to this sort of thing.

Since you are using WeatherLink, it is safe to assume you have a Davis station.  In this case there is one additional consideration.  You want the Davis data logger to sample at the same rate as WeatherCat so that you get continuous data even if your computer is down.  Somewhere we have a way to calculate exactly how many hours can be stored in a standard Davis data logger.  However, I've gone well beyond 12 hours without losing any data using a 5 minute sampling interval on my Davis logger.  Unless you hope to leave your station without your computer connected for extended periods of time, setting the Davis data logger to 5 minutes will also work fine.

Of course it will depend on where you are at and the sort of weather you encounter there.  Hopefully other folks will provide additional perspective on this question.

Feel free to ask additional questions if anything is unclear.

Cheers, Edouard  [cheers1]

Steve

  • Global Moderator
  • Storm
  • *****
  • Posts: 1263
    • DW8454
    • KOHAVON11
    • Avon Weather
  • Station Details: Davis Wireless VP2 Plus w/24 hr FARS, Leaf & Soil Station, 2017 27" iMac 3.8 GHz i5, 40GB RAM, Mac OS 10.13, WeatherCat 3
Re: Newbie: Sampling Interval?
« Reply #2 on: July 10, 2012, 10:33:00 PM »
Hi Steve, and welcome to the WeatherCat forum!

I convinced Steve to give WeatherCat a try when he was looking for a way to get his webcam images onto Weather Underground. I first suggested EvoCam, but then suggested dumping WL altogether and joining us here!

One thing to try before you jump in whole hog, Steve. If your data logger is from a WeatherLink for PC give it a try with LWC or WeatherCat first. I think it works with both, but I'm not absolutely positive. You will also need to install the latest driver from Silicon Labs here: http://www.silabs.com/products/mcu/Pages/USBtoUARTBridgeVCPDrivers.aspx

Welcome to the asylum,
Steve
Steve - Avon, Ohio, USA


CWOP: DW8454 • WU: KOHAVON11 • AWEKAS • CoCoRaHS • Skywarn •  UK Met Office
MWWN •PWSweather • WeatherBug • WeatherCloud • Twitter • Facebook

atnbirdie

  • Calm
  • *
  • Posts: 6
    • DW8300
    • KMDMIDDL9
  • Station Details: Davis Wireless Vantage Pro2 w/Envoy & Data Logger; TRENDnet TV-IP110W Camera; iMac OS 10.6.8; WC v1.0.2
Re: Newbie: Sampling Interval?
« Reply #3 on: July 11, 2012, 04:41:32 PM »
Edouard,
Thanks for that perspective.  I had just read what the capacity was of the logger the other day.  Here it is:

The WeatherLink Data Logger can store up to 2560 archive records (one 52-byte record per archive interval) for later transfer to a computer. Archive records are stored in 128K of non-volatile memory; protecting the data even if the console loses power. Maxima, minima, averages, and totals are taken over the archive interval, which is user-selectable in increments of 1, 5, 10, 15, 30, 60, or 120 minutes. The data logger archive storage capacity (the amount of time before the archive is completely filled) is as follows:

1 Minute Archive Interval - 42 hours
5 Minute Archive Interval - 8 days
10 Minute Archive Interval - 17 days
15 Minute Archive Interval - 26 days
30 Minute Archive Interval - 53 days
60 Minute Archive Interval - 106 days
120 Minute Archive Interval - 213 days

Looks like you have an 8 day buffer at a 5 minute archiving interval.

Steve,
I'm sure my logger will work as I purchased it with the WeatherLink for Mac software (it's a model 6520).   So I just need to return to the Silabs site and ensure our iMac gets the latest driver.

Thanks again to all.
Steve B.

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4805
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Thanks for crunching the numbers! (Re: Sampling Interval?)
« Reply #4 on: July 11, 2012, 09:29:41 PM »
Dear Steve and WeatherCat fans,

5 Minute Archive Interval - 8 days

Thanks for going through all the math to figure that out.  I had gotten this info somewhere, but honestly, I run my weather station computer 24/7, so it didn't seem worth too much effort to figure out the duration.  However, back in the Winter of 2010, the house was hit by lightening and I my computer was damaged as well as the link to the weather station.  I made sure to download the data every 12 hours just to be safe.  It is good to know that in an extreme emergency I can go over a week and still not lose any data.

Thanks again!

Cheers, Edouard  [cheers1]

atnbirdie

  • Calm
  • *
  • Posts: 6
    • DW8300
    • KMDMIDDL9
  • Station Details: Davis Wireless Vantage Pro2 w/Envoy & Data Logger; TRENDnet TV-IP110W Camera; iMac OS 10.6.8; WC v1.0.2
Re: Newbie: Sampling Interval?
« Reply #5 on: July 11, 2012, 09:42:52 PM »
LOL!  The only trouble I went through to do get the numbers was to copy and paste from a web site I'd run across.    [computer]   However, the basic logger capacity and an example calculation or two are also included in the WC manual.

I too run my system 24/7 but over this first year of operation have been logging a lot less frequently- 30 minutes.  I'm going to drop that down to 5 for better resolution which as you noted will give me about a week's leeway in case I am away from home in case a situation arises while I'm away.  What I may do is on those occasions when I'll be gone for more than a week, just adjust the logger/sampling rates to cover the period I'll be gone.  If I through a note in my monthly maintenance reminder, that should jog my memory to make the adjustment as such trips arise.

Steve B

The Grand Poohbah

  • Strong Breeze
  • ***
  • Posts: 235
  • Developer of WeatherCat for iOS, tvOS, and watchOS
    • EW6355 KCANEVAD43
    • Hopeful Hill Ranch
  • Station Details: Vantage Pro 2, aspirated, solar radiation, uv, soil temp and moisture
Re: Newbie: Sampling Interval?
« Reply #6 on: July 12, 2012, 07:19:59 PM »
I'm a bit confused about the differences in sampling between the Davis Console and WeatherCat. My assumption is that the Console and the WeatherCat sample independently. That is, the Console does its sample periodically and the WeatherCat samples the Console's data periodically. The sampling intervals for the Console and WeatherCat may be different. Is this correct?

WCDev

  • WeatherCat Developer
  • Administrator
  • Storm
  • *****
  • Posts: 2783
    • CW9739
    • ISCOTLAN25
    • Trixology
  • Station Details: Main Station: Vantage Pro-2, 24hr fars, solar, soil/leaf station, extra temp stations, no U.V. WeatherLink IP.
Re: Newbie: Sampling Interval?
« Reply #7 on: July 12, 2012, 08:01:15 PM »
I'm a bit confused about the differences in sampling between the Davis Console and WeatherCat. My assumption is that the Console and the WeatherCat sample independently. That is, the Console does its sample periodically and the WeatherCat samples the Console's data periodically. The sampling intervals for the Console and WeatherCat may be different. Is this correct?

Independent - they don't need to match. My logger interval is set to 5 minutes, my WeatherCat sampling rate is 3 minutes, with adaptive enabled.

Adaptive sampling only monitors some primary variables - temperature, humidity, pressure and rain - if any of these change, WeatherCat will record a sample irrespective of the sampling rate.

So what you're effectively setting with the sample period if adaptive is enabled is the recording rate for the channels that aren't monitored for change.

Hope this helps answer the question - oh and welcome to the forum Steve!

atnbirdie

  • Calm
  • *
  • Posts: 6
    • DW8300
    • KMDMIDDL9
  • Station Details: Davis Wireless Vantage Pro2 w/Envoy & Data Logger; TRENDnet TV-IP110W Camera; iMac OS 10.6.8; WC v1.0.2
Re: Newbie: Sampling Interval?
« Reply #8 on: July 12, 2012, 09:39:50 PM »
Thank you all for the warm [sun2] welcome and the great help, especially from Steve M.   Here's an update.

I have successfully converted and imported my year's worth of WeatherLink (Windows) data by passing it through LWC and on seamlessly into WeatherCat.  I had thought I'd wait until the weekend when I'd have more time, but I did afterall have a new toy (WC) to play with so started tinkering last night.  As a result, as of today I am a proud [cat] owner now and am already happy I made the switch.  If the ease with which the transition went is any indication, I'm going to love this software!  Wunderground and CWOP integration was ridiculously simple and effective. From the time I opened the WL data in LWC until I had WC operational and uploading to Wunderground and CWOP was under an hour and that's only because I was plodding along being sure I had drivers in place, previous programs shut down, etc. before launching WC.

Enough gushing.  I would like to ask another on topic question.  This has to do with Wunderground update rates.  I have rapid fire turned on and know I can update as frequently as every few seconds.  My question is what's the point?  Just because we can?  At what point are we exceeding the granularity of weather change with updates of 1-5 seconds?  The only thing that occurs that quickly is probably wind gusts, but even if you updated WU every 15 seconds, the averaging that would go on in that time shouldn't dampen the uploaded value that much would it?  My reason for asking is that I wonder about how taxing uploading stacks of data every 5 seconds or so is in terms of slowing down performance of the host computer. 

As a new comer, I realize this may have been discussed elsewhere.  If so, just send me the link to the thread or web source and I'll educate myself.  However, personal opinions are always welcome!  ;)
Steve

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4805
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Rapidfire very low impact (Re: Newbie: Sampling)
« Reply #9 on: July 12, 2012, 10:55:57 PM »
Howdy Steve and WeatherCat fans,

Enough gushing.  I would like to ask another on topic question.  This has to do with Wunderground update rates.  I have rapid fire turned on and know I can update as frequently as every few seconds.  My question is what's the point?  Just because we can?  At what point are we exceeding the granularity of weather change with updates of 1-5 seconds?  The only thing that occurs that quickly is probably wind gusts, but even if you updated WU every 15 seconds, the averaging that would go on in that time shouldn't dampen the uploaded value that much would it?  My reason for asking is that I wonder about how taxing uploading stacks of data every 5 seconds or so is in terms of slowing down performance of the host computer. 

Golly never thought about this.  All I can tell you is that turning it on and off seems to have very little effect on WeatherCat.  I just ran a quick check with Activity Monitor and I couldn't see any difference in CPU load turning Rapid Fire on and off.

Stu has done a really good job of making WeatherCat multi-threaded, and modern Macs have more muscle than the mainframes of 30 years ago that handled hundreds of users.  [computer]  So you probably have nothing to worry about.  Just turn it on and live it up a little!!  [bounce]

Cheers, Edouard  [cheers1]