Author Topic: Coping with Weather Underground "issuez"  (Read 1677 times)

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4804
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Coping with Weather Underground "issuez"
« on: January 09, 2018, 12:20:10 AM »
Dear WeatherCat data submission junkies,

I'm sure many of you have noticed that Weather Underground is not very reliable at the moment.  You might simply turn the service off for a while.  When they get their servers working once more, you should be able to upload the missing data as a historical upload.

Alas, my "WC Status Growler" AppleScript dutifully reports all of the error codes that Stu exposed to AppleScript.  One of them is disabling uploads to Weather Underground.  So if I disable it, I still a flurry of Growl notifications.  So I've opted for a Plan-B.  I've disabled webcam uploads and turned off Weather Underground rapid-fire.  I now upload infrequently - once every 10 minutes.  At least that way, I don't blast my computer screen with Growl notifications.

Until Weather Underground gets back to its former stability, we'll simply have to cope as we can!

Oh well, . . . . . Edouard

Blicj11

  • Storm
  • *****
  • Posts: 2831
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP™ Data Logger | iMac (late 2013), 3.5 GHz Intel Core i7, 24 GB RAM, macOS High Sierra | Sharx SCNC2900 Webcam | Supportive Wife
Re: Coping with Weather Underground "issuez"
« Reply #1 on: January 09, 2018, 07:32:07 AM »
I have been getting FTP, password, connection, and missing webcam errors since the day they announced they were not going to cancel webcam uploads after all. I don't get them all at once. They are dispersed randomly throughout the day. WU has never been this bad since the very early days. and then they had a good excuse. Then nobody knew what they were doing and it didn't matter. Now nobody there knows what they are doing and it impacts a whole lot more people. Now that they have made redundant everyone who knew what they were doing they are left with only the overwhelmed. What an inspiring way to run a company,
Blick


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4804
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Disabled it for now. (Re: Coping with Weather Underground "issuez")
« Reply #2 on: January 10, 2018, 12:26:52 AM »
Dear Blick and WeatherCat faithful,

I have been getting FTP, password, connection, and missing webcam errors since the day they announced they were not going to cancel webcam uploads after all. I don't get them all at once. They are dispersed randomly throughout the day.

No matter what I tried, I couldn't get better than around 47% of the packets to be received by Weather Underground.  The situation was generating a zillion warnings from my WC Status Growler AppleScript.  This afternoon I finally got fed up with the whole thing, so I modified WC Status Growler to no longer consider disabling Weather Underground to be an error.  I'm much happier not sending anything that fighting their servers all the way.

WU has never been this bad since the very early days. and then they had a good excuse. Then nobody knew what they were doing and it didn't matter. Now nobody there knows what they are doing and it impacts a whole lot more people. Now that they have made redundant everyone who knew what they were doing they are left with only the overwhelmed. What an inspiring way to run a company,

It doesn't help, but I don't think we can blame Jeff Masters and the original founders of Weather Underground for this.  Weather Underground took a risk and allowed itself to become part of the Weather Channel.  Once they lost their independence, they were at the mercy of their owners.  IBM is, after all, one of the original "suits" culture and couldn't even take advantage of the PC revolution that they started with that iconic IBM PC.  I'm sure Jeff and the original Weather Underground team would dearly like to be spun off from IBM at this point.  I don't know if IBM would be kind enough to consider it or if Weather Underground could survive as an independent company once more.

Alas, this is definitely another case of . . . . . . . . . .

Such are da' conditions that prevail . . . . .

Oh well,  Edouard

Weatheraardvark

  • Strong Breeze
  • ***
  • Posts: 245
  • Here are tornado sirens, lets go outside look
    • C0005
    • KIADESMI1
    • Des Moines Weather and Climate
  • Station Details: Davis VP2 Plus, is that better?
Re: Coping with Weather Underground "issuez"
« Reply #3 on: January 10, 2018, 04:13:57 AM »
I contacted Davis Instruments and received this today:

Thanks you for your email to Support. Customers with accounts who send data to Weather Underground Personal Weather Station (PWS) from your weatherlink.com account may notice data is missing or not being sent to Weather Underground in a timely fashion. WeatherLink Software users who have installed Weather Underground’s plugin are also affected. We’ve been in contact with technical staff at The Weather Channel, who operates Weather Underground, and they’ve informed us their RapidFire servers are experiencing a temporary performance problem that should be resolved in the next couple days. When services are available again, we’ll make an attempt to back-fill missing data, but some data points may be lost. Operations staff at Davis Instruments are doing everything possible to make sure the most complete set of data is sent to Weather Underground as soon as possible.
 
We’ve made available a status page operated by Pingdom, a third party external monitoring service, that measures availability for Weather Underground’s RapidFire service.
 
http://stats.pingdom.com/zw9vxjhrr9ze/1850219
 
Davis VP2 Plus; 24h  FARS; Extra Temp Humid sensor (2); Extra Temp Station (2); Soil Moisture/Temp Station;Weatherlink IP;; MAC Sierra; Webcam
https://www.desmoinesweather.org

Blicj11

  • Storm
  • *****
  • Posts: 2831
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP™ Data Logger | iMac (late 2013), 3.5 GHz Intel Core i7, 24 GB RAM, macOS High Sierra | Sharx SCNC2900 Webcam | Supportive Wife
Re: Coping with Weather Underground "issuez"
« Reply #4 on: January 10, 2018, 09:17:00 AM »
Very helpful information and link. Thanks for posting it here.
Blick


Felix

  • Gale
  • ****
  • Posts: 388
  • Station Details: Davis Wireless Vantage Pro-2 Plus with FARS, Leaf & Soil Moisture/Temp Sta + WeatherLink IP. Two prototype Sharx 3905 HD wide-angle webcams and one production model 3905, all using EvoCam v5.0. WeatherCat 3.01, macOS 10.14 B3 Build 18A326h (Mojave).
Re: Coping with Weather Underground "issuez"
« Reply #5 on: January 10, 2018, 10:43:55 AM »
I wonder if turning off RapidFire for the present time would make any difference for those having upload problems to the WU servers?


Quite frankly, I've stopped paying much attention to WU with all their issues so I haven't really been affected.

xairbusdriver

  • Storm
  • *****
  • Posts: 2054
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.13.4/WC 3.0
Re: Coping with Weather Underground "issuez"
« Reply #6 on: January 10, 2018, 04:11:04 PM »
Quote
I wonder if turning off RapidFire for the present time would make any difference
I think if 90% of us just stopped uploading to WU, their servers could probably handle things. I may have under estimated that percentage, of course... [rolleyes2] [lol] Maybe things will improve when Spring arrives and the interweb pipes thaw? [banghead] It's nice that Davis seems to get some direct responses from WU. Apparently, there are still some at WU who answer inquiries. I'm fairly sure the folks at WU know they are reaching the point of losing the volunteer uploading group. The question is, do they have restoration of that service as any kind of priority? I suspect that they are breaking down/fine tuning their 'eyeball' counting algorithm and don't see us as worth the effort to increase profits. I think many companies are finding it difficult to make money via "web services" not directly involving retail sales.

Weatheraardvark

  • Strong Breeze
  • ***
  • Posts: 245
  • Here are tornado sirens, lets go outside look
    • C0005
    • KIADESMI1
    • Des Moines Weather and Climate
  • Station Details: Davis VP2 Plus, is that better?
Re: Coping with Weather Underground "issuez"
« Reply #7 on: January 10, 2018, 08:47:03 PM »
Quote
I wonder if turning off RapidFire for the present time would make any difference
I think if 90% of us just stopped uploading to WU, their servers could probably handle things. I may have under estimated that percentage, of course... [rolleyes2] [lol] Maybe things will improve when Spring arrives and the interweb pipes thaw? [banghead] It's nice that Davis seems to get some direct responses from WU. Apparently, there are still some at WU who answer inquiries. I'm fairly sure the folks at WU know they are reaching the point of losing the volunteer uploading group. The question is, do they have restoration of that service as any kind of priority? I suspect that they are breaking down/fine tuning their 'eyeball' counting algorithm and don't see us as worth the effort to increase profits. I think many companies are finding it difficult to make money via "web services" not directly involving retail sales.

I wrote about this over on Wxforum, you get the short version.
I am guessing that WU is moving to a cheaper costing server.  Probably Amazon or something like that.   in the process they screwed up the incoming feed.  Have you ever moved to a new server,  the school district I taught would do this, not during summer break, but right in the middle of the year, around grade time.

Bloomsky last fall decided to move everything over to the Amazon Cloud.  things were screwed up and a lot of folks were pissed for lack of a kinder word.  Bloomsky units are internet based, no way to look at a console directly,   their wireless information goes either through a dongle or wirelessly to their server, then it is sent back either to their online map or to an app.  If the server craps out, everyone is off line. Not a good system.

those of us who have good stations do not worry about this stuff because we can look at the console or software and be happy.   

If you want to tell WU that you no longer are going to send them data, and cost them big buck loss and loss of readership etc,   they will wish you well on your new pathway. 

I am going to be light the flasher.  You know the story about the flasher, he was tired of flashing people. He was tired of wearing that trench coat in hot weather and sweating, and in winter freezing .   After some thought he decided to stick it out a bit longer.
Davis VP2 Plus; 24h  FARS; Extra Temp Humid sensor (2); Extra Temp Station (2); Soil Moisture/Temp Station;Weatherlink IP;; MAC Sierra; Webcam
https://www.desmoinesweather.org

xairbusdriver

  • Storm
  • *****
  • Posts: 2054
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.13.4/WC 3.0
Re: Coping with Weather Underground "issuez"
« Reply #8 on: January 10, 2018, 10:52:54 PM »
I don't see how WU would lose any money, much less "big buck(s)" by us ceasing to upload our data. OTOH, a large percentage of frustrations and thread posts here could be eliminated by such actions. We now have a nice link that is easy to use and check on the status of the WU servers. By judicious use of that site, those complaining/blaming their Systems/WC could end. Frankly, I doubt more than a one hand full of (real) people ever look at my data on WU. The web cam image is near worthless, in my humble opinion when I can get a nice time-lapse, with data from each frame. ThU5:-) [cheer]

My comment about 90% of us stopping uploads was a lame attempt at satire, not a recommendation. "Don't take anything I say as important or even factual, unless it includes the word FIRE in it."

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4804
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Thanks^2! (Re: Weather Underground "issuez")
« Reply #9 on: January 10, 2018, 11:43:43 PM »
Dear Weatheraardvark, Blick, Felix, X-Air, and WeatherCat faithful,

We’ve made available a status page operated by Pingdom, a third party external monitoring service, that measures availability for Weather Underground’s RapidFire service.
 
http://stats.pingdom.com/zw9vxjhrr9ze/1850219

As Blick has said (and I said on another thread) Thank you!  [tup]

Cheers, Edouard  [cheers1]

Felix

  • Gale
  • ****
  • Posts: 388
  • Station Details: Davis Wireless Vantage Pro-2 Plus with FARS, Leaf & Soil Moisture/Temp Sta + WeatherLink IP. Two prototype Sharx 3905 HD wide-angle webcams and one production model 3905, all using EvoCam v5.0. WeatherCat 3.01, macOS 10.14 B3 Build 18A326h (Mojave).
Re: Coping with Weather Underground "issuez"
« Reply #10 on: January 11, 2018, 10:42:06 AM »
This thread prompted me to go look at my WU feed only to find a webcam pic which was over 12 hours old.


A peek at the FTP log showed the following which is just another reason why I rarely click on WU anymore. I should just stop uploading to WU and be done with the site.


 Connecting to webcam.wunderground.com:21
Opening FTP session
> 421 50 users (the maximum) are already logged in, sorry
< QUIT
Closing FTP session

Blicj11

  • Storm
  • *****
  • Posts: 2831
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP™ Data Logger | iMac (late 2013), 3.5 GHz Intel Core i7, 24 GB RAM, macOS High Sierra | Sharx SCNC2900 Webcam | Supportive Wife
Re: Coping with Weather Underground "issuez"
« Reply #11 on: January 11, 2018, 05:11:26 PM »
I had received that webcam message off and now for a week now. However, WU servers have been up and running solid for 24 hours now, so perhaps they have tipped the scales in favor of emerging from this nightmare server transition.
Blick


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4804
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
Dear Felix, Blick, and WeatherCat longtime users of Weather Underground,

I had received that webcam message off and now for a week now. However, WU servers have been up and running solid for 24 hours now, so perhaps they have tipped the scales in favor of emerging from this nightmare server transition.

According to pingdom testing, there are still some outages going on, but it is much more stable than it was 24 hours ago.  I decided to compromise and upload data just once a minute and no webcam.  At first I wasn't getting anything, but now WeatherCat is successfully uploading historical uploads.  It doesn't seem that Weather Underground is back to normal but indeed perhaps we are heading out of the nightmare at last.

Cheers, Edouard

xairbusdriver

  • Storm
  • *****
  • Posts: 2054
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.13.4/WC 3.0

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 4804
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, MacBook Pro (Early 2011)
PingDom gives da' "all clear."(Re: Weather Underground "issuez")
« Reply #14 on: January 12, 2018, 11:45:35 PM »
Dear X-Air and WeatherCat data submission junkies,

According to the PingDom webpage, Weather Underground data submissions are indeed operating reliably once more.  I just turned Rapid Fire back on and started uploading my webcam image once more.  Perhaps we are finally "back to normal!" - I hope!!

Cheers, Edouard  [cheers1]