Trixology

WeatherCat => WeatherCat General Discussion => Topic started by: wx-watch on November 28, 2020, 11:10:25 AM

Title: Bannergate
Post by: wx-watch on November 28, 2020, 11:10:25 AM
Any updates on the "Banner Generation Not Working Under Big Sur" problem?????
Title: Re: Bannergate
Post by: WCDev on November 29, 2020, 10:24:26 AM
I've been looking into here but don't see the problem - is anyone else seeing this issue?

Cheers,
Stu.
Title: Re: Bannergate
Post by: wx-watch on November 29, 2020, 01:32:01 PM
Its mentioned a few times in this thread, by me as well as others

https://athena.trixology.com/index.php?topic=3206.0
Title: Re: Bannergate
Post by: ColdnFrosty on November 29, 2020, 11:24:06 PM
Yup. Mentioned it under bugs and elsewhere.

I've been looking into here but don't see the problem - is anyone else seeing this issue?

Cheers,
Stu.
Title: Which hardware? Any log info? (Re: Bannergate)
Post by: elagache on December 02, 2020, 12:00:34 AM
Dear wx-watch, Stu, ColdnFrosty, and WeatherCat early adopters,

I've been looking into here but don't see the problem - is anyone else seeing this issue?
. . . .

If Stu having trouble reproducing this problem, it would really help if you could provide Stu with as much detailed information about what you are observing and on what hardware.  Please include as many details as you can about the Mac you are running WeatherCat on.  Perhaps there is a memory or other unique hardware issue involved.

Also, if you could go through the WeatherCat logs to see if any error messages are being generated when the banner is not being generated.  Even the confirmation that no error messages are generated would be helpful to Stu.

Cheers, Edouard
Title: Re: Bannergate
Post by: ColdnFrosty on December 02, 2020, 12:13:39 AM
Hope this helps - not much else to report:

When I click on the Tools > Banner Generator I display the Banner Generator pop-up with correct current weather data. (see attachment)

The Banner Generator never actually generates the file. The log shows the following and repeats every 5 minutes as that is the timeframe I am using for updates:

2020/11/28 10:18:45:290  Aux. FTP: FTP upload starting.
2020/11/28 10:18:45:332  Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/mydisk/Applications/WeatherCatJPG/WeatherCatBanner.jpg
2020/11/28 10:18:55:297  Aux. FTP: Finished. Uploaded 0 bytes in 11 seconds at 0 bytes per second.

Note that in my second attachment, all other .jpg files successfully create and upload, but not the weathercatbanner.jpg.

Big Sur 11.0.1
iMac Pro 2017

Previously worked fine on the same machine running Catalina.
Title: Re: Bannergate
Post by: wx-watch on December 02, 2020, 08:16:08 AM
And this is exactly the same situation for me.
Title: File permissions? (Re: Bannergate)
Post by: elagache on December 03, 2020, 11:52:09 PM
Dear ColdnFrosty, wx-watch, and WeatherCat troubleshooters,

Sorry, my life is an extreme case of parallel processing and this forum is on the low end of the processing priority ranking.

Hope this helps - not much else to report:

When I click on the Tools > Banner Generator I display the Banner Generator pop-up with correct current weather data. (see attachment)

The Banner Generator never actually generates the file. The log shows the following and repeats every 5 minutes as that is the timeframe I am using for updates:

2020/11/28 10:18:45:290  Aux. FTP: FTP upload starting.
2020/11/28 10:18:45:332  Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/mydisk/Applications/WeatherCatJPG/WeatherCatBanner.jpg
2020/11/28 10:18:55:297  Aux. FTP: Finished. Uploaded 0 bytes in 11 seconds at 0 bytes per second.

Note that in my second attachment, all other .jpg files successfully create and upload, but not the weathercatbanner.jpg.
. . . . .

The path that you have for your custom web files is unusual.  Do you have a user named: mydisk?  I'm a bit puzzled that it would be trying to save the files in the Application folder for this user.  When you go to the path:

/Users/mydisk/Applications/WeatherCatJPG/

Do you find all the other files generated by WeatherCat custom web in this folder?  The banner file should end up in exactly the same folder as all the other custom web files.  In fact, there should be the last WeatherCatBanner.jpg file that was generated while running under Catalina somewhere on your hard drive.  If that file is somewhere else than:

/Users/mydisk/Applications/WeatherCatJPG/WeatherCatBanner.jpg

Then the problem has something to do with why WeatherCat has changed the saving path for WeatherCatBanner.jpg.  If these folders don't exist, it is no surprise that the banner generator no longer works.

This is a long shot I'll be the first to admit, but have you tried repairing the folder and file permissions using Disk Utility?  If there was some sort of permission issue, that might be why WeatherCat is unable to save the file.

Sorry if this isn't much more help, but do let us know what you discover.

Edouard
Title: Re: Bannergate
Post by: ColdnFrosty on December 04, 2020, 01:19:27 AM
"mydisk" is an acronym for my drive name. Names of folders are part of the FTP definition that I have created in WeatherCat Preferences. My generated files are stored in this folders and they have been since I started using WeatherCat many years ago through multiple releases of MACOSX. The folder names are not relevant to the issue. Multiple folks here who moved from Catalina to Big Sur have the same issue - no banners are being generated and saved. I am sure the remedy will be forthcoming!
Title: Re: Bannergate
Post by: xairbusdriver on December 04, 2020, 04:48:04 AM
Quote
The path that you have for your custom web files is unusual.
True, the default location (The Manual, p. 133 and secondarily 218) for Custom stuff is: ~/Library/Application Support/WeatherCatCustomWeb/ but it is user definable so almost any location should work. :)

Of course, even using the default location won't fix the Big Sur problem. It doesn't help that all the other images (jpg and png) seem to be uploading. Are any of you Big Sur users also successfully using/creating/uploading any Custom Graphs or any of the built-in jpg images (all_dials, barometer, cloudbase, heatindex, etc., etc.) WC creates automatically? That info might help Stu know if his jpeg creation functions are working. [banghead]
Title: Re: Bannergate
Post by: wx-watch on December 04, 2020, 07:32:44 AM
All my custom stuff gets generated and uploaded apart from the Banner

https://wx-watch.co.uk - is my custom site

https://wx-watch.co.uk/weathercat/  - is the "standard" WeatherCat site

I can confirm my custom stuff is in ~/Library/Application Support/WeatherCatCustomWeb/
Title: Re: Bannergate
Post by: xairbusdriver on December 04, 2020, 03:34:12 PM
Quote
I can confirm my custom stuff is in ~/Library/Application Support/WeatherCatCustomWeb/
OK, I see that both your sites use some of the 'standard' graphics. You don't appear to use any of the custom images. My hope was that we/Stu could see that his custom image creation functions were working or not. We can't really tell that since you aren't using any of the forms/frameworks that define those graphic's features. The only WC graphics creation form you are using is the Banner Generator. Only the dev knows if that image creation function is the same as the one for the Custom Gauge's/Graph's.

You may notice that the Console has two kinds of "FTP" entries: 'regular' FTP and "Aux. FTP", eg:
Quote
2020/11/27 15:06:41:077  FTP: Uploading customgauge15.png. (2 files left to upload.)
2020/11/27 15:06:41:154  FTP: Uploading custom-gauges-graphs.txt. (1 files left to upload.)
2020/11/27 15:06:41:232  FTP: Uploading station.html.
2020/11/27 15:06:41:310  CustomWeb: Finished parsing.
2020/11/27 15:06:41:388  CustomWeb: Finished processing custom web pages.
2020/11/27 15:06:42:125  Aux. FTP: FTP upload starting.
2020/11/27 15:06:42:204  Aux. FTP: Uploading Computer-Up-Time.txt.

The only things that use "Aux. FTP" (after a cursory search) are the things I list in the "Additional Files" and the WeatherCatBanner.jpg.

Are there any other instances of Aux. FTP in your Console logs?

A couple of tedious questions, if you don't mind:
1. What is the date/time on the WeatherCatBanner file in your Application directory?
2. Is there even a file by that name in ~/Library/Application Support/WeatherCatCustomWeb/? I suspect there is, but it has zero bytes.
Title: Re: Bannergate
Post by: wx-watch on December 04, 2020, 04:52:17 PM
I can't help re the size of the Banner file as I deleted it when I was trying to troubleshoot - it was there but has never been re-created.

I uploaded an additional file and the log is as follows

Aux. FTP: FTP upload starting.
4:43:19 PM 04-Dec-2020: Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/MYNAME/WeatherCat/WCWebCustom/WeatherCatBanner.jpg
4:43:24 PM 04-Dec-2020: CustomWeb: Processing additional custom web file: /Users/MYNAME/Documents/MAP.jpg
4:43:27 PM 04-Dec-2020: CustomWeb: Finished parsing.
4:43:27 PM 04-Dec-2020: CustomWeb: Finished processing custom web pages.
4:43:29 PM 04-Dec-2020: Aux. FTP: Uploading MAP.jpg.
4:43:31 PM 04-Dec-2020: Aux. FTP: Finished. Uploaded 145,030 bytes in 13 seconds at 10,835 bytes per second.

The file MAP.jpg is successfully uploaded to my website.
Title: Re: Bannergate
Post by: xairbusdriver on December 04, 2020, 05:35:34 PM
Quote
it was there but has never been re-created.
I assumed you would have a zero sized WeatherCatBanner file in the ~/Library/Application Support/WeatherCatCustomWeb/ directory and the one that was had been created still in the /Users/mydisk/Applications/WeatherCatJPG/ spot.

So, to be clear, there are no WeatherCatBanner.jpg files anywhere on your drive now? That seems logical, otherwise it would be uploading. And it tends to prove that your "strange" path for it is irrelevant.

I don't have a "MAP.jpg" file, don't even know how to enable/request it. [blush] Don't see any mention of it in the Manual. Perhaps it is a url set in the "Imagery" Prefs?

Just now followed some of your links and I now see that you are using several Custom Gauges. That might be a bit of helpful info for Stu. Those are png's the Banner is a jpeg, as are Custom Graphs. [coffee]

Thanks for the quick replies! [cheer]
Title: Re: Bannergate
Post by: wx-watch on December 04, 2020, 06:13:55 PM
The Map was was just an additional test file I uploaded. This as you can see was uploaded successfully by Aux. FTP

There are no WeatherCatBanner.jpg files anywhere on my system. When WeatherCat failed to upload the file after I upgraded to Big Sur I deleted the file in the hope it would be re-created anew. It wasn’t.

It looks like it would be uploaded if it was created but it isn’t being created
Title: Re: Bannergate
Post by: xairbusdriver on December 04, 2020, 07:22:40 PM
Yep, anything (text, html, php files, images, your contacts list, assorted plist backups, etc. :-* ) you put in the ~/Library/Application Support/WeatherCatCustomWeb/ directory get uploaded to your web space. I think most of us use the Prefs/On-Line/Custom Web/Additional Files list to put stuff in there. That list also allows one to set a time interval for each file. I update my Statistics page every 360 minutes, but my webcam page gets updated every 30 minutes. Computer-Up-Time.txt is uploaded every 1.3 minutes. It is then read by a php script and converted/echoed to the page that needs it. How they are used there is up to you. [tup] Of course, the Custom Gauges/Graphs already "know" to put their images in that location. ThU32:-)

Hope this problem can be corrected, perhaps after the Seasonal rush? :)
Title: Re: Bannergate
Post by: wx-watch on December 04, 2020, 07:25:03 PM
As a further test I downloaded the Banner file from an earlier post and copied it to

/Users/MYNAME/WeatherCat/WCWebCustom/WeatherCatBanner.jpg

From the log

7:16:13 PM 04-Dec-2020: Aux. FTP: Uploading WeatherCatBanner.jpg.
7:16:15 PM 04-Dec-2020: Aux. FTP: Finished. Uploaded 34,955 bytes in 2 seconds at 12,483 bytes per second.

And it is successfully uploaded to my site.

So if a valid file is present it will get uploaded .

I presume therefore WeatherCat running under Big Sur for some reason is not generating the file

Title: Re: Bannergate
Post by: xairbusdriver on December 04, 2020, 07:42:37 PM
Yes. And "presuming" is slightly less dangerous than "assuming". [lol]
Title: Incorrect path to hard drive volume. (Re: Bannergate)
Post by: elagache on December 04, 2020, 11:23:55 PM
Dear ColdnFrosty, X-Air, wx-watch, and WeatherCat troubleshooters,

"mydisk" is an acronym for my drive name.
. . . .

Of course, you can call your hard drive anything you like and you can store these files in any convenient folder path.  Clearly the banner file isn't being generated.  What I am wondering is if there is some sort of errant path that WeatherCat is somehow generating on your installations.

Checking my hard drive the path of my hard drive in UNIX notation is:

/Volumes/Odysseus HD

However, what the log output from WeatherCat is proposing a path in the Users directory:

/Users/mydisk/

That is the incorrect path and if WeatherCat was attempting to save a file there, it would fail.  I'm not sure why WeatherCat could get this wrong, but this appears to be at least a possible cause.

It continues to be a long shot, but you would be able to restore preference files from backups before you upgraded to Big Sur?  If so I would suggest trying to restore the main WeatherCat preference file:

~/Library/Preferences/com.trixology.weathercat.plist

and the WeatherCat banner preference file:

~/Library/WeatherCatData/BannerGeneratorData/BannerGeneratorSettings.plist

These are in UNIX convention with the ~ standing for the path to your home account directory.

In order to restore these files, you must first quit WeatherCat, restore the files from backup, reboot, and only then launch WeatherCat once more.  Unless you reboot, the restored preference files won't be used.

Sorry for the hassles and once more please let us know what you discover.

Cheers, Edouard
Title: Re: Bannergate
Post by: wx-watch on December 05, 2020, 03:57:28 PM
Instructions to restore the two .plist files followed exactly and no joy! Log excerpts follow...

3:46:54 PM 05-Dec-2020: WeatherCatPlugInManager: Starting updates for Banner Generator
...
3:46:55 PM 05-Dec-2020: Banner Generator initialising.
...
3:52:05 PM 05-Dec-2020: Aux. FTP: FTP upload starting.
3:52:05 PM 05-Dec-2020: Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/MYNAME/WeatherCat/WCWebCustom/WeatherCatBanner.jpg
3:52:15 PM 05-Dec-2020: Aux. FTP: Finished. Uploaded 0 bytes in 11 seconds at 0 bytes per second.

So in short - still no Banner  [roll]
Title: Re: Bannergate
Post by: xairbusdriver on December 05, 2020, 09:56:38 PM
Just for (more) grins... :P

See what happens if you (wx-watch):
Title: Try moving custom web folders. (Re: Bannergate)
Post by: elagache on December 05, 2020, 10:43:46 PM
Dear wx-watch, X-Air, and WeatherCat troubleshooters,

Instructions to restore the two .plist files followed exactly and no joy! Log excerpts follow...
. . . . .

Sorry, alas I did describe it as a long-shot.  I do have one more rather strange suggestion.  Try moving all your custom web folders to a location inside your home folder (~ in UNIX convention.)  Obviously then update WeatherCat with the new locations of these folders.  To avoid conflicts, it might be easier to make copies of your current folders and change the locations in WeatherCat.  That way there is no risk of WeatherCat attempting to access a relocated folder.

If that solves your problem, that strongly suggests that there is a bug in macOS daemon that caches preference settings and it is that bug which is giving WeatherCat a nonexistence location for your banner to be saved to.

Please do let us know what you discover.

Cheers, Edouard
Title: Re: Bannergate
Post by: ColdnFrosty on December 05, 2020, 11:10:53 PM
It does not.
Title: Re: Bannergate
Post by: wx-watch on December 06, 2020, 03:40:07 PM
Neither of the above two fixes worked for me either
Title: Do the log errors look any different? (Re: Bannergate)
Post by: elagache on December 07, 2020, 10:01:25 PM
Dear ColdnFrosty, wx-watch, and WeatherCat troubleshooters,

It does not.

Neither of the above two fixes worked for me either

Sorry to hear that, but it was a long shot at best.  Could you retrieve the log error messages in this case?  It might still provide some useful information for your fearless leader - Stu.

Edouard
Title: Re: Bannergate
Post by: ColdnFrosty on December 07, 2020, 10:10:54 PM
Already posted the error log earlier in the thread.
Title: Re: Bannergate
Post by: wx-watch on December 08, 2020, 09:28:06 AM
Same error as before. As I said earlier in this thread in an attempt to troubleshoot I deleted all instances of WeatherCatBanner.jpg from the system and it has never been re-created anywhere since.
.
.
9:15:13 AM 08-Dec-2020: WeatherCatPlugInManager: Starting updates for Banner Generator.
.
9:16:10 AM 08-Dec-2020: Aux. FTP: FTP upload starting.
9:16:10 AM 08-Dec-2020: Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/MYNAME/WeatherCat/WCWebCustom/WeatherCatBanner.jpg
9:16:20 AM 08-Dec-2020: Aux. FTP: Finished. Uploaded 0 bytes in 11 seconds at 0 bytes per second.
.
.
Title: Hunch explanation. (Re: Bannergate)
Post by: elagache on December 09, 2020, 12:07:14 AM
Dear ColdnFrosty, wx-watch, and WeatherCat troubleshooters,

Already posted the error log earlier in the thread.

Same error as before. . . .

Unfortunately, what this shows is that you didn't follow my second suggestion.  What I was asking you is to move all your custom template processing folders inside your home folder.  In my case this is (again in UNIX notion:)

/Users/elagache

Here is my reasoning for the experiment.  I checked the contents of the general WeatherCat preference file and the unique one for the Banner generator.  All the paths are only in the general preference file.  What that means is that if there was some problem reading the preference file, all custom web processing should fail, not just the banner generator.  Unfortunately, Apple has a daemon that caches the contents of the preference file.  When WeatherCat requests a preference, Apple provides the data instead of WeatherCat going directly to the preference file.

The log files you are providing me show that the path to save the WeatherCatBanner.jpg file is an impossible path.  The daemon is mixing a user directory with the path to your hard drive.  Not surprisingly, that fails.  Your hard drive cannot be accessed off us the /User branch of the UNIX file system.

That is why I am suggesting that you move all your custom web folders to your home folder.  Given that the Daemon is prepending /User erroneously to path, it - might - work because the path would be off of /User.  Another possibility is that it will still insist on putting that on the front of the path which would generate an erroneous path that would look start this:  /User/User.

Either way, that would provide Stu with a much clearer idea of what is going wrong.  It might be enough for him to be able to make a precise bug report for Apple and it certainly should help him find a way to work around this bug in the Apple daemon.

Cheers, Edouard
Title: Re: Hunch explanation. (Re: Bannergate)
Post by: ColdnFrosty on December 09, 2020, 12:59:37 AM
As noted in Reply #5, all of my WeatherCat folders are in my Home folder. In the /Applications folder. In Mac OS X, the home folder is /users/[myusername]

For the purposes of the discussion, I am using "mydisk" as a pseudonym for [myusername]

In that home folder is the /Applications folder and two subfolders, /WeatherCatHTML and /WeatherCatJPG

All of my generated .JPG files (33 of them) are saved there according to the time value set in preferences. There are also two .MP4 that are saved there.

In the /WeatherCatHTML folder, there is one subfolder called WCWeb. In that folder there are five .HTML files, eleven .JPG files, and one .TIF file used for created a web page.

The FTP settings in WeatherCat copy all of those files over to aShare on my router, for easy access. The last time WeatherCatBanner.jog was generated was on November 15, 2020 following my installation of Big Sur. (see below)

I think the developer(s) need to install Big Sur so they have a more direct view of what is working and what is not. :-)


2020/11/28 10:18:45:290  Aux. FTP: FTP upload starting.
2020/11/28 10:18:45:332  Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/[myusername]/Applications/WeatherCatJPG/WeatherCatBanner.jpg
2020/11/28 10:18:55:297  Aux. FTP: Finished. Uploaded 0 bytes in 11 seconds at 0 bytes per second.

Dear ColdnFrosty, wx-watch, and WeatherCat troubleshooters,

Already posted the error log earlier in the thread.

Same error as before. . . .

Unfortunately, what this shows is that you didn't follow my second suggestion.  What I was asking you is to move all your custom template processing folders inside your home folder.  In my case this is (again in UNIX notion:)

/Users/elagache

Here is my reasoning for the experiment.  I checked the contents of the general WeatherCat preference file and the unique one for the Banner generator.  All the paths are only in the general preference file.  What that means is that if there was some problem reading the preference file, all custom web processing should fail, not just the banner generator.  Unfortunately, Apple has a daemon that caches the contents of the preference file.  When WeatherCat requests a preference, Apple provides the data instead of WeatherCat going directly to the preference file.

The log files you are providing me show that the path to save the WeatherCatBanner.jpg file is an impossible path.  The daemon is mixing a user directory with the path to your hard drive.  Not surprisingly, that fails.  Your hard drive cannot be accessed off us the /User branch of the UNIX file system.

That is why I am suggesting that you move all your custom web folders to your home folder.  Given that the Daemon is prepending /User erroneously to path, it - might - work because the path would be off of /User.  Another possibility is that it will still insist on putting that on the front of the path which would generate an erroneous path that would look start this:  /User/User.

Either way, that would provide Stu with a much clearer idea of what is going wrong.  It might be enough for him to be able to make a precise bug report for Apple and it certainly should help him find a way to work around this bug in the Apple daemon.

Cheers, Edouard
Title: Re: Bannergate
Post by: wx-watch on December 09, 2020, 12:34:09 PM
Like this??

12:29:44 PM 09-Dec-2020: Aux. FTP: FTP advice - file wasn't ready to upload - will try again shortly. Path was: /Users/MYNAME/WCWebCustom/WeatherCatBanner.jpg
12:29:54 PM 09-Dec-2020: Aux. FTP: Finished. Uploaded 0 bytes in 10 seconds at 0 bytes per second.
Title: Re: Bannergate
Post by: xairbusdriver on December 09, 2020, 10:59:31 PM
I'm getting confused by the exact path you are wanting to save the Banner (should it ever be created!). I see the following in your various posts:
The actual default and recommended (please see p. 133 & 218 in The Manual) name/path is:~[user]/Library/Application Support/WeatherCatCustomWeb/

I think there is at least one other person reporting this missing Banner problem, but it might make things simpler and easier for Stu if we kept the variables to a minimum. Since you are not getting the file loaded anywhere, would it be asking too much to create the correctly named directory at the default location? [computer] jis' fer grins? [cheers1] If you could do that, I promise not to tell you to completely duplicate my entire folder hierarchy on your volumes! :-X
Even though I'm sure my way is best...
:-* cmu:-)
Title: Apologies (Re: Bannergate)
Post by: elagache on December 09, 2020, 11:14:25 PM
Dear ColdnFrosty, wx-watch, X-Air, and WeatherCat troubleshooters,

Okay my apologies.  I didn't understand your substitution of a disk name for your user name.  As a result, my hunch of what the problem was was completely off the beam.  At this point we can only wait to see what Stu discovers because there is nothing we can easily observe from the outside that might help in the diagnosis.

Oh well, . . . . . . . . Edouard
Title: Re: Bannergate
Post by: ColdnFrosty on December 10, 2020, 12:33:35 AM
Web Pages and Custom Web Pages are saved to folders defined in the preferences. There are other generated files that are stored elsewhere in folders that you can see in the attachment below.

No point in changing folder locations in preferences that WeatherCatBanner.jpg copied to fine under Catalina, but not under Big Sur.

I'm getting confused by the exact path you are wanting to save the Banner (should it ever be created!). I see the following in your various posts:
  • /Users/[myusername]/Applications/WeatherCatJPG/
  • /Users/MYNAME/WCWebCustom/
The actual default and recommended (please see p. 133 & 218 in The Manual) name/path is:~[user]/Library/Application Support/WeatherCatCustomWeb/

I think there is at least one other person reporting this missing Banner problem, but it might make things simpler and easier for Stu if we kept the variables to a minimum. Since you are not getting the file loaded anywhere, would it be asking too much to create the correctly named directory at the default location? [computer] jis' fer grins? [cheers1] If you could do that, I promise not to tell you to completely duplicate my entire folder hierarchy on your volumes! :-X
Even though I'm sure my way is best...
:-* cmu:-)