FTP vs Aux. FTP - Console logs

[xairbusdriver]

I've been watching Console for various reasons lately. Nothing much on TV and the grass has about stopped growing. Don't see much of anything here or in the Manual about these two things. What is the difference between them? Answering that will probably explain why both are needed and why they (at least the "Aux." version) exist and possibly how to eliminate the latter since it seems to be doing the same thing as the 'main' version. [interesting]to me, anyway...

Thanks! I gotta go watch the electric meter spin...

[Blicj11]

Aux FTP identifies those files that need to be uploaded more frequently than the main FTP upload frequency, or are uploaded from a different directory than the Main FTP uploads. In either case, they are identified in the Custom Web Preferences as Additional Files.

[xairbusdriver]

Thanks, Blick!

Hhmm... That will require additional investigation. I'll double-check later, but I think they files are the same in both uploads. That seems a waste of time and micro-joules. I'm not understanding (among hundreds of other things!) where or why WC uploads files. There are only six actual "Additional Files", 15 gauges, 10 graphs, and one movie. Not seeing any problems, just like to know what, when, where, why, and whow (SP?)!

[elagache]

Dear X-Air, Blick, and WeatherCat yard and TV watchers, . . . 

I've been watching Console for various reasons lately. Nothing much on TV and the grass has about stopped growing.

  . . . What do ya' mean da' grass stopped growin'! ? ??   Our grass just started! . . . . 

Of course that might have something to do with the ground fault interrupter tripping and taking down both the sprinkler control and the power for the weather station.  I found this out da' hard way. When I plugged in my super-deluxe PC case fan - nothing happened!    It too me a few minutes of troubleshooting before I realized what happened.

As to television, your problem is that you didn't buy a TiVo years ago and stocked it with all sorts of weird and wonderful goodies.  We are going back to the first year of The curse of Oak Island.  It is always fun to watch suckers spend their money and the weirdest part of this story is that somehow it has to be true and very strange! 

Thanks! I gotta go watch the electric meter spin...

Did you know you can get devices that attach to your electric meter so that you can watch it spin from the comfort of your own computer? . . . 

But get the TiVo first, much more entertainment value!

Cheers, Edouard 

[xairbusdriver]

No TiVo required here. PBS still has new programs that are worth watching, repeats are also available with a little searching. I'm sure there are worthwhile programs on commercial TV, just not something I'm interested in. Then there are these forums...

[elagache]

Dear X-Air and WeatherCat connoisseurs of fine entertainment . . .

I'm sure there are worthwhile programs on commercial TV, just not something I'm interested in. Then there are these forums...

  . . . . Well now, if you are looking for some real entertainment, maybe you should water your lawn!

You know what they say about sailboat racing . . . . .  it is about as exacting as watching the grass grow! . . . 

Cheers, Edouard 

[xairbusdriver]

Not to discourage your "entertainment" activities...

I've been watching Console all afternoon, specifically the WeatherCat logs. Since ~1:30pm when I moved a file out of the "Set HTML Source" path. I posted details about it in the '<WC workload>' thread. Apparently, WC uses the 'plain FTP' method with files in that path that are not specified in the "Additional Files" list. Any files in that list use the "Aux. FTP" method. At least, that's what I'm seeing here. If that's the case, at least I now have cut way back on the uploading needs.

[xairbusdriver]

Aux FTP identifies those files that need to be uploaded more frequently than the main FTP upload frequency, or are uploaded from a different directory than the Main FTP uploads. In either case, they are identified in the Custom Web Preferences as Additional Files.

OK, I've now earned a Boy Scout Merit Badge in Console Monitoring! Unfortunately, my membership expired several decades ago...

I was still perplexed/confused about why WC was basically using "Aux. FTP" and plain "FTP" both for practically the whole site! When I finally re-read Edouard's post, the "different directory" part reminded me that I have intentionally moved everything out of the "Set HTML Source" path in the prefs, except for four html and two php files. Any entry/text/tag used in any of the them is unique to each file, none of those entries/texts/tags are used in any other of the six files. So, why the doubles up FTP reports?! Within 35 seconds I saw ~180MB uploads with "Aux. FTP" and then plain "FTP". Numbers were not exactly the same but ? 1MB!

Here is a snapshot of the files structure on the drive. Please tell me how/why WC is uploading the CUSTOMstuff$ so often. The "CustomWebUploads.plist" has exactly the same six files (as mentioned above) mentioned above and listed in the "Additional Files" prefs.

[WCDev]

180MB sounds like a video being uploaded?

[xairbusdriver]

180MB sounds like a video being uploaded?

I'm not sure. The movie is ~30MB, but the 15 gauges average ~49KB for a total of ~746KB. The 10 graphs  total of~592KB. But they are being uploaded twice. I can see an entry just after each hour when the actual movie is being uploaded.

Here's the console entries just before the last movie upload, but I don't see a figure corresponding to the total bytes that would include all the gauges, graphs and the movie.

Code: [Select]

?10/18/15 3:00:07.167 PM WeatherCat[5376]: DailyMovieController: Renamed /Users/xairbusguy/Library/Application Support/WeatherCatMedia/Movies/Today/WeatherCatDailyMovie.mov to /Users/jameschaffin/Library/Application Support/WeatherCatMedia/Movies/Today/WeatherCatDailyMovie_14_00.mov
 10/18/15 3:00:08.748 PM WeatherCat[5376]: CWOP: Starting connection...
 10/18/15 3:00:08.769 PM WeatherCat[5376]: CWOP: Trying to connect to cwop.aprs.net.
 10/18/15 3:00:08.854 PM WeatherCat[5376]: CWOP: Connecting to cwop.aprs.net.
 10/18/15 3:00:12.857 PM WeatherCat[5376]: CWOP: Sending data to cwop.aprs.net. Success.
 10/18/15 3:00:15.862 PM WeatherCat[5376]: CWOP: Finished.
 10/18/15 3:00:15.883 PM WeatherCat[5376]: CWOP: Next transmission in 10 minutes and 23 seconds.
 10/18/15 3:00:23.538 PM WeatherCat[5376]: CustomWeb: Primed; waiting for weather sync.
 10/18/15 3:00:29.934 PM WeatherCat[5376]: Fetching data from weather station...
 10/18/15 3:00:30.571 PM WeatherCat[5376]: Temperature: 66?F  Dewpoint: 32?F  Windchill: 66?F  Pressure: 30 inHg  Av. Windspeed: 3 MPH from 52.6?  Current Conditions: Clear
 10/18/15 3:00:32.949 PM WeatherCat[5376]: CustomWeb: Processing custom web pages...
 10/18/15 3:00:36.267 PM WeatherCat[5376]: CustomWeb: Generating gauges.
 10/18/15 3:00:40.171 PM WeatherCat[5376]: CustomWeb: Finished parsing.
 10/18/15 3:00:40.193 PM WeatherCat[5376]: CustomWeb: Generating web cam picture.
 10/18/15 3:00:40.216 PM WeatherCat[5376]: CustomWeb: Waiting for web cam picture to stabilise.
 10/18/15 3:00:40.237 PM WeatherCat[5376]: FTP: FTP upload starting.
 10/18/15 3:00:40.257 PM WeatherCat[5376]: FTP: Uploading index.html. (31 files left to upload.)
 10/18/15 3:00:40.278 PM WeatherCat[5376]: FTP: Uploading statistics.html. (30 files left to upload.)
 10/18/15 3:00:40.300 PM WeatherCat[5376]: FTP: Uploading customgauge1.png. (29 files left to upload.)
 10/18/15 3:00:40.325 PM WeatherCat[5376]: FTP: Uploading customgauge2.png. (28 files left to upload.)
 10/18/15 3:00:40.346 PM WeatherCat[5376]: FTP: Uploading customgauge3.png. (27 files left to upload.)
 10/18/15 3:00:40.368 PM WeatherCat[5376]: FTP: Uploading customgauge4.png. (26 files left to upload.)
 10/18/15 3:00:40.389 PM WeatherCat[5376]: FTP: Uploading customgauge5.png. (25 files left to upload.)
 10/18/15 3:00:40.410 PM WeatherCat[5376]: FTP: Uploading customgauge6.png. (24 files left to upload.)
 10/18/15 3:00:40.431 PM WeatherCat[5376]: FTP: Uploading customgauge7.png. (23 files left to upload.)
 10/18/15 3:00:40.452 PM WeatherCat[5376]: FTP: Uploading customgauge8.png. (22 files left to upload.)
 10/18/15 3:00:40.477 PM WeatherCat[5376]: FTP: Uploading customgauge9.png. (21 files left to upload.)
 10/18/15 3:00:40.499 PM WeatherCat[5376]: FTP: Uploading customgauge10.png. (20 files left to upload.)
 10/18/15 3:00:40.522 PM WeatherCat[5376]: FTP: Uploading customgauge11.png. (19 files left to upload.)
 10/18/15 3:00:40.547 PM WeatherCat[5376]: FTP: Uploading customgauge12.png. (18 files left to upload.)
 10/18/15 3:00:40.569 PM WeatherCat[5376]: FTP: Uploading customgauge13.png. (17 files left to upload.)
 10/18/15 3:00:40.592 PM WeatherCat[5376]: FTP: Uploading customgauge14.png. (16 files left to upload.)
 10/18/15 3:00:40.618 PM WeatherCat[5376]: FTP: Uploading customgauge15.png. (15 files left to upload.)
 10/18/15 3:00:40.640 PM WeatherCat[5376]: FTP: Uploading upload-gauges.html. (14 files left to upload.)
 10/18/15 3:00:40.666 PM WeatherCat[5376]: FTP: Uploading customgraph1.jpg. (13 files left to upload.)
 10/18/15 3:00:40.688 PM WeatherCat[5376]: FTP: Uploading customgraph2.jpg. (12 files left to upload.)
 10/18/15 3:00:45.781 PM WeatherCat[5376]: FTP: Uploading customgraph3.jpg. (11 files left to upload.)
 10/18/15 3:00:45.803 PM WeatherCat[5376]: FTP: Uploading customgraph4.jpg. (10 files left to upload.)
 10/18/15 3:00:45.826 PM WeatherCat[5376]: FTP: Uploading customgraph5.jpg. (9 files left to upload.)
 10/18/15 3:00:45.849 PM WeatherCat[5376]: FTP: Uploading customgraph6.jpg. (8 files left to upload.)
 10/18/15 3:00:45.872 PM WeatherCat[5376]: FTP: Uploading customgraph7.jpg. (7 files left to upload.)
 10/18/15 3:00:45.897 PM WeatherCat[5376]: FTP: Uploading customgraph8.jpg. (6 files left to upload.)
 10/18/15 3:00:45.922 PM WeatherCat[5376]: FTP: Uploading customgraph9.jpg. (5 files left to upload.)
 10/18/15 3:00:45.947 PM WeatherCat[5376]: FTP: Uploading customgraph10.jpg. (4 files left to upload.)
 10/18/15 3:00:45.971 PM WeatherCat[5376]: FTP: Uploading upload-graphs.html. (3 files left to upload.)
?10/18/15 3:00:45.998 PM WeatherCat[5376]: FTP: Uploading WeatherCatDailyMovie.mov. (2 files left to upload.)
 10/18/15 3:00:46.023 PM WeatherCat[5376]: FTP: Uploading upload-movies.html. (1 files left to upload.)
 10/18/15 3:00:46.046 PM WeatherCat[5376]: FTP: Uploading WX-date-time-String.php.
 10/18/15 3:01:00.301 PM WeatherCat[5376]: CustomWeb: Getting web cam image files.
 10/18/15 3:01:00.372 PM WeatherCat[5376]: CustomWeb: Finished processing custom web pages.
 10/18/15 3:01:00.660 PM WeatherCat[5376]: WeatherBug:  dateutc=2015-10-18%2020%3A01%3A00&winddir=61&windspeedmph=1&windgustmph=11.0&humidity=29&tempf=65.8&rainin=0.00&dailyrainin=0.000&baromin=30.417&monthlyrainin=0.04&Yearlyrainin=23.95
 10/18/15 3:01:05.169 PM WeatherCat[5376]: CustomWeb: Processing additional custom web file: /Applications/WeatherCat 2.2/Web Templates/Mid-South/index.html
 10/18/15 3:01:08.449 PM WeatherCat[5376]: CustomWeb: Finished parsing.
 10/18/15 3:01:08.473 PM WeatherCat[5376]: CustomWeb: Finished processing custom web pages.
 10/18/15 3:01:08.671 PM WeatherCat[5376]: CustomWeb: Processing additional custom web file:   /Applications/WeatherCat 2.2/Web Templates/Mid-South/WX-date-time-String.php
 10/18/15 3:01:08.902 PM WeatherCat[5376]: Aux. FTP: FTP upload starting.
 10/18/15 3:01:08.929 PM WeatherCat[5376]: Aux. FTP: Uploading index.html.
?10/18/15 3:01:11.648 PM WeatherCat[5376]: Aux. FTP: Finished. Uploaded 5224 bytes in 3 seconds at 1394 bytes per second.

The "upload-movies.html" file is nothing more than this:

Code: [Select]

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <meta name="generator" content="BBEdit 10.1.1" />
    <title>Upload WeatherCatDailyMovie.mov and the 640px backyard image</title>
</head>
<body>
MOVIE$
WCAM640$
</body>
</html>
According to a BBEdit Multi-file search, of the entire "WeatherCat 2.2" directory (which is displayed in post #7, the tag appears exactly twice: In the "upload-movies.html" file and in Documentation/WeatherCat_all_tags/index.html. That latter file is not being uploaded and certainly not the entire catalog of tags listed in that file.

From the com.trixology.weathercat.plist:

Code: [Select]

<key>CustomWebPageHTMLPath</key>
    <string>/Applications/WeatherCat 2.2/Web Templates/Mid-South</string>That seems correct to me.

And the list of six files as seen in the "CustomWebUploads.plist" is correct. The entry for the "upload-movies.html also looks correct (31 minutes, not 2):

Code: [Select]

<dict>
    <key>FTP</key>
    <true/>
    <key>enabled</key>
    <true/>
    <key>path</key>
    <string>/Applications/WeatherCat 2.2/Web Templates/Mid-South/upload-movies.html</string>
?   <key>periodMins</key>
?   <real>31</real>
</dict>
Thia post is getting too long and complicated.  Perhaps I should simply Shut down the mini and see if things work more as I think they should?

[xairbusdriver]

Anyone interested in seeing some Console logs?
Am I the only one seeing this 'double-uploading'?
I am concerned that I am wasting bandwidth with this behavior which I simply do not understand. Nor can I find any info about why it is occurring.

Thanks!

[awilltx]

Hello xair,


Would you post screenshots of your WC pref Custom Web "Main" and "Additional files" panes and then zip up those files so I could take a look.


Thanks,


Alan

[xairbusdriver]

The "Additional Files" list:

The files in the actual HTML Source path:

• /Applications/WeatherCat 2.2/Web Templates/Mid-South/
• index.html
• statistics.html
• upload-gauges.html
• upload-graphs.html
• upload-movies.html
• WX-date-time-String.php

The HTML Source path info:


Hope this helps. I note that the correct html & php files are in the WeatherCatCustomWeb directory. Also the newly created custom gauges and graphs. It appears that WC also creates the basic images for things such as "all-dials", "all_dialsc", "barometer", etc., but they are not being uploaded, which is correct, I think.

[xairbusdriver]

Here are the three html files that "request" the custom gauges, graphs, and the QuickTime movie. Perhaps WC is 'confused' by these files? I can only assume that something I am doing or the way I have things set up is causing this behavior.

[awilltx]

Thanks for the info. Based on your "main" screenshot, you are processing ALL of the files in your Mid-South directory every two minutes, then your additional files (the same files!) are being processed every two minutes (except for your movie file, which is set to every 51 minutes). Basically, you are uploading your (big, I presume) movie file every two minutes as well as every 51 minutes. Additionally, you are double uploading all of the others at two minute intervals.


To rectify:


1. Remove all of your "additional files"


2. Move your upload-movies.html to another folder (other than "Mid-South") then add this one file to the "additional files" pane at 51 minutes.


Hope this helps!


Alan