Author Topic: Video formats  (Read 10916 times)

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Need to tweak the 'sed' string (Re: Video formats)
« Reply #15 on: July 12, 2017, 11:55:20 PM »
Dear X-Air and WeatherCat explorers of the wild world UNIX utilities

I think this script can be expanded to do the trick.
Code: [Select]
tell application "Finder"
set filename to name of (thisFile as alias)
set filename to (do shell script "echo " & quoted form of filename & " | sed 's/^[0-9_-]*//'")
set name of thisFile to filename
end tell

That isn't going to do exactly what you want for reasons I'll explain in a minute.

Need to learn more about the "echo" command.

echo does what it says, it echoes back the string that has been given to it to "standard out" (in the UNIX lingo.)  Here is the man page if you are curious:

https://ss64.com/osx/echo.html

Some details I need to work on:
  • Automatically change the name of the converted file to a 'generic' name like WC does for the hourly vids. I think that should be relatively simple with AppleScript. I'm thinking just (a) delete the previous vid and (b) remove the "_Hr_Mn" part of the new one.

The sed command that you have is removing numbers underscores from the front of the string, instead of the back.  So it won't do what you want.

sed is an extremely complex program.  Here is the man page for it:

https://ss64.com/osx/sed.html

My UNIX shell use is very rusty but with a some searching on the web this seems to work for what you want:

Code: [Select]
tell application "Finder"
set filename to name of (thisFile as alias)
set filename to (do shell script "echo " & quoted form of filename & " | sed 's/[0-9_-]*$//'")
set name of thisFile to filename
end tell

Do let us know if this finally accomplishes your objectives!

Cheers, Edouard  [cheers1]


Blicj11

  • Storm
  • *****
  • Posts: 3941
    • EW3808
    • KUTHEBER6
    • Timber Lakes Weather
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP Data Logger | iMac (2019), 3.6 GHz Intel Core i9, 40 GB RAM, macOS Ventura 13.6 | Sharx SCNC2900 Webcam | WeatherCat 3.3 | Supportive Wife
Re: Video formats
« Reply #16 on: July 13, 2017, 12:49:41 AM »
I think I can 'encourage' WC to simply upload the vid every 60 minutes by pointing to the file name as an "Additional Files" entry. The only problem is that I have no way to ask/tell WC when to do that. I assume (as dangerous as that may be!) that the 'timer', for any item in the "Additional Files" list, starts when one completes/edits an entry for that item. Perhaps they all start again when the last entry/edit is made? I'll check The Manual! [lol]

Additional Files is for uploading files that have to be updated more often than the interval you have set in Main files. If you set an interval in Additional longer than than the one you have in Main, it will ignore the longer one. However, I think that assumes that all of these various files to be uploaded are in the "upload" directory. I've never tried to upload a file from some other folder in the Additional Files. Let us know if this works.
Blick


xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #17 on: July 13, 2017, 05:17:22 PM »
It's been another very busy day, kept me away from the computer too much! I was about half through 'composing' this and now see that another post got ahead of me. First things first...  :)

Edouard,
I'm aware of what the AS is doing, but it should be simple to delete the underscores and numbers anywhere in the name/text. The script should be simpler since the name/text is always the same length and the part I want to delete is always in the same place. ;)

Blick,
Thanks for pointing the limitation of what WC uploads. At least I know it won't be an "Additional file", DOH!! The converter app can place the file anywhere I choose, but I'll have to review The Manual to confirm that WC could upload it at all. I already did an SFTP upload of a file with Automator...  :)

The converter app has been running for about 36 hours without a problem. I am too busy today to do any testing, but will see what happens with a Folder Action that simply uploads when 'seeing' a new addition to a folder. I hope it simply looking for the OS Event and not be 'confused' that one file is replacing another one (from 24 hours ago). If that simple task can be done, I can just SFTP the file and change the name on the server with php. As a matter of fact, since php is already installed in OS X/10, I could just use it on the file on the mini! It has PHP 5.5.38 installed 10.11.6 while I have 5.6.30 with macOS 10.12.5
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
The sed command works on command line (Re: Video formats)
« Reply #18 on: July 13, 2017, 11:53:16 PM »
Dear X-Air, Blick, and WeatherCat tinkerers,

Edouard,
I'm aware of what the AS is doing, but it should be simple to delete the underscores and numbers anywhere in the name/text. The script should be simpler since the name/text is always the same length and the part I want to delete is always in the same place. ;)

I tested the sed command that I gave you on a UNIX command line and it removed the trailing part of the filename as you were seeking.  So with any luck at all, I've solved that problem and you should be good to go with respect to getting the filename without the hour and minute stamp.

Cheers, Edouard

xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #19 on: July 21, 2017, 08:57:33 PM »
I was browsing through my files on my weather site space and noticed a collection of video's. They were all uploaded just a short time before I saw them. My first thought was, "Why are they here?" My second was, "How did they get here?"

A couple of days this week, I also got an email from WC warning that things were falling off the FTP train. "Files are being generated faster than they can be uploaded..." I didn't think much of those warnings until I saw those fairly large video files (24 on each upload!).

Even though I have a certificate claiming I have a degree in Accounting, I hadn't been able to put two and two together until seeing the extra files and the warnings.

While I can't use the "Additional Files" function to upload these vids, one can, even if accidentally, upload just about anything one might want by simply putting them in a certain folder. My WC prefs/On-Line->Main->Set HTML Source is a particular folder on the mac mini. I'm sure everyone else, with a website, has a similar path there. The fact is, that folder is exactly where I had told the converter app to place its converted QuickTime movies! WC then, dutifully Uploaded the html files, text, php AND the MP4 files for me.

I changed the settings in the video converter app as that's just wasting bandwidth and storage space plus adding to the workload of WC. I have just verified what is probably in The Manual about this behavior. I created three useless little files, placed them in the WC Prefs specified folder and saw them uploaded to my web space in a few minutes. Those 24 video files were uploading every two minutes, in my case. No wonder WC was complaining about its FTP workload! cmu:-) Sorry, Cat, it's my fault! [blush]
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Oops! (Re: Video formats)
« Reply #20 on: July 21, 2017, 11:50:55 PM »
Dear X-Air and WeatherCat tinkerers who occasionally tinker too much!  :-[

Those 24 video files were uploading every two minutes, in my case. No wonder WC was complaining about its FTP workload! cmu:-) Sorry, Cat, it's my fault! [blush]

Yup, that certainly could be a reason why WeatherCat would be in an FTP overdrive condition!

Glad you finally got to the root of the cause!

Cheers, Edouard

xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #21 on: July 31, 2017, 04:06:57 AM »
OK, we have an app that can convert the .mov files to .mp4. It has crashed twice, however, in the ~2 weeks it's been running. I've actually seen an AS that uses functions already in the OS to do this, need further study/investigation.

I have an Automator script that renames the .mp4 files and uploads them to my web space. It then moves that file to the Trash.

The last problem is emptying the Trash. Lots of ways to do that, of course, but Apple wants to protect simple-minded users (like me) from deleting anything without verifying that's really what they want to do. So, all the methods I know of, at least in Terminal, require the users password. You can set the "sudo" time-out to be just about any value you want (default is 5 minutes). Leaving your computer that way also leaves it almost wide open for hackers! Right now, I'm thinking I'll simply let the Trash get as full as I can stand it and then manually empty the Trash. [banghead] ...unless someone knows of a way to get around the password requirement somehow...
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
File ownership? (Re: Video formats)
« Reply #22 on: July 31, 2017, 10:24:00 PM »
Dear X-Air and WeatherCat tinkerers,

The last problem is emptying the Trash. Lots of ways to do that, of course, but Apple wants to protect simple-minded users (like me) from deleting anything without verifying that's really what they want to do. So, all the methods I know of, at least in Terminal, require the users password. You can set the "sudo" time-out to be just about any value you want (default is 5 minutes).

You shouldn't need to use sudo to delete the file.  These files should be owned by you and therefore at least on the terminal a standard UNIX rm command should work without the sudo.  Try running this command on one of your files that you trying to delete:

ls -l

Then paste the output for us to look at.

Cheers, Edouard

xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #23 on: August 01, 2017, 02:46:19 AM »
Quote
Try running this command on one of your files that you trying to delete:

ls -l
"ls" simply lists the files in any directory, it doesn't 'act' on a file.

After "cd"ing to .Trash, "ls" and "ls -1" do the same basic thing; they list all the files in that directory. the -1 just puts each file/directory on a single line so they are easier to see.

To delete anything the "rm" command is needed. (rm - remove files or directories) As in "rm WeatherCatConvertedMovie.mp4"

By George/Edouard, it does work! I was using an rm command that was needed to delete a "stuck" file; one that Finder refused to delete/empty from Trash.

Thanks for pointing out the obvious! [banghead] I'll edit the Automator workflow/app ASAP! [tup]

Changed the last step from "Move Finder Items to Trash" to "Run Shell Script". That script is:
Code: [Select]
rm /Users/myname/Pictures/WC_TimeLapse/WeatherCatConvertedMovie.mp4I think that is specific enough to not accidentally wipe the drive! [lol]

later...

Worked once in a row! [cheer] We'll see how it does over-night... the way I used to work! ThU32:-)
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #24 on: August 01, 2017, 10:01:37 PM »
Almost works. There was a problem last night after the 11pm vid was uploaded. The Automator app/workflow seems to either work or stop. Unfortunately, the error log is empty. I suspect it may have something to do with the last WC video of the day taking longer than others to get rendered. I've changed the launchd plist to only run from 6:05 am through 10:05 pm. We'll see how it goes tonight.

I'm using Launch Control.app to create the launchd plist files. When I discovered that one can specify the various times a daemon should run, I was faced with creating ~18 unique settings. Fortunately, Launch Control anticipated this and allows one to enter these settings in the format cron uses and then it generates the multiple entries required by launchd! Great app! ThU5:-)

I don't see any methods in Automator for any kind of error checking. I think that is more flexible in AS with its "try" function which can switch between (at least) two options if/when a problem arises. Is that correct?
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Wrong command line option. (Re: Video formats)
« Reply #25 on: August 01, 2017, 10:45:06 PM »
Dear X-Air and WeatherCat Mac users who have to play UNIX once in a while,

Quote
Try running this command on one of your files that you trying to delete:

ls -l
"ls" simply lists the files in any directory, it doesn't 'act' on a file.

Sorry, not ls -1 but ls -l .  The command line argument is a low-case 'L' (because of course those clever UNIX devils use the upper-case for something else.  The abbreviation is for long listing.  It will display information like file ownership.  Here is an example on my computer:

Code: [Select]
Odysseus:~ elagache$ ls -l Dropbox/Three\ cars\ at\ Orinda\ Motors/
total 52680
-rw-------@ 1 elagache  staff  5720599 Apr  3 11:25 Coquette and Cadette - front.JPG
-rw-------@ 1 elagache  staff  7322494 Apr  3 11:26 Coquette and Cadette - rear.JPG
-rw-------@ 1 elagache  staff  6538927 Apr  3 11:44 Three cars at Orinda Motors - back.JPG
-rw-------@ 1 elagache  staff  7382372 Apr  3 11:44 Three cars at Orinda Motors - side.JPG
Odysseus:~ elagache$

Glad that you are closing in solving your problem.  Perhaps you won't need this after all.  Still, it is a very useful command to be aware of.

Cheers, Edouard

xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #26 on: August 01, 2017, 11:46:45 PM »
The font that displays here may be controlled by the users settings. That's why it's good to use the "code" tags to display things like ... uhmn ... code! You can see the difference when you used the "code" tags and the lowercase "L" looks much different than a one "1".
Code: [Select]
lowercase "l" looks much different than a one "1"
I've added the mp4 vid to my "Webcam" page and I must say it is amazing how much faster it displays, even without the "poster" attribute! One reason is that it i/5 th the size of the .mov version (3.0MB vs. 15.4). Of course, it's probably has a much smaller resolution... It's also supported in every modern browser without any plugin.

Still, it would be great to have a 'converter' app that was reliable, not to mention my launchd 'creations' having some kind of error handling. [computer]

Is that kind of error handling possible in AS, perhaps when a "try" fails? ???
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


xairbusdriver

  • Storm
  • *****
  • Posts: 3126
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.15.7/WC 3.0.5
Re: Video formats
« Reply #27 on: August 10, 2017, 11:35:10 PM »
I think I have found a solution to the intermittent converter app crashing. I created a daemon that Quits the app and then reopens it a few minutes later. If it was a memory leak, I think that allows its memory blocks to be purged. That daemon (and the app) has been running for almost a week now without any crashes.

I also created a daemon that stops converting and uploading videos created after 10 pm until 6 am local. The black and white videos (IR) may be interesting to possibly capture a raccoon, but even then, only if it happens to be in view when the 'every two minute' jpeg is captured. Perhaps I need a small sign asking critters to stand/sit in a certain spot for at least two and a half minutes to assure a good image... and even then, I'll only capture critters that can read... in the dark!

I discovered another reason for using mp4 vids; they automatically allow "Picture-in-a-Picture" as well as Airplay! I can right now view the video while typing this.
THERE ARE TWO TYPES OF COUNTRIES
Those that use metric = #1 Measurement system
And the United States = The Banana system


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 6494
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Congratulations! (Re: Video formats)
« Reply #28 on: August 10, 2017, 11:52:02 PM »
Dear X-Air and WeatherCat "tinkerers,"

Congratulations on finally coming up with a solution that pleases you!  ThU32:-)  This is something I thought of doing in much the manner you have done.  Alas, these days I just don't have the free time for this sort of tinkering.

Cheers, Edouard  [cheers1]