Author Topic: Secure FTP (TLS) Uploads  (Read 716 times)

ELO

  • Gentle Breeze
  • **
  • Posts: 73
    • KCOBRECK14
    • Weather in the Highlands
  • Station Details: Old Davis WMII, Heated Rain Collector, Mac Mini
Secure FTP (TLS) Uploads
« on: May 02, 2020, 03:21:10 pm »
I am running WC 3.0.5 under Mojave on an older MacMini. For several years and WC versions, I've been uploading the Steel Gauges & WC pages via the Custom Web FTP function to my website (oshlo.net/weather) at MacHighway (my hosting service). As of the end of April, MacHighway began requiring that all FTP connections to their servers be secure via TLS v1.2.

That seems to have broken WC's FTP upload process and I can't figure out or find any info on doing a secure FTP scheduled upload via WC under the FTP with TLS protocol. I attempted to change the ftp://... to ftps://... but received an error notice that the ftp:// prefix was required.

Has anyone else encountered this issue? If so, how have you resolved it? Is secure FTP with TLS even possible within WC?

Thanks for your help or suggestions!!

Blicj11

  • Storm
  • *****
  • Posts: 3382
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP Data Logger | iMac (2019), 3.6 GHz Intel Core i9, 40 GB RAM, macOS Mojave | Sharx SCNC2900 Webcam | Supportive Wife
Re: Secure FTP (TLS) Uploads
« Reply #1 on: May 02, 2020, 04:14:18 pm »
ELO:

Edouard knows more than I do about this, but I'm pretty sure Stu has not been able to implement secure FTP because of some issue on the macOS end of things. There was a post somewhere on the forum a year ago about macOS not using anywhere near a current version of Curl. Edouard will fill in the blanks when he checks in.
Blick


ELO

  • Gentle Breeze
  • **
  • Posts: 73
    • KCOBRECK14
    • Weather in the Highlands
  • Station Details: Old Davis WMII, Heated Rain Collector, Mac Mini
Re: Secure FTP (TLS) Uploads
« Reply #2 on: May 02, 2020, 04:22:26 pm »
Blick,

Thanks. I was afraid of that. Hopefully, there is an elegant simple solution, otherwise I'll need to 1) find a new host that still allows straight FTP uploads (although that may be a temporary solution since security seems to be the name of the game); 2) think about trying to jump through hoops to set up my own server; 3) find a good Secure FTP local/remote automated file sync app that will run in the background; or 4) just shut down my site, which is my last choice!!

Rick

Blicj11

  • Storm
  • *****
  • Posts: 3382
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP Data Logger | iMac (2019), 3.6 GHz Intel Core i9, 40 GB RAM, macOS Mojave | Sharx SCNC2900 Webcam | Supportive Wife
Re: Secure FTP (TLS) Uploads
« Reply #3 on: May 02, 2020, 04:33:43 pm »
We are all in the same boat, sooner or later. I have worried about the same thing. I don't know about Catalina; maybe its Curl is a modern version. It's a weird situation because WeatherCat is not transmitting anything close to confidential stuff you don't want anyone to know; in fact, it's just the opposite, it transmits what you want to share with anyone.

Outside of WeatherCat, my hosting service does not allow secure FTP on their shared Linux hosting platform, which I find very strange. 
Blick


elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 5487
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Problem with curl. (Re: Secure FTP (TLS) Uploads)
« Reply #4 on: May 02, 2020, 09:34:46 pm »
Dear Rick, Blick, and WeatherCat troubleshooters,

I don't know about Catalina; maybe its Curl is a modern version.
. . . .

Last time I heard from Stu about this the problem was that the version of curl that came with macOS was really ancient.  Can anybody get the version number of curl on Catalina?  You can get the version number using this command:

curl --version

This is the information provided for the version of curl on Mojave:

Code: [Select]
curl 7.54.0 (x86_64-apple-darwin18.0) libcurl/7.54.0 LibreSSL/2.6.5 zlib/1.2.11 nghttp2/1.24.1
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz HTTP2 UnixSockets HTTPS-proxy

Oddly enough, it supports many secure protocols but not sftp.

Outside of WeatherCat, my hosting service does not allow secure FTP on their shared Linux hosting platform, which I find very strange.

I have basically the same problem with my GoDaddy account.  Supposedly they do support sftp but they only supported a very old version that my FTP client could not work with.  GoDaddy is supposed to upgrade my account soon.  Perhaps I'll finally get sftp support back once more.

Cheers, Edouard  [cheers1]

ELO

  • Gentle Breeze
  • **
  • Posts: 73
    • KCOBRECK14
    • Weather in the Highlands
  • Station Details: Old Davis WMII, Heated Rain Collector, Mac Mini
Re: Secure FTP (TLS) Uploads
« Reply #5 on: May 02, 2020, 09:55:29 pm »
Edouard,

My ISP (MacHighway.com) is now requiring FTP with explicit TLS/SSL, which I believe is a secure extension of FTP. The protocol is FTPS, which the Mojave version of Curl DOES appear to support according to your list. I think the SFTP protocol is an SSH File Transfer Protocol, which is completely different from FTP.

Given that Curl apparently supports ftps, perhaps that would be something that Stu could implement. I haven't tried to contact him directly.

Rick

xairbusdriver

  • Storm
  • *****
  • Posts: 2467
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.14.4/WC 3.0.4
Re: Secure FTP (TLS) Uploads
« Reply #6 on: May 03, 2020, 12:32:15 am »
I'm not on Catalina macOS 10.15 yet, but here is some info about which version of cURL may be current:

Blicj11

  • Storm
  • *****
  • Posts: 3382
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP Data Logger | iMac (2019), 3.6 GHz Intel Core i9, 40 GB RAM, macOS Mojave | Sharx SCNC2900 Webcam | Supportive Wife
Re: Secure FTP (TLS) Uploads
« Reply #7 on: May 03, 2020, 12:35:50 am »
You can see from X-Air's giant post for people with impaired vision that Apple is not the source for the highlighted line. They are down a few lines.

From my son's iMac running Catalina:
curl 7.64.1 (x86_64-apple-darwin19.0) libcurl/7.64.1 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.39.2
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets

From curl, the most recent version is 7.70.0 released earlier this week:
Supports: DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, HTTP/3, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more.

Might be worth contacting Stu directly and asking again about secure FTP.
Blick


xairbusdriver

  • Storm
  • *****
  • Posts: 2467
    • EW7115 (E7115)
    • KTNGERMA20
    • Mid-South Weather
  • Station Details: Davis VP2 wireless + remote Anemometer/2014 Mac min - 10.14.4/WC 3.0.4
Re: Secure FTP (TLS) Uploads
« Reply #8 on: May 03, 2020, 12:42:58 am »
Giant?! I use a less than 1200 pixel wide Safari window (approximately half my screen width) and the image above does not cause a horizontal scroll bar. I very much dislike those things, myself. If you don't mind, perhaps you can tell me how wide your Safari window normally is? And thanks for reminding us all to do a bit of editing before uploading "giant" images. :P

Seriously, I did not notice that the image was actually 1200 pixels wide until I checked my post. I then edited it (three times!) until I could get rid of the horizontal scroll bar. Hopefully, I can remember that 700 pixels is the max I can use at this forum with my Safari window size. YMMV. Likely that "quick draw blicj" saw only the original post only. [lol]

ELO

  • Gentle Breeze
  • **
  • Posts: 73
    • KCOBRECK14
    • Weather in the Highlands
  • Station Details: Old Davis WMII, Heated Rain Collector, Mac Mini
Re: Secure FTP (TLS) Uploads
« Reply #9 on: May 03, 2020, 07:15:49 pm »
Ha! [biggrin] [biggrin]

Thanks guys, I've sent Stu a note.

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 5487
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Our fearless leader has been busy. (Re: Secure FTP (TLS) Uploads)
« Reply #10 on: May 03, 2020, 11:41:26 pm »
Dear Rick, X-Air, Blick, and WeatherCat troubleshooters,

. . . . .
Thanks guys, I've sent Stu a note.

You've done the most prudent thing.  Alas, our fearless leader (Stu) has been overwhelmed by his day-job since before COVID-19 showed up.  I have no idea how this virus has further messed up his karma.  So please be patient.  It may take him a while to get back to you.

Cheers, Edouard  [cheers1]

ELO

  • Gentle Breeze
  • **
  • Posts: 73
    • KCOBRECK14
    • Weather in the Highlands
  • Station Details: Old Davis WMII, Heated Rain Collector, Mac Mini
Re: Secure FTP (TLS) Uploads
« Reply #11 on: May 04, 2020, 02:28:06 am »
Edouard,

Thanks. I hear you. Being a type A personality, patience normally isn't one of my virtues; particularly, when something breaks. Nevertheless, that being said and being 70+ (albeit fortunately in very good health, an avid cyclist, etc.) I have learned some new skills and virtues in the current environment; patience being one. Being healthy, normally an introvert and on the right side of the grass is a blessing in today's world!  [bounce]

Stay safe and sane,

Rick

elagache

  • Global Moderator
  • Storm
  • *****
  • Posts: 5487
    • DW3835
    • KCAORIND10
    • Canebas Weather
  • Station Details: Davis Vantage Pro-2, Mac mini (2018), macOS 10.14.3, WeatherCat 3
Hoping for 50% at least!! (Re: Secure FTP (TLS) Uploads)
« Reply #12 on: May 04, 2020, 11:55:10 pm »
Dear Rick and WeatherCatters caught in a decidedly crazy world, . . . . . .

. . . . .
Stay safe . . . . . .

Well, I am trying to follow all the correct procedures to avoid COVID-19.

. . . . . and sane,

 [wink]. . . . Alas, that's a much more difficult endeavor.  You see, way too many people in the area seem to be stark raving crazy!   [goofy] and alas, this too is to some degree an infectious condition! . . .  lol(1)

. . . . . Being a type A personality, patience normally isn't one of my virtues . . . .

 ;). . . . Well, yes I am trying to wait in joyful hope for the Second Coming,   but I must confess no small amount of impatience as this point!  . . . . .

. . . . . .  [biggrin]

Cheers, Edouard  [cheers1]

ELO

  • Gentle Breeze
  • **
  • Posts: 73
    • KCOBRECK14
    • Weather in the Highlands
  • Station Details: Old Davis WMII, Heated Rain Collector, Mac Mini
Re: Secure FTP (TLS) Uploads
« Reply #13 on: May 05, 2020, 05:43:37 pm »
Gang,

Well, heck if I know why but WeatherCat suddenly began successfully uploading to my hosting service mid day Monday [interesting]. Whether they relaxed requirements for the time being, fixed an internal glitch or something else, I don't know. The net result is WC has again been reliably & routinely uploading for 24 hours, so this seems to have been a false alarm - at least for the time being.

I've sent Stu an update. The current situation and all of this are just further examples of "Life is what happens while you are making other plans..."  [cheers1]

Rick

Blicj11

  • Storm
  • *****
  • Posts: 3382
    • EW3808
    • KUTHEBER6
    • Timber Lakes Utah
  • Station Details: Davis Vantage Pro2 Plus | WeatherLinkIP Data Logger | iMac (2019), 3.6 GHz Intel Core i9, 40 GB RAM, macOS Mojave | Sharx SCNC2900 Webcam | Supportive Wife
Re: Secure FTP (TLS) Uploads
« Reply #14 on: May 05, 2020, 05:45:37 pm »
I love it when a WeatherCatter finds a solution. Let's hope this one keeps on working.
Blick