Author Topic: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)  (Read 20164 times)

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #15 on: October 30, 2013, 04:29:22 AM »
My script works!!! If you have been using Weather Display and sending the data to a MySQL database you can run this script, do some copy and pasting, and be good to go. I've only done 2013 so far and it took about 15 minutes.

On the line with the mysql_connect statement, change DB path to the path to your DB, user to your MySQL user name and password to your password. On the next line, mysql_select_db("dbname",$db);, change dbname to your database. And last, change table in $equipsearch = "SELECT * FROM table ORDER BY month, day, hour, minute"; to your table.

Run the script, it could take awhile. The output will be by year and month. Copy the data, a month at a time, and paste it in to the proper .cat file, i.e. 10_WeatherCatData.cat in the 2013 folder. restart Weathercat and you should be good to go.

Steve

  • Global Moderator
  • Storm
  • *****
  • Posts: 1555
    • DW8454
    • KOHAVON11
    • Avon Weather
  • Station Details: Davis Wireless VP2 Plus w/24 hr FARS, Leaf & Soil Station, 2023 Mac mini M2 Pro, 32GB RAM, Mac OS 14, WeatherCat 3.2
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #16 on: October 30, 2013, 12:40:17 PM »
Hi Steve,
Welcome to the WeatherCat forum, Steve, and thanks for putting this together!

Stu probably has a better handle on requests, but to me it seems like an import tool for bringing in data from either another application of from Weather Underground is the number one feature request. Often you see someone ask about it in their first post, and then never hear from them again when they find out that you can't. You have to assume they went on to buy a different weather station software product.

Were you previously using the Mac or PC version of Weather Display. I'd assume that Brian would write the MySQL database the same for both, but was just curious.

Thanks!
Steve
Steve - Avon, Ohio, USA


CWOP: DW8454 - WU: KOHAVON11 - AWEKAS - CoCoRaHS
PWSweather - WeatherCloud - Twitter - Facebook

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #17 on: October 30, 2013, 02:10:44 PM »
Steve,

I had been using the PC version for years then tried the Mac version when I got my iMac a couple weeks ago.  The Mac version was nice, but just seemed like a bad port of the PC one. The MySQL database seemed to be different on the Mac. When I started it up, it told me I was missing colomns, I don't remember which ones, so I figured I could live without the database.  Then when I upgraded to Mavericks, Weather Display would suddenly stop gathering data. It thought it was but the numbers were never changing.  I tried all sorst of fixes, reinstalling WD, the USB drivers, taking a hub off, but nothing worked. That's when I discovered WeatherCat.

At first I was looking at all the log files that WD created and tried to figure out a way to parse them and create the cat files but was overwhelmed by the amount of data.  After I wrote the script to convert the database over it hit me that I could create the cat files from the database, like some people have done with Wunderground.

You do see a lot of people asking for a converter and other people saying they made a database or did it manually, but they don't say how or post the file.  Obviously they will if you ask, like Rick did, but I figured it would be helpful to just post it.  If anyone has a DB other than WD that they would like a script to convert, either to the WC database or cat file, I'd be more than happy to modify the one's I've writen for them.

Steve

Randall75

  • Storm
  • *****
  • Posts: 1332
  • CWOP-CW6734 WeatherUnderground-KOHNEWAR6
    • CW6734
    • KOHNEWAR6
    • Randy's Weather On The Hill
  • Station Details: Davis Vantage Pro 2 Plus.iMac i5 OS High Sierra 10.13.6 8GB Ram, WeatherCat 3,Logitech 9000 Pro Web Cam
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #18 on: October 30, 2013, 08:46:59 PM »
Hi Steve
 First welcome to WeatherCat
 And yes I would love to have one that would  convert my WeatherGround  data to WeatherCat cat files so if you can get around to it please do


 I set down and manually converted 1 year but it took me for ever having to convert everything metric then putting it in each column manually


Thanks

 

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #19 on: October 30, 2013, 09:05:22 PM »
If anyone downloaded the file I posted, ther was a typo in the math for the dew point. It's has been corrected and updated above.

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #20 on: October 30, 2013, 09:08:35 PM »
Randall,

My script would reqiure the data in be in a MySQL database.  WeatherUnderground would be in a csv file I believe.  If you have downloaded all the data you need, you can send me the file and I'll make a database out of it and see what I can do.

Steve

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #21 on: October 30, 2013, 09:30:22 PM »
Here is the PHP script to convert a Virtual Weather System MySQL database to the .cat format.

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #22 on: October 31, 2013, 12:52:52 AM »
I have added the ability to create the files directly on your web server, as long as it's Unix based.  The files below will still output to the web browser, but will also create a directory on on your server for each year processed and the monthly file for that year. Once it's done, just copy all the directories and files over to where ever you have WeatherCat storing your .cat files. Restart WC and you should be good to go.

WCDev

  • WeatherCat Developer
  • Administrator
  • Storm
  • *****
  • Posts: 2911
    • CW9739
    • ISCOTLAN25
    • Trixology
  • Station Details: Main Station: Vantage Pro-2, 24hr fars, solar, soil/leaf station, extra temp stations, no U.V. WeatherLink IP.
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #23 on: October 31, 2013, 04:48:33 PM »
Nice work  [tup]

Randall75

  • Storm
  • *****
  • Posts: 1332
  • CWOP-CW6734 WeatherUnderground-KOHNEWAR6
    • CW6734
    • KOHNEWAR6
    • Randy's Weather On The Hill
  • Station Details: Davis Vantage Pro 2 Plus.iMac i5 OS High Sierra 10.13.6 8GB Ram, WeatherCat 3,Logitech 9000 Pro Web Cam
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #24 on: November 02, 2013, 06:07:12 PM »
Hey Seve
 You wouldn't happen to have a converter from Excel to .cat files do you? [banghead]  see attahed


Can these be converter into WeatherCat.cat files
thanks
cheers
 [cheers1]

AstroSteve

  • Gentle Breeze
  • **
  • Posts: 58
    • KJ6IIW
    • KCAACTON3
  • Station Details: Davis Vantage Pro 2 Wireless, iMac 27"
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #25 on: November 03, 2013, 03:03:31 AM »
Randall,

Which file do you want converted, the CSV or SQL, or both?

The good news is, I have done the CSV, which looks like Wunderground data. At some point, I may pretty it up and have the script ask for data fields and units, but for now it's easier to write them for specific files.

To run the script, you need to upload the script and the csv file to the same directory on your server and run the script. The csv file needs to be called filename.csv. 

If you want the sql converted too, let me know and I'll do it next week.

By the way, these scripts aren't pretty but they get the job done.

Steve




Randall75

  • Storm
  • *****
  • Posts: 1332
  • CWOP-CW6734 WeatherUnderground-KOHNEWAR6
    • CW6734
    • KOHNEWAR6
    • Randy's Weather On The Hill
  • Station Details: Davis Vantage Pro 2 Plus.iMac i5 OS High Sierra 10.13.6 8GB Ram, WeatherCat 3,Logitech 9000 Pro Web Cam
Re: Importing Data from Weatherlink - Datafile Format (Standard v. Metric)
« Reply #26 on: November 03, 2013, 10:37:53 AM »
Hi Steve
 Thanks Steve I will play with this one and see how it goes
but I think the one for sql that I have would work netter let me try this and see what happens
thanks again