Trixology

WeatherCat => WeatherCat General Discussion => Topic started by: AstroSteve on December 19, 2013, 04:43:05 PM

Title: NOAA style files
Post by: AstroSteve on December 19, 2013, 04:43:05 PM
Up until October, I was running WeatherDisplay on a Windows PC. My website uses the template created by Ken at Saratoga Weather.  When I switched to a Mac, I started running WeatherCat because I didn't like the Mac version of WD.  The only down side of switching is I lost the ability to view some pages in Ken's template that use the dailynoaa and climatedataout files that WD creates, specifically the Detail (wx...detail.php), Summary (wx....summary.php) and Monthly Records (wxnoaarecords.php) pages.

While poking around my site, I discovered that Ken had written a script to parse the STMONTHLY ouptput and put it in a style similar to the NOAA report and had an idea. What if I took the script that Ken wrote and add to it to create a dailynoaareport and climatedataout.html file on a regular basis, both as the current month and with the month and year too.  The attached stmonthlytoWDfiles.php file is the result of that thought.

I have a cron job run every hour at 5 minutes before the hour that runs the script.  The top line of the file is required for the script to run on my hosting company. You can remove it if it's not needed.  As written, the script need to be in the same directory that your reports will be in, if you want to have it somewhere else, just update the path in the file_put_contents line around line 308-312. You will also need to adjust the path to the settings and common file to point to the proper location.

After I had that script working, I needed to get the data from past months so there were no holes in my reports.  I still had the files from when I was running WD, but was missing 2 months worth. I have WeatherCat upload the MySql so the data was available, I just needed to write a script to create the files.  That's handled by the createclimate.php and createnoaa.php scripts.

You need to put your database information in the proper spots around line 36 in both files then upload them to the directory where you are storing your noaa files. When you run the, enter the year and month that you want to convert and hit enter.  The resulting report will be printed on you screen and a matching file on your server. Just keep running the report for all the months and years you want to convert.

DISCLAIMER:  I am self taught in programing.  I'm sure there are better and more efficient ways to do what these scripts do, but they work and that's all I care about.  :)  Feel free to modify and improve them. 

I would not have been able to create these scripts without the work done by Ken, thank you Ken!

Steve
Title: Re: NOAA style files
Post by: WCDev on December 19, 2013, 05:15:28 PM
Nice work Steve  [tup]
Title: Thanks Steve!! (Re: NOAA style files)
Post by: elagache on December 19, 2013, 08:45:10 PM
Dear Steve, Stu, and WeatherCat fans,

Nice work Steve  [tup]

Yeah, just what Stu said!!  [bounce]

Seriously, Thanks for the effort!  (http://www.canebas.org/Weather/LWC_forum/Custom_emoticons/thankyou.gif)

If we all contribute a little something we end up so much better off than if were all by ourselves.  That's what makes this community - GREAT!

Cheers, Edouard  [cheers1]
Title: Re: NOAA style files
Post by: monymony on January 10, 2014, 07:35:05 PM
AstroSteve,


Can you provide a link to your website to see these scripts in action?


I am been thinking of trying to create these scripts myself but this might just save me from recreating the wheel.


Thanks!
Title: Re: NOAA style files
Post by: AstroSteve on January 10, 2014, 08:23:58 PM
My website is http://www.actonastro.com/weather/current.php and the results of the scripts can be seen by clicking on Temperature (or Rail, Wind or Barometer), then selecting Detail, Summary or Records. 

What you'll see on my website is actually the results my scripts, displayed by a script from Wildwood Weather, http://weather.wildwoodnaturist.com/downloads.php, in a template from Saratoga Weather, http://saratoga-weather.org/template/index.php.  The output of my script goes in to dailynoaareport*.htm and climatedataout*.html files formatted like Weather Display creates so Wildwood's script can parse it and create the pages you see. You can see the actual files at http://www.actonastro.com/weather/wc/climatedataout12014.html and http://www.actonastro.com/weather/wc/dailynoaareport12014.htm.  Replace the month and year to see other ones or take it out to see the daily ones, which are identical to the current month files.
Title: Re: NOAA style files
Post by: monymony on January 10, 2014, 08:47:47 PM
This is exactly what I have been looking for to add to my site! THANK YOU !!!


BTW, I happened to notice that your NOAA IOD page is not working.  Saratoga put a fix out for this at the end of December that you might want to look at.  It can be found under http://saratoga-weather.org/scripts-images.php#NOAAIOD and look for the December 28th update.
Title: Re: NOAA style files
Post by: AstroSteve on January 10, 2014, 11:00:32 PM
Glad to help. Thank you for the heads up on the NOAA IOD.
Title: Re: NOAA style files
Post by: monymony on January 12, 2014, 11:00:44 PM
Hi Steve,


Great scripts!


I have been able to generate the noaa and climate files from your scripts for my records back to 2007.  I did notice that the initial setting of MaxHi=0 caused some of my records to not import properly when the high temperature was less than 32?F for any particular day.  Fix was easy where I just changed the initial variable setting to make it set to -50 instead of 0 and everything worked properly. 

I did notice on your wxnoaarecords.php that you are showing the barometer high and lows for each day.  Did you do something else to get this data as your scripts don't seem to cover this? 


Thanks again!!


Jay
Title: Re: NOAA style files
Post by: AstroSteve on January 13, 2014, 02:41:05 AM
Jay,

I'm glad you found the scripts useful.  I guess I should have taken in to account people that live in areas where the high doesn't get above freezing sometimes. LOL!

The wxnoaarecords.php script comes from Wildwood Weather. It parses every climatedataout file and figures out the records for all the different readings.

Steve
Title: Re: NOAA style files
Post by: monymony on January 13, 2014, 10:06:26 PM
Hi Steve,


I got it working.   I had to update the wxnoaarecords-include.php file to change all the references from "WD" (and 'WD') to "WCT" (and 'WCT').


At some point I will probably play with this more as just showing the barometer as an average for the day as it is currently missing my Hurricane Sandy extreme low pressure of 28.23in but any coding by me is down the road.  My first task coding-wise is to  first get my manual snow totals to show on my webpage.  I am really hoping this feature to store snow totals gets added to WC v2.  I am guessing snow information is not something you have to worry about in California!


Jay






Title: Almost don't need rain data in CA!! (Re: NOAA style files)
Post by: elagache on January 14, 2014, 10:02:27 PM
Dear Jay, Steve, and WeatherCat extreme weather observers,

I am guessing snow information is not something you have to worry about in California!

Snow he says . . . . .   I could almost get rid of my rain gauge!!  The grand total of rainfall for January 2014 to date is . . . . 0.01"!  (0.25mm)

Us Californians sure are singing . . . . . . how dry I am!! (http://www.canebas.org/Weather/LWC_forum/Custom_emoticons/desert-smiley.gif)

Cheers, Edouard  [cheers1]
Title: Re: NOAA style files
Post by: AstroSteve on January 14, 2014, 10:18:07 PM
Wow, your drought is over! I have 0.00" so far, 0.06" in December, the second driest in 19 years.
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 07:05:26 PM
All,

I must be missing something with all of this. Where is the common.php file from? Settings I'm guessing is the Weather-Settings.php from the template files, but I'm not 100% on this.

Thanks!
Title: Re: NOAA style files
Post by: AstroSteve on March 31, 2014, 07:21:47 PM
No, the Settings.php is a different file then the Settings-weather.php file.  Both the commpn.php and Settings.php are in the same directory as the Settings-weather.php file.
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 07:42:48 PM
Interesting...

I'm seeing Settings here: http://saratoga-weather.org/wxtemplates/Settings-config-USA.php

Ah, okay, I found common.php in the Base-USA here: http://saratoga-weather.org/wxtemplates/install.php

Guess I didn't realize you needed the whole Base-USA directory... We should write up some directions for this... or just get Stu to implement NOAA outputs... =)
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 08:11:47 PM
okay, think I got this working, except that it only seems to pull the first day of the month I give it.

Any thoughts?
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 08:54:37 PM
hmm, so, has something to do with the CONVERT_TZ part of the mysql query... though I'm not sure why yet...
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 09:08:36 PM
Ah ha! So in Mysql 5.5, or Mac OS 10.8.5, when you import the tz data (see here: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_convert-tz), UTC is not just "UTC" it's "Etc/UTC". You need to change that with the date query on line 38.

Hopefully that helps someone. =)
Title: Re: NOAA style files
Post by: AstroSteve on March 31, 2014, 09:17:54 PM
Hmmm, must be Mac OS that's doing it because my provider is running MySQL 5.5.32 and it works fine. I imagine another fix would be to use '+00:00' instead of UTC.  The manual also states that "To use named time zones such as 'MET' or 'Europe/Moscow', the time zone tables must be properly set up."  That could be the problem sice you had to prefix it with what looks like the path.
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 09:43:56 PM
yeah, guessing that's it.

anyway, thanks! still need to do some work, but seems to be working. Any tip on where the "NAME: Saratoga-weather.org" is set?
Title: Re: NOAA style files
Post by: staze on March 31, 2014, 10:08:44 PM
It's also not pulling the lat/long/elevation (which I'm guessing are in Settings.php). Though, I don't see those variables in createnoaa being set anywhere...

was this based on the V2 Saratoga-Weather stuff, and V3 changed enough that things are broken?
Title: Re: NOAA style files
Post by: AstroSteve on March 31, 2014, 11:41:20 PM
That actually comes fro the WCTtags.php file.  I must have had it hard coded when I ran it and forgot to add the include line when I changed it and posted it here.  Oops! None of the variables in the header will not populate without that file.

Add include("WCTtags.php"); after require_once("../common.php"); at the top of both the createnoaa.php and createclimate.php files. You may have to adjust the path to point to wherever your WCTtags.php file is.
Title: Re: NOAA style files
Post by: staze on April 01, 2014, 12:03:33 AM
okay, have that... still being weird about it.

It pulled the name, but not the Elevation, lat, long, which is odd.
Title: Re: NOAA style files
Post by: AstroSteve on April 01, 2014, 12:19:08 AM
Have a look at your WCTtags.php file and make sure all the variables are being save to it.  Search for them without the WX and brackets, i.e Llat or Llong.  They get put in the $WX array when the WCTtags.php file is read.  If they aren't in the file, you will have to edit the file on your computer that creates and uploads that file to your server. in my case it's in the /Applications/WeatherCat/Web Templates/Saratoga/templates dir.  You can find it under Preferences, On-Line, Custom Web tab.
Title: Re: NOAA style files
Post by: staze on April 15, 2014, 12:23:48 AM
Got that all working.

So, how do the NOAA files/renderings work at this point? I got the script to generate the climate and noaa data going, but I'm unsure how the rendering files work... looks like they need data from another set of scripts? Then cron should be running some generation script routinely?

Thanks!
Title: Re: NOAA style files
Post by: JosBaz on April 15, 2014, 04:16:31 PM
Although I admire what some people manage to achieve, this thread shows it is a bit cumbersome (for me at least) to get a simple NOAA style text file generated.

For this reason, and this reason only (getting simple NOAA monthly reports), I switch on my old Weatherlink PC with Windows every couple of weeks.

Stu - what are the chances of getting this functionality in the next release?   [bounce]

Thanks,

Jos
Title: Re: NOAA style files
Post by: staze on April 15, 2014, 04:18:14 PM
Totally agree, and really makes me consider switching to weewx. :(
Title: Re: NOAA style files
Post by: AstroSteve on April 16, 2014, 02:34:45 PM
To generate the daily NOAA files, you need a cron job that runs the createnoaa.php and createclimate.php.  I run it hourly at 5 minutes before the hour.

As to how they work, that's up to you.  The pages on my website use the scripts from http://weather.wildwoodnaturist.com/downloads.php.

Title: Re: NOAA style files
Post by: AstroSteve on October 19, 2014, 04:57:05 PM
I posted this information in a new thread, but figured I better post it here too in case any one coes here looking for help.

When V2 of WeatherCat came out, I noticed that my wind data has been reporting incorrectly.  With the option to report Snow added in this version, the format of the WCTtags file changed, everything after Rain in the STMONTHLY section was shifted over one column to make room for the Snow data.  If you have been using my script to create the NOAA files, you need to replace it with this updated one. Here is the new script.

If, like me, you didn't notice this change till after the end of last month, you need to recreate Septembers files from your database using the files in the first post of this thread.
Title: Re: NOAA style files
Post by: staze on October 23, 2014, 09:29:50 PM
AstroSteve,

See my post here (http://athena.trixology.com/index.php?topic=1422.msg12067#msg12067). Any thoughts?
Title: Re: NOAA style files
Post by: AstroSteve on October 24, 2014, 01:31:09 AM
Wow, that's strange.  Did you build all those reports using the scripts it the first post of this thread? Cause they look messed up from the beginning of time.
Title: Re: NOAA style files
Post by: staze on October 24, 2014, 01:52:10 AM
Yup. Wrote a simple shell script that just fires them over and over (starting with my oldest data in 2009).

Code: [Select]
#!/bin/sh
for ((a=2009; a<=2014; a++))
do
 for ((b=1;b<=12;b++))
 do
  if (($b<10)); then
    c=0$b
  else
    c=$b
  fi
  d="$a-$c"
  curl -d "reportdate=$d" http://127.0.0.1/createnoaa.php
  curl -d "reportdate=$d" http://127.0.0.1/createclimate.php
  #echo $d
 done
done

So... something is definitely wonky here. Looks like the issue truly fixed itself on/around 10/06/2013 (http://www.staze.org/weather/noaa/dailynoaareport102013.html). I don't see the issue reappear after that date. Looks like that's around the time WC 1.2.2 was in beta.

OH! That's the exact day I got my new weather station... before that, I was having my weather data drop in and out, so it was producing NULL values for certain time periods for temp/humidity, etc. But, right before that time, the "rawrain_mm" in mysql is indeed set to 372.4mm, which would be the total rain for... the year? No, it stays in that same range until 2013-08-21 21:18:44, at which point it jumps to 1117.8 for a few hours, then back down to 219.

So, WCDev/Stu, if you read this, any idea what might be going on? It would seem WC might have been getting data from my previous weather station, and some of the values might have been off/missing, which caused WC to insert data into the DB wrong? I guess I could not have reports from before that, but it seems like a huge waste to throw away 4 years of data... Did a DB column get deleted/added/moved during the 1.2.x period?
Title: Re: NOAA style files
Post by: AstroSteve on October 24, 2014, 02:42:06 PM
The rain seems fixed but you're still missing wind data.  Is the wind data in your DB?
Title: Re: NOAA style files
Post by: staze on October 24, 2014, 05:52:35 PM
It is, in fact, the Wind data is in the "text" NOAA files: http://www.staze.org/weather/noaa/dailynoaareport102014.html but missing from the "pretty" output http://www.staze.org/weather/noaa/climatedataout102014.html

So... I'm unsure.
Title: Re: NOAA style files
Post by: AstroSteve on October 24, 2014, 07:47:20 PM
I just ran my creatclimate.php script on a previous month and it created every thing like it's supposed to.  What happens if you run the createclimate scrip direct? Pick a month and just run it for that month without your shell script and see what the data looks like for that month after.
Title: Re: NOAA style files
Post by: staze on October 24, 2014, 09:23:39 PM
I just ran my creatclimate.php script on a previous month and it created every thing like it's supposed to.  What happens if you run the createclimate scrip direct? Pick a month and just run it for that month without your shell script and see what the data looks like for that month after.

Wind data is still missing. Tried on 2014-09. Also manually ran the query against the WindData table, and it returns all the values expected (all 43193 rows).

Also of interest, after and "explain", it might be better to do the Timezone conversion in PHP rather than MySQL. I added an index for the WC tables on Time, but because you're doing CONVERT_TZ in mysql, it won't use that index, so it searches the whole db for what it needs. If you remove that convert, mysql will use the index, and only looks at the values it needs.

Just a thought (I can probably make that change myself pretty quickly and upload the result this weekend).
Title: Re: NOAA style files
Post by: AstroSteve on October 25, 2014, 03:47:56 PM
The only thing I can think of is the query is returning empty on the wind data.  That comes from a different table called WindData.  But the dailynoaareport shows the correct data so that rules that out because the SELECT statements are the same. 

The data is put in the file differently in the climate report.  The noaa report puts it a line at a time while the climate one gets aall the data and puts it a column at a time. For some reason, your data is failing the the IF statement in function Put_row at the end of the script.  I check to see if there is something in the value passed and we are not past the last day in the month. so 1 of 2 things is happening.  Either your days_in_month is getting set wrong, which can't be because all the other reading get put in correctly, or your query is returning nothing. The data is put in an array, it the case of the wind, they are $Ave_windspd, $Ave_wind_dir, $gust and $gust_dir. Yours seems to be getting set to "" for each day.  You can check the query by putting something like:

if (!$result) {
    $message  = 'Invalid query: ' . mysql_error() . "\n";
    $message .= 'Whole query: ' . $query;
    die($message);
}

after the query at line 115. 

If that passes, I would put an echo statement somewhere in the while loop that starts at line 121 to output some of the values that were returned, like echo $row['Windspeed_kmh']."<br />"; , or any other variable you want to check the value of. Your file would look messed up while testing but if you only do it on one month, once you find where the data is getting corrupt, you would fix the problem then run that month again.

Title: Re: NOAA style files
Post by: staze on October 26, 2014, 02:25:37 AM
Turns out I had most my error reporting turned off... See the output when I turned it all on. This is for generating my 2014-10 data...

Code: [Select]
Notice: Undefined variable: c_day in /var/www/html/weather/noaa/createclimate.php on line 51
Notice: Undefined index: 1 in /var/www/html/weather/noaa/createclimate.php on line 67
Warning: Division by zero in /var/www/html/weather/noaa/createclimate.php on line 171
Warning: Division by zero in /var/www/html/weather/noaa/createclimate.php on line 172
Notice: Undefined index: 1 in /var/www/html/weather/noaa/createclimate.php on line 173
Notice: Uninitialized string offset: 1 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 1 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 1 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 1 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 1 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Uninitialized string offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 31 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 2 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 3 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 4 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 5 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 6 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 7 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 8 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 9 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 10 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 11 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 12 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 13 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 14 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 15 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 16 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 17 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 18 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 19 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 20 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 21 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 22 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 23 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 24 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 25 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 26 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 27 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 28 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 29 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined offset: 30 in /var/www/html/weather/noaa/createclimate.php on line 275
Notice: Undefined variable: gust_dir in /var/www/html/weather/noaa/createclimate.php on line 254

BTW, this is PHP 5.3.28.
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 06:53:35 AM
Found it. It's in the Convert_Tz part of the sql query. I made the change in noaa, but not in climate for wind. You have to change "UTC" to "Etc/UTC".

That said, still bad data in the rain field... will have to figure that one out.
Title: Re: NOAA style files
Post by: WCDev on October 26, 2014, 02:20:50 PM
Did a DB column get deleted/added/moved during the 1.2.x period?

No columns added or removed - there's a variable in the private table called dataVersion - if something was changed this would be incremented.
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 02:29:42 PM
WCDev,

Thanks. Any thoughts on the weird rain data? Did you used to store rain totals for a given period rather than amounts for that day? Because as I mentioned earlier, these values are in the MySQL db.

Thanks!
Title: Re: NOAA style files
Post by: WCDev on October 26, 2014, 02:39:00 PM
On a Vantage/Vue the RawRain_mm column is the daily rain as received from the console - there's no processing performed on it (apart from to convert it to mm's). It should start at 0 at the start of the day and increment from there (assuming there's rain on that day).

On a different station type, again it's whatever is available from the console - typically a continuously incrementing amount (until the rain counter resets).
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 02:41:36 PM
On a Vantage/Vue the RawRain_mm column is the daily rain as received from the console - there's no processing performed on it (apart from to convert it to mm's). It should start at 0 at the start of the day and increment from there (assuming there's rain on that day).

On a different station type, again it's whatever is available from the console - typically a continuously incrementing amount (until the rain counter resets).

Ah... That makes sense then. Any knowledge on when that counter resets for the 23xx stations (that's what I had previous to the Vue). That's going to be a bear to re-process. :/
Title: Re: NOAA style files
Post by: WCDev on October 26, 2014, 03:00:41 PM
2499 mm is the max quoted in the 2300 manual.
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 03:04:57 PM
Hmm.  We never get that much rain in a day, so hopefully I can just go day by day and calculate deltas.

That does make things more challenging.

Any insight why you chose to just store data as the console gave it rather than normalize it for all station types?
Title: Re: NOAA style files
Post by: WCDev on October 26, 2014, 03:19:41 PM
I didn't want any processing on the data - that's why the column is called RawRain. It's the raw data feed which means it hasn't been processed beyond converting it to a known set of units - I did debate not converting it but think getting it into known units is worth it.
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 03:21:35 PM
I didn't want any processing on the data - that's why the column is called RawRain. It's the raw data feed which means it hasn't been processed beyond converting it to a known set of units - I did debate not converting it but think getting it into known units is worth it.

Duh. :p should have assumed since it's called raw. Okay. Will have to figure out some post-processing.

Thanks!
Title: Re: NOAA style files
Post by: AstroSteve on October 26, 2014, 06:09:01 PM
I'm glad you found it, Staze.  After you mentioned that, I remembered hearing that before.  You posted that in this thread back in March. LOL.
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 06:11:07 PM
Yup. I just didn't see it missing in the code until I removed that whole chunk. :p

The bigger problem will indeed be dealing with the raw rain values. I assume AS you have a Davis station (confirmed looking at your profile info)? :p
Title: Re: NOAA style files
Post by: AstroSteve on October 26, 2014, 06:22:06 PM
My rain values seem ok, as does yours in your more recent file.  I assume it's just messed up on your older, pre-vantage data.
Title: Re: NOAA style files
Post by: staze on October 26, 2014, 06:38:07 PM
Correct (see WCDev's explanation). I'll just need to figure out rain daily deltas. :/

WCDev, see any reason I shouldn't just overwrite the values for rawrain?