Friday, July 26, 2013

How to download 'weather data' for your analysis?

As I felt difficult to find this URL, I thought this discovery may help someone.

Weather Data set was used as an example to explain the concepts of Hadoop Framework in Tom White's Book (Hadoop: The definitive guide, 3rd Edition)

# get the data from the following URL (till 1st July 2013)


# get the data from the following URL (after 1st July 2013)


And much more data is available at the website for further analysis, some are free - enjoy.

Caution: Loads of data is available there. Please know upfront about what you are downloading. As the data was captured under small files in large number, It takes a lot of time to download. I prefer downloading a year data at a time. (Ex: Year 2007 has around 12000 files).

Note: the below source didn't provide the exact location as the forum thread closed an year ago. Now, this took nearly a day effort for me to locate the URL. Therefore, enjoy the pleasure, you saved the time.

source of infohttp://mail-archives.apache.org/mod_mbox/hadoop-common-user/201202.mbox/%3CC33D2F48-22E6-4F9B-BFEC-0CEB3419E5F6@doddington.net%3E