OzPolFeeds (cont…)

I identified a few things with the feeds that have now been fixed. Dates are properly added to the Democrats feed, the author is now listed at the top of the ALP feed, the Liberals feed has been renamed to reflect that fact that it is the PM media statements being listed.

I have checked the legal note on each site I am scraping from and it all seems ok for the type of purpose I am using it for. The ALP explicitely includes a note allowing this kind of use so no probs there.

And now I am starting to be able to see the benefit from this work. The Democrats comment on the government’s planned fuel excise. The Prime Minister announces the fuel excise. The Democrats respond here and here.

Further, Labor calls for the government to release the Keniry report, then after the release of the report Labor attacks the government, and the Democrats call for banning of live exports. Meanwhile, the government is silent (in fact the only thing the PM seems to want to discuss is Steve Waugh).

There are similar trends around the hunger strikers in Nauru.

What’s cool about all this is that you can see a lot of the dialogue occurring around the issues of the day without worrying about what a journalist has left out – seeing a fuller picture than the mainstream press allows. This to me is the most important benefit of this type of system – even with far from perfect inputs. And hopefully these inputs will improve over time.

Early days, but very interesting…

OzPolFeeds (cont…)

OK. After much black magic and HTML wrangling I’ve managed to hack together a feed for the Liberal Party and a full article feed for the Democrats (remembering that the Dems also have headline only feeds of their own).

So, with another hours hacking I’ll get a feed up for the National Party as well (they have reasonably straight forward HTML, so shouldn’t take too long), and we’ll have objective number one (as outlined in my previous post) out of the way.

So, the complete list of feeds is Democrats (Full Article), Greens, Labor, and Liberals

These feeds will be automatically updated approx. every 4 hours from 6 am to 6 pm each day. Enjoy. Now, where’s my dinner…

Political RSS Feeds

Been playing some more… And I think it’s time to fess up.

I’m in the process of building a system to generate RSS feeds for all the major political parties in Australia. You can already get the Australian Democrats feed, albeit a headline only feed (I intend to whip up a full article feed soon), but no other party seems to be providing feeds. As I use an aggregator (I use and love NetNewsWire) for all my news, and as RSS is such a great format for then sharing and repurposing information, I decided to do somthing about this problem (it’s a problem in mine, and others, view anyway).

So, I have started with some work-in-progress versions for the ALP and The Greens. These are derived from the respective news pages on each of the parties’ sites, just doing a screen-scrape. The Greens site is a mess in terms of the HTML, and the ALP site is only slightly better, but there’s enough hooks to at least generate these feeds.

I’ll be augmenting the feeds to include the auther details soon. The code is developed in .NET, and once I have a chance to clean it up I’ll be making it available under a Creative Commons licence – probably the attiribution-share alike licence.

This is all unofficial. I have yet to approach the parties about whether this is OK. I will in time, and that may result in one or more of these feeds being pulled. Hopefully, given the information I am using is publicly available, and the fact I am doing a programatic pull of the data (not modifying content in any way), will mean this won’t be a problem. We’ll see…

The ultimate goal is to create reasonably useful RSS feeds for all of the major political parties in Australia, then produce an aggregator site for all these feeds, listing entries in chronological order. I am completely open to minor parties or independents also having their feeds listed – be they candidates for seats in this year’s election, sitting members, independent, left-wing, right-wing, I don’t care. I only have so many hours in the day, so I am focussing on the major parties – ALP, Greens, Dems and Liberals. If someone else wants to put the time in to build and/or add a feed, that’s cool – I’m party neutral in that regard.

Once the feeds are in place I hope to be able to add value through categorisation, comments and trackback. But that’s some way off yet (as I am doing this in my very limited spare time).

I am completely open to others getting involved in this project, which I am dubbing “OzPolFeeds” for the moment. If anyone wants to build a feed for a party or candidate, just drop me a line via this weblog (the email link or commenting/trackback to this entry) pointing me to the feed location.

Things afoot…

I’ve been playing. (Those of you that don’t know what RSS is, or aren’t interested in Australian politics probably won’t find that link all that interesting.)

More to come…

NOTE: That link is almost guaranteed to change, but if you want to subscribe, go ahead anyway.