Republish a feed (or other data) protected by HTTP basic access authentication

Some services let you access or download data on a url that is protected by username and password-popup, or “basic access authentication“, for example your twitter replies at http://twitter.com/statuses/replies.rss …:

basic-authentication-example-twitter-replies

… or your bloglines subscriptions in opml format at http://rpc.bloglines.com/listsubs :

basic-authentication-example-bloglines-subscriptions

PHP script to access authenticated URL

With the following simple php script you can access those files and make them available without authentication:

download Authentication Script zipped together with Snoopy Class file.

Unzip and configure the file/page to fetch (e.g. http://twitter.com/statuses/replies.rss for Twitter) and your username and password:

// Configuration! Change values here, 
// leave rest of script untouched.
$url_to_be_fetched           = "http://someserver/somepage/";
$username_to_be_sent        = "your_user_name";
$password_to_be_sent        = "your_password";

Upload it to a directory on your (php-capable) webserver together with the (included in the zip file) Snoopy PHP class file.

Use Cases

Keeping Private Feeds private

In case of personalised RSS feeds, most services do not protect the feed with authentication, but provide the user with a freely accessible feed at a “secret url”. As soon as you use that feed in Web-based feed readers (Bloglines, Google Reader…) chances are other users will stumble on them when searching.

Which is why Bloglines introduced an <access:restriction> element:

 <access:restriction relationship="deny" />

If a feed contains this element, it won’t show up in searches within Bloglines. Which is why I’ve included the restriction by default in the feed, although you can switch it off by setting the $feed_preferably_private to false:

// in case you use this to be able to read an authenticated feed 
// via Bloglines, you probably do not want it to show up in search 
// results
$feed_preferably_private    = true;

// end configuration

Support for access:restriction element

I haven’t found any mentions of Google Reader supporting it as well, but I have a feed in Google reader with this restriction for more than a month now, and it doesn’t show in the results when I search for it (using a different account). Some people have pointed out though that Google does allow you to share these “protected feeds” via Google Shared Items or Feedburner.

From the services with “personalised/secret feeds” I am using, Facebook supports it, most others like Google Reader Shared Items, the del.icio.us “for” feed and Wakoopa alerts don’t.

Feel free to leave your comments!

12 Responses to “Republish a feed (or other data) protected by HTTP basic access authentication”

  1. [...] http-authentication.  If you have your own webspace, you could de-authenticate the feed with a php script I wrote and host that on a “secret” place2.  That is in fact how most services who publish [...]

  2. mandrill says:

    How do I get a hold of the output? Do I have to use a feed reader or can I use it to populate a webpage?

  3. Pascal says:

    Both is possible…

  4. Batmagnai says:

    thank you for your content i use it and solve one of problem.but when i read file url from rss using snoopy i can’t download file because i requires authentication login. how solve it . do know any way

  5. Pascal says:

    You need to fill out username and password in the script, search on “Configuration” in this posting

  6. Kerry says:

    Great content! Very useful in our experiment. Social Networking and art.

  7. Chad says:

    Awesome. Is there any way to format the results like in the original feed?

  8. Pascal says:

    Hey Chad,

    most feeds don’t care about format (format being layout, not the flavour of rss/atom), some feeds do include a stylesheet.
    it’s possible a feed uses a relative link to a stylesheet, in which case that stylesheet is not found if you republish it from another server.

    Haven’t looked into the problem, since I always consume my feeds in a feedreader, in which case it doesn’t matter.

  9. [...] Republish a feed (or other data) protected by HTTP basic access authentication « PHP Notebook (tags: php httpbasicaccessauthentication rss) [...]

  10. Ashwin says:

    Hi,

    I configured and hosted the script in my server. But when i run it nothing is printed…how am i supposed to read the conent of the rss via this script???

    (I specifically want to use the twitter replies rss only)

  11. Pascal says:

    @Ashwin: not sure what problem is – are you sure your script is able to fetch something over http (some server configs forbid this)

Leave a Reply