authenticated site support

[bdarcus]

Not sure how hard this would be, but it'd be nice if feedability could grab content behind paywalls (say the Wall Street Journal, or soon again, the New York Times).

[4poc]

How should this work? Feedability can only fetch articles that are available to you anyways.

[bdarcus]

Well, assume a subscription. But beyond that, I really don't know; just thought I'd ask.

[4poc]

Ahh okay now I understand what you meant. It depends on the way the authentication is handled. I guess most sites would use some kind of cookie session management, the easiest solution would be to just use the existing cookie jar of the browser (like firefoxs cookies.sqlite). That way feedability should be able to see the articles. The more difficult way would be to actually implement the login procedure itself, so the user just provides the login credentials in the settings. However, since I don't own an account, the latter would require someone with an subscription providing me with information in order for me to implement a specific login procedure.
I tend to implement the first solution sometime, do you know if the authentication is in-fact cookie based? Anyways interesting idea, thanks for sharing.

[bdarcus]

Yes, I think they typically use cookies.

I was trying to figure out how to implement login of, for example, the new york times, using https://github.com/jnunemaker/httparty (mechanize can do this as well) but couldn't figure out how.

[4poc]

I've just added firefox 3/4 cookie support, it is now possible to use the cookies from a firefox profile in feedability. If you are (in firefox) signed-in a news site with subscriptions, and you configure the cookies.sqlite path in the (user_)settings.json file, feedability should be able to see the articles behind the paywall. (at least in theory, I've not tested it against any subscription)

For more detailed documentation on how to setup the new cookies configuration options: https://github.com/4poc/feedability/wiki/Settings (section cookies)