Avoiding re-parsing of unchanged feeds

I've recently been thinking of an idea to speed up feed parsing in gPodder in cases where the feed contents don't change (i.e. no new or changed episodes), but the server doesn't support E-Tag or If-Modified-Since headers.

Right now, whenever gPodder updates a podcast feed, it sends the E-Tag and Last-Modified headers to feedparser, which will use it to make a conditional HTTP request. In case the server supports it, this will avoid parsing (and downloading) the whole feed, and the server will just tell the feedparser "Nothing new", which in turn will finish the feed update for gPodder - avoiding both the download of the whole feed data, and the parsing inside feedparser and gPodder.

Now, the idea here is that on mobile devices like the N9, we want to avoid CPU usage as much as possible (in addition to keeping the traffic down if possible). One idea now would be to calculate a hash over the downloaded feed content, and remember that hash. In cases where the server doesn't support E-Tag or If-Modified-Since, this could effectively avoid having to parse the feed again. Of course, a hash of the feed contents can only be calculated if the content is downloaded first, so this won't allow for avoiding re-downloads, but in general, I guess that it could improve performance on devices where parsing takes up a non-trivial amount of time.

This might need some experimentation if it is effective in the general case. One idea could be to check the gpodder.net Top 100 podcasts lists and see how many of them support E-Tag and If-Modified-Since and how many of them would benefit from having this special "content hash" in place to avoid re-parsing.

The issue has been filed here, let's see if it's something that could go upstream: feedparser issue 370

Update: Patch added for gPodder and feedparser, both patches can be found in gPodder bug 1634.