I suggest you ...

stop noisy feeds drowning out quiet feeds

At the moment, feeds which get a lot of posts quickly push out articles from less busy feeds. This is because, when the number of articles in the reading list exceeds the maximum, the oldest ones are deleted regardless of which feed they come from.

If we could specify a maximum number of articles for individual feeds, the latest posts in every feed would always be available, no matter what the frequency of posts may be.

239 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    alex.comer shared this idea  ·   ·  Admin →

    20 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • AdminMariano Kamp (Admin, newsrob) commented  · 

        @Alex.

        Sorry, your post slipped by me.

        Anyway, sounds interesting.

        I am very concerned about how to explain this to a user. I try to avoid help screens, but prefer either intuitive approaches or clear naming instead. So would you deal with that? How would you call those parameters?

        I have no first hand experience with high traffic feeds that drown out other feeds, so correct me if I am wrong. My understanding is that the drowning feeds are the issue, but your solution suggest managing the drowned feeds. That would mean managing, say 50 feeds instead of 1, right?

        It would be possible to define a maximum number of articles to keep on a per-feed basis, with unlimited as the default. However this is not totally intuitive, but more importantly it is a bit ugly/inefficient as it would fetch the meta data for an article, including the article body, just to delete them right away again.

        Also I don't see right now how it should work with my current approach to fetch older unread articles when the user specified s/he wants a maximum number of articles of 1,000, but currently there are only 500 articles in the database at the moment.
        NewsRob then asks for 500 more unread articles. Unfortunately I cannot ask GR for 500 more, but only 22 of those can be of the big bad high traffic feed.

      • AdminMariano Kamp (Admin, newsrob) commented  · 

        @Matt, yes, that would be technically possible. At least at first glance.

        Currently NewsRob fetches older unread messages when the configured capacity (max no of articles) is not reached. NewsRob could try a special label first and when the capacity is not reached after that, then go for the other older unread articles.

        However this solution is not intuitive, which is a major downside.

      • Matthew Snell commented  · 

        I commented earlier on this one. I was thinking maybe an alternative is to have the NewsRob tag only option expanded to have a alternative of NewsRob tag 1st.
        Understand there maybe API issues with this request regardless of the implementation method.

      • MD commented  · 

        Pleeeeeease add this, I hate the fact that noisy sites drown out the quieter sites.
        As Alex suggested, I'd be quite happy to discard unread items from the noisy sites as new items come in from them. MD.

      • Alex Comer commented  · 

        If you can't specify a number to sync per source, how about we turn the problem around and have a _minimum to keep_ setting for each feed? You sync the same number you normally would, but instead of just deleting the oldest posts to make room, you skip any which would take their feed below the minimum.

        You could treat the minima as additional to the global limit, so whilst you sync up to the global limit, you actually store up to the global limit PLUS the sum of the minima. This way, you always have room to keep all the posts you sync.

        Alternatively, you could swap in any new posts in the feeds which were down to their minimum, and discard the oldest posts which don't fit. If the sum of the minima exceeds the global limit, you would have to store up to the sum of the minima, or automatically increase the global limit to fit them.

        This way, users who have problems with busy feeds can specify exactly how many posts they want in each feed. Slow feeds will rarely reach their limit, so they'll be kept safe until we're ready to read them.

        What you reckon? ;o)

      • rajah commented  · 

        Yes , I'm also looking for a similar feature. Setting per feed limit would be very helpful.

      • Moshe Bergman commented  · 

        I am a new NewsRob user and planning to get the pro version.

        I would vote for fetching all of them (basic fetching), and marking anything over X as read. This is what I currently do manually (reading some items then marking all as read)

        The other alternative, is to fetch and fetch top X, when marking all as read to just send mark all as read to google reader?

        If this feature is not for everyone you can enable it via advanced options.

        http://groups.google.com/group/fougrapi/msg/c5e3bfa4d03fde00

      • AdminMariano Kamp (Admin, newsrob) commented  · 

        Steven, also a sound idea, but not possible with the current API.

        I can just sync each feed/label individually (which would equal to many many requests) or completely the reading list and within that reading list I cannot specify a maximum number per source.

        The only thing I could do is fetch them all, and then mark those as read that are over the individual feed limit (or over the global limit per feed).

      • AdminMariano Kamp (Admin, newsrob) commented  · 

        Matthew, this is a sound idea per se, but to gimmicky to use for the "general population". I suspect that also the newsrob-only syncing is rarely used, because of the same issue.

        Implementation wise it would not be an issue however, and actually I already use something very similar which you can see labeled in the progress bar with "fetching *older* unread articles". This I also do to fill up until the configured capacity is met.

        It might be worth a shot as my issue from above should not affect the unsuspecting user as s/he wouldn't see it.

        I'll have to think more about this. I would go for it, if I could sync the reading list except for articles from the "newsrob-fill" tag, but this I can't do because of the API. Instead I would really need to explicit tags: newsrob-prio, newsrob-fill ;(

      • Matthew Snell commented  · 

        I was thinking of a similar request for a slightly different issue. As mentioned, some feeds are noisy and/or have a short currency. I would consider these feeds 'fillers' - I want them because they are are worth scanning for nuggets of quality/relevant articles, but are generally lower quality/lower success rate. My core feeds tend to have a longer currency and higher relevancy rate. My suggestion is to have two newsrob Reader tags. In addition to the normal newsrob tag, a newsrobpriority (or similar) tag that will be synced 1st, any balance in the download quantity will be drawn from the newsrob tag. Example, if you set NR to 250 Articles, if the number of unread articles in the newsrobpriority tag is 300, then on the 1st sync all 250 articles will only come from newsrobpriority. If later in the day, I resync, newsrob would have 250 articles marked as read, and 50 new articles may have arrived, so in the 2nd sync I will get 100 articles from newsrobpriority, and 200 articles from the newsrob tag. Assumption is that if a feed is tagged with both, the duplication will be accounted for & newsrobpriority takes precedence.

      • Steven commented  · 

        This is a very important feature that I would love to have. Even with 1000 limit on my new phone, I still don't see the all the feeds that I wish to see. This could be solved by Google as well by letting you limit the number of items in each feed, but alas they don't currently offer that functionality.
        Mariano - could this be added as a general feature, instead of a Per-Feed setting? Perhaps the general All-Feed limit (250, 500, 1000 etc...) as it currently stands, and then a Individual-Feed limit of 10, 25, 50, 100, 500???

      • Max commented  · 

        Yes, I'm starting to feel the impact of this problem too.
        I believe it is vital not only to have a "count limit", but also a "time limit", i.e. a shelf life after which articles are marked read on a FIFO "first in - first out" basis. Especially news feed articles often lose their relevance after a day, and I would really like to set the days/hours after which they are discarded.
        Luckily, this should be fairly easy to include in the existing architecture, with a "default" setting for initially all feeds and a derivative "manage feed" setting, to fine-tune both time and count limit on on a per feed basis.

      • saurabh kumar commented  · 

        In my case feed from www.delicious.com stops other feeds from loading up. I would want to limit articles from delicious to top N articles only

      • søren commented  · 

        @Mariano: Fantastic! It works good for me. Thanks for the advice.
        I'll keep the votes for the subject however it still seems a good idea to limit the noisy feeds in some way.

      • compaholic commented  · 

        I agree. Fore example: Engadget, Theregister, and Techcrunch are killing my smaller Autoblog, Motortrend, and Hot deals feeds. Even when maxing out the article downloads to 1000. There should be a setting to allow max articles per feed.

      • AdminMariano Kamp (Admin, newsrob) commented  · 

        Søren,

        you can add a widget just for a specific label. This label could then group all your feeds that you consider important enough to be part of the count.

      • søren commented  · 

        In addition it would be nice if i could choose that these noisy feeds don't add to the counter in the widget. Case in point is news feeds that i read when i have the time, i dont really need to track when new items are added.

        Stefan: i bought it anyway :-)

      • Stefan Curtis commented  · 

        Up to now, I've been using the free version, for this I would upgrade to the paid version... This is what's stopping me using Newsrob on a daily basis. Would prefer to specify a time period rather than number of articles.

      • RashaMatt commented  · 

        YES!! :-)
        > Be able to limit the per-feed number of articles to cache, such that on
        > these high-traffic feeds, I only get the freshest stuff, and it doesn't
        > impact the lower-trafficked feeds

      Feedback and Knowledge Base