BUG: Pagination is broken
When using the 'count' and 'offset' options to paginate results, the 'totalEstimatedMatches' value changes after a certain number of pages.
For example, a search for "cottage with hot tub" on our site initially shows 1630 matches. But after paging through the first 21 pages of results (offset 440), the number of matches drops to 499 for no apparent reason.
Surely the number of matches should stay the same regardless of where within the results you are viewing them?
Ross Lammas
shared this idea
6 comments
-
Anonymous
commented
I have the issue as well. Initial page says 133 results, I go to page 2, and I now have a total of 64 results and the query did not change. It is maddening to say the least when we use this and our client is like what the **** happened to the number I saw....
-
Zach Wood-Doughty
commented
I don't think it's an issue with the estimated matches varying but rather what appears to be an upper limit for how many results can possibly be scraped. In hundreds of API searches, I have not been able to get more than 1000 unique results from any query.
Here's an example. If I go to Bing.com and search for "cat," it estimates that there are 68 million responses. If I search for "cat" with an API query and ask for 50 results and offset 0, I get 47 results and the response tells me that totalEstimatedMatches is 69,100,000. So far, so good.
My second request is for another 50 results for my "cat" query, now with an offset of 50. This time I get 48 results and totalEstimatedMatches of 69,200,000. Great!
Now I keep making queries, every time incrementing the offset by 50.
The totalEstimatedMatches number fluctuates mildly, but is always around 69 million. The number of actual results returned also fluctuates, but is always between 40 and 50. In other words, everything is great......until my offset parameter passes 1000 (the 21st page of results). As soon as that happens, I get a response with only 2 result and totalEstimatedMatches of 1000. Further queries only produce duplicates from the <1000 results I've already received, and increasing the offset further does not help.
This I believe is the issue that everyone below is complaining about -- not the small fluctuations during the first several responses, but the sudden clamp down to 500-1000 matches. If totalEstimatedMatches begins by offering millions of results, it should not suddenly change to offering a thousand or fewer results., especially since it's obvious that Bing has more than 1000 results for "cat." Furthermore, this happens for every query I have tried, and for other queries I get fewer than 1000 results. I cannot find any documentation for this issue in Bing API v7. This appears to affect both Bing Custom Search and Bing Search, making them useless for many simple use-cases.
-
totalEstimatedMatches is a number which is matched with the query however to make API performance better there is a timeout which only show top results for API call and not all the estimated matches. Since timeout occurs at different time period, estimated matches value also vary with time out. Its not a performance issue in custom search, it is a normal behavior of Bing API.
-
Matt Reinert
commented
Yeah, that first number is total BS. No matter what value you get, there are never so many results to fetch. If it says totalEstimatedMatches=10000, try to set the offset to 10000. Or even 9000. I get totalEstimatedMatches values like 91000, but can only page to about 700 or so. You guys really need to fix this.
-
Thomas Brower
commented
I'm experiencing the same issue as well.
-
rcasdorph
commented
I am having the same issue with totalEstimatedMatches changing when reloading for the offset.
