SERP APIs and rank tracking: why my clients' positions did not add up
SERP APIs return a different page from the one you see, and a different one from one request to the next. Good for reading intent, not for tracking positions: for those, use Search Console.
Davide Ruggeri
Where I started
In kwrm, the tool I use to map my clients’ pages, there is a section that keeps track of the queries I have already assigned a page to. To find out where each page ranked in the SERP, I queried SerpApi.
The positions didn’t add up. When I searched the same queries from my own browser, I found the clients’ sites in different places, sometimes very different from the ones kwrm recorded. And a ranking report that doesn’t match what the client sees on their phone is worse than no report.
I rechecked the parameters, then built a benchmark to work out who was right: 30 keywords searched by me and, right after, with SerpApi and DataForSEO. I collected the numbers in a separate article. Here I care about what I took from it: the answer was less clear-cut than I expected.
There is no “true” SERP
The first mistake was mine: I assumed there was a correct results page and that the API should reproduce it. It doesn’t work that way.
Google personalises the page based on history, account and location. Even in incognito mode it locates you from your IP address. In my first test, searching “pizzeria napoli”, the first result was a pizzeria in Bergamo with “Napoli” in its name, a few kilometres from my home. No API could see it, because it simulated a search from Italy in general.
An SEO describes the same problem in a Local Search Forum thread: automated tools often show different results from the ones people see. In the replies, someone points out that every person and every tool gets a different page. DataForSEO, too, in its page on the differences between API and SERP, lists personalisation, location and algorithm updates as causes.
The biggest surprise came when I repeated the searches. With the same settings, the same API returned a different page every time: about half of the results changed from one request to the next, and the first one often changed too.
So the SERP changes even when you change nothing. Any comparison based on a single search, mine or the API’s, contains a good deal of chance.
Why rank tracking is the worst case
Tracking a position means asking the API for exactly what it does worst: a precise number, repeated over time. I ran into three problems.
The first is absolute position. Even the most faithful API in my test often got the first result wrong, and only in a minority of cases did a result sit in exactly the same position I saw. If you tell a client “you’re third for this keyword” based on an API, they search from their phone, find themselves fifth, and you lose credibility.
The second is monitoring. In kwrm I made one request per keyword at every check. With a SERP that changes by half from one request to the next, that data mostly measured noise: a three-position jump could be a real change or a request that landed on another version of the page.
The third is vague location. In kwrm, on a national query with the location set only to Italy, SerpApi returned pages from Swiss institutions. I can’t say whether precise coordinates would have done better, but giving only the country isn’t enough even for national queries. On local queries the problem is even more obvious, as in the pizzeria case. In the benchmark I passed the coordinates of my town, but even so a difference remains, because Google still estimates the location from the IP.
Where they do work: understanding intent
If you stop looking at individual URLs and look at what kind of pages are there, the picture changes. In my searches the URLs changed often, the nature of the SERP almost never.
On “best robot vacuum cleaner”, both I and the APIs saw guides, reviews and products for sale. On “how to bake bread at home”, recipes and videos. On “pizzeria napoli”, rankings and local results. The positions were different, but anyone would have drawn the same conclusion about intent.
Special elements also hold up better than organic results. Local pack, video, products and AI Overview appeared in most cases both on my page and on the APIs’ pages, often citing the same sources.
That’s why I use them to answer questions like these:
- does this query lead to service pages, e-commerce or guides?
- is there a local pack, a shopping block, an AI Overview?
- who, broadly speaking, are the players occupying the first page?
It’s what another tool I’m building needs, one that decides whether a Google Ads keyword matches the intent of a landing page. One URL more or less doesn’t change the verdict. If a SERP looks ambiguous, a second request costs a few cents and removes the doubt.
How I use them now
After the benchmark I split my questions into two groups: those a SERP API answers well and those that need another source.
- For intent and the type of results I use SERP APIs, always with the location stated explicitly. I judge what kind of pages are there, not which sites.
- For the position of clients’ sites, the number that used to end up in kwrm’s monitoring, the right source is Search Console: the average position comes from the real impressions of people who searched, not from a simulation.
If you want to understand how to set up keyword research or campaigns for your site, tell us about your project.