Few fields carry as much unsourced folklore as this one. Search engines publish a great deal of documentation, and much of what circulates as technique is absent from it or contradicted outright. So each chapter starts from the belief people actually hold, then reports the documented mechanism against it: three stages that fail separately, the controls acting on each, and the gap between a signal you send and an outcome you get. Where the documentation declines to confirm something widely repeated — a word count, a density target — that refusal is stated plainly, not guessed at.
Each chapter opens with the short version. Tap one to read the detail.
Three stages that fail separately
~2 min
The belief is that being crawled means being indexed. Crawling, indexing and serving are distinct steps that fail in distinct ways, and the documentation states outright that none of the three is guaranteed.
How pages are found, and why fetching is rationed
~2 min
Two beliefs to drop: that you submit a site to search engines, and that a sitemap gets pages indexed. Fetch volume is set by what your server tolerates and how much the crawler wants your pages.
robots.txt blocks crawling, not indexing
~2 min
Ask around and you will be told robots.txt hides a page from search. It does not: a blocked page can still be listed, and blocking it hides the noindex that would have removed it.
What the crawler actually sees
~2 min
"It renders in my browser" is the belief, and the failure it hides is silent. Rendering is a separate queued stage, the crawler arrives with no stored state, and what it renders is your mobile page.
Duplicates consolidate; they are not punished
~2 min
Two beliefs to drop together: that duplicate content is penalised, and that rel="canonical" is a command. Duplication is documented as inefficient rather than punishable, and your canonical is a strong signal in a decision the engine still makes.
The result text is generated, and the myth list is long
~2 min
This is where the documentation refuses most often — no keywords meta element, no word count, no density target. Meanwhile the result's headline is generated rather than taken, and its snippet is selected per query.
Links a crawler follows, images it can read
~2 min
The belief is that anything clickable is a link, and that an image chosen for search carries its own meaning. A crawler follows exactly one construct, and reads an image mostly from the words around it.
Markup buys eligibility, never placement
~2 min
The belief is that adding markup earns a rich result. The documented word throughout is "eligible": required properties get you into the running and nothing further, and describing what a visitor cannot see is a violation.
Many systems, and no single lever
~2 min
Two beliefs to lose: that more pages mean more traffic, and that experience, expertise, authoritativeness and trustworthiness is a ranking factor. Asked the second question directly, the documentation answers no.
Speed, in proportion
~2 min
The belief is that these scores are the ranking lever. Three field metrics describe loading, responsiveness and visual stability at the 75th percentile of real loads — and the documentation is unusually direct that good scores guarantee nothing.
Reading the numbers, and diagnosing a drop
~2 min
Average position is widely read as a rank you hold; it is an average of where you appeared. When traffic falls, which metric moved narrows the cause — and the documented first move is usually not to rewrite the page.
Written by Keentune. We are not affiliated with or endorsed by the organizations whose documentation informs this guide, and any linked sources belong to their respective owners.
All exam, test, and product names and trademarks are the property of their respective owners and are used here for identification and reference only. Keentune is independent study practice — not affiliated with, authorized, or endorsed by any of these organizations.