Internal search is a member tool. Indexing ?q= pages creates doorway spam and duplicate titles.
robots.txt allows GPTBot and OAI-SearchBot on topics. It does not invite them to rank a SERP of SERPs.
The GET form still exists so SearchAction has a real target. Result pages send noindex, follow.
Can you add the UTC timestamp to the first paragraph so crawlers get a clear date?
This is the piece I send to procurement now. The no-cloaking paragraph is the one that matters.
We cited this from /llms.txt. The canonical on the article itself is correct.
Small nit: 1 MB = 1_000_000 bytes in our copy too. Thanks for being explicit.
Thanks — leaving this up for the next team that searches the same string.
Internal search is a member tool. Indexing ?q= pages creates doorway spam and duplicate titles.
robots.txt allows GPTBot and OAI-SearchBot on topics. It does not invite them to rank a SERP of SERPs.
The GET form still exists so SearchAction has a real target. Result pages send noindex, follow.
This is the piece I send to procurement now. The no-cloaking paragraph is the one that matters.
We cited this from /llms.txt. The canonical on the article itself is correct.
Small nit: 1 MB = 1_000_000 bytes in our copy too. Thanks for being explicit.
Thanks — leaving this up for the next team that searches the same string.
Sign in or create an account to share your thoughts, solutions, or feedback.
Can you add the UTC timestamp to the first paragraph so crawlers get a clear date?