SingRank3 of 10 Tuition Pages Served a Bot Check, Not a Page
Contents13
- What we measured, and what came back
- Why three pages never showed us anything
- Does an open robots.txt mean crawlers get in?
- How far this evidence goes, and where it stops
- How many pages open by saying what they do?
- Who shows a fee, and who makes you ask?
- Question headings and structured data
- Who actually holds these searches
- What we would do this week
- What it costs, including the part against us
- Method and limits — read these before quoting
- Want us to run this on your centre?
- FAQ
A parent in Tampines types "tuition centre singapore tampines" into their phone, taps your result, and your page loads. Their phone runs JavaScript and clears whatever check sits in front of your site, so they never know the check was there. Then someone asks an AI assistant the same question, and your centre is not in the answer.
On 21 September 2026 we ran four real buyer searches for Singapore tuition and measured ten of the centre websites that held a slot. Three of the ten never showed us a page. They returned a bot check instead: a challenge screen, a JavaScript reload, a captcha redirect. We fetched each page three times and got the same thing every time.
Two of those three publish a robots.txt that blocks nothing at all. The door is open on paper and shut in practice. That is the finding, and it is what we would fix before anyone touches a keyword.
What we measured, and what came back
Every figure below comes from our own fetches on 21 September 2026. We took the four searches through SingRank's self-hosted SearXNG metasearch, which blends Bing and Yandex and is not Google's ranking. Then we requested each ranked page directly, without running JavaScript, because that is how a plain crawler sees it.
| # | What we checked | The number |
|---|---|---|
| 1 | Ranked pages that returned readable HTML | 7 of 10 |
| 2 | Pages that returned a bot check instead of the page | 3 of 10 |
| 3 | Sites blocking GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot or ClaudeBot in robots.txt | 0 of 10 |
| 4 | Readable pages opening with a paragraph that names the service | 4 of 7 |
| 5 | Readable pages whose opening paragraph names Singapore or an area | 4 of 7 |
| 6 | Readable pages showing a lesson fee | 3 of 7 |
| 7 | Readable pages with at least one question heading | 5 of 7 |
| 8 | Readable pages carrying a business-entity schema | 4 of 7 |
| 9 | Readable pages carrying no structured data at all | 3 of 7 |
| 10 | Results across the four searches that were a centre's own website | 30 of 55 |
Row 2 is the one that costs money, so it goes first.
Why three pages never showed us anything
Because something in front of the website answered instead of the website. Each of the three did it differently, and each did it on all three attempts.
One returned HTTP 403 with a challenge page titled "Just a moment...", carrying the meta tag robots: noindex, nofollow. One returned HTTP 200 and a near-empty document whose only instruction was a script that reloads the window after five seconds. One returned HTTP 202 and a meta-refresh pointing at /.well-known/sgcaptcha/, a host-level captcha gate.
A browser passes all three without the visitor noticing. A plain fetch does not, because it runs no scripts, solves no challenges, and does not arrive from a home broadband address.
Nobody did this on purpose. Bot protection gets switched on at the host or the CDN, often by whoever built the site, and it works quietly. No owner sees a problem, because every human who visits has a browser.
Does an open robots.txt mean crawlers get in?
No. Robots.txt states a preference, and it is read before the page is requested. Whatever sits in front of the page still decides.
We checked all ten. Seven served a real robots.txt; one returns an HTML page at that address, one returns 404, and one keeps its robots.txt behind the same captcha gate as the rest of the site. Not one of the ten blocked any of the five AI crawlers we tested. One file even names GPTBot and ClaudeBot, then gives them the same path-level rules as every other crawler rather than a block.
So the blocking, where it happens, is not happening in robots.txt.
Google is explicit about this. Its guidance on AI features lists "ensuring that crawling is allowed in robots.txt, and by any CDN or hosting infrastructure" among the best practices for AI features, and says on the same page that you do not need new machine-readable files, AI text files or special structured data to appear. Its crawl budget documentation adds that when a site "responds with server errors (5xx HTTP status codes) or rate-limiting signals (such as HTTP 429), the limit goes down and Google crawls less."
On the assistant side, OpenAI's crawler documentation describes OAI-SearchBot as the bot "used to surface websites in search results in ChatGPT's search features". A bot that cannot read the page cannot surface it.
Our earlier study of 109 Singapore local business pages found none of 69 websites blocking those crawlers in robots.txt, and said the figure was a floor rather than a clean bill of health. These three pages are what that caveat looks like in practice.
How far this evidence goes, and where it stops
Less far than it looks, and we would rather say so than let you quote it too hard.
Our requests came from a data centre, with a browser user-agent string and no cookie history. That is the exact profile bot protection is built to challenge, and a host can let OpenAI's published crawler IP ranges through while challenging everyone else. So we did not prove that GPTBot is blocked on those three sites. We proved that an ordinary request for the page does not get the page.
That still matters, because it means the owner cannot tell from the outside what any crawler sees. The answer lives in the server logs, and almost nobody reads them.
How many pages open by saying what they do?
Four of the seven readable pages. We read the first paragraph of fifteen words or more, outside the header, menu and footer, and checked whether it names tuition, tutoring, enrichment, lessons or classes within 120 words.
The three that missed opened on something else: a founder's publishing history, a line about helping a child adapt to a changing world, and an offer for a free grammar pack. All three are reasonable things to say. None answers the question the parent arrived with.
Four of the seven also named Singapore or a specific area in that opening paragraph, and all seven named a location somewhere on the page. The location is on the site. It usually sits below the part a machine is most likely to quote.
Who shows a fee, and who makes you ask?
Three of the seven readable pages published a lesson fee. The rest ask you to enquire.
The three that publish are specific. One lists a trial at $99 for four lessons with a $25 materials fee, then $129 and $149 for four lessons by level. One lists $83 to $113 per two-hour lesson by level, before GST, with trial classes from $90 to $155. One lists $30, $35 and $45 an hour for lower secondary, upper secondary and JC.
We nearly reported four. A fourth page carried a dollar figure that turned out to be a pre-order price for a workbook. Worth mentioning, because an automated price check will make the same mistake on your site.
Every one of the seven offers a way to get in touch: a form or contact link on all seven, WhatsApp on four, a tappable phone number on three. Nobody is hiding. The gap is what a parent knows before they make contact.
Question headings and structured data
Five of the seven readable pages used at least one heading phrased as a question. One of them reads "Is Scoring Above 90 Marks Very Difficult for PSLE Math?" That is a question a parent actually asks, with the answer underneath it, which is what makes a passage quotable.
Structured data is thinner. Four of the seven declared a business entity, and three carried none at all. One page declared 22 distinct types, including FAQPage, Course and Review, and it was the only FAQPage in the sample. For this sample the order is obvious: a page that cannot be fetched gains nothing from richer markup.
Who actually holds these searches
Across the four searches, 55 results came back and 30 were a centre's own website. The split by search type is the part an owner can spend against.
| Searches | Results | Centre's own site |
|---|---|---|
| "best math ...", "best english ..." | 25 | 10 |
| "best secondary ...", "... tampines" | 30 | 20 |
The two "best subject" searches were held mostly by review sites, roundups and lists. The level search and the town search were held mostly by centres. One centre's own site appeared in three of the four searches.
In our view that is a budget decision. Chasing "best maths tuition centre Singapore" puts you in a queue behind publishers whose entire business is that page. A level page and a town page put you in a queue behind other centres. We looked separately at whether to pay for a slot on a best-of list.
What we would do this week
In this order, for a Singapore tuition centre:
- Fetch your own page the way a crawler does. From a computer that has never visited your site, run
curl -sSL -A "GPTBot" https://yoursite.com/and read what comes back. A challenge, a reload script or a captcha is your first job, and it is a setting at your host or CDN, not a rewrite. - Search your server logs for the crawler names. GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot. If they appear against 403 or 429 responses, you have your answer without guessing.
- Rewrite the first paragraph of the page you most want found. Name the subject, the levels and the area in the first sentence. Three of seven pages here spend that paragraph elsewhere.
- Put a fee or a fee range on the page. Four of seven do not. A range with conditions beats silence, and it filters enquiries you did not want.
- Turn two headings into the questions parents ask you on the phone, and answer each in the two sentences underneath.
Steps 3 to 5 are an afternoon in your CMS. Steps 1 and 2 may need whoever manages your hosting. The rest of the method is in our guide to SEO for tuition centres in Singapore.
What it costs, including the part against us
Bot protection exists for a reason. Tuition sites get scraped for fee tables and timetables, and contact forms get hammered with spam. Switching a challenge off wholesale trades one problem for another. The work is allowing the named crawlers while keeping the gate up for everything else, which is fiddly rather than hard.
Here is the part that costs us. Doing all five steps may not bring you one extra student. If your enrolment comes from Google Maps, from the parent chat group at your feeder school and from siblings of current students, your website is not your bottleneck, and a better website will not become one. The three gated sites in this sample are ranking perfectly well for humans today.
The honest case for fixing it is narrower than "AI is the future". It is that you currently cannot see what a machine reads on your site, that the fix is cheap, and that it gets expensive to reverse if assistants keep taking share of the "which centre near me" question. Our view on when search spending is the wrong call is in three times SEO is the wrong thing to buy.
Method and limits — read these before quoting
- Date and sample. 21 September 2026. Four searches, 55 results, 10 pages measured. Small numbers: one page either way moves a "of 7" figure by 14 points.
- Metasearch is not Google. Results came from SingRank's self-hosted SearXNG. Seven of its engines returned errors that day and Yandex supplied the results we kept. This is competitive shape, not Google's ranking and not its local pack.
- Which pages. The exact URL that held each slot, spread across all four searches, one page per business, each a tuition centre's own site. Directories, roundups, tuition agencies, a mall listing and a social page were excluded. One result was a business SingRank has worked for; it was excluded from the measured ten and is not named.
- Raw HTML only. No JavaScript was executed. Pages that build their content in the browser score worse here than they deserve, which is the point of the measurement rather than a flaw in it.
- Our requests are not a crawler's requests. A 403 to us is not proof of a 403 to GPTBot.
- Repeatability. Every page was fetched three times. The seven readable pages returned identical byte counts on passes two and three, and the three gated pages returned the same gate each time.
- No business is named as failing anything. Counts only.
Want us to run this on your centre?
Send us the one page that should win your subject and your area. We will fetch it the way a crawler does, tell you whether it comes back readable, and show you what a machine can quote from it. That starts as a free visibility check through our contact page. If you want us to do the work afterwards, our SEO, AEO and GEO programme starts from S$500/month.
What we will not promise is a citation. AI platforms choose their own answers, and no agency controls them.
These findings describe ten pages fetched on 21 September 2026. This article names no business as deficient and is not a rating of any company.
FAQ
How do I check whether my website is readable to an AI crawler?
Open a terminal on a machine that has not visited your site and run curl -sSL -A "GPTBot" https://yoursite.com/. Read the output. If your page text comes back, it is readable. If you see a challenge screen, a reload script or a captcha redirect, something in front of your site answered instead of your site.
Does an open robots.txt mean AI crawlers can read my site?
No. Robots.txt only states a preference, and it is read before the page is requested. Your CDN, firewall or host can still refuse. Google's guidance asks owners to ensure crawling is allowed in robots.txt and by any CDN or hosting infrastructure, which are two separate checks on two separate systems.
Should a Singapore tuition centre turn off bot protection?
Not wholesale. Scraping and form spam are real, and the gate is there for a reason. The better move is allowing the named AI crawlers by user-agent or published IP range while keeping the challenge for everything else. That is a hosting or CDN setting, not a website rebuild.
Should I put my fees on my website?
In our view yes, as a range with conditions. Three of the seven readable pages in this sample publish lesson fees and four do not. A published range answers the question a parent has before enquiring, and it filters out the enquiries that were never going to convert at your price.
Is it better to target "best maths tuition Singapore" or a town page?
Judge it by who holds the results. In our four searches, the two "best subject" queries returned 10 centre websites out of 25 results, while the level and town queries returned 20 out of 30. The best-of queries are held largely by publishers whose whole business is that page.
Can I cite these figures?
Yes. Please cite "SingRank, Singapore tuition website study, 21 September 2026" with a link to this page. Quote each figure with its base, such as 3 of 10 pages, and keep the metasearch caveat and the note that our requests came from a data centre address. Challenges to any number are welcome through our contact page.