Our SEO Results, Tested: 0 of 34 Changes Proved Significant

Our SEO results, tested: we logged every SEO change we made to singrank.com between 14 August and 2 October 2026, 145 changes in all. By 2 October, 35 were old enough to judge, giving 34 separate measurements. And 0 of 34 changes proved significant: none showed a click gain that survived a significance test, and not one reached even the loosest bar.

That isn't a confession that SEO doesn't work. It's what honest SEO results look like on a site that earns a handful of clicks a month: most single changes can't be measured at all. Below is the ledger, the test, the one-click swings that fooled our own threshold rule, and a method you can copy to test a change on your own site.

Key takeaways

• 0 of 34 matured changes showed a click gain that passed a significance test (p < 0.05) and then survived Benjamini-Hochberg correction (q < 0.10). None reached p < 0.05 even before correction; the lowest p-value was 0.375.
• 145 changes were logged from 14 August to 2 October 2026. 110 were too young to judge on 2 October; they mature between 7 and 26 October.
• 24 of 34 measurements had zero clicks both before and after the change. All 34 started from four clicks or fewer in their 28-day baseline, and 29 started from zero.
• Our own threshold rule called 13 measurements 'improved'. A control check against untouched pages contradicted 4 of 16 verdicts it flagged as improved or worse.
• 124 of 145 changes were logged as bundles of two or more edits. Just 1 of the 35 matured rows logged a lone edit.

What we logged, and why we're publishing a zero

Every time we change a page on singrank.com, we log it: the URL, the date and what changed. The logging tool saves the page's last 28 days of Search Console data as a baseline. From day 21 after the change, it compares the days since the change, up to 28 of them, with that baseline.

The ledger runs from 14 August to 2 October 2026. It holds 145 changes: new articles, title rewrites, internal links, redirect fixes, template code. Only 35 were 21 days old or more on 2 October. Two of those rows were logged for the same page on the same day and share every number, so there are 34 separate measurements.

We're publishing this because most owners aren't asking what SEO is. They're asking what one owner wrote in a small-business forum thread title: they had "spent $17k+ and still unsure if it was worth it".

A report with a percentage doesn't answer that. A test does, and for small sites the honest answer is often 'not measurable yet'. Our editorial policy explains how we handle our own data.

0 of 34: what the significance test found

For each measurement we asked one question: did clicks rise more than chance would explain? The test compares clicks before and after the change, allowing for the length of each window. It returns a p-value: roughly, the chance of seeing a difference this big if the change did nothing. The usual bar is 0.05, or 1 in 20.

We used an exact binomial test, the same one SciPy's binomtest runs. Run many tests at once and a few will clear the bar by luck. So we added a false discovery rate (FDR) correction, which raises the bar to allow for that. We used Benjamini-Hochberg, the fdr_bh method in statsmodels.

Only 10 of the 34 had any click in either window. Here they are in full:

Change date	Page	Clicks, 28 days before	Clicks after	p-value
20 Aug	AI SEO agency guide	2	0	0.5
21 Aug	Homepage	4	1	0.375
21 Aug	Healthcare SEO guide	0	1	1.0
30 Aug	Blog index	0	1	1.0
30 Aug	AI SEO agency guide	2	0	0.5
2 Sep	Homepage	4	1	0.375
2 Sep	Blog index	0	1	0.491
2 Sep	Pricing page	0	1	0.491
2 Sep	SEO package page	0	1	0.491
7 Sep	Healthcare SEO guide	1	0	1.0

The same move from 0 clicks to 1 scores 1.0 on 30 August and 0.491 on 2 September because the test weighs each window by its length: 28 days after the first change, 27 after the second, since Search Console data ended on 29 September. Neither is anywhere near the bar.

Across all ten, clicks went from 13 to 7. That isn't evidence we made things worse. At these numbers, a fall from 4 clicks to 1 looks dramatic in a report, yet no test can tell it apart from a quiet month. The other 24 measurements had zero clicks on both sides, so there was nothing to test.

For scale: from 30 August to 26 September, the 40 searches we appeared in most showed us 8,782 times and sent no clicks at all. That is how small this site is, and why the ledger works as a warning.

Why our tool said 'tested 0'

Our experiment tool reports a significance summary. For singrank.com on 2 October it read 'tested 0'. Before writing a word, we had to check whether that was a bug or the truth.

It was the truth, with a guard rail.

Only 11 rows carried a p-value at all (the 10 measurements above, with the duplicate homepage row counted twice), because a row needs at least one click to be tested.

The tool won't apply an FDR correction to fewer than 20 tests, because at that size the correction does almost nothing. So it declined to call anything significant. We ran the correction ourselves anyway. The result didn't change: the lowest corrected value was 0.71, far from the 0.10 bar.

The deeper problem is statistical power: whether a test could spot a real gain at all. From our numbers:

• A page that had zero clicks before a change needs 4 to 6 clicks in the next window, depending on its length, before a test can tell the gain from chance.
• A page that had 1 to 4 clicks needs roughly 3 to 8 times the clicks you'd expect without the change.

Put simply, a change would have to at least triple a page's clicks in four weeks to show up. So on a site our size, a real but ordinary improvement is invisible to any honest test. That, more than the zero itself, is the finding.

The threshold called 13 pages improved. The control check disagreed

Our tool also gives a simple verdict: 'improved' if average position gets at least one place better, or clicks rise by a fifth. That rule called 13 measurements improved, 18 flat and 3 worse. We don't trust it, and here's why.

Average position is an average over impressions. Google says Search Console "averages the position value for all impressions" (Search Console Help). So when a page stops showing for its weaker searches, its average jumps up, even though no ranking improved.

Our aeo-and-aio page fell from 114 impressions to 2 while its average position went from 24 to 4.5. Our geo-agency page fell from 347 impressions to 7 and 'improved' from 29.3 to 8.7. Of the 13 'improved' measurements, 7 lost more than half their impressions and 6 had zero clicks before and after.

Pages move on their own. Google's guide to traffic drops says "Small fluctuations in position can happen at any time (including moving back up in position, without you needing to do anything)" (Google Search Central). So we ran a control check on all 16 measurements the rule flagged as improved or worse. The check compares each page with untouched pages on the same site over the same weeks.

Control-check result	Flagged 'improved' (13)	Flagged 'worse' (3)
Indistinguishable from the rest of the site	7	1
Underperformed the site	3	1
Outperformed the site	3	1

Four of 16 verdicts were contradicted: three 'improved' pages underperformed the site, and one 'worse' page outperformed it. Four crossed the usual z = 1.96 line (the same 1-in-20 bar) in the direction the rule claimed. But each did so on a swing of three clicks or fewer. And in each case, the check's change-point test dated the shift in the page's numbers before our change went live.

124 of 145 changes were shipped in bundles

Even with enough clicks, most rows couldn't tell you what worked. 124 of 145 log two or more edits at once, averaging 3.87 changes per row. Think of a new title, three links and an FAQ, all on the same day. Only 1 of the 35 matured rows was a single change.

If a bundle wins, every part of it gets the credit, including the parts that did nothing.

New pages have a different blind spot. A new article has no baseline, so the threshold rule can never call it improved. All 10 matured new pages scored 'flat' automatically.

Why most 'we improved X%' claims can't be trusted

Our view, from the ledger above: when an SEO agency in Singapore, or anywhere, shows you a before-and-after percentage, it tells you almost nothing unless four things come with it.

1. The raw counts. 'Doubled' can mean 1 click became 2. On our homepage, 'down three quarters' meant 4 clicks became 1.
2. The dates of both windows. Google itself says a change can take effect within a few days or only after several months (Google Search Central). A short window catches noise, not effect.
3. A comparison with untouched pages. Without it, a site-wide rise or a seasonal swing gets credited to the change.
4. How many changes were tried. Make 34 changes and a few will look good by chance. NIST's engineering statistics handbook makes the same point about repeated comparisons: run them over and over and the overall significance level is no longer what one comparison promises (NIST). A case study that shows the best one without saying how many were made is a highlight reel, not evidence.

If a report gives you a percentage with none of these, ask for them. Our guide to SEO performance benchmarks sets out what a baseline should hold, and our page on what SEO returns covers the money side.

How to test a change on your own site

You don't need our tools. A spreadsheet and Search Console will do, plus a month of patience.

1. Write it down before you change anything

The page, the date, the one change, and the one number you expect to move.

2. Change one thing per page

If you must bundle, record every edit so you know what the result covers.

3. Check the page can be tested

Pull its clicks for the last 28 days. Fewer than about five clicks means only a large jump will show up in a test. Measure something with more volume, such as impressions for a fixed set of queries, or pool many similar changes and test them together.

4. Wait at least 21 days, ideally 28

Compare 28 days before with 28 days after. Our benchmarks guide recommends a 90-day baseline for judging the site as a whole.

5. Compare with pages you didn't touch

If they rose by the same amount, your change probably didn't cause it.

6. Run the test

In Python, scipy.stats.binomtest(after, before + after, p) with p = days after ÷ (days before + days after). Testing several changes? Correct the p-values with fdr_bh.

7. Write down the answer, including 'not measurable'

That's a result too, and it stops you repeating a change that never earned its credit.

If you split traffic between two versions of a page, follow Google's testing guidance: when a test sends some visitors to a different URL, Google asks for a temporary 302 redirect rather than a permanent 301. Google also notes that the time a reliable test needs depends on "how much traffic your website gets" (Google Search Central). On a small site, that's the whole story.

Method

We exported every singrank.com row from SingRank's experiment ledger on 2 October 2026: 145 changes, the first logged on 14 August. No client data is used anywhere.

We wrote the analysis plan before running our own tests: what counts as improved, which test, the correction and the minimum window. The plan records that we had already seen the tool's top-line counts. Its deviations log lists five later decisions, none of which changes the headline.

A change is matured once it has at least 21 days of Search Console data after it. The baseline is the 28 days logged before the change; the post window runs up to 28 days. The click test is a two-sided exact binomial test, and our script reproduces the tool's own p-value on all 11 rows that carry one. False discovery is controlled with Benjamini-Hochberg at q < 0.10.

The control check is a difference-in-differences test: it compares each page's before-and-after change with the change on untouched pages on the same site. We ran it for the 16 measurements the threshold rule flagged. Position isn't tested for significance, because Search Console gives an average with no spread. The data is in the public CSV listed under Cite this.

Limits

• One small site. singrank.com earns very few clicks, and that is the main reason nothing could be proved. A busier site could detect smaller effects.
• Not randomised. We chose which pages to change, and when.
• Bundles. Most rows log several edits, so even a clear result couldn't name the edit that caused it.
• Clicks only. We didn't test enquiries, AI citations or impressions, and a change aimed at those could work without moving clicks.
• 110 changes not yet judged. They mature between 7 and 26 October 2026. We'll add them when they do.
• Free-text logs. CSV categories come from a fixed rule applied to our own notes.

None of this says SEO changes don't work. It says these 34 didn't produce evidence either way, and it shows why. If your own site is this small, also read when SEO is the wrong spend. More of our studies, such as our census of the first screen of Google results, sit on the SingRank research page.

Want a second pair of eyes on your numbers?

Send us one change you made and the date it went live. We'll tell you whether your traffic is big enough to test it, and how long you'd need to wait. If you later want us to run the work, our SEO pricing for Singapore SMEs starts with Premium Starter at S$500 a month, with a 3-month minimum.

FAQ

How long should I wait before judging an SEO change?

At least 21 days, and 28 is better, then compare the same number of days before and after. Google says a change can show within days or take months to land, so a two-week check is likely to catch noise rather than effect. Judge the site as a whole against your own 90-day baseline, not against one month. For how long a whole programme tends to take, see our AI SEO results timeline.

How do I compare SEO agencies' results in Singapore?

Ask each agency for the same four things: raw click counts, the dates of both windows, a comparison with pages they didn't touch, and how many changes they made in total. A results page that can't supply them is showing you a highlight, not a measured outcome. If two agencies both can, compare those numbers, not their percentages.

How many clicks do I need to test a change?

More than most small pages get. In our ledger, a page starting from zero clicks needed 4 to 6 clicks in the following window before a test could separate the gain from chance. Pages with 1 to 4 clicks needed roughly 3 to 8 times their expected count. Below that, 'not measurable yet' is the honest answer.

Cite this

SingRank, "Own SEO Experiment Ledger 2026", https://singrank.com/blog/singrank-own-seo-experiment-ledger-2026/, exported 2 October 2026. Quote each figure with its base, such as 0 of 34 matured changes. The anonymised file, one row per logged change with no free-text notes, is singrank-own-seo-experiment-ledger-2026.csv.