M6.L5 · Search & SEO
Technical SEO without a developer
What you'll be able to doDiagnose your site's crawlability, indexing, Core Web Vitals and structured-data status using free tools, and prioritise the fixes you can make yourself.
Learn
Binary in a way content never is
Technical SEO sounds like a developer's discipline. For a site your size, it isn't: it's about 90% checkable and mostly fixable by you, with free tools, in an afternoon. And it deserves an afternoon, because it's binary in a way content never is — if crawlers can't reach or read your page, everything you built in Lessons 6.2–6.4 doesn't underperform. It doesn't exist. That includes the newest crawlers: the bots that feed AI assistants have to be able to read you too.
This lesson is a diagnosis routine, not a syllabus. You'll run it today and once a quarter after.
Can machines reach you? Crawlable and indexable
Four checks, in plain terms:
- robots.txt isn't blocking bots — the file that tells crawlers where they may go.
- Pages return 200 — they load, publicly, without a login wall.
- No accidental
noindex— the tag that says "don't list this page". Useful on a thank-you page; fatal on a product page. - An XML sitemap exists and is submitted — the machine-readable list of your pages.
On Carrd, Framer, Google Sites, Shopify, WordPress and every mainstream builder, most of this is automatic — which is exactly why the failures are silent. Builders have innocuous-looking toggles ("hide from search engines", "password protect", a theme's "exclude from sitemap") that quietly deindex pages. Nobody notices, because a page missing from search looks identical to a page ranking poorly — unless you check. Which brings us to the tool that checks.
Search Console is mission control
Google Search Console — free, and required from here on. Verify your site (each builder has a walkthrough in the lesson; usually pasting one tag or one DNS record), submit the sitemap, then read two reports:
- Pages / Indexing — which pages Google has indexed, and why it excluded the rest, stated in plain reasons ("Excluded by noindex tag", "Crawled — currently not indexed").
- Performance — the queries you appear for, with clicks and positions. You met this in Lesson 6.2 as a keyword source; it's also your measurement instrument for the whole module.
If you set up one tool in this entire Program, this is the tool.
Core Web Vitals, plainly
Google measures how pages feel to real users, with three published thresholds — the numbers are Google's own, and stable enough to print:
| Metric | Means | Good |
|---|---|---|
| LCP — Largest Contentful Paint | Main content visible | ≤ 2.5 s |
| INP — Interaction to Next Paint | Responds when tapped | ≤ 200 ms |
| CLS — Cumulative Layout Shift | Nothing jumps around | ≤ 0.1 |
Measure free with PageSpeed Insights — paste your URL, read the mobile scores. Honest framing: Core Web Vitals are a modest ranking input and a large conversion input. A page that stutters loses buyers regardless of where it ranks. And for a no-code site, the culprit is nearly always the one you met in Module 4: enormous images. Compress them (Squoosh, free) and most red numbers turn green. Mobile is the score that counts — Google indexes your mobile page, full stop — and HTTPS is table stakes every recommended builder includes free.
Structured data, honestly
Schema.org markup is machine-readable labelling — JSON-LD snippets that state "this is a LocalBusiness, at this address" or "this is a Product, price €68, rated 4.7". It helps search engines and AI systems parse who and what you are, and it can unlock rich results (stars, prices, FAQs) that improve click-through.
What it is not: a rankings cheat code. Schema makes true things parseable; it does not make pages rank. Most builders and e-commerce themes output the basics automatically — your job is usually to verify, not write. The lesson provides copy-paste JSON-LD templates for the three cast archetypes (LocalBusiness for Herzog, Organization plus SoftwareApplication for Storkflow, Product for Ostara), and everything gets checked with Google's free Rich Results Test.
The new robots.txt decision: AI crawlers
A decision that didn't exist a few years ago: whether the bots that feed AI systems may read your site — GPTBot, ClaudeBot, PerplexityBot, Google-Extended and their successors (the names will change; the decision won't).
For a business that wants to be discovered — that's you, or you'd not be in this module — blocking them is self-harm: an assistant cannot cite a page its crawler was refused. The counter-case (publishers protecting paid content) is real and isn't yours. What matters is deciding knowingly: check what your builder's robots.txt does by default — some added AI-bot blocks without asking. Next lesson turns this access into strategy.
Ostara's audit, timestamped
Mateus runs the routine on the Shopify store, clock running:
- GSC verified, sitemap submitted (Shopify generates it automatically) — 25 minutes, mostly the DNS record.
- Indexing report shows two size-guide pages "Excluded by noindex tag" — a theme setting shipped that way. One toggle. The pages that answer size questions — the exact queries in his Lesson 6.2 care-and-durability cluster — had been invisible for two years and nobody knew, because nothing looks broken.
- PageSpeed Insights, mobile: LCP 4.8 s — the Sal collection's product photos are uncompressed multi-megabyte files. An hour with Squoosh: 2.2 s. Green.
- Product schema: the theme outputs it; Rich Results Test confirms it validates. Star ratings become eligible to appear under his listings.
- robots.txt: Shopify's default reviewed; AI crawlers not blocked. Decision recorded: allowed, deliberately.
Total: about three hours, €0, no developer — and one genuinely material find. That ratio is typical: most checks pass, one silent failure repays the whole afternoon.
What actually needs a developer — and what you can ignore
Honesty in both directions.
You, alone, free: everything above, plus titles, headings, internal links, alt text, image compression, redirects inside your builder's settings.
A developer (or your platform's support): custom schema beyond what the builder outputs, JavaScript-rendering problems (content that only appears after scripts run — rare on mainstream builders), site migrations, server-level redirects on some platforms. If you have these, hire by the hour with a specific brief — this lesson is the brief.
Nobody, at your size: crawl-budget optimisation, log-file analysis, hreflang (unless you genuinely run multiple language versions), pagination tuning. These are enterprise problems. Knowing what not to do is the skill — every hour spent on them is an hour taken from content that could rank.
Do
Exercise 6.5.1 — Your technical audit
Run your own technical audit with Search Console and PageSpeed Insights, record your real numbers, and fix at least one issue before moving on. If Search Console has no data yet (new site), verification plus sitemap submission counts as your fix.
Write these down — in your plan document, or on the worksheet at the end of this lesson. Ostara's completed audit above is your worked reference.
| What to record | Guidance |
|---|---|
| Search Console verified | Yes, with the date. Each builder's verification is usually one pasted tag or one DNS record |
| Sitemap | Submitted · builder generates it automatically and you confirmed it's present · not applicable — single page, noted as fine |
| Indexing status | Up to 30 words: pages indexed vs excluded, from the GSC Pages report, with the excluded reasons. "No data yet" is fine for a new site |
| Core Web Vitals | LCP (seconds), INP (milliseconds) and CLS from PageSpeed Insights, mobile — each marked pass / needs improvement / fail against the published thresholds above |
| Schema status | None yet · builder adds it automatically, verified · template added, validation passed — for the last two, keep the Rich Results Test link |
| AI crawler status | AI bots allowed (the default) · some blocked, knowingly, with your reason · unknown — in which case find out before Lesson 6.6 |
| The one issue fixed | 20–60 words: what you fixed today, with before/after evidence where applicable |
| Correctly ignored | Confirm you did NOT spend time on: crawl budget · log files · hreflang · pagination tuning. All four is the right answer at your size |
Where this goes: section 7.5 — Technical health check — of your Marketing Plan.
Check
Rubric
Mark your own work against these criteria.
| Criterion | 8–10 | 5–7 | 1–4 |
|---|---|---|---|
| Real numbers recorded | CWV and indexing figures from actual tool runs today | Numbers present but partial | Estimated or copied from the example |
| A genuine fix shipped | Fix named with before/after evidence | Fix named, evidence thin | "Will fix later" |
| Decisions made knowingly | AI-crawler and schema status checked and stated with reasons | Stated but unverified | Left as "unknown" without attempting the check |
| Proportionality shown | Enterprise topics correctly ignored | Some anxiety about non-issues | Time spent on crawl budget for a 6-page site |
Pass: 5+ on every criterion. Distinction: 8+ on all four.
Quiz — four questions. Pick an answer to see whether you were right.
1. Ostara's size-guide pages get zero search impressions. GSC's Pages report says "Excluded by noindex tag". What happened?
- a) The pages need more backlinks
- b) A theme or builder setting is telling search engines not to index them — remove the noindex ✔
- c) Core Web Vitals are too slow
- d) The sitemap is missing
Why: the report states the cause in plain text. No amount of content or authority work can rescue a page that carries an instruction not to list it — which is why the indexing check comes before everything else.
2. Your mobile PageSpeed test shows LCP of 5.1 s. Which is the most likely single fix?
- a) Add FAQ schema
- b) Compress or resize the oversized hero and product images ✔
- c) Buy a faster domain
- d) Reduce word count
Why: on no-code sites, huge images cause most slow paints. Squoosh is free and Ostara's 4.8 s → 2.2 s took an hour. "Faster domain" is not a thing; schema doesn't affect speed.
3. What is the honest description of what structured data does?
- a) It's a ranking cheat code Google hasn't patched
- b) It replaces the need for good content
- c) It makes true facts about you parseable — enabling rich results and cleaner entity understanding for search engines and AI systems ✔
- d) It's obsolete now that AI reads pages
Why: schema labels reality; it doesn't outrank it. The reward is eligibility — stars, prices, FAQs in results, and a clearer machine picture of who you are, which Lesson 6.4's entity work depends on.
4. A business that wants customers to find it via AI assistants discovers its builder blocks GPTBot and friends by default. What does this lesson advise?
- a) Leave it — blocking bots always improves security
- b) Unblock them — an assistant can't cite a page its crawler was refused; blocking is a publisher's defence, not a small business's ✔
- c) Block Google too, for consistency
- d) It doesn't matter; assistants don't use crawlers
Why: retrieval requires access. The decision deserves to be made knowingly — and for a business seeking discovery, access is the whole point. The bot names will change; the decision is durable.
Advance
Your site is now verified, measured, and readable by every machine that matters — and you know which three-letter acronyms you're entitled to ignore. Most technical SEO anxiety dissolves on contact with an afternoon like this one.
Next: M6.L6 — Being found by AI assistants. Everything in this module — the clusters, the brief, the trust signals, the crawler access — converges on the newest surface: the answer an assistant gives when a customer asks it to recommend someone. Next lesson you audit what assistants currently say about you, and build the plan to change it. It closes the module and its project.
Mark your own work
| Good | Not yet | |
|---|---|---|
| Search Console live | Verified, sitemap handled, reports read | "I'll set it up when the site is finished" |
| Numbers are yours | Today's real LCP / INP / CLS, mobile | The example's numbers, or desktop's flattering ones |
| One thing fixed | A before/after you can show | A list of things to fix someday |
| Access decided | You know what your robots.txt says to AI bots | You've never looked |
| Proportionate | Enterprise topics ignored on purpose | An evening lost to crawl-budget articles |
Worksheet
THE SCHOOL OF NET MARKETING
Lesson 6.5 — Technical SEO without a developer
REACHABLE?
☐ Search Console verified date: ________
☐ Sitemap submitted / auto-generated & present
☐ Pages report read — indexed: ____ excluded: ____
excluded reason(s): _____________________________
☐ No accidental noindex / "hide from search" toggles
FAST ENOUGH? (PageSpeed Insights — MOBILE)
mine good
LCP ____ s ≤ 2.5 s ☐ pass ☐ fix
INP ____ ms ≤ 200 ms ☐ pass ☐ fix
CLS ____ ≤ 0.1 ☐ pass ☐ fix
Usual culprit: huge images → compress (Squoosh, free)
PARSEABLE?
☐ HTTPS on (builders include it)
☐ Schema present (builder/theme or template)
validated with Rich Results Test: ☐ yes
☐ robots.txt read — AI crawlers:
☐ allowed (deliberate) ☐ blocked (deliberate)
☐ unknown → find out before Lesson 6.6
THE ONE FIX SHIPPED TODAY
What: ______________________________________________
Before → after: ____________________________________
CORRECTLY IGNORED (at this size)
✓ crawl budget ✓ log files ✓ pagination tuning
✓ hreflang (unless truly multilingual)
NEEDS A DEVELOPER (hire by the hour, with this sheet)
custom schema · JS-rendering issues · migrations ·
server-level redirects
Next: Lesson 6.6 — Being found by AI assistants.
theschoolofnetmarketing.com/learn/being-found-by-ai-assistants