How guests and AI assistants find The Wentworth
Prepared by Ethan Hawkes · checked 30 October 2026 (12:18 pm ET) · PDF copy
What we checked
- Your home page and contact page, fetched signed out
- Your robots.txt and sitemap
- The structured data (JSON-LD) that search engines and AI assistants read from your home page
- Every finding below is quoted exactly, with the page it came from and the time it was fetched
What we found on your site
“<title>404 Not Found</title>”
Source: https://thewentworth.com/robots.txt, fetched 2026-10-30 12:18 ET, re-checked about 12:20 ET
robots.txt is the first file search engines and AI crawlers ask for. Its absence isn't fatal, but it means they get no directions and no sitemap pointer.
“https://thewentworth.com/sitemap_index.xml → HTTP 200”
Source: https://thewentworth.com/sitemap_index.xml, fetched 2026-10-30 12:18 ET
With no robots.txt, the usual way to tell crawlers about this sitemap is missing.
Why it matters
Crawlers for search and for AI assistants start with robots.txt to learn what to read and where your sitemap is. A 404 there leaves them to discover pages on their own, so new rooms, packages and dining pages can take longer to show up.
3 fixes, ranked
- Publish a robots.txt.Two lines are enough: "User-agent: *" and "Sitemap: https://thewentworth.com/sitemap_index.xml". Most website platforms have a setting for it.Web · 15 minutes
- Submit the sitemap in Google Search Console and Bing Webmaster Tools.This confirms both engines can read it and shows any pages they can't.Web / marketing · 30 minutes
- Re-check in a week.Fetch https://thewentworth.com/robots.txt and confirm it returns the file, not a 404 page.Web · 5 minutes
If it's useful, I'd welcome 20 minutes to compare notes on how guests are finding independent hotels now. — Ethan Hawkes, Signalry