“How do the algorithms that find websites actually work?” Many businesses ask themselves this as soon as their own website is on the agenda. Google, OpenAI and others keep their exact formulas secret, but the basic principles are well documented. Here we explain them in plain language, with tips for SMEs and practices.
Two ways to be found
Classic Google search still matters. Now there is a second route: more and more people ask AI assistants such as ChatGPT or Perplexity directly. For many searches, Google also shows an AI summary above the results, called AI Overviews. There, a ready-made answer with a few linked sources takes centre stage.
So you want to be one of those sources too. The good news: both routes rest largely on the same steps. A machine has to find, understand and rank your pages, and consider them trustworthy.
There is no secret trick – not for Google and not for AI assistants. A technically sound website with clear, credible and up-to-date content is the foundation for both routes.
Crawling: how search engines discover your pages
It all starts with the crawler, a program that automatically fetches web pages and follows the links on them. Google’s crawler is called Googlebot, as Google explains in its overview of how Search works.
So what isn’t linked is hard to find. Good internal linking makes every important page reachable through the menu or through links in relevant text.
An XML sitemap is a table of contents of your website for machines. Common website platforms such as WordPress create one automatically, and you can submit it in the free Google Search Console.
The robots.txt text file (Google’s guide) tells crawlers which areas they should not fetch. Important: it only controls fetching. A blocked page can still show up in search results, for instance if other pages link to it.
Readable addresses help too, such as /physiotherapy/ instead of /index.php?id=372.
Rendering and indexing: what makes it into the index
Next, the search engine displays the page as a browser would, a step called rendering. It then decides whether the page goes into the index, the database behind the search results.
On many modern websites, JavaScript loads the content after the page opens. Google can run JavaScript, but not every crawler does, especially among AI services. Ideally, text and links are already in the HTML your server delivers. A slow or error-prone server also holds Googlebot back.
A noindex tag in the code keeps a page, such as a thank-you page, out of the search results. It becomes a problem when one left over from development hides the entire website. For Google to see the tag, the page must not be blocked in robots.txt (Google on noindex).
Duplicate content arises easily, for example when a page works both with and without “www”. As a rule, this is not penalised, but Google then picks a version itself. Permanent redirects help, as does a canonical tag that marks the authoritative version.
Search Console’s Page indexing report shows whether everything is indexed.
Ranking: who comes out on top
This is the part most people mean by “algorithm”: Google puts the most suitable pages from its index in order. The most important signals:
Relevance: Google has long understood synonyms and context, so repeating a keyword achieves nothing. What counts is answering the question well, in your customers’ own words.
Helpful content: Google aims to favour content written for people. According to Google’s guide to helpful content, its yardstick is E-E-A-T, short for experience, expertise, authoritativeness and trustworthiness. It is not a single score. So show who is behind the website and back up your experience with real examples. Links from reputable websites, such as associations or local media, still count too.
Mobile first: Google generally evaluates the mobile version of your website, so anything missing on a smartphone may go unseen.
Page experience: The Core Web Vitals measure a page’s loading speed, responsiveness and visual stability. LCP (Largest Contentful Paint) shows how quickly the largest visible element appears; up to 2.5 seconds is good. INP (Interaction to Next Paint) shows how quickly the page responds to clicks, taps and key presses; up to 200 milliseconds is good.
INP replaced the older FID metric in March 2024. CLS (Cumulative Layout Shift) shows how much content shifts unexpectedly; 0.1 or less is good. You can check your pages for free with PageSpeed Insights.
A fast page will not overtake better content, but when content is similarly good, page experience can tip the balance.
Local signals for SMEs and practices
For searches such as “physiotherapy Winterthur”, Google often shows a map with listings first. Google names three main factors: relevance, distance and prominence. You cannot change the distance, but you can influence the other two.
The foundation is your free Google Business Profile. Fill it in completely: the right category, opening hours including public holidays, photos, services and a link to your website.
Your company name, address and phone number should be identical everywhere: website, profile, Swiss directories such as local.ch and search.ch, and social media, ideally exactly as in the commercial register.
According to Google, more reviews and positive ratings can help your local ranking. Actively ask satisfied customers for reviews and respond to criticism, too. Buying reviews or rewarding them with incentives breaches Google’s policies.
Structured data: plain language for machines
Structured data is additional information in your website’s code, using the shared schema.org vocabulary. It tells a machine explicitly: this is our company name, our address, our opening hours. For SMEs, the Organization or LocalBusiness types are the most useful.
This helps search engines interpret your content and can make a page eligible for enhanced displays known as rich results. Neither a better position nor such a display is guaranteed, though, and the information must match the visible content.
The rules change, too: in 2023, Google limited expandable FAQ answers to well-known, authoritative government and health websites, and since May 2026 it no longer shows them at all. How heavily AI assistants use such data is not disclosed in detail.
AI search and GEO: how assistants choose their sources
For current answers, AI assistants search the web live, using their own index or a search engine. They then summarise what they find and link to some of it. So a page no crawler can fetch will not be cited.
GEO, short for generative engine optimisation, means preparing content so that AI systems can find, understand and cite it. Much of it is simply good SEO (search engine optimisation).
Google’s AI Overviews draw on the regular search index. According to Google, nothing extra is needed: the page must be indexed and eligible to appear in Search with a snippet.
ChatGPT and Perplexity have their own crawlers: OAI-SearchBot for search in ChatGPT and PerplexityBot for Perplexity (see the documentation from OpenAI and Perplexity). GPTBot, by contrast, collects content that OpenAI may use to train its models. In robots.txt, you can control each crawler separately, so you can appear in ChatGPT search and still opt out of training.
Google-Extended controls whether Google may use your content for Gemini, for example for training. It has no effect on Google Search or AI Overviews. Also check whether your hosting blocks AI crawlers across the board.
No provider discloses exactly how sources are chosen, but three principles follow from how these systems work:
Quotable statements: Answer questions in the first sentence. “We service heating systems of all brands in the Winterthur area, and you’ll get a quote within two working days” is easy to pick up; “We deliver solutions with passion” is not.
Consistent details: The more consistently your website, profiles and directories describe your business, the more clearly an AI can tell who you are and what you offer.
Up-to-date content: Outdated prices or opening hours can lead to wrong answers about you.
And llms.txt? This file is meant to give language models an overview of your website. It is a 2024 proposal, not an official standard, and whether major providers use it is still open.
What you can do now
- Search for your main services on Google and ask ChatGPT or Perplexity about them. Who gets mentioned?
- Set up Google Search Console, submit your XML sitemap and check the Page indexing report.
- Check your
robots.txt, forgottennoindextags and your redirects. - Test your most important pages with PageSpeed Insights.
- Complete your Google Business Profile and ask satisfied customers for reviews.
- Make your company name, address and phone number consistent across all platforms.
- Add structured data about your business, using the schema.org vocabulary.
- Decide deliberately which AI crawlers you allow.
- State the key points on your service pages clearly and keep them current.
Tricks designed to outwit algorithms rarely last. What counts is a website that machines can read easily and that gives people reliable answers: less magic, more craft.
If you would like support: with our website development with SEO and GEO, we build these foundations in from the start. Get in touch for a free first consultation.
