AI search engines

When someone asks ChatGPT “where to stay in Guatapé” or “which additive do I put in the engine”, the answer brings four or five names. There is no page two. Understanding how those names got into the answer is the difference between being there or not existing for that customer.

It is not an index: it is a reading

Google has an index: a list of pages ranked by relevance, and you compete for a position on that list. An AI assistant does not give you a position. It reads what it finds, checks it against other sources and writes an answer with room for few names. Your goal is not to “climb”; it is to be a source the AI can read, understand and trust.

There are two paths by which an assistant reaches your business:

1. Real-time crawling. ChatGPT with search on, Perplexity and Gemini query the web when they get the question. They request pages, read them at that moment and cite what they find. If your site does not allow crawling or loads so slowly that the crawler gives up, you are not in.
2. What was learned. Models are trained on large amounts of public text. What your site, your reviews and the mentions about you said when that text was collected is what the model “knows” about you without searching. It is slow to change, but it accumulates.

In both cases the raw material is the same: clear, structured, consistent text about who you are, what you offer and where you are.

Who visits you (and who you are leaving out)

Each assistant has a crawler with its own name, and your site can admit or block it from robots.txt:

Crawler	Whose	What it is used for
GPTBot · OAI-SearchBot · ChatGPT-User	OpenAI	training · search in ChatGPT · visits when a user requests a page
ClaudeBot · Claude-User	Anthropic	crawling · visits when a user asks for it
Google-Extended	Google	use in Gemini and in generated answers (Googlebot is still the search one)
PerplexityBot · Perplexity-User	Perplexity	crawling and visits in response to questions
Bingbot	Microsoft	Bing search, which feeds Copilot
Applebot-Extended · CCBot · Bytespider · Amazonbot · meta-externalagent	Apple · Common Crawl · ByteDance · Amazon · Meta	training and their own assistants

What we see most often when auditing a site: the block is in place and nobody decided it. Some CDN providers switch on a “managed” robots.txt by default that bans all AI crawlers; some security plugins do the same. The site looks fine, Google indexes it, and for ChatGPT it does not exist. It is the first thing we check, and for more than one client it was the only intervention needed to start showing up.

What an assistant reads from your site

When the crawler comes in, it does not “see” your page like a person. It reads the HTML. That changes what matters:

• Headlines and structure. An h1 that says what you are and where you are is worth more than a clever headline that says nothing. “19-room boutique hotel in the town center of Guatapé, with its own pier” is a quotable sentence; “Awaken your senses” is not.
• Structured data (schema.org). A block that declares business type, address, coordinates, rooms, policies and frequently asked questions saves the AI from guessing. It is the part of the site written for machines.
• Real text, not images. A price, a schedule or an advantage that only exists inside a photo or a scanned PDF does not exist for a model.
• Consistency across sources. If your name, address or phone number differ between your site, Google and Booking, the AI does not know which one is true and tends not to cite you.
• llms.txt. A plain-text file at the root of the domain, like robots.txt, that describes the business in clear language: what it is, what it offers, where, with links to the important pages. Not every assistant reads it yet, but it costs an hour and it is the most direct way to talk to a model.

What makes it mention you

Being verifiable. An AI prefers to cite what it can confirm in more than one place: your site says you have your own pier, Google reviews mention the pier, the Booking listing lists “lake access”. Three sources that agree turn a claim into a fact, and facts are what an AI answer is made of.

The opposite also works: a site full of adjectives (“unique”, “unforgettable”, “the best”) with no data gives the AI nothing to cite, and it does not cite you.

Where to start, today

1. Ask ChatGPT and Perplexity about your category in your city. Note who comes up and who does not.
2. Open yourdomain.com/robots.txt and look for GPTBot, ClaudeBot, Google-Extended. If they are blocked, make that your decision.
3. Read your own homepage as if you were a crawler: text only. Does it say what you are, where and why?
4. Check that name, address and phone are identical on your site, Google and your listings.

What follows —schema, llms.txt, concrete content— is weeks of work and gets measured before and after. At Porto Marina those four things took the GEO score from 62 to 80 out of 100 without touching the design. The next lesson explains why a website should be built for AI from day one.