Skip to main content
Conductor customers: Review this lesson in our Community
dynamic contentWelcome, SEO & Content Pros!If you’ve felt like you need a reliable compass to navigate the new, super-fast, and sometimes chaotic world of AI-powered search, you are in precisely the right place.The relationship between brands and their customers is undergoing a massive shift. For decades, we’ve optimized our content to rank on a list of ten blue links. But today, the game has changed. We are now optimizing for a synthesized, generated answer.This is the shift from a traditional “Answer Index” to an AI “Answer Engine”.

​
What to Expect in This Course

We know the industry is flooded with new acronyms right now (LLM, GenAI, AIO, GEO…). We aren’t going to just throw more jargon at you—there’s enough of that on LinkedIn already.Instead, this course is designed to be your strategic playbook. We’re going to take the SEO skills you already have and evolve them to help you win in this new era.By the end of this course, you will be able to:
  • Understand the “Why”: Grasp the fundamental technical shift from “Crawl, Index, Rank” to “Ingest, Embed, Synthesize”.
  • Master the Strategy: Learn the AEO (Answer Engine Optimization) framework and its three core pillars: Authority (Quality + Quantity), Content (Structuring for AI), and Experience (Technical accessibility).
  • Measure Success: Move beyond rankings and learn to track the metrics that matter now: AI Visibility (Mentions & Citations), AI Share of Voice, and AI Referral Traffic.
  • Take Action: Apply tactics to audit your technical health for AI crawlers, build authoritative content ecosystems at scale, and optimize content structure to ensure you are the answer the AI cites.
Let’s get started!🚨Note: Please make sure to click the “start course” button at the bottom of the course and the “complete course” button at the end to get credit for completing the course 🚨
Conductor customers: Review this lesson in our Community
Before we dive in, take 10 seconds and think about your own experience. What’s the biggest difference you’ve noticed between a ‘traditional’ Google search and asking a question to a tool like ChatGPT or Gemini?

​
What is AI Search? From Answer Index to Answer Engine

Let’s clear something up. As savvy content and SEO pros, we know that modern search hasn’t been about simple “keyword matching” for a long time. We’ve already shifted our focus to crucial concepts like E-E-A-T, user intent, and topical authority to prove our value.The traditional search engine model you’ve been mastering is an answer index. It excels at understanding the meaning and intent behind a query to find and rank the most relevant, authoritative webpages to answer it.AI search represents the next powerful evolution. It’s an answer engine.It uses a similar deep understanding of meaning to find the best information, but then it takes a critical next step: it synthesizes information from multiple sources to generate a single, new, conversational answer.This is the fundamental shift. Your goal is no longer just to be the #1 ranked link. Your goal is to be the undeniable source of truth that the AI trusts, consumes, and cites in its generated answer.dynamic content

​
The New Process: From “Crawl, Index, Rank” to “Ingest, Embed, Synthesize”

In our SEO world, we learned the simple model:
  • Crawl: Bots discover your pages.
  • Index: Google stores a copy of those pages.
  • Rank: An algorithm sorts those pages when a user searches.
An AI Answer Engine works on a different, but parallel, logic.
  1. Ingest & Embed (The New “Index”)
Instead of just storing a copy of your page, an AI (like an LLM) reads it and converts the words and sentences into numerical representations called embeddings (or vector embeddings).Think of these as a “language of meaning.” Words or phrases with similar meanings get mapped to points that are numerically “close” to each other. For example, the embedding for ‘kitten’ would be very close to ‘cat,’ ‘feline,’ ‘cat memes,’ (and even ‘grumpy cat’) even though the words are different.
  1. Store in a Vector Database (The New “Database”)
These numerical embeddings are stored in a specialized vector database.
  1. Retrieve & Synthesize (The New “Rank”)
When you ask a question, the AI converts your prompt into its own embedding. It then rapidly searches that vector database to find the pieces of content whose embeddings are numerically closest in meaning to your prompt. This allows the AI to retrieve information that is conceptually relevant, even if it doesn’t contain your exact keywords.And finally, it doesn’t just show you those sources. It synthesizes them into a new, single, conversational answer.dynamic contentThis “how it works” is precisely why the strategies we’ll cover in Lesson 3 are so critical. Your goal is to create such comprehensive, authoritative, and clearly structured content that the AI has no choice but to use your insights for its answer.(Optional Deeper Dive:) So, Where Does the “Intelligence” Come From?You might be wondering, “But how does the AI understand and write the answer in the first place?”You’re right. The “Ingest, Embed, Synthesize” process is only half the story. That’s the retrieval part (like a “search tool”). The other half is the generative part (the “brain”).
  • The “Brain” (The LLM): This is the core “AI model” (like Google’s Gemini or OpenAI’s GPT). It was “trained” using machine learning on a massive snapshot of the internet—the “training data.” This is how it learned grammar, context, reasoning, and how to write.
  • The “Search Tool” (The Retrieval): This is the embedding process we discussed.
When you ask a question, the “Brain” uses the “Search Tool” to find new, relevant content (like your page), and then uses its own “intelligence” to synthesize a new answer for you based on those new facts.Why this matters for AEOYour strategy needs to do two things:
  1. Impress the “Search Tool” with clear, well-structured content (lists, tables) so you are retrieved and cited.
  2. Impress the “Brain” by being a well-known, authoritative source in general. This ensures that even if you’re not cited, the AI’s “foundational knowledge” includes your brand as a trusted entity.

​
From SEO to AEO: Your New Playbook

The world of search is suddenly flooded with new terms: LLM, GenAI, AIO, GEO… it’s an alphabet soup, and it’s easy to get lost in the jargon.But here’s the good news: you don’t need to be a data scientist to win. You just need a new playbook.dynamic contentFor years, we’ve all lived and breathed SEO (Search Engine Optimization). It’s the craft of making our content visible and authoritative for traditional search engines, with the goal of ranking #1.But as we just covered, we’re no longer just optimizing for an “answer index.” We’re now optimizing for an “answer engine” that synthesizes information.This new landscape requires a new strategy. At Conductor, we call it AEO: Answer Engine Optimization.This is the fundamental shift you need to grasp:
  • SEO (Search Engine Optimization): Your goal is to rank #1. You optimize for Google’s algorithm to win the top link on the results page.
  • AEO (Answer Engine Optimization): Your goal is to be the answer. You optimize for your content to be consumed, trusted, and cited by the AI in its generated response.

​
The Three Pillars of AEO

This might sound like a big shift, but your current expertise is the perfect foundation. AEO is built on three core pillars that amplify the principles of quality you already know:
  1. Authority: This is your E-E-A-T in overdrive. The AI engine must recognize you as a trustworthy, expert, and authoritative source, or it simply won’t use your content.
  2. Content: Your content must be structured to provide answers, not just target keywords. This means using natural language and creating comprehensive discussions that thoroughly cover a topic, including all the related sub-topics and user questions.
  3. Experience: This is the technical side of AEO. Your site’s performance, structure, and usability matter more than ever. The AI needs to be able to crawl, understand, and extract your content with zero friction.
dynamic content

​
“My Traffic is Down. Should I Panic?”

Let’s address the fear in the room. You’re starting to see these new AI answer engines providing direct answers, and you’re worried that your top-of-funnel “informational” traffic is going to disappear.Your first instinct is to panic. Don’t.This isn’t the end of traffic; it’s a fundamental shift in traffic quality.Think about the user’s journey. Before, a user looking for “best paint finish for kitchens” had to click 5 different links, read 5 different articles, and figure out the answer for themselves.Now, the AI does that research for them.This means the user who does click on a link from an AI-powered search is no longer a casual researcher. They are a pre-qualified, high-intent lead. They’ve already got their basic answer, and now they’re clicking for one of two reasons:
  1. To buy from the brand the AI recommended.
  2. To go deeper with the source the AI used as the ultimate authority.
In both cases, that click is now infinitely more valuable.This is the very core of your new AEO strategy. You are shifting your focus from winning all the clicks to winning the clicks that matter.It’s about building such undeniable authority that when a high-intent user is ready to act, your brand is the one they trust. The rest of this course will show you exactly how to build that authority and win those high-quality conversions.

​
The AI Landscape: Two Major Shifts Are Happening Now

To build a winning AEO playbook, we need to understand the new landscape. It’s not just one change; it’s two major shifts happening at the same time.Shift 1: AI in the Search Results (Google)This is the change that’s being pushed to billions of users, whether they ask for it or not.
  • What it is: Google’s AI Overviews.
  • How it works: Google now automatically provides a synthesized, AI-generated answer at the top of many search results, powered by its Gemini model.
  • Why it matters: Because of Google’s massive scale, this is where millions of mainstream users are having their first (and most frequent) interaction with generative AI. They are still typing in traditional, keyword-driven queries, but they’re getting a new kind of result.
dynamic contentShift 2: AI as the Destination (The Chatbots)This is the new behavior that’s driving a more profound, long-term shift. This is where users are purposefully going to have a conversation with an AI.Instead of just keywords, these users are bringing complex, multi-step prompts. They’re not just searching; they’re collaborating, researching, and creating.dynamic content

​
Meet the Key Players

This “Destination AI” space is where we see a ton of new players. Here’s a quick breakdown of some major ones:dynamic content

​
Two Channels, One Core Strategy

This new landscape gives you two powerful new channels to reach your audience: the passive AI on Google and the active AI in these destinations.Here’s the simple breakdown:dynamic content(Note: See what we did there? The goal is effectively the same. Whether it’s Google or a Chatbot, you need to be the known entity that provides the cited facts.)

​
Don’t Get Lost: It’s Still About Your Customer

It’s easy to get lost in the sauce of new tools, tactics, and acronyms. But let’s bring it back to basics.The fundamentals have not changed.You are still trying to help people find what they’re looking for. Your goal is to be the best, clearest, and most authoritative answer to their questions.Whether it’s a traditional link, an AI Overview, or a chatbot response, you win by proving you understand what your customer really needs and by giving them the highest-quality content to help them.AEO is just a new playbook for doing exactly that.

​
Lesson 1 Key Takeaways

  • AI Search is an “answer engine,” not just an “answer index.” It synthesizes new answers.
  • Our new job is AEO (Answer Engine Optimization), and our goal is to “be the answer.”
  • AEO is built on three pillars: Authority, Content, and Experience.
  • Don’t panic about traffic loss. AEO focuses on capturing higher-quality, higher-intent leads.
  • The AI landscape has two channels: AI in search (Google Overviews) and AI as a destination (Chatbots).
Conductor customers: Review this lesson in our Community
Welcome to your new AEO measurement toolkit. Now that we have our new playbook (AEO) and our pillars (Authority, Content, Experience), we need to know how to keep score.In our traditional SEO world, we lived by rankings, clicks, and traffic. In a world where a user can get a perfect answer without clicking, our old set of metrics is missing the full story.This lesson will walk you through the five key metrics in your new toolkit. We’ll show you how to find them, what they mean, and how to use them to prove your AEO success.

​
AI Visibility

dynamic contentWhat It Is: This is the most fundamental AEO metric. It answers the question, “Are we even part of the AI conversation?” It’s measured in two ways:
  1. Mentions: When the AI names your brand or product in its answer.
  • Example: A user asks, “What are the best running shoes for flat feet?” The AI Overview might say, “Some of the top-rated brands for this include [Your Brand], Brooks, and Asics.”
  • This is a powerful signal that the AI recognizes your brand as a relevant and authoritative entity for that topic.
  1. Citations: When the AI directly links to your website as a source for its answer.
  • Example: After providing its synthesized answer, the AI will display a list of sources that it used, with a direct link to your article on “How to Choose Paint Finishes.”
  • This is the source of a new, high-value click. It’s the “link” you win that gives a user the chance to become AI Referral Traffic.
Why It’s Valuable: These are the new “rankings.” It’s your top-of-funnel for AEO. It tracks your brand’s presence and authority for your most important topics. Are you growing? Are you losing ground? This is where you find out.dynamic contentHow to Track in Conductor:
  • Tool: AI Search Performance in Conductor Intelligence.
  • What You’ll See: This report will show you your total number of Mentions and Citations for the topics you track. You can monitor your visibility over time to see if your AEO efforts are helping you show up more often for important topics.

​
AI Share of Voice

dynamic contentWhat It Is: This metric takes your AI Visibility and puts it in context. It’s your total Mentions or your total Citations compared to your top competitors’ for a given set of topics.Why It’s Valuable: Visibility is great, but Share of Voice tells you if you’re winning. Knowing you have 100 mentions is good. Knowing you have 100 mentions and your top competitor only has 20 is great. This is how you measure your market share in the AI landscape.dynamic contentHow to Track in Conductor:
  • Tool: AI Search Performance in Conductor Intelligence.
  • What You’ll See: The AISP report is built for this. You can add your top competitors and see a direct, side-by-side comparison of your market share. This is perfect for competitive analysis and proving your growing authority to leadership.

​
Brand Sentiment in AI Responses

dynamic contentWhat It Is: This is a more advanced, qualitative metric. It’s not just if you’re mentioned, but how you’re mentioned. Is the AI positioning your brand as a trusted leader? Is the sentiment positive, neutral, or negative?Why It’s Valuable: Not all mentions are equal. A mention like, “[Your Brand] is a trusted, industry-leading solution…” is far more valuable than just being in a list. A negative mention is a fire you need to put out. This is a crucial metric for brand health.dynamic contentHow to Track in Conductor:
  • Tool: AI Search Performance in Conductor Intelligence.
  • What You’ll See: The AISP report analyzes the context of your mentions to assign a sentiment (positive, negative, neutral). This lets you move beyond simple counts and understand the quality and impact of your AI visibility.

​
AI Referral Traffic

dynamic contentWhat It Is: This is the metric that answers, “Are people clicking on those citations?” This is the traffic that lands on your site from an AI-generated answer.Why It’s Valuable: This is the “money” metric. It’s the high-intent, pre-qualified traffic we talked about in Lesson 1. By tracking this, you can prove that your AEO visibility is turning into high-quality site visitors who are more likely to convert. This is how you connect AEO to your core business goals.dynamic contentHow to Track in Conductor:
  • Tool: The [new “Pages” report](/intelligence/2025-pages-reporting-alpha-program) in Conductor Intelligence.
  • What You’ll See: This report can help you identify traffic coming from AI sources, and compare AI referral to traditional search traffic trends.
  • Important Note: The new Pages report is currently in an opt-in Alpha. If you’re interested in early access, please speak to your account team or reach out to [email protected].

​
AI Crawler Bot Activity

dynamic contentWhat It Is: This is a technical, diagnostic metric. It’s the measure of how often AI-specific crawlers (like ChatGPT-User, PerplexityBot, etc.) are visiting your site.Why It’s Valuable: This is your technical ‘check engine’ light. If your visibility metrics in AISP are low, this is the first place to look. Are the bots not crawling you? Is there a technical issue? Or are they crawling you, but just not using your content? This metric is the key to troubleshooting your technical AEO.dynamic contentHow to Track in Conductor:
  • Tool: Conductor Monitoring (as part of the Log File Analysis feature).
  • What You’ll See: This feature allows you to integrate your server’s log files. You can see exactly which bots are visiting, how often they come, and which pages they’re looking at.
  • Important Note: This is an advanced feature available to enterprise package customers.

​
Lesson 2 Key Takeaways

  • Your new AEO Measurement Toolkit includes five key metrics, all trackable in Conductor.
    • AI Visibility (Mentions & Citations) is your new “ranking,” tracked in AI Search Performance.
    • AI Share of Voice is your competitive benchmark, also in AI Search Performance.
    • Brand Sentiment tracks the quality of your mentions in AI Search Performance.
    • AI Referral Traffic measures the high-intent visitors coming from citations, trackable in the new Pages Report (Alpha).
    • AI Crawler Bot Activity is your technical diagnostic tool, found in Conductor Monitoring.
Conductor customers: Review this lesson in our Community
Now we move from the “what” to the “how.” In this lesson, we’ll focus on the first two pillars of AEO: building Authority and creating Content that AI engines love.

​
Pillar 1: Building Authority (Quality + Quantity)

For years, the SEO mantra has been “quality over quantity.” But AI search introduces a challenging new twist. To be seen as a true authority by an AI, you now need both.Here’s why:
  • Quality is your ticket to the game. It proves your E-E-A-T and builds the trust required for an AI to cite you.
  • Quantity (of that high-quality content) is how you win. You need to build a wide and deep content ecosystem that covers the millions of specific, long-tail, and conversational questions users are now asking AI engines.
Your goal is to create an undeniable web of expertise. Here’s what this new “quality at scale” standard looks like:Depth and BreadthGo deep on topics. Cover all angles and answer all potential sub-questions. Don’t just create one “ultimate guide”; aim to build an “ultimate resource library” on your subject.Accuracy is ParamountFact-check everything. AI engines are getting better at spotting errors. If your content is inaccurate, it will be ignored, and your authority will drop.Originality and Unique InsightsDon’t just regurgitate what’s already out there. Offer fresh perspectives, original research, or unique angles that make your content stand out. This is what signals your “E-E-A-T” (Experience, Expertise, Authoritativeness, and Trustworthiness) to the AI.dynamic content

​
The “AI Trap”: Why You Can’t Just “Set It and Forget It”

Using AI to scale your content quantity is smart. Trusting it blindly is dangerous. To build real Authority, you need to watch out for the “Big Three” pitfalls of GenAI:Hallucinations (The Confident Liar)AI doesn’t just make mistakes; it hallucinates. It can invent facts, citations, and data with total confidence. If you publish a hallucination, your digital credibility could tank. Rule #1: Fact-check everything.Knowledge Cutoffs (The Time Capsule)Most models are stuck in the past. Their “training data” has a cutoff date, meaning they might not know about the product you launched last week or the news story from this morning.Generic Output (The “Beige” Factor)Without specific guidance, AI defaults to the average. It produces bland, templated content that sounds like everyone else. “Generic” does not build a brand, and it certainly doesn’t signal E-E-A-T.

​
AEO in Action: Building Authority with Writing Assistant

These are precisely the challenges Conductor’s Writing Assistant was built to solve.Generic, “horizontal” AI tools are good at creating generic text, but they lack specialized SEO data and can’t capture your unique brand voice. Writing Assistant is a purpose-built tool designed to create high-performing website content at scale, without sacrificing quality.Here’s how it helps you build this pillar:
  • For Quality & Brand Voice (Your E-E-A-T): Writing Assistant is powered by Content Profiles. This is your central hub to define your brand’s unique voice, style, and compliance rules. The AI then generates content already aligned with your brand identity, ensuring consistency and quality every time.
  • For Accuracy & Originality (Your Expertise): You can use Knowledge Sources to upload your own documents, whitepapers, or research notes as source material for the AI. This ensures the AI’s output is grounded in your unique facts and expertise, not just the generic web. This is critical for demonstrating unique insights.
  • For Quantity & Scale: The Personalized AI Content Generation feature creates full, data-driven drafts in seconds. It uses Retrieval-Augmented Generation (RAG), which means it infuses drafts with real-time search insights and competitor data, which is great for hallucination prevention and something that static AI models can’t do.
dynamic content

​
Pillar 2: The Content Strategy (Writing for the AI)

This is one of the most practical AEO strategies you can start using today.“Prompt engineering” is what your customer does when they talk to an AI. Your job is to reverse-engineer their prompts and create content that is perfectly structured to be the answer.The AI’s job is to find and synthesize information. Your job is to make your information the easiest, clearest, and most logical choice for it to grab. Let’s break down how to structure your content to match the most common types of prompts.
  1. The “How-To” Prompt
  • User Asks: “How do I paint a room?”
  • How You Write: Use a numbered list with strong verbs under a clear header.
  • Content Example: Your Step-by-Step Painting Guide
    1. Prep Your Space: Cover all furniture and floors.
    2. Tape Your Trim: Use painter’s tape to protect baseboards.
    3. Cut In: Paint the corners and edges with a brush first.
    4. Roll the Walls: Use a roller to fill in the main wall sections.
  1. The “Comparison” Prompt
  • User Asks: “Compare satin vs. semi-gloss paint.”
  • How You Write: Create a dedicated section for the comparison. A simple HTML table is the single best format for this.
  • Content Example: Satin vs. Semi-Gloss: Which is Right for You?
dynamic content
  1. The “What Is” or “Summary” Prompt
  • User Asks: “What is E-E-A-T?” or “Explain paint finishes like I’m a beginner.”
  • How You Write: Provide a clear, simple answer right at the top. Use blockquotes or “Key Takeaway” boxes to signal to the AI that this is a definitive summary.
  • Content Example: What Are Paint Finishes? A Simple Guide The takeaway? A paint’s “finish” just means how shiny it is. This affects its look and durability. Finishes range from flat (no shine, hides flaws) to high-gloss (very shiny, durable, and easy to clean).
  1. The “Pros and Cons” Prompt
  • User Asks: “What are the pros and cons of using satin paint?”
  • How You Write: Create a dedicated section with clear, scannable “Pros” and “Cons” subheadings. Bulleted lists are the best format.
  • Content Example: The Pros and Cons of Satin Paint
    • Pros:
      • Great Durability: It’s easier to clean than flat or eggshell finishes.
      • Subtle Sheen: It offers a velvety look that isn’t as shiny as semi-gloss.
    • Cons:
      • Shows Flaws: The slight shine can highlight imperfections on the wall.
      • Harder Touch-Ups: It’s more difficult to touch up scuffs.
  1. The “Best of” / Recommendation Prompt
  • User Asks: “What are the best tools for painting a room?”
  • How You Write: Use a bulleted list where each item is a clear “thing.” Start each item with a bolded title followed by a short, descriptive “why.”
  • Content Example: The 5 Must-Have Tools for Painting
    • A Quality Roller: This is your main tool for walls.
    • An Angled Brush: This is for “cutting in”—painting the trim and corners.
    • Painter’s Tape: Use this to protect your trim for clean, sharp lines.
    • Drop Cloths: Protect your floors and furniture from spills.
    • A Paint Tray: This is essential for loading your roller with an even amount of paint.
  1. The “Why” Prompt
  • User Asks: “Why is it important to use primer?”
  • How You Write: Answer the question directly in the first sentence. Then, use a bulleted list to break down the “why” into easy-to-digest reasons.
  • Content Example: Why You Should Never Skip Primer Using a primer is crucial because it ensures your final paint job looks professional and lasts longer. Here’s exactly what it does:
    • It Blocks Stains: Primer stops old stains from bleeding through your new paint.
    • It Improves Adhesion: It gives the paint a perfect surface to stick to.
    • It Evens Out the Surface: It creates a smooth, uniform base.
    • It Hides the Old Color: This is essential if you’re making a dramatic color change.
dynamic content

​
AEO in Action: Structuring Content with Writing Assistant

This is where Writing Assistant becomes your co-pilot. It’s designed not just to write, but to write in these specific, AI-friendly structures.
  • Generate Structured Drafts from the Start: When you use Generate Draft or Generate Outline, the AI is already using these principles. It’s pulling from top-ranking competitor data and “People Also Ask” questions to build a logical structure that’s optimized for AEO.
  • Use the “Revise” Action for Micro-Adjustments: This is the clutch feature. Have a dense paragraph you need to fix? Highlight it, and a menu will appear. Select “Revise Draft” and type a natural language command like, “Turn this into a bulleted list of pros and cons” or “Rephrase this to directly answer ‘Why is this important?’”.
  • Stay on Target with the AI Content Score: As you write, the AI Content Score gives you a real-time grade (from 0-100) on your content’s quality. It specifically evaluates you on Topical Coverage and Intent Alignment —the very essence of this pillar. It gives you actionable feedback to ensure your content is as comprehensive and relevant as possible before you ever hit “publish.”
dynamic contentReady to master these tools?For a step-by-step guide, watch the Draft new and existing content with Writing Assistant tutorials and read our Writing Assistant Best Practices Guide.

​
Lesson 3 Key Takeaways

  • To build Authority, you need both Quality (trust) and Quantity (comprehensive coverage).
  • Your Content strategy should “reverse-engineer” common AI prompts.
  • Structure your content to be the answer: use tables for comparisons, numbered lists for “how-to”s, and bulleted lists for “pros/cons” and “why.” Provide clear, quotable summaries for “what is” prompts.
Conductor customers: Review this lesson in our Community
Welcome to our final pillar: Experience. This is the technical side of AEO. Your amazing content and authority won’t matter if the AI crawlers can’t easily access, understand, and extract your information.Good news! Many of the best practices for traditional SEO still apply, but with a renewed, critical importance.dynamic content

​
AEO in Action: Validating Your Technical Experience

How do you know if your technical AEO efforts are working? How can you be sure AI bots can even see your content in the first place?This is where log file analysis becomes your key diagnostic tool. You must validate that AI crawlers are successfully visiting your site.dynamic contentHow to Track in Conductor:
  • Tool: AI Crawler Activity in Conductor Monitoring (part of the Log File Analysis feature).
  • What It Is: This feature analyzes your server’s log files to show you exactly which AI-specific crawlers (like ChatGPT-User, GPTBot, PerplexityBot, etc.) are visiting your site, which pages they’re crawling, and how often.
  • Why It’s Your Key Diagnostic: This is the early warning system we talked about in Lesson 2. It helps you answer a critical question:
    • “Is my technical setup the problem?” If you see no AI bot activity, you know you have a foundational problem, like an incorrect robots.txt rule or a log file filter that’s blocking the bots.
    • “Is my content the problem?” If you see high crawl activity (bots are visiting) but you still have low AI Visibility (you’re not getting mentions/citations), it’s a strong signal that your technical setup is fine, and you need to focus on improving your Authority and Content (Pillars 1 & 2).
  • Important Note: This feature is available to enterprise package customers.

​
Deeper Dives: Key Technical Challenges

Let’s take a closer look at a few of the most critical and confusing technical points.

​
Deep Dive: Do LLMs render JavaScript?

This is a big one, especially for those of us who have worked hard to optimize JavaScript-heavy sites for Google.The simple answer is: Not yet, at least not consistently.While Googlebot has become incredibly sophisticated at rendering JavaScript (it essentially acts like a headless browser), many independent LLM crawlers (like those powering some chatbot web-browsing features) are more lightweight for now. They often fetch the source HTML and move on.What does this mean for you?If your crucial content relies entirely on client-side JavaScript to load, those LLMs might see a blank page or incomplete information. It’s like inviting the AI to a dinner party but locking the food in the kitchen.Our Strategic Advice: Don’t Panic, Just Be AwareThis is most likely a temporary problem. The expert consensus is that AI crawlers for tools like ChatGPT are in a race to catch up with Google and Perplexity.
  • We do not advise investing in a massive, expensive site re-architecture just to be seen by these crawlers today.
  • We do advise that the best practice for universal accessibility is to use server-side rendering (SSR), static site generation (SSG), or hybrid approaches for your most critical content.
The takeaway? If you’re already using or planning to use SSR, you’re in a great position. If you have a script-heavy site, don’t panic. Just be aware that for right now, some of your content might be invisible to parts of the AI ecosystem.For more details on rendering capabilities by AI engine, see our LLM JavaScript Rendering Capability Matrix.

​
Deep Dive: What’s an LLM.txt and do I need one?

You may have heard about LLM.txt, an emerging, proposed standard (similar to robots.txt) designed to give Large Language Models specific instructions on how to interact with your website.In theory, it’s a file you’d place in your site’s root directory. Unlike robots.txt (which is for blocking crawlers), LLM.txt would be for guiding them.But let’s be very clear: this is a super early stage test.As of now, none of the major LLMs—like Google or OpenAI—officially support or read this file.So, do you need one?The short answer is no, not right now. While some SEO tools are starting to support its creation, it’s a proactive experiment, not a recognized standard. Think of it as a way to be ready if it ever becomes supported, but it will not impact your AEO performance with the major engines today.

​
A Final Note: The Messy Reality of Controlling Access

Everything we’ve discussed so far is about AEO—our main goal is to get AI to see, trust, and cite our content. We want these bots to visit.But what about the content you don’t want to be cited? For this, you have a tool you already know: robots.txt.This is the global standard, supported by all cooperative crawlers (Google-Extended, ChatGPT-User, PerplexityBot, etc.), for telling bots which parts of your site to stay out of. You can use it to block specific AI user agents from:
  • Paid or members-only sections
  • Internal search results
  • Staging or test pages
The Critical Caveat (A Word of Warning)Now, it’s important to be realistic. robots.txt is not a magic shield, especially in the world of AI.The AI world has a “data history” problem. Many models were trained on massive web scrapes that were collected before these new AI-specific robots.txt rules were put in place.This means that even if you block an AI crawler today, your content might already be in its training data from a crawl last year. This is why you might see content you thought was blocked still appearing in AI answers.The takeaway?Your AEO strategy should be to let crawlers in. That’s how you get mentions and citations.You should only use robots.txt to block specific sections for clear business reasons. When you do, understand that it’s a go-forward instruction for cooperative bots, not a retroactive “delete button” for data that is already in a trained model.

​
Lesson 4 Key Takeaways

  • Technical AEO is about making your content easy for bots to crawl, understand, and extract.
  • Prioritize clean HTML, fast page speed, and strong internal linking.
  • Many AI crawlers do not render JavaScript reliably yet. SSR/SSG is the safest bet, but don’t panic-invest in a total re-architecture.
  • LLM.txt is an unsupported, “early-stage test” and is not currently read by major LLMs.
  • Use robots.txt to block crawlers from specific areas, but understand it’s a “go-forward” instruction and not a retroactive “delete button” for data already in a trained model.
Conductor customers: Review this lesson in our Community
dynamic contentYou’ve made it! This new landscape of AI search can feel overwhelming, but as you’ve seen, it’s not about throwing out the old playbook. It’s about evolving it.Your skills in understanding users, creating quality content, and proving E-E-A-T are more valuable than ever.You now have the framework: AEO (Answer Engine Optimization). You have the pillars: Authority, Content, and Experience. And you have the new metrics: Mentions and Citations.This is an opportunity, not a threat. It’s a chance to move from just chasing clicks to building true, undeniable authority—and capturing the high-value, high-intent traffic that comes with it.You have the compass. You’re ready to start optimizing.

​
Your AEO Action Plan: What to Do This Week

You’ve got the knowledge. Here are three simple things you can do this week to put it into action.✅ Run a “Prompt Audit” on One PagePick one of your high-performing blog posts. Review it against the “Writing for the AI” checklist (Lesson 3, Pillar 2). Does it have a clear summary for a “What is” prompt? Does it use a table for a “Comparison” prompt? Take 15 minutes to re-format one section. (Bonus points: Use Writing Assistant ↗️ to score your content and revise sections with the help of AI.)✅ Find One “Authority” GapPick one of your most important topics. Look at it in Conductor’s AI Search report ↗️. Find one “cluster” of related, long-tail questions you aren’t answering (i.e. prompts where you are neither mentioned nor cited). That’s your next content brief.✅ Check Your robots.txt FileGo to yourdomain.com/robots.txt and make sure you aren’t accidentally blocking the very bots you’re trying to impress, like ChatGPT-User or PerplexityBot. This is a crucial starting point for a strategic conversation with your team.
Conductor customers: Review this lesson in our Community
dynamic contentYou have completed this course! To keep learning more workflows, check out our additional courses.