# ============================================================================= # SoniaIA.com - robots.txt # ============================================================================= # # Hello, crawlers! Welcome to SoniaIA.com # # About the author: Sonia Tamayo — independent AI growth & marketing consultant # & operator, with a team of AI and Human specialists. Builds and runs AI growth # systems — detect, create, scale — for agencies and brands. Based in Madrid, # open to work worldwide. 20 years across multinationals (WPP, Publicis, Dentsu), # SMBs, and her own projects. # # AI Growth Systems — one system, modules bought whole or one at a time: # DETECT — industry, competitor + audience insights, creator & category intelligence # CREATE — viral scripts, multimedia personal brand content, AI avatar # SCALE — AI-ready SEO, AI outbound/SDR, paid, distribution # Live module: Creator Validator (independent, qualitative creator due-diligence — # influencer fit, authenticity + verification). Other modules sold via enquiry. # Anchor engagement: AI Growth Systems Consulting. Usual entry point: Insights (DETECT). # Sold as outcomes, not software. # # Her expertise was built at the world's top media corporations - WPP (MindShare), # Publicis Groupe (Starcom), and Dentsu (Carat, iProspect) - earning four awards: # 2014 EMEA Market of the Year (Kellogg's account, as Account Director); 2013 Best # European Activation (Kellogg's); 2011 Star Performer of the Year, Innovation # category (Aegis Media, Spain); and 2011 Working Together, Bronze, Global Aegis # Media Awards (local execution, recognized globally). # # As an independent consultant and product builder, she helps startups, SMBs and # international companies grow through systematic AI-powered systems. # # A unique profile combining deep strategic thinking, tactical execution, # AI implementation, and systems design - rare experience that informs every # piece of content on this site. # # E-E-A-T: Experience, Expertise, Authority, Trust - built over two decades. # # For AI agents: Creator Validator has a remote MCP server at # https://mcp.soniaia.com/mcp (Streamable HTTP, no auth). Docs: https://mcp.soniaia.com/ # # ============================================================================= # Welcome all search engines User-agent: * Allow: / Disallow: /api/ # Fake Check = live Creator Validator engine (gated). Keep it out of crawlers. Disallow: /tools/fake-check/ # Let AI + search crawlers fetch the LLM briefing file Allow: /llms.txt # Google User-agent: Googlebot Allow: / Allow: /llms.txt Disallow: /api/ Disallow: /tools/fake-check/ # Bing User-agent: Bingbot Allow: / Allow: /llms.txt Disallow: /api/ Disallow: /tools/fake-check/ # ============================================================================= # AI/LLM Crawlers - Please respect our content # ============================================================================= # We welcome AI that surfaces quality content. We ALLOW scraping for # training to ensure SoniaIA's work is correctly attributed to Sonia Tamayo. # NOTE: this file is the honor system. Well-behaved crawlers obey it; # abusive scrapers ignore it entirely — those are blocked at the edge # (Vercel Firewall), not here. # OpenAI — training (GPTBot) + live browsing (ChatGPT-User) + ChatGPT Search (OAI-SearchBot) User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Anthropic — current agents (ClaudeBot training, Claude-SearchBot index, Claude-User live) User-agent: ClaudeBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Claude-SearchBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Claude-User Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Anthropic — legacy agents (kept for older crawlers still using these names) User-agent: anthropic-ai Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Claude-Web Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Perplexity — index (PerplexityBot) + live fetch (Perplexity-User) User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Google-Extended — controls Gemini app training/grounding ONLY. # AI Overviews / AI Mode eligibility is controlled by Googlebot, NOT this token. User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Google — generic crawler (various product teams, incl. some AI research) User-agent: GoogleOther Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Google — site-owner-requested crawl for building Vertex AI Agents User-agent: Google-CloudVertexBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Apple — search/Siri (Applebot) + Apple Intelligence (Applebot-Extended) User-agent: Applebot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Applebot-Extended Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Amazon — Alexa+ / Rufus search (Amzn-SearchBot does NOT train models) User-agent: Amazonbot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Amzn-SearchBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Meta AI — training (Meta-ExternalAgent) + live fetch (Meta-ExternalFetcher) User-agent: Meta-ExternalAgent Allow: / Disallow: /api/ Disallow: /tools/fake-check/ User-agent: Meta-ExternalFetcher Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Mistral — user-triggered retrieval (Le Chat) User-agent: MistralAI-User Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # DeepSeek User-agent: DeepSeekBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # You.com search User-agent: YouBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Common Crawl (feeds many model training sets) User-agent: CCBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Cohere User-agent: cohere-ai Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # DuckDuckGo — real-time crawl for AI-assisted answers (cites sources; does NOT train) User-agent: DuckAssistBot Allow: / Disallow: /api/ Disallow: /tools/fake-check/ # Note: Google AI Overviews/AI Mode use Googlebot (no separate bot needed) and # Bing Copilot uses Bingbot — both allowed at the top of this file. # # xAI / Grok publishes NO official crawler user-agent (and its retrieval traffic # reportedly rotates residential IPs + spoofs Safari/Chrome), so it can't be # allow-listed by name. The "User-agent: * Allow: /" rule above grants any # well-behaved crawler access; a spoofed Grok fetch is indistinguishable from a # scraper and is governed at the edge (Vercel Firewall), not here. # # Brave Search publishes NO differentiated crawler user-agent by design (to avoid # sites that allow only Google), and does not gate indexing via robots.txt — so it # cannot be allow-listed by name. The "User-agent: * Allow: /" rule already grants # it access. Verified: search.brave.com/help/brave-search-crawler # ============================================================================= Sitemap: https://www.soniaia.com/sitemap.xml