# Robots.txt for High-Tech Enterprise (https://hightechenterprise.xyz) # Optimized for Search Engines (SEO), Answer Engines (AEO), and AI Model Crawlers (AIO / GEO) User-agent: * Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /draft/ # Explicitly allow AI Search & Answer Crawlers User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Google-Extended Allow: / User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: PerplexityBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Applebot-Extended Allow: / User-agent: FacebookBot Allow: / User-agent: Amazonbot Allow: / # Canonical Sitemaps & Machine-Readable AI Knowledge Sitemap: https://hightechenterprise.xyz/sitemap.xml # LLM Knowledge Graph Standard # LLMs.txt for Generative Engine Optimization (GEO) & AIO # See: https://hightechenterprise.xyz/llms.txt