Public API

The Public API provides unauthenticated endpoints for retrieving published content, navigation data, and site metadata. This guide covers all public endpoints and their usage.

Base URL

The Public API is available at:

https://api.{rootDomain}

Example: https://api.example.com

Authentication

Public API endpoints do not require authentication. They are designed for public access to published content.

Endpoints

Get Page Content

Retrieve rendered page content by slug.

Endpoint: GET /sites/{siteId}/pages/{slug}

Parameters:

  • siteId (path): Site identifier (UUID)
  • slug (path): Page slug (e.g., about, blog/post-1)

Response: 200 OK

{

  "pageId": "550e8400-e29b-41d4-a716-446655440000",

  "title": "About Us",

  "description": "Learn about our company",

  "slug": "/about",

  "content": "<html>...</html>",

  "publishedAt": "2024-01-15T10:30:00Z",

  "tags": ["company", "about"]

}

Response fields:

  • pageId: Unique page identifier
  • title: Page title
  • description: Page description
  • slug: Page slug
  • content: Rendered HTML content
  • publishedAt: Publication timestamp (ISO 8601)
  • tags: Array of page tags

Error responses:

  • 404 Not Found: Page not found or not published
  • 403 Forbidden: AI crawler blocked (if site has AI crawler access disabled)

Example:

curl https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/pages/about

Get Navigation

Retrieve site navigation data (list of published pages).

Endpoint: GET /sites/{siteId}/navigation

Parameters:

  • siteId (path): Site identifier (UUID)

Response: 200 OK

{

  "pages": [

    {

      "pageId": "550e8400-e29b-41d4-a716-446655440000",

      "title": "Home",

      "slug": "/",

      "parentPageId": null

    },

    {

      "pageId": "660e8400-e29b-41d4-a716-446655440001",

      "title": "About",

      "slug": "/about",

      "parentPageId": null

    },

    {

      "pageId": "770e8400-e29b-41d4-a716-446655440002",

      "title": "Our Team",

      "slug": "/about/team",

      "parentPageId": "660e8400-e29b-41d4-a716-446655440001"

    }

  ]

}

Response fields:

  • pages: Array of published pages
    • pageId: Unique page identifier
    • title: Page title
    • slug: Page slug
    • parentPageId: Parent page ID (null for top-level pages)

Error responses:

  • 404 Not Found: Site not found

Example:

curl https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/navigation

Get Site Metadata

Retrieve site metadata and settings.

Endpoint: GET /sites/{siteId}/meta

Parameters:

  • siteId (path): Site identifier (UUID)

Response: 200 OK

{

  "siteId": "550e8400-e29b-41d4-a716-446655440000",

  "domain": "mysite.example.com",

  "displayName": "My Awesome Site",

  "defaultLanguage": "en",

  "aiCrawlerAccess": true,

  "organizationJsonLd": {

    "@context": "https://schema.org",

    "@type": "Organization",

    "name": "My Company",

    "url": "https://mysite.example.com"

  }

}

Response fields:

  • siteId: Unique site identifier
  • domain: Site domain (FQDN)
  • displayName: Site display name
  • defaultLanguage: Default language code (ISO 639-1)
  • aiCrawlerAccess: Whether AI crawlers are allowed
  • organizationJsonLd: Organization structured data (optional)

Error responses:

  • 404 Not Found: Site not found

Example:

curl https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/meta

Submit Form

Submit form data from a published page.

Endpoint: POST /sites/{siteId}/forms

Parameters:

  • siteId (path): Site identifier (UUID)

Request body:

{

  "pageId": "550e8400-e29b-41d4-a716-446655440000",

  "fields": {

    "name": "John Doe",

    "email": "john@example.com",

    "message": "Hello, I'd like to learn more."

  }

}

Request fields:

  • pageId: Page where form was submitted
  • fields: Object with form field names and values

Response: 201 Created

{

  "submissionId": "880e8400-e29b-41d4-a716-446655440003",

  "submittedAt": "2024-01-15T10:30:00Z"

}

Response fields:

  • submissionId: Unique submission identifier
  • submittedAt: Submission timestamp (ISO 8601)

Error responses:

  • 400 Bad Request: Invalid request body
  • 404 Not Found: Site or page not found
  • 429 Too Many Requests: Rate limit exceeded

Rate limiting: 100 requests per minute per IP address

Example:

curl -X POST https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/forms \

  -H "Content-Type: application/json" \

  -d '{

    "pageId": "550e8400-e29b-41d4-a716-446655440000",

    "fields": {

      "name": "John Doe",

      "email": "john@example.com"

    }

  }'

Search Pages

Search published pages with filtering and pagination.

Endpoint: GET /sites/{siteId}/search

Parameters:

  • siteId (path): Site identifier (UUID)
  • q (query, required): Search query string
  • tags (query, optional): Comma-separated list of tags to filter by
  • page (query, optional): Page number (default: 1)
  • limit (query, optional): Results per page (default: 10, max: 100)

Response: 200 OK

{

  "results": [

    {

      "pageId": "550e8400-e29b-41d4-a716-446655440000",

      "title": "About Our Company",

      "description": "Learn about our history and mission",

      "slug": "/about",

      "tags": ["company", "about"],

      "score": 100,

      "publishedAt": "2024-01-15T10:30:00Z"

    },

    {

      "pageId": "660e8400-e29b-41d4-a716-446655440001",

      "title": "Company History",

      "description": "Our journey from startup to industry leader",

      "slug": "/about/history",

      "tags": ["company", "history"],

      "score": 50,

      "publishedAt": "2024-01-14T09:00:00Z"

    }

  ],

  "totalResults": 2,

  "page": 1,

  "totalPages": 1,

  "filters": {

    "tags": {

      "company": 2,

      "about": 1,

      "history": 1

    }

  }

}

Response fields:

  • results: Array of matching pages
    • pageId: Unique page identifier
    • title: Page title
    • description: Page description
    • slug: Page slug
    • tags: Array of page tags
    • score: Relevance score (higher = more relevant)
    • publishedAt: Publication timestamp
  • totalResults: Total number of matching pages
  • page: Current page number
  • totalPages: Total number of pages
  • filters: Available filters with counts
    • tags: Tag counts for filtering

Relevance scoring:

  • Title exact match: 100 points
  • Title partial match: 50 points
  • Description/SEO metadata match: 25 points
  • Content match: 10 points

Error responses:

  • 400 Bad Request: Missing or invalid query parameter
  • 404 Not Found: Site not found
  • 429 Too Many Requests: Rate limit exceeded (100 requests per minute per IP)

Example:

# Basic search

curl "https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/search?q=company"

 

# Search with tag filter

curl "https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/search?q=company&tags=about,history"

 

# Search with pagination

curl "https://api.example.com/sites/550e8400-e29b-41d4-a716-446655440000/search?q=company&page=2&limit=20"

AI Crawler Detection

The Public API automatically detects AI crawlers by User-Agent:

Detected crawlers:

  • GPTBot (OpenAI)
  • ChatGPT-User (OpenAI)
  • Google-Extended (Google)
  • anthropic-ai (Anthropic)
  • ClaudeBot (Anthropic)
  • Bytespider (ByteDance)
  • Applebot-Extended (Apple)

Behavior:

  • If site has aiCrawlerAccess: true: Allow access, log activity
  • If site has aiCrawlerAccess: false: Return HTTP 403, log blocked attempt

Activity logging:

  • Crawler visits are logged to DynamoDB
  • Records include: crawler name, page ID, timestamp, IP address
  • Records expire after 90 days (TTL)
  • Accessible via Admin Tool analytics

CORS Configuration

GET Endpoints

All GET endpoints allow cross-origin requests:

Access-Control-Allow-Origin: *

Access-Control-Allow-Methods: GET, OPTIONS

Access-Control-Allow-Headers: Content-Type

POST Endpoints

POST endpoints (form submission) restrict origin to the site domain:

Access-Control-Allow-Origin: https://mysite.example.com

Access-Control-Allow-Methods: POST, OPTIONS

Access-Control-Allow-Headers: Content-Type

Rate Limiting

Rate limits protect the API from abuse:

Endpoint
Limit
GET /pages/{slug}
No limit
GET /navigation
No limit
GET /meta
No limit
POST /forms
100 requests per minute per IP
GET /search
100 requests per minute per IP

Rate limit headers:

X-RateLimit-Limit: 100

X-RateLimit-Remaining: 95

X-RateLimit-Reset: 1642252800

Rate limit exceeded response: 429 Too Many Requests

{

  "error": "Rate limit exceeded",

  "code": "RATE_LIMIT_EXCEEDED",

  "retryAfter": 60

}

Error Responses

All error responses use a consistent format:

{

  "error": "Human-readable error message",

  "code": "MACHINE_READABLE_CODE"

}

Common error codes:

  • PAGE_NOT_FOUND: Page not found or not published
  • SITE_NOT_FOUND: Site not found
  • INVALID_REQUEST: Invalid request parameters
  • RATE_LIMIT_EXCEEDED: Rate limit exceeded
  • AI_CRAWLER_BLOCKED: AI crawler access denied

Usage Examples

Building a Navigation Menu

async function buildNavigation(siteId) {

  const response = await fetch(

    `https://api.example.com/sites/${siteId}/navigation`

  );

  const data = await response.json();

  

  // Build hierarchical navigation

  const topLevel = data.pages.filter(p => !p.parentPageId);

  const children = data.pages.filter(p => p.parentPageId);

  

  return topLevel.map(page => ({

    ...page,

    children: children.filter(c => c.parentPageId === page.pageId)

  }));

}

Fetching Page Content

async function getPage(siteId, slug) {

  const response = await fetch(

    `https://api.example.com/sites/${siteId}/pages/${slug}`

  );

  

  if (!response.ok) {

    if (response.status === 404) {

      throw new Error('Page not found');

    }

    throw new Error('Failed to fetch page');

  }

  

  return response.json();

}

Submitting a Form

async function submitForm(siteId, pageId, formData) {

  const response = await fetch(

    `https://api.example.com/sites/${siteId}/forms`,

    {

      method: 'POST',

      headers: {

        'Content-Type': 'application/json'

      },

      body: JSON.stringify({

        pageId,

        fields: formData

      })

    }

  );

  

  if (!response.ok) {

    if (response.status === 429) {

      throw new Error('Too many requests. Please try again later.');

    }

    throw new Error('Failed to submit form');

  }

  

  return response.json();

}

Searching Pages

async function searchPages(siteId, query, tags = [], page = 1) {

  const params = new URLSearchParams({

    q: query,

    page: page.toString(),

    limit: '10'

  });

  

  if (tags.length > 0) {

    params.append('tags', tags.join(','));

  }

  

  const response = await fetch(

    `https://api.example.com/sites/${siteId}/search?${params}`

  );

  

  if (!response.ok) {

    throw new Error('Search failed');

  }

  

  return response.json();

}

Best Practices

Caching

  • Cache navigation data: Navigation changes infrequently
  • Cache page content: Pages are immutable once published
  • Respect cache headers: Use Cache-Control headers
  • Invalidate on rebuild: Clear cache when site is rebuilt

Error Handling

  • Handle 404s gracefully: Show user-friendly error pages
  • Retry on 429: Implement exponential backoff for rate limits
  • Log errors: Log API errors for debugging
  • Fallback content: Show fallback content when API fails

Performance

  • Minimize requests: Batch requests when possible
  • Use CDN: Content is served via CloudFront CDN
  • Lazy load: Load content as needed, not all at once
  • Prefetch: Prefetch likely next pages

Security

  • Validate input: Validate all user input before submitting
  • Sanitize output: Sanitize API responses before rendering
  • Use HTTPS: Always use HTTPS for API requests
  • Rate limiting: Implement client-side rate limiting

Related Topics