> ## Documentation Index
> Fetch the complete documentation index at: https://docs.promptingcompany.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloudflare Logpush

> Stream HTTP request logs from Cloudflare to The Prompting Company and watch AI crawler traffic in your dashboards

Cloudflare Logpush pushes batches of your zone's HTTP request logs directly to The Prompting Company. Once the job is enabled, every AI crawler and agent visit (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, and others) shows up in the [AI traffic](/guides/metrics#ai-traffic) metric automatically — classification happens on our side, so you send everything and we keep what matters.

Not on Cloudflare Enterprise? Use the [raw events API](/guides/analytics/log-exporters/raw-events) to stream from any log source instead, or [export a one-off CSV](/guides/analytics/cf-traffic-export) for history.

## Before you start

* You need a The Prompting Company **API key** (organization or service-account key) from **Settings → API Keys**.
* The zone must be on a Cloudflare **Enterprise** plan — Logpush for the HTTP requests dataset is Enterprise-only.
* Your DNS records must be **proxied** (orange cloud) — Logpush only captures traffic that flows through Cloudflare.

## Configure the destination

```text theme={null}
https://logs.promptingco.com/v1/events/cf-logpush?header_Authorization=Bearer%20YOUR_API_KEY
```

Cloudflare converts `header_*` query parameters into request headers, so this sends your key as an `Authorization: Bearer` header with every push. Keep the value URL-encoded (`%20` for the space).

### Using the dashboard

1. In the Cloudflare dashboard, open your zone and go to **Analytics & Logs → Logpush**.
2. Click **Create a Logpush job** and select the **HTTP requests** dataset.
3. Choose **HTTP destination** and paste the destination URL above.
4. Select these fields: `ClientIP`, `ClientRequestHost`, `ClientRequestMethod`, `ClientRequestReferer`, `ClientRequestSource`, `ClientRequestURI`, `ClientRequestUserAgent`, `WorkerSubrequest`, `EdgeStartTimestamp`, `EdgeResponseContentType`, `EdgeResponseStatus`.
5. Save and enable the job. Cloudflare validates the destination with a test message — this succeeds automatically.

### Using the API

```bash theme={null}
curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer $CF_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{
    "name": "tpc-crawler-logs",
    "dataset": "http_requests",
    "enabled": true,
    "destination_conf": "https://logs.promptingco.com/v1/events/cf-logpush?header_Authorization=Bearer%20YOUR_API_KEY",
    "max_upload_bytes": 5000000,
    "output_options": {
      "field_names": [
        "ClientIP", "ClientRequestHost", "ClientRequestMethod",
        "ClientRequestReferer", "ClientRequestSource", "ClientRequestURI",
        "ClientRequestUserAgent", "WorkerSubrequest", "EdgeStartTimestamp",
        "EdgeResponseContentType", "EdgeResponseStatus"
      ],
      "timestamp_format": "rfc3339"
    }
  }'
```

Any `timestamp_format` works (`unixnano`, `unix`, or `rfc3339`); `rfc3339` is the easiest to debug.

## What gets ingested

You don't need to pre-filter traffic — send everything and The Prompting Company classifies AI crawlers and agents on ingestion. A few line types are skipped automatically so visits aren't double-counted: Cloudflare's destination-validation test message, Worker subrequests (`WorkerSubrequest: true`), non-eyeball requests (`ClientRequestSource` values like `edgeWorkerFetch` or purges), and lines without a `ClientRequestHost`. You can optionally add a Logpush filter to reduce volume, but it isn't required for correctness.

## Responses

| Status | Meaning                                                                               |
| ------ | ------------------------------------------------------------------------------------- |
| `200`  | Batch stored. Body reports `{"ok": true, "received": N, "ingested": N, "skipped": N}` |
| `401`  | Missing or invalid API key — check the key in the destination URL                     |
| `429`  | Key verification was rate limited — Cloudflare retries                                |
| `400`  | Body could not be decoded                                                             |
| `413`  | Decompressed batch too large — lower the job's `max_upload_bytes`                     |
| `503`  | Temporary ingestion outage — Cloudflare retries automatically                         |

Delivery is at-least-once: failed batches are re-pushed in full, and occasional duplicates are expected and tolerated by the traffic metrics.

## Verify it's working

Real traffic starts appearing in the [AI traffic](/guides/metrics#ai-traffic) dashboard within a few minutes of enabling the job. To exercise the pipeline immediately without waiting for a crawler visit, send one test event through the [raw events API](/guides/analytics/log-exporters/raw-events#verify-its-working).