# Content Sync Explained

> Source: https://agilitycms.com/docs/developers/content-sync-explained

Agility gives you three ways to read content: the **Content Fetch API** (REST), the **GraphQL API**, and **Content Sync**. The first two answer a question each time you ask. Content Sync copies your content into a store you own and keeps that copy up to date, so your app reads locally instead of calling Agility.

This guide explains when to use each one, how the sync token works, and how to drive incremental syncs from webhooks. For the endpoint reference, see [Content Sync API](/docs/developers/content-sync-api). For the JavaScript SDK, see [Content Sync JS SDK](/docs/javascript/content-sync-js-sdk).

## Sync, Fetch or GraphQL?

| | Content Fetch API (REST) | GraphQL API | Content Sync |
| --- | --- | --- | --- |
| What you get | One item, list, page or sitemap per request | Exactly the fields you select, several queries in one request | Every content item and page that changed since your last sync |
| Where your app reads from | Agility, on each request (through the CDN) | Agility, on each request | Your own store (files, database, cache) |
| Filtering and sorting | `filter`, `sort`, `take` (max 250), `skip` | Same filter syntax, per list | Whatever your store supports |
| Counts towards API rate limits | Uncached requests do | Uncached requests do | Only the sync calls themselves |
| Best for | Server-rendered and statically built sites, most apps | Pulling many related lists and fields in one round trip | Offline use, very high read volume, copying content into another system |

Some rules of thumb:

- **Start with the Fetch API or GraphQL.** For most websites, requesting content when you render (with your framework's cache in front, and webhooks to clear it) is simpler than keeping a copy.
- **Choose GraphQL** when a page needs fields from several lists at once and you want one request with only those fields.
- **Choose Content Sync** when you need content without calling Agility at read time: an offline-capable app, a search index or data warehouse, a system such as Redis that other services read from, or a static build that should not re-download everything each time.

You can mix them. A common pattern is Sync for a search index or cache, and the Fetch API for preview.

## How sync works

The Sync API is part of the Content Fetch API. It has two endpoints, one for content items and one for pages:

```text
GET https://api.aglty.io/{guid}/{fetch|preview}/{locale}/sync/items?syncToken={token}&pageSize={n}
GET https://api.aglty.io/{guid}/{fetch|preview}/{locale}/sync/pages?syncToken={token}&pageSize={n}
```

Both take your `APIKey` header like any other Fetch API request. `pageSize` defaults to 500. Each response is an object with the items (or pages) in that batch and the next token. The shape (values are placeholders):

```json
{
  "items": [
    { "contentID": 0, "properties": { "state": 2, "referenceName": "..." }, "fields": { } }
  ],
  "syncToken": 0
}
```

Each item has the same shape as an item from the Fetch API, so the field shapes in [Field Types and What the APIs Return](/docs/developers/field-types-api-reference) apply.

Use the `fetch` API type and a fetch key to sync published content, or `preview` and a preview key to sync the latest saved content for a preview environment.

## The sync token lifecycle

The sync token is a number that marks how far through the change history you have read. Think of it as a bookmark.

1. **First sync.** Call with `syncToken=0`. You get the first batch of everything, plus a new token.
2. **Keep paging.** Call again with the token you just received. Each call returns the next batch and a newer token.
3. **Stop when you are caught up.** When a call returns no items and the token no longer advances, you have everything. The `@agility/content-sync` SDK stops as soon as the returned token is not greater than the one it sent.
4. **Store the token.** Save the last token with your synced data (per locale, and separately for items and pages).
5. **Next time, resume.** Start from the stored token, not from `0`. You only receive what changed since then.

Things to get right:

- **Keep one token per locale and per endpoint.** Items and pages have separate tokens, and each locale is synced separately.
- **Save the token after you save the content.** If your process stops between the two, you re-read a batch rather than skip one. Re-reading is safe because each item simply overwrites the copy you already have.
- **Treat `0` as "rebuild".** To start over (for example after you clear your store, or if your store and token get out of step), sync from `0` again.

### Deleted and unpublished content

Sync returns removals as well as changes. An item that was deleted or unpublished comes back with `properties.state` set to `3` (Deleted). Remove it from your store when you see that state. The `@agility/content-sync` SDK does this for both content items and pages.

## Incremental sync driven by webhooks

You don't need to poll. Let Agility tell you when something changed, then sync from your stored token:

1. **Create a webhook** in **Settings > Webhooks** for **Content Publish Events** (and **Content Save Events** if you sync preview content). See [Webhooks](/docs/developers/webhooks).
2. **When a delivery arrives,** acknowledge it with a 2xx response straight away, then run a sync in the background from your stored token.
3. **Don't trust the payload as the content.** The webhook tells you that something changed (`contentID`, `referenceName`, `state`, `languageCode`); the sync call gives you the current data, including removals.
4. **Use the payload's `languageCode`** to sync only the locale that changed.
5. **Collapse bursts.** Publishing many items sends many webhooks. Queue them, and run one sync for the batch. The SDK runs a sync at most once per second per locale.

Webhook delivery is at-least-once, so the same event can arrive twice. Syncing from a stored token is naturally safe here: a second sync with nothing new returns nothing. For signatures, retries and the `webhook-id` header, see [Verifying Signed Webhooks](/docs/developers/verifying-signed-webhooks) and [Webhook Events and Payload Reference](/docs/developers/webhook-events-and-payloads).

> [!TIP]
> Run a scheduled sync as well (for example nightly) as a safety net in case a webhook delivery is missed. Because it starts from your stored token, it costs almost nothing when nothing changed.

## The @agility/content-sync SDK

The [`@agility/content-sync`](https://www.npmjs.com/package/@agility/content-sync) package (version 1.2.5 at the time of writing) runs this loop for you:

```bash
npm install @agility/content-sync
```

```js
import agilitySync from "@agility/content-sync"

const syncClient = agilitySync.getSyncClient({
  guid: process.env.AGILITY_GUID,
  apiKey: process.env.AGILITY_API_FETCH_KEY,
  languages: ["en-us"],
  channels: ["website"],
  isPreview: false
})

await syncClient.runSync()
```

What `runSync()` does, per locale:

- Reads the stored tokens (starting at `0` the first time) and syncs content items, then pages, 100 at a time.
- Saves each item to the store, and removes items whose state is Deleted.
- Waits and retries if the API reports that it is busy, for up to 10 minutes.
- If anything changed, downloads the flat and nested sitemaps for each channel you list.
- Checks URL redirections and saves them when they have changed.
- Saves the new tokens.

By default the store is the local filesystem, under `.agility-files`. You can plug in your own store (a database, Redis, or anything else) by implementing the store interface. See [Content Sync JS SDK](/docs/javascript/content-sync-js-sdk) for the interface and for reading content back with `syncClient.store`. Call `clearSync()` to empty the store and start from scratch.

## Full re-sync and recovery

Sync from `0` again when:

- you change how you store or transform content,
- your store was lost or partly written,
- you switch the same store between published (fetch) and preview content.

With the SDK, call `clearSync()` and then `runSync()`. With your own code, clear your store and the stored tokens, then sync from `0`. A full sync reads every item, so on a large instance run it outside peak hours.

## Related

- [Content Sync API](/docs/developers/content-sync-api)
- [Content Sync JS SDK](/docs/javascript/content-sync-js-sdk)
- [Content Fetch API](/docs/developers/content-fetch-api)
- [GraphQL API](/docs/developers/graphql-api)
- [Webhooks](/docs/developers/webhooks)
- [Fetch API Status Codes and Caching](/docs/developers/fetch-api-status-codes-and-caching)