slacrawl 💾 — Your Slack history, on your terms.

slacrawl mirrors Slack workspace data into local SQLite for full-text search, terminal browsing, reporting, and read-only SQL. It is for people who want a queryable Slack archive that stays under their control.

Install

Homebrew is the shortest path on macOS or Linux:

brew install openclaw/tap/slacrawl

The tap can trail the newest release. GitHub Releases provides signed and notarized macOS archives, Linux archives, and Debian/RPM packages for AMD64 and ARM64.

To build the latest source, install Go 1.27.0 or newer, then run:

git clone https://github.com/openclaw/slacrawl.git
cd slacrawl
make build
./bin/slacrawl --help

macOS binaries require macOS 13 or newer.

The repository also includes a Dockerfile for container builds.

Quick start

Set SLACK_BOT_TOKEN, or SLACK_USER_TOKEN for user-only API sync, to a token that can read the workspace. Then replace the example workspace and channel IDs:

slacrawl init --workspace T01234567
slacrawl doctor
slacrawl sync --source bot --channels C01234567
slacrawl search "incident"
slacrawl tui

init writes ~/.slacrawl/config.toml; database-backed commands use ~/.slacrawl/slacrawl.db by default. A configured bot takes precedence; failed bot authentication never falls back to the user token. Without a bot, the user owns discovery, profiles and history. Live Socket Mode tailing still requires both bot and app tokens.

Slacrawl prefers the user token for historical replies and optional DMs. With no user token configured, API sync fetches channel replies with the bot where its scopes and membership permit; workspace thread coverage stays partial. A rejected user credential never triggers bot reply fallback.

Already have a Slack export? Import its ZIP or extracted directory instead of syncing from the API:

slacrawl import ~/Downloads/slack-export.zip --workspace T01234567

Set [sync].include_dms = false in the selected config to exclude DM and group-DM conversations from workspace JSON export imports. Unqualified types or ambiguous conversation ownership stop the import. Omitted/true retain DM inclusion. Dry-run checks the export without initializing or repairing the archive. See Slack export admission for supported formats, file-change detection, and privacy limits.

Choose an ingestion source

Every source feeds the same SQLite archive and search index.

Source	Command	Use it for
Slack API	slacrawl sync --source bot	Token-backed channel history, users, threads, and incremental refreshes
Slack Desktop	slacrawl sync --source wiretap	Read-only recovery from local macOS or Linux desktop caches
MCP connector	slacrawl sync --source mcp --workspace T01234567	Connector-backed history without a direct Slack API integration
External provider	slacrawl sync --source provider:archive --workspace T01234567	A trusted local JSONL adapter for another archive
Slack export	slacrawl import <path> --workspace T01234567	ZIP or directory exports

Treat one config and database as one visibility boundary. Keep personal and company archives in separate configs, database paths, and share remotes. See Configuration for tokens, multiple workspaces, MCP, external providers, media caching, and source precedence; see Desktop mode for local-cache coverage and limitations.

Ordinary API sync also revisits retained thread roots, keeping unfinished replies work across restarts and token changes. Explicit --since and Tail repair leave that backlog alone. See Retained API threads for scope and partial-coverage behavior.

External provider v1 sync rejects [sync].include_dms = false before reading its checkpoint or launching the adapter. Use API sync or a supported Slack workspace JSON export when excluding DMs. Omitted/true retain provider intake behavior.

Ordinary MCP sync also preserves unfinished replies across history overwrites and restarts. Explicit --since, including --full --since, reads only threads whose roots are returned in history, including roots identified by archived children, and leaves older pending work alone. See Retained MCP threads for missing thread tools, cancellation and partial-coverage behavior.

Native MCP history/replies responses that report more pages or Slack history/message limits retain valid fetched writes but stop with an error before advancing successful sync state. See native response coverage.

Desktop recovery includes unsent drafts by default. Set [slack.desktop].include_drafts = false to exclude future draft ingestion; drafts already archived remain unchanged. Set [sync].include_dms = false to also exclude future desktop DM intake; unclassified cache records are omitted and reported. See Desktop admission limits.

Explore the archive

Search uses SQLite FTS5 with a substring fallback. The same archive is available through the TUI, structured commands, and read-only SQL:

slacrawl report
slacrawl messages --limit 20
slacrawl sql 'select channel_id, count(*) from messages group by channel_id;'
slacrawl --json status

Use --format text, --format json, or --format log on commands that support structured output. Color turns off when stdout is not a TTY; --no-color and NO_COLOR=1 force plain output.

Schema 9 retains a private, key-only native change journal and exposes its generation and sequence anchors in status --json. This is the foundation for later incremental delivery; no delivery worker or journal compaction is enabled. See the journal contract.

Library callers can capture a bounded, read-only six-table native seed at one consistent journal head and project it through the private archive profile. This does not deliver data or qualify an incremental baseline; see native seed capture.

The internal delivery API upgrades recognized version-1 outboxes to version 2 on open, preserving pending requests and receipts exactly. Older binaries reject the upgraded file. Explicit library callers can stage a complete private seed and resume exact requests through the existing delivery driver; only the final verified receipt qualifies its native baseline. Ambiguous keys retain evidence but block completion. No scheduler is enabled. See the outbox contract.

Explicit operator commands are available as slacrawl delivery init, stage-seed, stage-changes, and deliver. They require a separately qualified private scope, explicit native/outbox paths and bounded operations; only deliver contacts the registered service using CRAWL_REMOTE_TOKEN (or --token-env). They neither sync nor migrate the source/outbox and reject --config. See the command and scope contract before use. Incremental staging uses the latest receipt-qualified baseline; unchanged H is a no-op, while excluded changes still require an empty part and receipt.

Keep it current

Run an incremental API refresh after the first sync:

slacrawl sync --source bot

Use --latest-only to update only channels that already have local history, tail for Socket Mode events, or watch for recurring desktop-cache refreshes. Ordinary incremental sync preserves retention cutoffs; --full, an older explicit --since, desktop ingestion, or imports can deliberately restore older records.

API sync retains valid fetched pages but reports an error when Slack still signals more results without a continuation cursor, or reports a history/message limit. The attempted interval stays pending instead of advancing successful coverage. See API history completeness.

Share an archive

One machine can publish compressed, git-backed snapshots while other machines subscribe and query locally without Slack credentials. Routine updates merge safely and preserve destination-only rows; exact replacement requires update --restore.

See Git archive sharing for publisher and subscriber setup, snapshot tags, media behavior, and restore semantics.

Retention

purge previews its effect unless --force is supplied:

slacrawl purge --older-than 90d
slacrawl purge --older-than 90d --force --vacuum

Threads are retained or removed as a unit based on the parent timestamp. See Retention purge for workspace scoping, message-event compaction, cached media, and git-backed archive behavior.

Commands

Area	Commands
Setup and health	init, doctor, status, check-update
Ingest and refresh	sync, import, tail, watch, files
Browse and query	search, tui, messages, mentions, users, channels, sql
Reports	report, digest, analytics
Offline projections	export prepare, export build, export verify
Git snapshots	publish, subscribe, update
Automation	metadata, completion

The command reference describes every command, output modes, shell completion, and common workflows. slacrawl <command> --help is the authoritative flag reference.

Development

make build
make test
make check

make check runs the local CI gates, including formatting, vet, vulnerability and dead-code checks, tests, CLI smoke tests, and a release snapshot. Read CONTRIBUTING.md before changing behavior and SPEC.md for the implementation contract.

License

MIT. Built by Vincent Koc.