egret-data/platform

What is Egret

Egret is an enterprise data platform that lets many teams work on one shared data foundation. A single deployment gives the organization one source of truth, gives each team an isolated workspace, and governs and audits every access to data.

Most organisations end up with one BI tool, one catalog, one access-policy stack and one workflow tool per department: silos, copies of the data, and access rules nobody can audit. Egret replaces that with one deployment:

• One data foundation. A shared Iceberg lakehouse and the query engines on top of it. Teams do not copy data; they get a workspace.
• Isolated workspaces. Dashboards, datasets, charts, saved queries, connections and APIs live inside a workspace and never leak across it.
• Governance in the engine, not around it. One authorizer (Cedar), per-user delegated queries, Ranger masking and row filters, and an audit log of every decision.

It is a Rust modular monolith (one binary, 37 crates) with a React front end, and it brings together what you would otherwise assemble from Metabase or Superset (BI), Lakekeeper (Iceberg catalog), Apache Ranger (access policy), a data-API builder and a workflow engine.

Highlights

Area	What you get
Explore	SQL Lab, datasets with metrics, 19 chart types, dashboards with filters and drill-through, alerts and reports (email and Slack, PNG and PDF).	Docs
Data API	Publish a parameterised SELECT as a REST endpoint with OAuth2 clients, versions, rate limits and an activity log.	Docs
Data Apps	marimo apps that run as the person viewing them, so the data policy follows the viewer.	Docs
Workflows	A workflow engine compatible with Kestra's flow syntax: YAML flows with schedules, webhooks and triggers, executions with logs, namespaces and files.	Docs
Governance	Cedar authorization, Ranger-compatible policy for Trino and StarRocks, tag-based masking, classifications, audit log, SSO (OIDC).	Docs
Engines and catalog	Provision Trino (on Docker or Kubernetes) and StarRocks (on Kubernetes) from the UI; an embedded Iceberg REST catalog; query federation over 30+ connectors.	Docs

Screenshots

A dashboard in a workspace: the demo stack seeds this one, with 22 charts on three tabs.

Every chart type, on sample data: donut, box plot, bars, a US map, scatter, funnel, Sankey and more.

Quick start

Run the whole platform on Docker, with no Kubernetes: Egret, Postgres, object storage, Keycloak SSO, a Trino engine, Data Apps, a mail catcher, a PDF/PNG renderer, and a sample workspace with a dataset, charts and a dashboard.

git clone https://github.com/egret-data/platform.git
cd platform
just demo-up

Then open http://egret.localtest.me:8080 and sign in with Admin sign-in (break-glass): [email protected] / Admin12345!. Single sign-on users admin-sso and member-sso are there too, and every mail the demo sends lands in Mailpit at http://localhost:8025.

• Needs Docker with Compose v2, just, curl and python3, and 6 to 8 GB of free RAM.
• The first run builds the backend (20 to 40 minutes); later runs reuse the images.
• Demo only. The stack uses fixed public secrets. Never expose it to a network.

The full guide, with ports, engines, Data Apps and how to reset, is docs/en/guides/demo-stack.md. The Compose stacks and the Helm chart (deploy/helm) are development tooling, not a production topology: read the security hardening checklist before exposing a deployment, and the configuration keys.

How it fits together

• Backend: a Rust workspace (axum, tokio, sqlx) built into one binary that serves the API, the control plane and every workspace.
• Front end: one React single-page app (Vite, Tailwind, Radix) with two areas: Explore (per workspace) and Control Plane and Governance (platform admins).
• State: PostgreSQL for metadata and S3-compatible object storage (RustFS in the demo) for the lakehouse.
• Engines: Trino runs as containers (Docker) or pods (Kubernetes) and StarRocks as pods, all provisioned and watched by Egret; each query runs as the calling user.
• Identity: OIDC single sign-on for people; Egret mints short-lived per-user tokens for engines, so there are no shared engine passwords.

Deeper reading: platform overview (data plane, federation, the governed query lifecycle, isolation, security) and the full architecture design.

Documentation

Get started	What Egret is, the concepts, a first dashboard.
Run the demo stack	The Docker quick start in detail.
Explore, Data API, Workflows	Page-by-page user guide with screenshots.
Governance and Control Plane	Access policy, engines, catalogs, storage, secrets.
Reference	Configuration keys and the REST API.
Known issues	What is open, with severity, and what is planned.

The documentation lives in egret-data/docs; a change that alters what users see ships with its docs update.

Repository layout

crates/       37 Rust crates (server, auth, policy, catalog, engines, query, bi, workflow, apps, ...)
frontend/     React + Vite single-page app
migrations/   sqlx migrations
deploy/       Dockerfiles, the Docker demo stack, Helm chart, Keycloak and engine configs
apps-image/   the Data Apps (marimo) runtime image
renderer/     the PDF and PNG renderer used by reports
plugin-host/  the JVM host that runs workflow plugins
docs/         architecture notes, specs and plans (English canonical, Vietnamese reference)

Status

Egret is under active development. Authentication and SSO, workspaces, unified authorization, Ranger enforcement, the secrets vault, BI and Explore, the Iceberg catalog, engine provisioning, Data Apps, the Data API and workflows are built and in use. Open defects and gaps are tracked, with severity, in Known issues.

Contributing and security

Read CONTRIBUTING.md: Conventional Commits, tests with every change, and docs with every user-visible change. Report a vulnerability as SECURITY.md describes, not in a public issue. Everyone taking part follows the Code of Conduct.

License

Licensed under the Apache License 2.0.