Skip to content
Projects

artabramov/hidden-s3 has 60 GitHub stars and 4 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-18 12:57 UTC. Last push 2026-11-10.

How these counts are measured

Measured star growth

in 24h
0
in 7 days
Collecting history
in 15 days
Collecting history
in 1 month
Collecting history

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Encrypted self-hosted S3-compatible object storage.

Self-hosted

README.md

Original

Hidden S3 — encrypted self-hosted S3 storage

This is S3-compatible object storage for sensitive data on privately managed infrastructure. It provides filesystem-level encryption independently of the S3 layer while maintaining full S3 compatibility.

Built on gocryptfs and VersityGW, the system combines encrypted storage with an S3-compatible gateway. VersityGW provides the S3 API, IAM, and WebGUI, while FastAPI manages the storage lifecycle through an OpenAPI interface.

It runs as a Docker container with persistent state stored in two volumes: cipherdir for encrypted data and a bind-mounted secrets volume for internal secrets. The secrets volume can reside on removable media or a separate remote host.

The cipherdir is mounted through gocryptfs using an encrypted passphrase stored in secrets and unlocked with a master password. When unmounted, the data remains encrypted and inaccessible.

The encrypted filesystem uses a POSIX layout, storing S3 objects as regular files and directories together with IAM data. This allows files and directories to be transferred directly into or out of the storage without conversion.

For emergency recovery, the cipherdir can be mounted directly with gocryptfs using the decrypted passphrase. The data is not locked into the application and can be accessed or exported independently.

version test coverage security checks license

If you like it, star it ⭐ — it helps discoverability. Thank you!

Documentation

Quick start

Before installation, ensure the host meets the following requirements:

  • Linux host with Docker and FUSE support.
  • Docker permissions required for FUSE mounts.
  • Shared mount propagation for external secrets storage.
  • GNU Make for installation using the provided Makefile.

Mount remote storage or removable media at /mnt/hidden/secrets to store the application secrets, then build and run the container:

make install

Or, use force mode in a trusted environment (the /mnt/hidden/secrets path will be created automatically as a local directory on the host filesystem):

make install FORCE=1

By default, the management interface is exposed on port 8081. Use it to initialize and mount the encrypted storage and manage its state:

http://localhost:8081/docs

Once the storage is mounted, the S3 API is available on port 9000 and the WebGUI on port 8080:

http://localhost:8080

That's it — use your preferred S3-compatible client or the WebGUI.

How it works

S3 data and IAM state reside within the encrypted filesystem. Decrypted data is available only while the filesystem is mounted and is accessible externally only through the S3 gateway.

Initialization, mounting, manual unmounting, and other operations on the encrypted storage require both the secrets volume to be available and the master password to be provided. If either is unavailable, the storage remains protected.

flowchart LR
    subgraph VOL["External volumes"]
        direction TB
        STORAGE["<b>Encrypted storage</b><br/><small>hidden-cipherdir</small>"]
        SECRET["<b>Encrypted secret</b><br/><small>/mnt/hidden</small>"]
    end

    subgraph INT["Container internals"]
        direction TB
        H["<b>Hidden S3</b><br/>Mounts encrypted storage,<br/>runs the S3 gateway, and<br/>automatically shuts down<br/>and unmounts when required."]
    end

    subgraph EXT["External interfaces"]
        direction TB
        REST["<b>REST API</b><br/><small>:8081</small>"]
        S3["<b>S3 API</b><br/><small>:9000</small>"]
        GUI["<b>WebGUI</b><br/><small>:8080</small>"]
    end

    STORAGE --> H
    SECRET --> H

    H --> REST
    H --> S3
    H --> GUI

    classDef blue fill:#07539c,color:#fff,stroke:#07539c,stroke-width:2px;
    classDef interface fill:#fff,color:#000,stroke:none;

    class H,SECRET,STORAGE blue;
    class REST,S3,GUI interface;

    style EXT fill:none,stroke:#888,stroke-dasharray:5 5
    style INT fill:none,stroke:#888,stroke-dasharray:5 5
    style VOL fill:none,stroke:#888,stroke-dasharray:5 5

When an object is uploaded through the S3 API, VersityGW writes it to the mounted filesystem using standard POSIX operations. These operations pass through FUSE to gocryptfs, which transparently encrypts file contents and filenames before storing them on the underlying filesystem. Downloads follow the same path in reverse, with data decrypted on the fly.

sequenceDiagram
    participant Client as S3 Client
    participant GW as VersityGW
    participant FS as FUSE
    participant GC as gocryptfs
    participant Disk as Filesystem

    Client->>GW: PUT Object
    GW->>FS: POSIX write()
    FS->>GC: FUSE write request
    Note over GC: Encrypt file contents<br/>and filenames
    GC->>Disk: Write encrypted data

    Note over Client,Disk: Downloads follow the reverse path with transparent decryption

As an additional protection layer, a watchdog monitors critical runtime conditions. If the application process fails, the secrets volume becomes unavailable, or the storage itself becomes inaccessible, it automatically unmounts the decrypted filesystem and stops the S3 gateway.

What is protected

The protection model covers physical access to stored data, storage volumes, or the host system. Encryption remains independent of the S3 layer and application-level access controls.

  • Stolen host or unauthorized disk access. The cipherdir contains encrypted data only. Physical access to the host or storage, raw disk access, filesystem recovery tools, or direct inspection of the encrypted volume do not expose plaintext file contents or names.

  • Leaked volume data. The cipherdir and secrets volumes are protected independently. A copied or exposed cipherdir remains encrypted without the corresponding gocryptfs passphrase, while the passphrase stored in secrets is itself encrypted and cannot be used without the master password. Even possession of both volumes is insufficient to access the stored data without the master password.

  • Unexpected runtime conditions. A watchdog monitors the application process, the secrets volume, and the encrypted storage. If a critical condition is detected, it unmounts the decrypted filesystem and stops the S3 gateway.

  • Container loss or replacement. Persistent data and secrets are stored outside the application container, allowing the container to be replaced without affecting the stored data. The cipherdir can also be mounted directly with gocryptfs using the decrypted passphrase, providing access to its POSIX files and directories independently of the application.

What is not protected against

The protection model does not cover compromised runtime environments or recovery from a lost master password.

  • Runtime host or container compromise. While the encrypted filesystem is mounted, decrypted data exists inside the container. An attacker with sufficient privileges on the host or inside the container may be able to access the decrypted filesystem directly.

  • Forgotten master password. The master password is not stored anywhere. If lost, the encrypted passphrase cannot be unlocked and the storage cannot be recovered.

  • Unsecured network access. TLS is disabled by default. It can be enabled for the S3 API and WebGUI with a certificate and private key. The management API still needs TLS at the infrastructure level. Without TLS, network traffic may be intercepted.

Security audit

The project uses three security analysis tools:

  • Bandit — scans Python source code for potential security issues.
  • Semgrep — performs static analysis using Python and security audit rules.
  • pip-audit — checks Python dependencies for known vulnerabilities.

Run all checks against the running container:

make audit

The audit tools are installed inside the container when invoked.

See also

The project relies on the following core components:

License

This project is licensed under the Apache License 2.0, allowing use, modification, and distribution in accordance with its terms.

See LICENSE for details.