Vyntr.com - the independent search engine.

Privacy Policy | Terms of Service | License | YouTube video

Vyntr is a search engine project with multiple components:

Components

• Genesis - Web crawler and content analyzer
• Pulse - Search indexing system using Tantivy
• Lexicon - WordNet-based dictionary lookup service
• Website - Frontend interface at vyntr.com

Setup

1. Create a .env file in the root directory:

# Database
PRIVATE_DB_URL="postgresql://postgres:your_password@serverip:port/postgres"

# AWS S3/Compatible Storage
S3_ENDPOINT="https://s3.eu-central-1.amazonaws.com"
S3_REGION="eu-central-1"
S3_BUCKET="vyntr"
AWS_ACCESS_KEY_ID="your-key-id"
AWS_SECRET_ACCESS_KEY="your-secret-key"

1. Set up the database:

cd genesis/tools/database
docker compose up -d

1. Set up individual components:

• Genesis crawler: Follow genesis setup
• Lexicon service: Follow lexicon setup
• Website: Follow website setup

Pipeline

1. Genesis crawler collects and analyzes web pages
2. Data is stored in partitioned JSONL files in S3
3. Content is cleaned through dataset.
4. Content is processed through embedding tools (vector), or Pulse (full-text).
5. Website frontend provides search interface.

Requirements

• Python with uv package manager
• Node.js
• PostgreSQL with pgvector
• Docker
• Bun runtime (for Lexicon service)
• Rust toolchain

Dataset

The Vyntr dataset is not publicly available. For licensing inquiries, please contact [email protected].

You may also use the official API provided at https://vyntr.com/api.

License

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). See the LICENSE file for details.

Individual components may have additional licensing requirements. See their respective directories for specific licensing information.

WordNet data used in Lexicon is subject to the WordNet License.