Skip to content
Project categories

Data · GitHub

Store, explore, and understand your data.

Useful GitHub projects

4,541 projects

data-engineering

sophie-nguyenthuthuy

56stars

100+ data engineering projects from scratch — streaming, CDC, table formats, query engines, consensus, governance. 2,500+ tests, mypy strict.

DataPythonGitHub

sparkdq

sparkdq-community

77stars

A lightweight, declarative PySpark framework for data quality validation — check columns, rows, and entire datasets directly in your Spark pipelines

DataPythonApache-2.0GitHub

sparklabx

sparklabx

31stars
sparklabx preview

Self-hosted multi-user Apache Spark notebooks — PySpark + Scala, per-user kernel pods, MinIO IAM storage, OAuth. One Helm install. Apache-2.0.

  • Spark notebooks in the browser — PySpark, Scala (Almond), with Monaco editor, Markdown cells, kernel restart, and a Libraries dialog that wraps spark.jars.packages / import $ivy so users add Maven coordinates without editing code.
  • Embedded Spark UI + DataFlint — open the kernel's Spark Web UI (DAGs, stages, SQL plans, metrics) in-app, no port-forward. DataFlint (a Spark performance plugin) is on by default for PySpark and Scala, adding a performance tab that flags slow stages, skew, and oversized scans.
DataTypeScriptApache-2.0GitHub

play-reactive-slick

spiderveer1-byte

149stars
play-reactive-slick preview

Play + Postgres starter template with a clean UI

  • Play Framework backend with a clean, ready-to-use UI out of the box
  • PostgreSQL wired in via Slick for functional, type-safe database access
DataHTMLMITGitHub

spot2

spotorm

592stars

Spot v2.x DataMapper built on top of Doctrine's Database Abstraction Layer

DataPHPBSD-3-ClauseGitHub

zorm

springrain

84stars

Go轻量ORM,支持达梦(dm),金仓(kingbase),神通(shentong),南大通用(gbase),TDengine,mysql,postgresql,oracle,mssql,sqlite,db2,clickhouse...

DataGoApache-2.0GitHub

sqg

sqg-dev

48stars

Generate typesafe code from SQL queries, DuckDB, SQlite, Postgres. ⭐️ Star to support our work!

DataTypeScriptGitHub

sqlite-js

sqliteai

569stars

Create custom SQLite functions in JavaScript. Extend your database with scalars, aggregates, window functions, and collations directly in JavaScript.

DataCApache-2.0GitHub

sqlobject

sqlobject

157stars

SQLObject, an object-relational mapper for Python

DataPythonLGPL-2.1GitHub

datasheets

Squarespace

623stars

Read data from, write data to, and modify the formatting of Google Sheets

DataPythonGitHub

squeryl

squeryl

588stars

A Scala DSL for talking with databases with minimum verbosity and maximum type safety

DataScalaApache-2.0GitHub

the-apache-ignite-book

srecon

86stars
the-apache-ignite-book preview

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

DataJavaGPL-3.0GitHub

goobie-sql

Srlion

27stars

Easily connect to MySQL in Garry's Mod using Rust binary!

  • ✅ Dual Database Support - MySQL and SQLite
  • ⚡ Async & Sync - Both asynchronous and synchronous queries
DataLuaMITGitHub

data-engineering-patterns

ssanjaychandra123

93stars
data-engineering-patterns preview

600+ free patterns and concepts for data engineers on Azure Databricks, Microsoft Fabric, and PySpark. 12 books covering the full stack. Free forever.

DataGitHub

spark-pdf

StabRise

83stars
spark-pdf preview

PDF DataSource for Apache Spark, allow to read PDF files directly to the DataFrame and ocr it

  • Read PDF documents to the Spark DataFrame
  • Support efficient read PDF files lazy per page
DataScalaAGPL-3.0GitHub

French-Postal-Code

Stanislas-Poisson

87stars
French-Postal-Code preview

Dataset of the regions, departments, communes and postal codes of France, with one GPS point per postal code and the history of changes. Built from INSEE, La Poste, BAN and geo.api.gouv.fr.

DataPHPMITGitHub