Skip to content
Project categories

Data · GitHub

Store, explore, and understand your data.

Useful GitHub projects

6,841 projects
36stars
rivianmate preview

Self-hosted data logger and dashboard for Rivian vehicles: drives, charging sessions, vehicle health, and more stored in your own database

DataTypeScriptGPL-3.0GitHub

node-gtfs

BlinkTagInc

506stars
node-gtfs preview

Import, validate, query, and export GTFS Schedule and Realtime data with SQLite, PostgreSQL, and MySQL.

DataTypeScriptMITGitHub

spirit

block

204stars

Online schema change and data operations for MySQL 8.0+

DataGoApache-2.0GitHub

bitcoin-etl

blockchain-etl

462stars

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

DataPythonMITGitHub

ethereum-etl

blockchain-etl

3,132stars

Python scripts for ETL (extract, transform, and load) jobs involving Ethereum blocks, transactions, ERC20/ERC721 tokens, transfers, receipts, logs, contracts, and internal transactions. Data is available in Google BigQuery: https://goo.gl/oY5BCQ

DataPythonMITGitHub

ethereum-etl-airflow

blockchain-etl

443stars

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee

DataPythonMITGitHub

polygon-etl

blockchain-etl

106stars
polygon-etl preview

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

DataPythonMITGitHub

scan

blockloop

614stars

Tiny lib to scan SQL rows directly to structs, slices, and primitive types

tsfresh

blue-yonder

9,495stars
tsfresh preview

Automatic extraction of relevant features from time series:

  • Christ, M., Braun, N., Neuffer, J., and Kempa-Liehr A.W. (2018). Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh -- A Python package). Neurocomputing 307, p. 72-77, doi: 10.1016/j.neucom.2018.03.067.
  • Christ, M., Kempa-Liehr, A.W., and Feindt, M. (2017). Distributed and parallel time series feature extraction for industrial big data applications. ArXiv e-print 1610.07717, https://arxiv.org/abs/1610.07717.
DataJupyter NotebookMITGitHub

turbodbc

blue-yonder

658stars

Turbodbc is a Python module to access relational databases via the Open Database Connectivity (ODBC) interface. The module complies with the Python Database API Specification 2.0.

Tree

BlueM

237stars

PHP library for handling tree structures based on parent IDs, e.g. a self-joined database table

serverless

bluenimble

32stars
serverless preview

BlueNimble is a Hybrid Serverless Platform focusing on developer productivity and application portability. Create and run scalable APIs and applications without coding or by coding less. Focus on application business logic without any knowledge of the underlying microservices architecture.

DataHTMLApache-2.0GitHub

Meerschaum

bmeares

154stars
Meerschaum preview

Create and manage data pipes with Meerschaum.

  • ⚡️ Incremental by default — the sync engine fetches only new or changed rows and concurrently updates many streams at once. Duplicate rows are ignored; rows with existing keys are updated.
  • 📊 Built for data scientists and analysts — integrate with Pandas, Grafana, and friends; persist DataFrames and always get the latest data. Skip pandas overhead and read rows as plain dicts with Pipe.get_docs().
DataRustApache-2.0GitHub