Skip to content
项目

tradewelltech/beavers has 41 GitHub stars and 2 forks. GitHot measured 0 stars in 24 hours and 0 in 7 days. Last checked 2026-11-22 00:43 UTC. Last push 2026-07-28.

这些数字如何统计

可以做什么

  • Works in real time (eg: reading from Kafka) and replay mode (eg: reading from Parquet files).
  • Optimized for analytics, using micro-batches (instead of processing records one by one).
  • Similar to incremental, it updates nodes in a dag incrementally.
  • Stream: ephemeral micro-batches of events (cleared after every cycle).

实测星标增长

24小时
0
7天
0
15天
正在收集历史
1个月
正在收集历史

各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。

关于

Python stream processing for analytics

数据

README.md

原文

PyPI Version Python Version Github Stars codecov Build Status Documentation License Downloads Downloads Ruff prek snyk trackgit-views

Beavers Logo

Beavers

Documentation / Installation / Repository / PyPI

Beavers is a python library for stream processing, optimized for analytics.

It is used at Tradewell Technologies, to calculate analytics and serve model predictions, for both realtime and batch jobs.

Key Features

  • Works in real time (eg: reading from Kafka) and replay mode (eg: reading from Parquet files).
  • Optimized for analytics, using micro-batches (instead of processing records one by one).
  • Similar to incremental, it updates nodes in a dag incrementally.
  • Taking inspiration from kafka streams, there are two types of nodes in the dag:
    • Stream: ephemeral micro-batches of events (cleared after every cycle).
    • State: durable state derived from streams.
  • Clear separation between the business logic and the IO. So the same dag can be used in real time mode, replay mode or can be easily tested.
  • Functional interface: no inheritance or decorator required.
  • Support for complicated joins, not just "linear" data flow.

Limitations

  • No concurrency support. To speed up calculation use libraries like pandas, pyarrow or polars.
  • No async code. To speed up IO use kafka driver native thread or parquet IO thread pool.
  • No support for persistent state. Instead of saving state, replay historic data from kafka to prime stateful nodes.

Talks

探索相关项目

加载中…