altasin356/Transformer-Summarizer-Infosys-Springboard
altasin356/Transformer-Summarizer-Infosys-Springboard has 56 GitHub stars and 0 forks. GitHot measured 0 stars in 24 hours, 0 in 7 days, and 0 in 15 days. Last checked 2026-11-21 17:40 UTC. Last push 2026-11-18.
可以做什么
- 🤖 Dual-Engine Summarization — Abstractive Transformer and extractive rule-based pipelines under one API.
- 🌍 Multilingual Support — Summaries in 12+ languages, with automatic language detection and graceful fallback.
- 📱 Responsive UI — A mobile-first web console that resizes fluidly from phone to ultrawide monitor.
- ⚡ Streaming Inference — Token-by-token output so users see progress instead of a spinner.
实测星标增长
- 24小时
- 0
- 7天
- 0
- 15天
- 0
- 1个月
- 正在收集历史
各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。
关于
AI Text Summarization with Transformers and Rule-Based NLP 2026
机器学习README.md
原文🧠 LumenScribe — Abstractive & Extractive Summarization Toolkit
LumenScribe is a dual-engine summarization framework that distills long, noisy documents into crisp, faithful digests. Instead of forcing a single paradigm, it pairs a Transformer-based abstractive engine with a transparent, rule-driven extractive engine, letting you switch, blend, or benchmark both from one clean interface.
📖 Table of Contents
- Why LumenScribe?
- Core Concept
- Feature Highlights
- Architecture Overview
- Repository Layout
- Quick Start Guide
- Supported Languages
- Configuration Reference
- Evaluation & Benchmarks
- Responsive Web Interface
- API Surface
- SEO & Discoverability Notes
- Roadmap 2026
- Contributing
- License
- Disclaimer
🔍 Why LumenScribe?
Most summarization projects pick a side. Abstractive pipelines hallucinate beautifully; extractive pipelines stay faithful but read like a bag of spliced sentences. LumenScribe refuses that trade-off. It treats summarization the way a skilled editor treats a manuscript — sometimes rewriting a paragraph into one luminous line, sometimes simply underlining the sentence that already says it best.
The result is a toolkit that adapts to the document, not the other way around.
🧭 Core Concept
Imagine a library where two librarians work side by side:
- The Paraphraser — a neural Transformer that reads the whole text, grasps the latent meaning, and writes a fresh, condensed passage in its own words.
- The Curator — a deterministic, rule-based system that scores sentences by position, keyword density, and discourse markers, then assembles a summary strictly from verbatim source sentences.
You choose the librarian. Or you let LumenScribe choose for you through a hybrid arbitration layer that weighs document type, length, and target compression ratio.
✨ Feature Highlights
- 🤖 Dual-Engine Summarization — Abstractive Transformer and extractive rule-based pipelines under one API.
- 🌍 Multilingual Support — Summaries in 12+ languages, with automatic language detection and graceful fallback.
- 📱 Responsive UI — A mobile-first web console that resizes fluidly from phone to ultrawide monitor.
- ⚡ Streaming Inference — Token-by-token output so users see progress instead of a spinner.
- 🧩 Pluggable Preprocessors — Boilerplate stripping, section detection, and de-duplication as composable stages.
- 📊 Built-in Evaluation Suite — ROUGE, BERTScore, compression ratio, and faithfulness heuristics.
- 🕒 24/7 Customer Support — Round-the-clock triage desk with an average first-response window under two hours.
- 🔒 Privacy-First Design — Local-only mode keeps sensitive documents on your own hardware.
- 🧪 Experiment Tracking — Every run logged with hyperparameters, seeds, and artifacts for reproducibility.
- 🎛️ Batch & Single Modes — One document or ten thousand, same interface.
🏛️ Architecture Overview
The pipeline flows through five conceptual stages:
- Ingestion — Raw text, PDFs, or HTML are normalised into clean unicode.
- Segmentation — Sentences, paragraphs, and headings are labelled with structural roles.
- Scoring / Encoding — The extractive scorer assigns salience weights; the Transformer encoder builds contextual embeddings.
- Generation / Selection — Either the decoder writes a new summary, or the selector stitches source sentences.
- Post-Processing — Redundancy pruning, length calibration, and optional style transfer.
A thin orchestration layer routes each request to the appropriate engine based on the strategy parameter.
🗂️ Repository Layout
lumen-scribe/
├── lumen/ # Core library
│ ├── abstractive/ # Transformer summarization engine
│ ├── extractive/ # Rule-based summarization engine
│ ├── hybrid/ # Arbitration + blending logic
│ ├── preprocess/ # Cleaning, segmentation, language ID
│ ├── eval/ # ROUGE, BERTScore, faithfulness checks
│ └── cli/ # Command-line entry points
├── web/ # Responsive front-end console
├── api/ # HTTP service layer
├── configs/ # YAML experiment configs
├── notebooks/ # Exploratory and demo notebooks
├── tests/ # Unit + integration tests
└── docs/ # Extended documentation
🚀 Quick Start Guide
LumenScribe is designed to be welcoming from the very first command. The internal bootstrap script handles environment discovery and asset retrieval automatically.
- Prepare your workspace — create a project directory wherever you prefer.
- Launch the bootstrap routine — run the provisioning script shipped in the repository root; it resolves dependencies and caches language assets.
- Point it at a document — feed a plain-text or PDF file into the summarizer via the CLI or web console.
- Choose your strategy — set
strategy=abstractive,strategy=extractive, orstrategy=hybrid. - Read the digest — output lands in your console and, optionally, in a structured JSON file.
For orchestrated deployments, the same bootstrap routine supports container and serverless targets.
🌐 Supported Languages
LumenScribe ships with ready-to-use language packs for:
English, Spanish, French, German, Portuguese, Italian, Dutch, Hindi, Bengali, Tamil, Japanese, and Mandarin Chinese. Additional packs can be registered via the same manifest system used by the bundled ones.
⚙️ Configuration Reference
Every behavior is dictated by a YAML config. A condensed example of the tunables:
| Key | Purpose | Typical Value |
|---|---|---|
strategy |
Engine selection | hybrid |
max_length |
Summary token ceiling | 180 |
min_length |
Summary token floor | 40 |
compression_ratio |
Target reduction | 0.15 |
language |
Forced language code | auto |
beam_size |
Decoding width | 4 |
faithfulness_weight |
Penalty for unsupported spans | 0.3 |
Configuration files live in configs/ and can be layered — a base file plus per-experiment overrides.
📈 Evaluation & Benchmarks
The evaluation suite runs on demand and produces a compact report card per document set:
- ROUGE-1 / ROUGE-2 / ROUGE-L for n-gram overlap.
- BERTScore for semantic alignment.
- Compression Ratio to confirm the digest is genuinely shorter.
- Faithfulness Heuristic comparing abstractive spans against source evidence.
- Readability Indices for human-oriented quality.
Results are stored under runs/<timestamp>/ with full provenance so any number in a report can be traced back to the exact run that produced it.
📱 Responsive Web Interface
The web console is built with a mobile-first philosophy. It reflows from a single column on phones to a three-pane editorial layout on large displays. Dark mode is bundled. Keyboard shortcuts let power users move between documents without ever touching a mouse. The interface never blocks on a summarization call — the streaming layer paints partial output as it arrives.
🔌 API Surface
The HTTP service exposes a small, predictable set of endpoints:
POST /summarize— single-document summarization with a chosen strategy.POST /batch— queue a collection of documents and poll for results.GET /languages— list installed language packs.GET /health— liveness probe for orchestration platforms.GET /runs— enumerate historical runs with metadata.
All responses are JSON. Errors are structured, not free-text.
🔎 SEO & Discoverability Notes
This repository is written to be discoverable by people searching for abstractive text summarization, extractive summarization, Transformer summarization models, multilingual summarization tools, document digest generators, and rule-based NLP pipelines. Keywords appear where they genuinely help a reader — never as filler.
🛣️ Roadmap 2026
- Q1 2026 — Publish reference weights for three additional language packs.
- Q2 2026 — Introduce a fine-tuning recipe notebook with dataset schema docs.
- Q3 2026 — Ship a desktop companion app wrapping the local-only mode.
- Q4 2026 — Expand the arbitration layer with learned strategy selection.
🤝 Contributing
Contributions are welcome across code, docs, translations, and evaluation datasets. Open an issue describing the change before large pull requests so the direction can be agreed on early. All contributors are expected to follow the project's code of conduct and to keep commits scoped and readable.
📜 License
This project is distributed under the MIT License. The full text lives in the repository's license file and is also available at the canonical reference:
https://opensource.org/licenses/MIT
⚠️ Disclaimer
LumenScribe produces machine-generated summaries. While the engine is tuned for faithfulness, no automated system is infallible — generated digests may omit nuance, misattribute phrasing, or compress context in ways that change emphasis. Always review output before relying on it for legal, medical, financial, or editorial decisions. The maintainers provide this software as-is, without warranty of any kind, and are not liable for decisions made using its output.
探索相关项目
加载中…