Word Error Rate for Python

Meta
License
Security
Testing
Package

What is werpy?

werpy is an ultra-fast, lightweight Python package for calculating and analyzing Word Error Rate (WER) between two sets of text.

Built for flexibility and ease of use, it supports multiple input types such as strings, lists, and NumPy arrays. This makes it ideal for everything from quick experiments to large-scale evaluations.

With speed in mind at every scale, werpy harnesses the efficiency of C optimizations to accelerate processing, delivering ultra-fast results from small datasets to enterprise-level workloads.

It also comes packed with powerful features, including:

• 🔤 Built-in text normalization to handle data inconsistencies
• ⚙️ Customizable error penalties for insertions, deletions, and substitutions
• 📋 A detailed summary output for in-depth error analysis

werpy is a quality-focused package, built to production-grade standards for reliability and robustness.

Functions available in werpy

The following table provides an overview of the functions that can be used in werpy.

Function	Description
normalize(text)	Preprocess input text to put it into Unicode Normalization Form C, remove ASCII and Unicode punctuation, remove duplicated spaces, leading/trailing blanks and convert all words to lowercase.
wer(reference, hypothesis)	Calculate the overall Word Error Rate for the entire reference and hypothesis texts.
wers(reference, hypothesis)	Calculates a list of the Word Error Rates for each of the reference and hypothesis texts.
werp(reference, hypothesis, insertions_weight=1, deletions_weight=1, substitutions_weight=1)	Calculates a weighted Word Error Rate for the entire reference and hypothesis texts.
werps(reference, hypothesis, insertions_weight=1, deletions_weight=1, substitutions_weight=1)	Calculates a list of weighted Word Error Rates for each of the reference and hypothesis texts.
summary(reference, hypothesis, *, output="pandas")	Provides a comprehensive breakdown of the calculated results including the WER, Levenshtein Distance and all the insertion, deletion and substitution errors. Returns a pandas DataFrame, a polars DataFrame with output="polars", or a list of dictionaries with output="records".
summaryp(reference, hypothesis, insertions_weight=1, deletions_weight=1, substitutions_weight=1, *, output="pandas")	Delivers an in-depth breakdown of the results, covering metrics like WER, Levenshtein Distance, and a detailed account of insertion, deletion, and substitution errors, inclusive of the weighted WER. Returns a pandas DataFrame, a polars DataFrame with output="polars", or a list of dictionaries with output="records".

Installation

You can install the latest werpy release with Python's pip package manager:

# Install werpy from PyPi
pip install werpy

summary() and summaryp() return their results as a pandas DataFrame, a polars DataFrame or a list of dictionaries. To install pandas or polars with werpy, use its extra:

# Install werpy and pandas from PyPi
pip install "werpy[pandas]"

# Install werpy and polars from PyPi
pip install "werpy[polars]"

With a plain pip install werpy, output="records" returns the results as a list of dictionaries.

Usage

Import the werpy package

Python Code:

import werpy

Example 1 - Normalize a list of text

Python Code:

input_data = ["It's very popular in Antarctica.","The Sugar Bear character"]
reference = werpy.normalize(input_data)
print(reference)

Results Output:

['its very popular in antarctica', 'the sugar bear character']

Example 2 - Calculate the overall Word Error Rate on a set of strings

Python Code:

wer = werpy.wer('i love cold pizza', 'i love pizza')
print(wer)

Results Output:

0.25

Example 3 - Calculate the overall Word Error Rate on a set of lists

Python Code:

ref = ['i love cold pizza','the sugar bear character was popular']
hyp = ['i love pizza','the sugar bare character was popular']
wer = werpy.wer(ref, hyp)
print(wer)

Results Output:

0.2

Example 4 - Calculate the Word Error Rates for each set of texts

Python Code:

ref = ['no one else could claim that','she cited multiple reasons why']
hyp = ['no one else could claim that','she sighted multiple reasons why']
wers = werpy.wers(ref, hyp)
print(wers)

Results Output:

[0.0, 0.2]

Example 5 - Calculate the weighted Word Error Rates for the entire set of text

Python Code:

ref = ['it was beautiful and sunny today']
hyp = ['it was a beautiful and sunny day']
werp = werpy.werp(ref, hyp, insertions_weight=0.5, deletions_weight=0.5, substitutions_weight=1)
print(werp)

Results Output:

0.25

Example 6 - Calculate a list of weighted Word Error Rates for each of the reference and hypothesis texts

Python Code:

ref = ['it blocked sight lines of central park', 'her father was an alderman in the city government']
hyp = ['it blocked sightlines of central park', 'our father was an elder man in the city government']
werps = werpy.werps(ref, hyp, insertions_weight = 0.5, deletions_weight = 0.5, substitutions_weight = 1)
print(werps)

Results Output:

[0.21428571428571427, 0.2777777777777778]

Example 7 - Provide a complete breakdown of the Word Error Rate calculations for each of the reference and hypothesis texts

Python Code:

ref = ['it is consumed domestically and exported to other countries', 'rufino street in makati right inside the makati central business district', 'its estuary is considered to have abnormally low rates of dissolved oxygen', 'he later cited his first wife anita as the inspiration for the song', 'no one else could claim that']
hyp = ['it is consumed domestically and exported to other countries', 'rofino street in mccauti right inside the macasi central business district', 'its estiary is considered to have a normally low rates of dissolved oxygen', 'he later sighted his first wife anita as the inspiration for the song', 'no one else could claim that']
summary = werpy.summary(ref, hyp)
print(summary)

Results Output:

Example 8 - Provide a complete breakdown of the Weighted Word Error Rate for each of the input texts

Python Code:

ref = ['the tower caused minor discontent because it blocked sight lines of central park', 'her father was an alderman in the city government', 'he was commonly referred to as the blacksmith of ballinalee']
hyp = ['the tower caused minor discontent because it blocked sightlines of central park', 'our father was an alderman in the city government', 'he was commonly referred to as the blacksmith of balen alley']
weighted_summary = werpy.summaryp(ref, hyp, insertions_weight = 0.5, deletions_weight = 0.5, substitutions_weight = 1)
print(weighted_summary)

Results Output:

Example 9 - Get the complete breakdown as a list of dictionaries

Python Code:

ref = ['it was beautiful and sunny today']
hyp = ['it was a beautiful and sunny day']
records = werpy.summary(ref, hyp, output="records")
print(records)

Results Output:

[{'wer': 0.3333333333333333, 'ld': 2, 'm': 6, 'insertions': 1, 'deletions': 0, 'substitutions': 1, 'inserted_words': ['a'], 'deleted_words': [], 'substituted_words': [('today', 'day')]}]

Example 10 - Get the complete breakdown as a polars DataFrame

Python Code:

ref = ['it was beautiful and sunny today', 'she cited multiple reasons why']
hyp = ['it was a beautiful and sunny day', 'she sighted multiple reasons why']
summary = werpy.summary(ref, hyp, output="polars")
print(summary.select("wer", "ld", "inserted_words", "substituted_words"))

Results Output:

shape: (2, 4)
┌──────────┬─────┬────────────────┬───────────────────────┐
│ wer      ┆ ld  ┆ inserted_words ┆ substituted_words     │
│ ---      ┆ --- ┆ ---            ┆ ---                   │
│ f64      ┆ i64 ┆ list[str]      ┆ list[struct[2]]       │
╞══════════╪═════╪════════════════╪═══════════════════════╡
│ 0.333333 ┆ 2   ┆ ["a"]          ┆ [{"today","day"}]     │
│ 0.2      ┆ 1   ┆ []             ┆ [{"cited","sighted"}] │
└──────────┴─────┴────────────────┴───────────────────────┘

Dependencies

• NumPy - Provides an assortment of routines for fast operations on arrays

Optional dependencies

• Pandas - Powerful data structures for data analysis, time series, and statistics. The pandas extra installs it, pip install "werpy[pandas]", for the pandas DataFrame of summary() and summaryp().
• Polars - A DataFrame library. The polars extra installs it, pip install "werpy[polars]", for the polars DataFrame of summary() and summaryp() with output="polars".

Licensing

werpy is released under the terms of the BSD 3-Clause License. Please refer to the LICENSE file for full details.

This project uses standard scientific Python libraries: NumPy, with pandas and polars as optional dependencies. For license details, please refer to their official repositories:

• NumPy - https://github.com/numpy/numpy
• Pandas - https://github.com/pandas-dev/pandas
• Polars - https://github.com/pola-rs/polars