Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions .github/workflows/mypy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,28 +14,27 @@ jobs:
fail-fast: true
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

#----------------------------------------------
# ----- install uv -----
#----------------------------------------------
- name: Install uv
uses: astral-sh/setup-uv@v5
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
version: "0.7.3"

#----------------------------------------------
# ----- setup matchers & run mypy -----
Expand Down
12 changes: 5 additions & 7 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,19 +30,17 @@ jobs:
# check-out repo and set-up python
#----------------------------------------------
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python 3.9
uses: actions/setup-python@v5
uses: actions/checkout@v6
- name: Set up Python 3.10
uses: actions/setup-python@v6
with:
python-version: "3.9"
python-version: "3.10"

#----------------------------------------------
# ----- install uv -----
#----------------------------------------------
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
version: "0.7.3"
uses: astral-sh/setup-uv@v7

#----------------------------------------------
# install dependencies and build packages
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/ruff.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,17 @@ jobs:
fail-fast: true
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,21 @@ jobs:
# Check out repo and set up Python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v6

- name: Set up Python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

#----------------------------------------------
# Install uv
#----------------------------------------------
- name: Install uv
uses: astral-sh/setup-uv@v5
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
version: "0.7.3"

#----------------------------------------------
# Run Pytest
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
default_language_version:
python: python3.9
python: python3.10

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.0
rev: v0.9.21
hooks:
# Run the linter and fix issues ...
- id: ruff
Expand Down
47 changes: 25 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# `tab_err`

<h1 align="center">
Inject Realistic Errors Into Tables
</h1>
Expand All @@ -18,7 +19,6 @@
[![pytest](https://github.com/calgo-lab/tab_err/actions/workflows/tests.yaml/badge.svg?branch=main)](https://github.com/calgo-lab/tab_err/actions/workflows/tests.yaml)
[![Ruff](https://github.com/calgo-lab/tab_err/actions/workflows/ruff.yaml/badge.svg?branch=main)](https://github.com/calgo-lab/tab_err/actions/workflows/ruff.yaml)


`tab_err` injects realistic errors into tabular data such as database tables and DataFrames.
The library is developed and maintained by the [Cognitive Algorithms Lab](https://calgo-lab.de/) at BHT Berlin.

Expand All @@ -28,13 +28,15 @@ Researchers and data practitioners can generate errors in a controlled way, eval
## How it Works

The library's building blocks are `ErrorMechanism`s, `ErrorType`s, and `ErrorModel`s.
- An `ErrorMechanism` describes the error's distribution - that's *where* incorrect cells appear in the table. We support *erroneous at random* (EAR), *erroneous not at random* (ENAR) and *erroneous completely at random* (ECAR).
- An `ErrorType` describes *how* the value is wrong: a typo, an outlier, a category swap, and so on. Read the documentation for a [full list of supported error types](https://tab-err.readthedocs.io/latest/api/tab_err/error_type/index.html).

- An `ErrorMechanism` describes the error's distribution - that's _where_ incorrect cells appear in the table. We support _erroneous at random_ (EAR), _erroneous not at random_ (ENAR) and _erroneous completely at random_ (ECAR).
- An `ErrorType` describes _how_ the value is wrong: a typo, an outlier, a category swap, and so on. Read the documentation for a [full list of supported error types](https://tab-err.readthedocs.io/latest/api/tab_err/error_type/index.html).
- An `ErrorModel` is a set of mechanisms and types to perturb existing data with realistic errors. It is shareable as metadata.

`tab_err` is supported by a `pandas` backend.

## Examples

```python
from sklearn.datasets import load_iris

Expand Down Expand Up @@ -77,7 +79,7 @@ For a detailed guide and more examples, see our [Getting Started Notebook](https

## Where to get it

The source code is hosted on GitHub at <https://github.com/calgo-lab/tab_err>.
The source code is hosted on GitHub at <https://github.com/calgo-lab/tab_err>.
Binary installers for the latest releases are available at the Python Package Index (PyPI) <https://pypi.org/project/tab-err>.

```sh
Expand All @@ -96,25 +98,26 @@ Develop on feature branches and open pull requests when you're ready.
Make sure that your changes are tested, documented, and clearly described in the pull request.

## Citation

If you use the error model that's underlying `tab_err` for a scientific publication, we would appreciate your citation.

```
```bib
@article{10.1145/3774914,
author = {Jung, Philipp and J\"{a}ger, Sebastian and Chandler, Nicholas and Biessmann, Felix},
title = {Towards Realistic Error Models for Tabular Data},
year = {2025},
issue_date = {December 2025},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {17},
number = {4},
issn = {1936-1955},
url = {https://doi.org/10.1145/3774914},
doi = {10.1145/3774914},
journal = {J. Data and Information Quality},
month = dec,
articleno = {28},
numpages = {27},
keywords = {Tabular data, data quality, data errors, data error generation, error model, realistic error model, error type}
author = {Jung, Philipp and J\"{a}ger, Sebastian and Chandler, Nicholas and Biessmann, Felix},
title = {Towards Realistic Error Models for Tabular Data},
year = {2025},
issue_date = {December 2025},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {17},
number = {4},
issn = {1936-1955},
url = {https://doi.org/10.1145/3774914},
doi = {10.1145/3774914},
journal = {J. Data and Information Quality},
month = dec,
articleno = {28},
numpages = {27},
keywords = {Tabular data, data quality, data errors, data error generation, error model, realistic error model, error type}
}
```
```
2 changes: 1 addition & 1 deletion examples/1-Getting-Started.ipynb

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No changes here - maybe consider removing from PR

Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.20"
"version": "3.10.19"
}
},
"nbformat": 4,
Expand Down
Loading