Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: CI

on:
push:
branches:
- master
- dev
- language_support
pull_request:

jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4

- uses: astral-sh/setup-uv@v6
with:
python-version: ${{ matrix.python-version }}

- name: Sync dependencies
run: uv sync --extra classifier

- name: Smoke import
run: uv run python scripts/test_import_error.py

- name: Build generated assets
run: uv run python scripts/build.py

- name: Run test suite
run: |
uv run pytest \
quantulum3/tests/test_setup.py \
quantulum3/tests/test_hypothesis.py \
quantulum3/tests/test_no_classifier.py \
quantulum3/tests/test_parse_ranges.py \
quantulum3/_lang/en_US/tests/extract_spellout_values.py \
quantulum3/tests/test_classes.py \
quantulum3/tests/test_scripts.py \
quantulum3/tests/test_load.py

- name: Run classifier tests
run: uv run pytest quantulum3/tests/test_classifier.py -k "not test_wikipedia_pages"
48 changes: 48 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Publish

on:
release:
types: [published]
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read

steps:
- uses: actions/checkout@v4

- uses: astral-sh/setup-uv@v6
with:
python-version: "3.13"

- name: Sync dependencies
run: uv sync --extra classifier

- name: Build generated assets
run: uv run python scripts/build.py

- name: Build distributions
run: uv build

- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/*

publish:
needs: build
runs-on: ubuntu-latest
if: github.event_name == 'release'
permissions:
id-token: write

steps:
- uses: actions/download-artifact@v4
with:
name: dist
path: dist

- uses: pypa/gh-action-pypi-publish@release/v1
73 changes: 0 additions & 73 deletions .travis.yml

This file was deleted.

1 change: 0 additions & 1 deletion MANIFEST.in

This file was deleted.

24 changes: 0 additions & 24 deletions Pipfile

This file was deleted.

42 changes: 29 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# quantulum3

[![Travis master build state](https://app.travis-ci.com/nielstron/quantulum3.svg?branch=master "Travis master build state")](https://app.travis-ci.com/nielstron/quantulum3)
[![CI](https://github.com/nielstron/quantulum3/actions/workflows/ci.yml/badge.svg)](https://github.com/nielstron/quantulum3/actions/workflows/ci.yml)
[![Coverage Status](https://coveralls.io/repos/github/nielstron/quantulum3/badge.svg?branch=master)](https://coveralls.io/github/nielstron/quantulum3?branch=master)
[![PyPI version](https://badge.fury.io/py/quantulum3.svg)](https://pypi.org/project/quantulum3/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/quantulum3.svg)
Expand All @@ -26,12 +26,24 @@ the fork of [sohrabtowfighi](https://github.com/sohrabtowfighi/quantulum).
pip install quantulum3
```

Or with `uv`:

```bash
uv add quantulum3
```

To install dependencies for using or training the disambiguation classifier, use

```bash
pip install quantulum3[classifier]
```

Or with `uv`:

```bash
uv add "quantulum3[classifier]"
```

The disambiguation classifier is used when the parser find two or more units that are a match for the text.

### Usage
Expand Down Expand Up @@ -210,12 +222,18 @@ Use `quantulum3-training` on the command line, the script `quantulum3/scripts/tr
quantulum3-training --lang <language> --data <path/to/training/file.json> --output <path/to/output/file.joblib>
```

With `uv`, you can run the installed entry point without activating an environment:

```bash
uv run quantulum3-training --lang <language> --data <path/to/training/file.json> --output <path/to/output/file.joblib>
```

You can pass multiple training files in to the training command. The output is in joblib format.

To use your custom model, pass the path to the trained model file to the
parser:

```pyton
```python
parser = Parser.parse(<text>, classifier_path="path/to/model.joblib")
```

Expand Down Expand Up @@ -346,25 +364,23 @@ All fields are case sensitive.

### Contributing

`dev` build:
`dev` build:

[![Travis dev build state](https://travis-ci.com/nielstron/quantulum3.svg?branch=dev "Travis dev build state")](https://travis-ci.com/nielstron/quantulum3)
[![CI](https://github.com/nielstron/quantulum3/actions/workflows/ci.yml/badge.svg?branch=dev)](https://github.com/nielstron/quantulum3/actions/workflows/ci.yml)
[![Coverage Status](https://coveralls.io/repos/github/nielstron/quantulum3/badge.svg?branch=dev)](https://coveralls.io/github/nielstron/quantulum3?branch=dev)

If you'd like to contribute follow these steps:
1. Clone a fork of this project into your workspace
2. Run `pip install -e .` at the root of your development folder.
3. `pip install pipenv` and `pipenv shell`
4. Inside the project folder run `pipenv install --dev`
5. Make your changes
6. Run `scripts/format.sh` and `scripts/build.py` from the package root directory.
7. Test your changes with `python3 setup.py test`
(Optional, will be done automatically after pushing)
8. Create a Pull Request when having commited and pushed your changes
2. Run `uv sync --extra classifier`
3. Commit the generated `uv.lock` changes when dependencies change
4. Make your changes
5. Run `scripts/format.sh` and `uv run scripts/build.py` from the package root directory. The build step regenerates `common-words.json` and `clf.joblib`.
6. Test your changes with `uv run pytest`
7. Create a Pull Request when you have committed and pushed your changes

### Language support

[![Travis dev build state](https://travis-ci.com/nielstron/quantulum3.svg?branch=language_support "Travis dev build state")](https://travis-ci.com/nielstron/quantulum3)
[![CI](https://github.com/nielstron/quantulum3/actions/workflows/ci.yml/badge.svg?branch=language_support)](https://github.com/nielstron/quantulum3/actions/workflows/ci.yml)
[![Coverage Status](https://coveralls.io/repos/github/nielstron/quantulum3/badge.svg?branch=language_support)](https://coveralls.io/github/nielstron/quantulum3?branch=dev)

There is a branch for language support, namely `language_support`.
Expand Down
94 changes: 92 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,93 @@
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta:__legacy__"
requires = ["setuptools>=69", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "quantulum3"
version = "0.9.2"
description = "Extract quantities from unstructured text."
readme = "README.md"
license = "MIT"
authors = [
{ name = "Marco Lagi, nielstron, sohrabtowfighi, grhawk and Rodrigo Castro", email = "n.muendler@web.de" },
]
requires-python = ">=3.9"
dependencies = [
"inflect",
"num2words",
]
keywords = [
"information extraction",
"quantities",
"units",
"measurements",
"nlp",
"natural language processing",
"text mining",
"text processing",
]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"Natural Language :: English",
"Operating System :: MacOS :: MacOS X",
"Operating System :: Microsoft :: Windows",
"Operating System :: POSIX",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Scientific/Engineering",
"Topic :: Text Processing :: Linguistic",
]

[project.optional-dependencies]
classifier = [
"joblib",
"numpy",
"scikit-learn",
"scipy",
"stemming",
"wikipedia",
]

[project.scripts]
quantulum3-training = "quantulum3.scripts.train:main"

[project.urls]
Homepage = "https://github.com/nielstron/quantulum3"
Repository = "https://github.com/nielstron/quantulum3"
Issues = "https://github.com/nielstron/quantulum3/issues"
Changelog = "https://github.com/nielstron/quantulum3/blob/master/CHANGES"

[dependency-groups]
dev = [
"black",
"coverage",
"flake8",
"hypothesis",
"pylint",
"pytest",
]

[tool.setuptools]
include-package-data = true

[tool.setuptools.packages.find]
include = ["quantulum3*"]

[tool.setuptools.package-data]
quantulum3 = ["*.json", "*.joblib", "_lang/**/*.json", "_lang/**/*.txt", "_lang/**/*.joblib"]

[tool.pytest.ini_options]
testpaths = [
"quantulum3/tests",
"quantulum3/_lang/en_US/tests",
]

[tool.uv]
default-groups = ["dev"]
4 changes: 2 additions & 2 deletions quantulum3/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""quantulum3 init."""

VERSION = (0, 9, 2)
from importlib.metadata import version

__version__ = ".".join([str(i) for i in VERSION])
__version__ = version("quantulum3")
__author__ = "Marco Lagi, nielstron, sohrabtowfighi, grhawk and Rodrigo Castro"
__author_email__ = "n.muendler@web.de"
__copyright__ = "Copyright (C) 2016 Marco Lagi, nielstron,t sohrabtowfighi, "
Expand Down
Loading