HS code classification and unit price anomaly detection for Pakistani knitted garment exports.
A single notebook that takes an export declaration line, predicts the HS code its description actually points to, checks the declared unit price against a benchmark built from comparable shipments, and routes the doubtful lines to a human review queue.
When an exporter files a shipping bill, they give an HS code and a unit price for each item. If the HS code is wrong, the duty refund or rebate claim may be rejected and the shipment may be held at customs. If the unit price sits far from the normal price for that product, customs may need to examine the declared value. Today both of these are checked by hand, one line at a time.
That does not scale. This file alone holds 39,112 lines, and it covers one HS chapter for one month.
This notebook builds a pre-submission checker that reads each declaration line and returns three things:
- The HS code the description points to, two alternatives, and a confidence margin.
- Whether the declared unit price sits inside the normal band for that product.
- Rule checks that catch problems without needing a model at all.
The three are combined into a single routing decision, so that a reviewer reads the 29.4% of lines carrying a disagreement, a price outlier, or a rule violation, rather than all of them.
Pakistani export declarations for HS Chapter 61 (knitted and crocheted garments), February 2024.
The source file is not included in this repository. It is commercial trade data containing real exporter and importer names and tax registration numbers, so it is not published. Every cell in the notebook is saved with its output, which means the full analysis, all metrics and all charts can be read without the data. The notebook cannot be re-executed without it.
Identifying fields are masked in the saved outputs too. Exporter name, importer name, tax registration number and shipping bill number are replaced with pseudonyms wherever a row is displayed, and the exporter volume ranking is shown as counts without names. This costs nothing: the classifier reads the item description and nothing else, and the price benchmark is built per HS code, not per company.
| Raw lines | 39,112 |
| Lines after cleaning | 38,897 |
| Period | 1 to 29 February 2024 (one month) |
| HS6 classes / HS4 classes | 95 / 17 |
| Distinct item descriptions | 22,504 |
| Exporters / importers / destinations | 2,033 / 4,309 / 103 |
The dataset is not clean, and the notebook does not pretend otherwise. Every column was checked against the data rather than trusted by its name:
- The column named
ORIGINholds the destination country. Every shipment originates in Pakistan. U/PRICEandUNIT PRICE FCare the same column stored twice, identical in 100% of rows.EXP VALis the PKR value andTotal Value FCis the USD value. The ratio between them has a median of 279.3, which is the PKR/USD rate for February 2024.- The
CURRcolumn lists Deutsche Mark, Guilder, Peseta, Italian Lira and Belgian Franc, all of which ceased to exist in 2002. Each shows an implied rate of about 279, which is the USD rate. It is a legacy country-to-currency lookup, not the invoicing currency, so it was dropped rather than imputed.
Sixteen steps, in one continuous flow.
Data source → understanding → validation → cleaning → EDA → problem definition → split strategy → model comparison → final model → error analysis → confidence threshold → price benchmark → rule checks → the checker → results → limitations.
The checker combines three independent signals:
- HS code classifier — reads the item description, returns the predicted code, two alternatives, and a confidence margin.
- Price benchmark — compares the declared unit price against a robust band built from other shipments of the same HS code, using the median and median absolute deviation on log price so that outliers do not define the band they are being measured against.
- Rule checks — conditions that are true by definition of the tariff schedule or by arithmetic, and need no model at all.
Throughout this section, "accuracy" means agreement with the HS code the exporter filed. That is the only label available. It is not a measure of correct tariff classification, for the reason set out in Accuracy is measured against declared codes below.
Model selection, all on the same leakage-safe split:
| Model | Accuracy | Macro F1 | Fit time |
|---|---|---|---|
| TF-IDF word + char + LinearSVC | 0.746 | 0.556 | 40 s |
| TF-IDF word + LogisticRegression | 0.738 | 0.573 | 104 s |
| TF-IDF word + LinearSVC | 0.737 | 0.549 | 25 s |
| TF-IDF word + SGD (log loss) | 0.724 | 0.529 | 6 s |
| Majority class baseline | 0.129 | 0.003 | 1 s |
Final model, TF-IDF word plus character n-grams with LinearSVC:
| Metric | HS6 (95 classes) | HS4 (17 classes) |
|---|---|---|
| Top-1 accuracy | 0.746 | 0.806 |
| Top-3 accuracy | 0.848 | 0.917 |
| Macro F1 | 0.556 | 0.669 |
| Majority baseline | 0.129 | 0.215 |
Confidence-based ranking, HS6. The gap between the best and second-best class score, the decision margin, is used to rank lines. Every figure below is agreement between the model and the code the exporter filed — not verified correctness. The label is the filed code, so no number in this table can say which of the two is right when they differ.
| Share of lines by margin | Model–filing agreement on that share | Agreement on the remainder |
|---|---|---|
| Top 100% | 0.746 | — |
| Top 80% | 0.838 | 0.375 |
| Top 60% | 0.930 | 0.462 |
| Top 50% | 0.964 | 0.528 |
What this establishes is narrow but real: the margin works as a ranking signal, and disagreements concentrate in the low-margin lines. It does not establish that the top 50% are correctly classified.
Price and rule checks across the full dataset:
| Check | Lines | Share |
|---|---|---|
| Price outside the robust band for its HS code | 865 | 2.22% |
| At least one rule violation | 3,838 | 9.87% |
| Lines routed to review on the test set | 2,259 | 29.4% |
Two findings the checker surfaced on its own:
- The most extreme low-price lines have a quantity of 1, a price of one or two cents, and a
description that reads
FOR VALUE ADJUSTMENT. These are accounting corrections, not shipments. Nobody flagged in advance that such records existed in the file. - 602 lines inside a chapter that covers knitted goods only describe themselves as woven. Woven garments belong in Chapter 62.
There are 38,897 lines but only 22,504 distinct descriptions, because the same text is reused across many shipments. Under a plain random split, 1,772 descriptions would appear in both train and test. The model would score well by memorising text it had already seen, and the reported accuracy would be measuring recall of the training set rather than generalisation.
Grouping the split by description drops that overlap to zero. The accuracy in this repository is lower than a random split would show, and it is the number that means something.
The label is what the exporter filed, not what the tariff schedule requires. Where a filing was wrong, that error sits inside the training labels.
This is visible in the results. The single largest confusion pair, 105 lines, has descriptions that
read only JACKET or MENS JACKET. The model predicts 6103.39, jackets and blazers. The filed code
is 6112.12, track suits of synthetic fibres. Read against the text, the model looks more consistent
than the declaration — but it is counted as an error, because the declaration is the label.
The output should therefore be read as a disagreement flag for a human reviewer, never as a ruling on the correct tariff classification.
This is a triage system. Its job is to decide which lines a person should look at, not to decide what the code is. On that job it routes 29.4% of test lines to review and leaves the remaining 70.6% alone, which is the reduction in manual reading the project is actually claiming.
This is worth stating plainly, because the two are easy to confuse.
The margin threshold (Step 11) is an evaluation device. It asks: if we ranked lines by model confidence and only trusted the top half, how often would the model and the filing agree? Answer: 96.4%, against 74.6% across all lines. That is what justifies using the margin to rank the queue.
The checker's routing rule (Step 14) is the actual policy: a line is reviewed if the predicted code differs from the filed code, or the price falls outside its band, or a rule fires. The margin is reported to the reviewer but does not enter this decision.
These do not compose. Every line the checker lets through is a line where prediction and filing already agree, so agreement on that set is 100% by construction, not 96.4%. The 96.4% figure belongs to the ranking experiment and should never be quoted as the accuracy of what the checker passes.
Two quantities matter for deployment and neither can be computed from this data:
- Flag precision. Of the 2,259 lines routed to review, how many contain a real problem? Unknown. A disagreement with the filing is not evidence that the filing is wrong.
- Miss rate. Of the 70.6% of lines the checker passes, how many carry a misclassification that both the exporter and the model share? Unknown, and structurally invisible: the model is trained on filed codes, so a systematically mis-filed product will be predicted exactly as mis-filed.
Both require a sample of lines adjudicated by someone who can read the tariff schedule. That work has not been done here. Until it is, every figure in this repository describes agreement between a model and a filing, and nothing about correctness.
Each of these was ruled out by evidence in the data, not by preference.
| Not done | Reason |
|---|---|
| Trend, seasonality, forecasting | The data covers one month, 29 days |
| Customer recency, RFM, lifetime value | Recency cannot be computed inside a single month |
| Currency exposure analysis | The currency field was shown to be an unreliable legacy lookup |
| Predicting export value from quantity and price | Value is defined as their product; such a model measures an identity, not a relationship |
| Tree ensembles and gradient boosting | Four different linear learners land within 2.2 points of each other, which points to a ceiling in the labels rather than the model |
These are the main limitations of the dataset and the method.
The labels are declared codes, not verified codes. The model learns from the codes reported by exporters. If a declaration was wrong, that error is also included in the training data. Therefore, the model should only be used to identify lines for human review, not to decide the correct tariff classification.
Only one month of data was used. The dataset covers February 2024 only. Because of this, we cannot say whether the same patterns would appear throughout the year. We also cannot make seasonal or long term trend claims.
Rare codes have limited reliability. About one third of the HS6 codes have fewer than 30 lines. This is one reason why macro F1 is much lower than overall accuracy. The model should not be strongly relied on for these rare codes.
Very short descriptions are difficult to classify. Nearly 1,900 lines contain fewer than ten characters, such as SHIRTS. These descriptions provide very little information, so the model often predicts the most common code.
The price benchmark assumes that most prices are normal. Each declared price is compared with the typical price for its HS code. If low prices are common within a code, they may become part of the normal range and will not be flagged. Therefore, this method finds unusual prices, not necessarily incorrect prices.
A price flag is not proof of a problem. There can be valid reasons for a low unit price, such as samples, promotional goods, lower quality items, or different fabric weights within the same HS code. Each flag should therefore be treated as something to review, not as a confirmed problem.
No external tariff reference was used. The project does not include the official HS nomenclature or explanatory notes. Because of this, the checker can suggest that a code may be wrong, but it cannot explain why another code may be more suitable. Adding this reference in the future could make the system more useful by giving reasons for its suggestions.
Company names were not standardized. The same company can appear under different spellings. This does not affect the model because it uses only the item description. However, company level analysis would not be reliable without first matching and standardizing company names.
The notebook is committed with all outputs saved, so it can be read start to finish on GitHub without running anything.
To re-execute it you need the source data file, which is not published here for the reason given above. With the file present in the same directory:
pip install -r requirements.txt
jupyter notebook Export_Declaration_Quality_Checker.ipynbRun the cells top to bottom. Expected runtime is roughly 4 to 7 minutes, most of it in the model comparison step.
Export_Declaration_Quality_Checker.ipynb Complete project, sixteen steps, outputs included
make_figures.py Rebuilds the three README figures
figures/ Figures used in this file
requirements.txt Dependencies
LICENSE MIT, code and analysis only
README.md This file
make_figures.py needs no data. Every value in it is transcribed from a saved output cell in the
notebook, so the figures can be regenerated from a clean checkout with python make_figures.py.
Dependencies are pandas, numpy, matplotlib, scikit-learn and openpyxl. Nothing else is required.
The licence covers the code and the written analysis. It does not cover the export declaration data, which is commercial, is not distributed here, and is not mine to license.