diff --git a/curation/records/2024_lusaka_mobility_prediction_hackathon.md b/curation/records/2024_lusaka_mobility_prediction_hackathon.md index bb8b170c..d01d2116 100644 --- a/curation/records/2024_lusaka_mobility_prediction_hackathon.md +++ b/curation/records/2024_lusaka_mobility_prediction_hackathon.md @@ -3,10 +3,8 @@ unique_name: 2024_lusaka_mobility_prediction_hackathon name: 2024-lusaka-mobility-prediction-hackathon checked_by: - AI (UNVERIFIED) -suggestion: 'No' -decision_markers: -- Time-series (Forecasting) -- Missing source information +suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: @@ -16,10 +14,12 @@ year: '2024' domain: technology & internet required_split: - Temporal (NON-IID) -problem_type: TBD +problem_type: Regression source_links: -- https://zindi.africa/competitions/practice-challenge-lusaka/data - https://zindi.africa/competitions/2024-lusaka-mobility-prediction-hackathon/data +- https://zindi.africa/competitions/2024-lusaka-mobility-prediction-hackathon +- '# a different Yango dataset (average road speeds, Sept 2023), not this task''s data' +- https://zindi.africa/competitions/practice-challenge-lusaka/data source_row: 934 needs_review: - ai_unverified @@ -30,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: a per-trip ride-time regression (Yango's ETA), not demand forecasting; the competition page refutes the old `No` with `Time-series (Forecasting)` and `Missing source information`. +- Source: Zindi one-day hackathon "2024 Lusaka Mobility Prediction Hackathon Powered" by Yango, 26 Oct 2024 (https://zindi.africa/competitions/2024-lusaka-mobility-prediction-hackathon, read through Zindi's page API); as published: "a sampled dataset of trips ordered via the Yango App from January 2024 in Lusaka": Train.csv (15.6 MB, with the target), Test.csv (6.4 MB, no target), Lusaka_weather.csv (49 KB, joined by time), VariableDefinitions.csv. Coordinates geomasked "by at least X meters and no more than Y meters in a random direction"; "The straight-line distance is calculated based on the original points"; "excessively short trips" removed. Rows and columns not counted (the files need a login). +- Use case: "Yango uses a wide range of data in order to predict travel and arrival times of its partners' drivers and vehicles" (Description): the platform quotes a ride time when a trip is ordered; fit direct if every column is known at ordering; regime temporal (one month of trips; quoting future trips), as for `nyc_taxi_trip_duration`. +- Siblings and duplicates: the same Yango task in other cities: `2025_accra_mobility_prediction_hackathon_powered` (Jan 2025, Train.csv 7.9 MB) and `2025_maputo_mobility_prediction_hackathon_powered_by_yango` (2024 trips, 9.0 MB), plus a 2026 Yaounde edition without a record: different data, the same task and publisher, so take one city (this one has the largest train file). The road-speed hackathons (`accra_mobility_prediction_hackathon`, practice-challenge-lusaka, the cities in `mobility_prediction`) are another task. Ride-time precedents: `nyc_taxi_trip_duration` and `pkdd_15_taxi_trip_time_prediction_ii` (`Yes`), `maps_router_eta` (`Yes`, TabReD), `yassir_eta_prediction_challenge` (`TBD -> Yes`). External collections: none found. +- Papers and pages: no paper; the competition's Description, Data and Evaluation tabs; no public solution or column list found (web search 2026-10-22). +- Metric: RMSE on the ride time ("The evaluation metric for this competition is Root Mean Squared Error", Evaluation tab); a competition's leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the column list is unread: check that no column is known only after the trip (driven distance, drop-off time) (LEAK); the target's unit; only Train.csv is labelled; the second link (practice-challenge-lusaka) is a different dataset (road speeds, Sept 2023); access: login, and the competition "is only open to Zambian users residing in Zambia" (no criterion, EV-4). +- Priority: gap 0 (technology & internet, regression and temporal are not thin), use case + (Yango's ETA, named by the source), source 0 (a competition page, no codebook read), shape + (a month of trips with time and weather), risk 0. + A Zindi 'mobility prediction' hackathon for Lusaka with no comments in the record. Mobility-prediction challenges (often ride-hailing/transport demand) typically forecast trips/demand over space and time, which is excluded forecasting rather than a fixed IID/temporal-split tabular task. The record carries essentially no descriptive content, so the verdict is unverified. A human must confirm the exact target and whether it is a one-shot prediction or a multi-step forecast; if it is genuine forecasting it stays No, but if it is a fixed per-trip prediction it could be revisited. diff --git a/curation/records/2025_accra_mobility_prediction_hackathon_powered.md b/curation/records/2025_accra_mobility_prediction_hackathon_powered.md index e0a96f89..9b562a6b 100644 --- a/curation/records/2025_accra_mobility_prediction_hackathon_powered.md +++ b/curation/records/2025_accra_mobility_prediction_hackathon_powered.md @@ -3,10 +3,8 @@ unique_name: 2025_accra_mobility_prediction_hackathon_powered name: 2025-accra-mobility-prediction-hackathon-powered checked_by: - AI (UNVERIFIED) -suggestion: 'No' -decision_markers: -- Time-series (Forecasting) -- Missing source information +suggestion: TBD -> 2nd Tier +priority: 2 tags: - AI-Filled (Verify) collections: @@ -16,9 +14,10 @@ year: '2025' domain: technology & internet required_split: - Temporal (NON-IID) -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/2025-accra-mobility-prediction-hackathon-powered/data +- https://zindi.africa/competitions/2025-accra-mobility-prediction-hackathon-powered source_row: 946 needs_review: - ai_unverified @@ -29,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a per-trip ride-time regression (Yango's ETA), not forecasting; the same task as `2024_lusaka_mobility_prediction_hackathon` in another city, with a smaller train file. +- Source: Zindi one-day hackathon "2025 Accra Mobility Prediction Hackathon Powered" by Yango, 5 Jul 2025 (https://zindi.africa/competitions/2025-accra-mobility-prediction-hackathon-powered, read through Zindi's page API); as published: "a sampled dataset of trips ordered via the Yango App from January 2025 in Ghana": Train.csv (7.9 MB, with the target), Test.csv (3.3 MB), Accra_weather.csv, VariableDefinitions.csv; coordinates geomasked, straight-line distance from the original points, "excessively short trips" removed. Rows and columns not counted (login). +- Use case: "Yango uses a wide range of data in order to predict travel and arrival times" (Description): a ride time quoted at ordering; fit direct if every column is known then; regime temporal (one month of trips). +- Siblings and duplicates: `2024_lusaka_mobility_prediction_hackathon` (Jan 2024, 15.6 MB) and `2025_maputo_mobility_prediction_hackathon_powered_by_yango` are the same task and publisher on other cities' data: not copies, but one city is enough (this is the smallest of the three). Not a duplicate of `accra_mobility_prediction_hackathon` (road speeds per segment, Sept 2024), as the old note guessed. Precedents: `nyc_taxi_trip_duration`, `pkdd_15_taxi_trip_time_prediction_ii`, `maps_router_eta` (`Yes`). +- Papers and pages: no paper; the competition's tabs; no public solution found. +- Metric: RMSE ("The evaluation metric for this competition is Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: the column list is unread (post-trip columns, LEAK); the target's unit; access: login, "only open to Ghanaian users residing in Ghana" (no criterion, EV-4). +- Priority: gap 0, use case + (Yango's ETA), source 0, shape + (a month of trips with time and weather), risk - (a sibling of the Lusaka task; keep one city). + Zindi mobility-prediction hackathon for Accra (the Yango-family ride-hailing series). These challenges generally forecast transport demand/mobility over time and space, which falls under excluded time-series forecasting and is RecSys/operations-adjacent. The record has no comments or descriptive fields to confirm the precise target. A human must check whether the task is a fixed predictive label per row (possibly viable) versus a demand-over-horizon forecast (excluded). diff --git a/curation/records/2025_maputo_mobility_prediction_hackathon_powered_by_yango.md b/curation/records/2025_maputo_mobility_prediction_hackathon_powered_by_yango.md index 3664f271..91271a39 100644 --- a/curation/records/2025_maputo_mobility_prediction_hackathon_powered_by_yango.md +++ b/curation/records/2025_maputo_mobility_prediction_hackathon_powered_by_yango.md @@ -3,10 +3,8 @@ unique_name: 2025_maputo_mobility_prediction_hackathon_powered_by_yango name: 2025-maputo-mobility-prediction-hackathon-powered-by-yango checked_by: - AI (UNVERIFIED) -suggestion: 'No' -decision_markers: -- Time-series (Forecasting) -- Missing source information +suggestion: TBD -> 2nd Tier +priority: 2 tags: - AI-Filled (Verify) collections: @@ -16,9 +14,10 @@ year: '2025' domain: technology & internet required_split: - Temporal (NON-IID) -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/2025-maputo-mobility-prediction-hackathon-powered-by-yango/data +- https://zindi.africa/competitions/2025-maputo-mobility-prediction-hackathon-powered-by-yango source_row: 949 needs_review: - ai_unverified @@ -29,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a per-trip ride-time regression (Yango's ETA), not forecasting; the same task as `2024_lusaka_mobility_prediction_hackathon` in another city. +- Source: Zindi one-day hackathon "2025 Maputo Mobility Prediction Hackathon Powered" by Yango, 11 Oct 2025 (https://zindi.africa/competitions/2025-maputo-mobility-prediction-hackathon-powered-by-yango, read through Zindi's page API); as published: "a sampled dataset of trips via the Yango platform from March 2024 in Maputo" (Description), but "from April 2024 in Mozambique" (Data tab, which ends in a stray "xxx"): the month is to check. Train.csv (9.0 MB, with the target), Test.csv (3.7 MB), Maputo_weather.csv, VariableDefinitions.csv; coordinates geomasked, straight-line distance from the original points, short trips removed. Rows and columns not counted (login). +- Use case: "Yango uses a wide range of data in order to predict travel and arrival times" (Description): a ride time quoted at ordering; fit direct if every column is known then; regime temporal (one month of trips). +- Siblings and duplicates: `2024_lusaka_mobility_prediction_hackathon` (15.6 MB) and `2025_accra_mobility_prediction_hackathon_powered` are the same task and publisher on other cities' data: one city is enough. An older Maputo hackathon (maputo-mobility-prediction-hackathon, listed in `mobility_prediction`) is the road-speed task. Precedents: `nyc_taxi_trip_duration`, `pkdd_15_taxi_trip_time_prediction_ii`, `maps_router_eta` (`Yes`). +- Papers and pages: no paper; the competition's tabs; no public solution found. +- Metric: RMSE ("The evaluation metric for this competition is Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: the column list is unread (post-trip columns, LEAK); the data month (March or April 2024); access: login, "only open to Mozambique users residing in Mozambique" (no criterion, EV-4). +- Priority: gap 0, use case + (Yango's ETA), source 0 (the page contradicts itself on the month), shape + (a month of trips with time and weather), risk - (a sibling of the Lusaka task; keep one city). + Zindi mobility-prediction hackathon for Maputo, explicitly powered by Yango (a ride-hailing platform), so the task is almost certainly ride/trip demand forecasting. That is excluded forecasting and operations/RecSys-adjacent rather than a fixed tabular classification/regression task. The record contains no descriptive content. A human must verify the actual target and split regime before any reconsideration. diff --git a/curation/records/2030_vision_flood_prediction_in_malawi.md b/curation/records/2030_vision_flood_prediction_in_malawi.md index 6d7e472e..af60a705 100644 --- a/curation/records/2030_vision_flood_prediction_in_malawi.md +++ b/curation/records/2030_vision_flood_prediction_in_malawi.md @@ -4,20 +4,20 @@ name: 2030-vision-flood-prediction-in-malawi checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier -decision_markers: -- Missing source information -- Time-series (Forecasting) +priority: 2 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: environmental science & climate required_split: - '?' -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/2030-vision-flood-prediction-in-malawi/data +- https://zindi.africa/competitions/2030-vision-flood-prediction-in-malawi source_row: 915 needs_review: - ai_unverified @@ -28,4 +28,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the flooded fraction of each ~1 km cell in southern Malawi for a flood event, from elevation, land cover and weekly rainfall; a fixed regression, not a series forecast, but only one event is labelled in the public train file. +- Source: Zindi, "UNICEF Arm 2030 Vision #1: Flood Prediction in Malawi", Dec 2019 to May 2020, sponsored by Arm and UNICEF (2030Vision) (https://zindi.africa/competitions/2030-vision-flood-prediction-in-malawi, read through Zindi's page API); as published: one Train.csv (7.1 MB) of ~1 km x 1 km cells (0.01 degree, X and Y centres) with "the fraction (percentage) of that rectangle that was flooded in 2015", mean elevation, dominant land cover (MODIS LC_Type1), and weekly precipitation "for 18 weeks beginning 2 months before the flooding" for both the 2015 and 2019 events; the test is the same cells' extent for cyclone Idai, March 2019 ("target_2019"), scored on Zindi. Rows not counted. +- Use case: "a machine learning model that helps predict the location and extent of floods in southern Malawi" (Description), for humanitarian preparedness (UNICEF; Zindi blog "Zindi solutions will help Malawi prepare better for flooding"); actor and decision not named, fit approximate: the rainfall covers the flood weeks, so it estimates extent at the event rather than ahead. Regime: by event (train 2015, test 2019) with strong spatial autocorrelation; with only 2015 labelled, our split would be spatially grouped within one event, which does not mirror a next flood. +- Siblings and duplicates: none among the records (searched malawi, flood); external collections: none found. A participant's repo (kmezhoud, learn_by_example, Flood_prediction_Malawi) claims 2015 and 2019 datasets: whether the 2019 labels are public is to check. +- Papers and pages: no paper; the competition's tabs, discussions 644, 646, 1241, 1363; a Kaggle U-Net baseline ("zindi-flood-in-malawi-unet-baseline") treats the cells as a raster. +- Metric: RMSE on the fraction ("The error metric for this competition is the Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: the target is derived from satellite flood maps and the cells form a grid where a U-Net is a baseline: judge SEL-12 on what wins (the features are tabular); the 2019 labels; the split (one event, spatial blocks); many cells at 0 (a hurdle shape, per a moderator). +- Markers: the old `Missing source information` is refuted (Zindi, UNICEF and Arm, a documented page); `Time-series (Forecasting)` dropped: the target is an event's extent per cell, not future values of a series, though Zindi tags it "Forecast". +- Priority: gap + (a spatially grouped regime is thin), use case 0 (approximate, actor unnamed), source 0, shape + (thousands of cells, 20+ features, spatial structure), risk - (one labelled event, raster structure). + A Zindi flood-prediction challenge for Malawi with no descriptive comments. Flood prediction can be either a spatio-temporal forecast (excluded) or a fixed per-location/per-event risk classification on tabular hydrological/geospatial features (potentially viable). Without details the modality (it may rely on satellite/remote-sensing rasters rather than tabular features) and the target are unknown. A human must inspect whether the data is genuinely tabular and whether the task is a fixed prediction vs. a time-horizon forecast. diff --git a/curation/records/absa_corporate_client_activity_forecasting_challenge.md b/curation/records/absa_corporate_client_activity_forecasting_challenge.md index 5646b578..f9322110 100644 --- a/curation/records/absa_corporate_client_activity_forecasting_challenge.md +++ b/curation/records/absa_corporate_client_activity_forecasting_challenge.md @@ -28,6 +28,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): the source asks for a forecast of each corporate client's activity series, the 6-hour windows of the day after seven weeks of raw event logs; no fixed per-row task table is published. +- Source: Zindi, "Absa Corporate Client Activity Forecasting Challenge", 1-27 Nov 2022, Absa CIB (https://zindi.africa/competitions/absa-corporate-client-activity-forecasting-challenge, read through Zindi's page API); as published: Train.csv (58 MB) of events (`eventdatetime`, `event`, `userid`, `dayofweek`, `useremaildomain`, `userrole`, `companyprofileid`, `country`) for "4 200 customers, across five countries over a period of seven weeks"; Test.csv (0.7 MB) lists user x 6-hour slots of one day, e.g. `User2504_Wed_3_6`. +- Deciding words: subtitle "Can you forecast if a corporate banking client will be active in the following week?"; Data tab "The training set is 7 weeks of data, and the last day of the final week will be the test set, where the target is whether a client performed event 14 during that day at 6-hour intervals"; Zindi's problem type "Forecast". +- Use case: Absa CIB anticipating platform usage ("help Absa anticipate their clients' needs, allowing for better planning, delivery and customer service"); the prediction is the next values of each client's event series over a one-day horizon. +- Why not SEL-11: the label is built from the same event series the features come from, one horizon day is shared by all clients, and the older note records that the winning solution was an LSTM; a per-client churn-like reframing would be our task, not the source's (SEL-8). +- Siblings and duplicates: `absa_customer_income_prediction_challenge` is another dataset; none else. +- Metric: F1 on hard labels ("The error metric for this competition is the F1 score", Evaluation tab), MET-4. +- Reject check (AI, 2026-10-22): stands. SEL-10 (Must). Data tab: "The training set is 7 weeks of data, and the last day of the final week will be the test set, where the target is whether a client performed event 14 during that day at 6-hour intervals". Zindi type_of_problem ["Forecast"]. The target event is in the same raw event log as the features. All clients share one horizon day, no row table is published, and the older note says the winner used an LSTM. A TP-10 per-client reframing would be our own task (SEL-8). https://zindi.africa/competitions/absa-corporate-client-activity-forecasting-challenge + Predict whether a corporate client performs a specific action within a 6-hour window across a day, using seven weeks of prior event history. This is a sequential/temporal forecasting task over future windows (the winning solution used an LSTM), which is excluded time-series forecasting rather than a fixed IID tabular task. The temporal tag and the 'forecasting' framing both reinforce this. A human could double-check whether an aggregated per-client tabular reformulation exists, but as posed it should be rejected. --- diff --git a/curation/records/absa_customer_income_prediction_challenge.md b/curation/records/absa_customer_income_prediction_challenge.md index 523ec857..bbe5cdca 100644 --- a/curation/records/absa_customer_income_prediction_challenge.md +++ b/curation/records/absa_customer_income_prediction_challenge.md @@ -4,6 +4,7 @@ name: absa-customer-income-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 decision_markers: - Needs extensive data wrangling tags: @@ -37,6 +38,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: a bank predicting a retail customer's declared net income from 14 months of transactions; a real per-customer regression, held back by the income-band leak, a heavy join and access. +- Source: Zindi, "Absa Customer Income Prediction Challenge", 29 Nov 2022 to 26 Feb 2023, Absa (https://zindi.africa/competitions/absa-customer-income-prediction-challenge, read through Zindi's page API); as published: "5144 customers and 46926 unique accounts", transactions "from the start of July 2021 to the end of August 2022" (transactions.csv, 448 MB), Train.csv with "3600 customers along with their declared net income", Test.csv with 1,544 unlabelled customers, customer.csv (demographics), income_group.csv, employment_status.csv. +- Use case: "This solution will enable Absa to anticipate their customers' needs, and allow them to offer optimal solutions to their customers" (Description): the bank estimates income to tailor offers; fit direct for an income estimate at the declaration date; regime IID over customers (one declaration each). +- Siblings and duplicates: `absa_corporate_client_activity_forecasting_challenge` is another dataset (corporate client events). Other income records (`income_prediction_challenge_for_azubian`, `predictive_insights_youth_income_prediction_challenge`, `adult`) are other populations. External collections: none found. +- Papers and pages: no paper; discussions 14806, 15094, 15312, 16527 and the solution notebook in `source_links` (the 2026-10-14 assessment below has the details). +- Metric: RMSE on the declared income ("The error metric for this competition is the Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: `INCOME_GROUP_CODE` is a band of the same income: drop it (LEAK); `DATE_LAST_UPDATED` falls after `RECORD_DATE`; aggregate transactions up to each `RECORD_DATE` (LEAK, temporal); the target is self-declared (noise); only the 3,600 train customers are labelled; access: login, a South-Africa-only competition (no criterion, EV-4). +- Priority: gap 0 (finance, regression, IID are not thin), use case + (Absa names its use), source 0 (a competition page), shape + (3,600 customers, transactions plus demographics), risk - (the band leak and a 448 MB join). + Predict a bank customer's income from their transaction history, tagged 'New IID' by curators. This is a genuine real-world tabular regression task in finance with a clear, meaningful target. The main caveat is that transaction-level data must be aggregated to one row per customer, so it requires non-trivial feature engineering and the final row count after wrangling is unknown. A human must verify the post-aggregation table size, that the target is true income (not leakage from balance), and the absence of ethical issues around financial profiling. **Assessment (AI, 2026-10-14):** recommend keeping `TBD -> Yes` with `Needs extensive data wrangling`, on two conditions: `INCOME_GROUP_CODE` is dropped, and someone can download the data from Zindi. The human has the call. diff --git a/curation/records/accra_mobility_prediction_hackathon.md b/curation/records/accra_mobility_prediction_hackathon.md index f60c2f23..5cad80e1 100644 --- a/curation/records/accra_mobility_prediction_hackathon.md +++ b/curation/records/accra_mobility_prediction_hackathon.md @@ -3,21 +3,23 @@ unique_name: accra_mobility_prediction_hackathon name: accra-mobility-prediction-hackathon checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Time-series (Forecasting) -- Missing source information tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2024' domain: technology & internet required_split: - Temporal (NON-IID) -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/accra-mobility-prediction-hackathon +- https://zindi.africa/competitions/accra-mobility-prediction-hackathon/data source_row: 937 needs_review: - ai_unverified @@ -28,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: average road-segment speeds per weekday time slot from Yango's track logs; a spatio-temporal speed panel, so forecasting-like (SEL-11, TP-10 decide), but no clear SEL-10 failure from the page. +- Source: Zindi one-day hackathon "Accra Mobility Prediction Hackathon Powered" by Yango, 23 Nov 2024 (https://zindi.africa/competitions/accra-mobility-prediction-hackathon, read through Zindi's page API); as published: "aggregated statistics on the numbers of cars and speed of travel along Accra's main highways" from "track logs of transportation service providers (Yango partners) in September 2024"; two tables: statistics per time interval and `persistent_id` (a road section), and a road graph (geometry, name); Train.csv 37.1 MB, Test.csv 17.5 MB, Graph.csv 0.9 MB. A test ID reads `5405080210124813414_X_second_weekday_X_evening_rush_hour`: a road section x weekday x time slot. +- Use case: "Knowing the average traffic speeds helps in finding the quickest and most reliable routes. It also allows partners' drivers and Yango to provide accurate estimated time of arrival (ETA)" (Description); fit approximate: a router has measured speeds from the logs, so the task fills in sections and slots; regime unclear (grouped by road section, or a panel over slots; how Zindi chose the test rows is not stated). +- Siblings and duplicates: not a duplicate of `2025_accra_mobility_prediction_hackathon_powered` (per-trip ride times), as the old note guessed. The same road-speed format: practice-challenge-lusaka (Sept 2023, labelled "Forecast" on Zindi) and the Lusaka 2023, Luanda, Maputo and Kinshasa hackathons in `mobility_prediction` (`TBD -> 2nd Tier`): keep at most one of the family. +- Papers and pages: no paper; the competition's tabs; no public solution found. +- Metric: RMSE ("The evaluation metric for this competition is Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: forecasting-like panel: a model must beat each section's own speed profile (TP-10) and the speed counts of the same slot may give the target away (LEAK); the join to Graph.csv; the target's time grain (15-minute IDs in the description, named periods in the test IDs); access: login, open only to residents of Ghana (no criterion, EV-4). +- Priority: gap 0, use case 0 (approximate), source 0, shape + (a large panel with a road graph), risk - (forecasting-like panel, join). + Another Zindi mobility-prediction hackathon for Accra with no descriptive content in the record (likely a duplicate/earlier instance of the 2025 Accra Yango challenge). Mobility-prediction tasks generally forecast transport demand over space and time, which is excluded forecasting. The record gives nothing to confirm the target. A human should check for duplication with the 2025 Accra entry and verify the exact task; as it stands it is rejected. diff --git a/curation/records/adult.md b/curation/records/adult.md index 3abb9de6..352200eb 100644 --- a/curation/records/adult.md +++ b/curation/records/adult.md @@ -33,6 +33,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line, TabArena Reject) stands on SEL-3 and USE grounds; one counter-argument on a marker: `Duplicate` does not fit. +- Source: UCI Adult (doi 10.24432/C5XW20), donated in 1996 by Ronny Kohavi and Barry Becker, extracted by Becker from the 1994 Census database; 48,842 rows x 14 features (train 32,561, test 16,281), the target income above 50K dollars; first used in Kohavi (KDD 1996), see Reference. +- Use case: none fits, as the CC says: no actor decides on whether income exceeds 50K, and the data was not collected for that decision (SEL-3; `No Good Target / Scientific Discovery`). `Outdated`: 1994 data; Ding et al. (NeurIPS 2021, "Retiring Adult") rebuilt the task on American Community Survey data (Folktables, ACSIncome), for which no record exists. +- Siblings and duplicates: `census_income` (KDD 1994-95 CPS, 300k rows, `TBD -> 2nd Tier`, AI) is another extract of the same survey programme, not a copy of Adult, and Adult is the first publication; `income_prediction_challenge_for_azubian` and `predictive_insights_youth_income_prediction_challenge` are other populations. External collections: TabSTAR, OpenML 1590. +- Metric: Kohavi 1996 reports accuracy (error rate) only, a habit (MET-6), so the fallback had it shipped. +- Ethics: race and sex are features (noted by the curator); no objection from subjects or creators found (SEL-1 not triggered on the source's words). +- **Counter-argument (AI):** `Duplicate` (PROV-4 to PROV-7) needs a recorded original that Adult copies; none exists, and the OpenML link is Adult itself. Suggest dropping the marker, or naming the dataset it duplicates; the verdict does not depend on it. The curator has the call. + The classic 1996 UCI/Census 'Adult' income (>50K) dataset, already tagged 'TabArena Reject' and '2nd Tier / Scientific Discovery'. It is hugely overused/duplicated across benchmarks, derived from a 1994 census extract, and curators note the >50K target was never a real operational task (ethical concerns around race as a feature). While it is representative tabular data, it is not a meaningful real predictive task and is explicitly a prior reject. No additional verification is needed to keep it out. --- diff --git a/curation/records/africa_soil_profiles_database.md b/curation/records/africa_soil_profiles_database.md index 144f68d4..89b6cac2 100644 --- a/curation/records/africa_soil_profiles_database.md +++ b/curation/records/africa_soil_profiles_database.md @@ -4,6 +4,7 @@ name: Africa Soil Profiles Database (AfSP v1.2, ISRIC) checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 3 decision_markers: - Needs extensive data wrangling tags: @@ -32,6 +33,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 3: measured legacy soil profiles for Sub-Saharan Africa from ISRIC, documented and large, but with no target the source defines and contained in WoSIS. +- Source: ISRIC - World Soil Information, Africa Soil Profiles Database v1.2, November 2014, for the AfSIS project (Leenaars, van Oostrum and Ruiperez Gonzalez, ISRIC Report 2014/01); as published (ISRIC metadata record, read 2026-10-22): "18,532 unique soil profiles", "Soil analytical data are available for 15,564 profiles of which 14,197 are georeferenced", "profile site and layer attribute data", values "standardized according to SOTER conventions", reliability "subjectively rated, at the profile level, 1-4"; a 34.7 MB zip (AF-AfSP1.2.zip, served, not opened): profile and layer tables to join. +- Use case: no task in the source: it is a compilation for digital soil mapping. Candidate tasks: properties at a location from covariates (needs a remote-sensing join, not shipped), or a pedotransfer function (a costly property, such as CEC or water retention, from cheap ones, such as texture and organic carbon), which soil surveys use; fit approximate either way; regime grouped by profile (layers of one profile) or spatial blocks. +- Siblings and duplicates: `wosis_soil_profiles` (`TBD -> 2nd Tier`) ingests AfSP: keep one; `open_soil_data` (iSDA, mostly model predictions), `africa_soil_property_prediction_challenge`, `open_soil_spectral_library`, `lucas_topsoil` are the same family. External collections: none found. +- Papers and pages: ISRIC Report 2014/01; the AfSIS and iSDA soil maps trained on it. +- Metric: none found (the ISRIC metadata record and project page define no task), so the fallback (rmse). +- Risks for /add-dataset: the target and the task are ours to define (SEL-3, USE); joins of site and layer tables; methods harmonised after the fact across decades of surveys; many layers per profile (grouped split); licence CC BY-NC 3.0 noted (no criterion, EV-4). +- Priority: gap + (a grouped regime is thin), use case 0 (none in the source), source + (an institution's report and metadata), shape + (15,564 analysed profiles, many properties, grouped structure), risk - (no defined target, joins, overlap with WoSIS). + ISRIC compilation of legacy soil profiles for Sub-Saharan Africa, made for the AfSIS project (Leenaars, van Oostrum & Ruiperez Gonzalez, ISRIC Report 2014/01). Metadata record: "18,532 unique soil profiles", "soil analytical data are available for 15,564 profiles of which 14,197 are georeferenced", 37 countries, published 2014-11-03. Licence per the same record: "Attribution-NonCommercial 3.0 International (CC BY-NC 3.0)"; noted, not a factor at this stage. Measured laboratory values (the point where `open_soil_data` fails), but a profile/layer database from many surveys spanning decades, with no covariates: a digital-soil-mapping task needs the remote-sensing join and a spatially blocked split. WoSIS (`wosis_soil_profiles`) ingests AfSP, so the two are not independent; decide which one to keep. Data not inspected. Drafted 2026-09-23 as a follow-up to `open_soil_data` (iSDA), whose 49k rows are 96% spectroscopy-model predictions; this is one of the measured-data parents of that family (`open_soil_spectral_library`, `africa_soil_property_prediction_challenge`, `africa_soil_profiles_database`, `lucas_topsoil`, `wosis_soil_profiles`). diff --git a/curation/records/africa_soil_property_prediction_challenge.md b/curation/records/africa_soil_property_prediction_challenge.md index 871c4bd8..bfc4a9a2 100644 --- a/curation/records/africa_soil_property_prediction_challenge.md +++ b/curation/records/africa_soil_property_prediction_challenge.md @@ -4,14 +4,13 @@ name: Africa Soil Property Prediction Challenge (AfSIS, Kaggle 2014) checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Wrong Domain / Source Modality -- Too Small tags: - AI-Filled (Verify) - Many features - Multi-target -- Tiny Data collections: - New (BeyondArena) original_source: Kaggle @@ -34,6 +33,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a documented AfSIS spectroscopy task with a clear lab use case, but the inputs are raw mid-infrared spectral bins (SEL-15, weighed against) and about 1,158 samples. +- Source: Kaggle competition "Africa Soil Property Prediction Challenge" (afsis-soil-properties), opened 2014-08-27, closed 2014-10-21, 1,231 teams, research category; data from the Africa Soil Information Service (AfSIS) Phase I, 2009-2013, "Georeferenced samples were collected from 19 countries in Sub-Saharan African" (AWS registry page, managed by QED, ODbL). As published: train.zip 13.3 MB and test.zip 8.4 MB (Kaggle file list, not opened); 1,158 training and 727 test samples per the 2nd-place README (CharlyBi/Soil-Prediction); five targets (Mehlich-3 Ca, P, pH, SOC, Sand) from "diffuse reflectance infrared spectroscopy measurements" plus a Topsoil/Subsoil indicator and a few covariates (column count not verified). +- Use case: a soil laboratory replaces costly wet chemistry by a spectral prediction: "The two types of data can be paired to form a training dataset for machine learning", so "certain soil properties can be predicted through cheaper spectral methods" (AWS registry); fit direct for the lab, but the task is chemometrics on spectra; regime grouped: topsoil and subsoil of one location and samples clustered in sentinel sites, while the site id is not in the Kaggle table (only in the AfSIS georeferences on S3). +- Siblings and duplicates: same family as `open_soil_spectral_library` (`TBD -> 2nd Tier`, `Wrong Domain / Source Modality`; OSSL ingests AfSIS spectra, so the two overlap), `africa_soil_profiles_database` (`TBD -> 2nd Tier`), `lucas_topsoil` (`TBD -> Yes`), `open_soil_data` (`No`, model predictions, Andrej and Lennart); Zindi `amini_soil_prediction_challenge` is related soil work, not checked. External collections: none found. +- Papers and pages: Kaggle competition and discussions (not read); 2nd-place solution README; Stanford CS229 2014 project (Akinola and Dowd) on the same five properties (PDF not reachable). +- Metric: "Mean Columnwise Root Mean Squared Error" over the five targets (Kaggle API `evaluation_metric`); a leaderboard metric (MET-4); inputs none. A single-target definition would score the RMSE of that target, the column of the MCRMSE. +- Risks for /add-dataset: spectral modality (SEL-15, "Raw spectral bins" weigh against; follow the precedent of `open_soil_spectral_library`); five targets (Multi-target: pick one or keep the MCRMSE); the grouping key needs a join to the AfSIS georeferences (SPLIT); the old `Too Small` marker is dropped: about 1,158 regression samples is no `Too Small` under TP-5 without a measurement. +- Priority: gap 0 (environmental science and regression not thin; the grouped key is not in the table), use case + (AfSIS states the spectral-for-wet-chemistry use), source + (AfSIS/QED and Kaggle document it), shape 0 (about 1,158 rows of one feature type), risk - (spectral modality, grouping join, five targets). + Kaggle competition (2014) on AfSIS Phase I sentinel-site samples: predict five soil properties (Mehlich-3 Ca and P, pH, soil organic carbon, sand) from mid-infrared absorbance spectra plus a handful of remote-sensing covariates; metric MCRMSE. Sizes from the 2nd-place repo (CharlyBi/Soil-Prediction README): training 1,158, test 727; that solution downsampled the spectra by 8 to 391 features and ensembled small neural nets, noting "the training set and test set are relatively small in size". Exact spectral column count and covariate list not verified here (Kaggle page not fetchable). The small, famous version of the OSSL question: same modality call as `open_soil_spectral_library`, and probably too small on its own. The samples come from clustered LDSF sentinel sites, so a random split leaks spatial structure; the site id is not in the Kaggle table (only in the AWS `s3://afsis/` georeferences), so the appropriate grouped split needs a join. Kaggle discussions not read. Data not inspected. diff --git a/curation/records/african_snake_antivenom_binding_challenge.md b/curation/records/african_snake_antivenom_binding_challenge.md index 9415a4ff..548e4b97 100644 --- a/curation/records/african_snake_antivenom_binding_challenge.md +++ b/curation/records/african_snake_antivenom_binding_challenge.md @@ -4,9 +4,9 @@ name: African Snake Antivenom Binding Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- No Good Target / Scientific Discovery -- Needs extensive data wrangling +- Wrong Domain / Source Modality tags: - AI-Filled (Verify) collections: @@ -29,6 +29,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the proposed fast reject (SEL-12) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Wrong Domain / Source Modality`, SEL-12): the input is a 16-mer amino-acid string per row, a protein-sequence task where sequence models are the tool; the remaining columns (toxin id, positions, antivenom, taxonomy) only index the sequence. +- Source: Zindi "UmojaHack Africa 2022: African Snake Antivenom Binding Challenge (ADVANCED)", 2022-03-19 to 2022-03-20, InstaDeep, 252 participants (Zindi API); data from a high-density peptide microarray study of eight African antivenoms (PLOS NTD, doi 10.1371/journal.pntd.0008366): "16-mer peptides ... derived from toxin protein sequences representing 40 snake species and 481 proteins"; Signal "normalised and ... represented as Z-score". Columns: ID, Toxin_UniprotID, Position_start, Position_end, Antivenom, Toxin_K_mer, Signal (target), Genus, Species, ProteinFam, ProteinSubFam, ProteinSubSubFam; Zindi `data_type` "Unstructured". +- Use case: "predict what other toxins a given antivenom could be effective against" (research screening); real, but the information is the sequence. +- Siblings and duplicates: same pattern as `instadeep_kinase_classification_challenge` (`No`, `NLP (Text)`, `Wrong Domain / Source Modality`, AI unverified) and `instadeep_enzyme_classification_challenge`; no duplicate. +- Metric: Root Mean Squared Error on Signal (Zindi Evaluation page). +- The old markers `No Good Target / Scientific Discovery` and `Needs extensive data wrangling` are not the reason: the target is a measured signal and the table ships as one file. +- Reject check (AI, 2026-10-22): does not stand. SEL-12 is judged per dataset and needs modality-specific models to be clearly better. The input is a fixed-length 16-mer plus the antivenom name, so it vectorizes to 16 positional categoricals. The page shows only that sequence NNs were used, not that they are clearly better. The 2nd-place post: LSTM on residue embeddings, "I think the baseline was not bad after all", 1D CNNs "didn't improve the performance", ProtBERT too slow and not used. No tabular baseline is reported. Zindi tags it Unstructured/NLP. https://zindi.africa/competitions/umojahack-africa-2022-advanced-challenge/discussions/11087 +- For /add-dataset: TBD -> 2nd Tier, priority 1. The add stage must: (1) probe a GBM on the 16 positional residues plus antivenom (and the toxin taxonomy) against a small CNN/LSTM on the same rows. If the sequence model is clearly better, the verdict is No with SEL-12. (2) Use a split grouped by Toxin_UniprotID. Overlapping tiled k-mers share 15 of 16 residues, and the 2nd-place author found that a grouped CV lowered the score. (3) Check consistency with instadeep_kinase/enzyme (No, NLP), which take full-length sequences, unlike this one. + Predict how well eight commercial antivenoms bind/neutralize African snake toxins (UmojaHack 2022 advanced challenge). This is a scientific binding-affinity prediction that likely relies on protein/toxin sequence or molecular descriptors rather than naturally tabular features, and it is a scientific-discovery flavored task with potentially few distinct toxins. Both the modality (sequence-derived) and very small sample concerns make it questionable for representative tabular ML. A human must inspect whether features are genuinely tabular, the number of rows, and whether the multi-antivenom target is one regression task or a multi-output problem. --- diff --git a/curation/records/agile_bridge_beginners.md b/curation/records/agile_bridge_beginners.md index d6bbe3a5..a43061bd 100644 --- a/curation/records/agile_bridge_beginners.md +++ b/curation/records/agile_bridge_beginners.md @@ -3,9 +3,9 @@ unique_name: agile_bridge_beginners name: agile-bridge-beginners checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -26,4 +26,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a private one-day re-hosting of Zindi's "Financial Inclusion in Africa" (2019) for Agile Bridge users, with the parent's data and metric. +- Source: Zindi hackathon `agile-bridge-beginners`, titled "Financial Inclusion in Africa", 2023-02-03, "a private hackathon only open to Agile Bridge users"; the Zindi API names its parent `indabax-togo-2022`; first publication of the data: Zindi "Financial Inclusion in Africa" (https://zindi.africa/competitions/financial-inclusion-in-africa, opened 2019-07-29, 10,679 participants). As published: Train.csv, Test.csv, VariableDefinitions.csv; "approximately 33,600 individuals" from the FinScope and FinAccess surveys of Kenya 2018, Rwanda 2016, Tanzania 2017 and Uganda 2018; target `bank_account` (Yes = 1, No = 0). +- Use case: the parent's (predict who has or uses a bank account); nothing new here. +- Siblings and duplicates: `financial_inclusion_in_africa_for_botswana` (`TBD -> Yes`, `Duplicate`) carries the same data and lists eight more re-hostings; no record holds the 2019 parent itself under its own slug. The Data Description and Evaluation text of this page match the 2019 parent word for word. +- Metric: Mean Absolute Error on the 0/1 label (Zindi Evaluation page), as in the parent. +- Recommend: add https://zindi.africa/competitions/agile-bridge-beginners to the canonical Financial Inclusion record's `source_links` and settle which record is canonical. +- Reject check (AI, 2026-10-22): stands. PROV-4/PROV-5. The Zindi API names the parent indabax-togo-2022, titled "Financial Inclusion in Africa". The Data Description is 94% identical to the 2019 original (https://zindi.africa/competitions/financial-inclusion-in-africa): "approximately 33,600 individuals ... Finscope surveys ranging from 2016 to 2018". Same files and the same MAE metric. The original is kept: its link is in financial_inclusion_in_africa_for_botswana (TBD -> Yes). https://zindi.africa/competitions/agile-bridge-beginners + A Zindi 'agile-bridge-beginners' competition with no comments, description, target, or domain in the record. There is essentially no information to assess scope, modality, or task type, and the 'beginners' framing suggests it may be a tutorial/toy dataset. Per the guidelines, insufficient info alone should not force a hard reject. A human must open the competition page to determine the actual target, modality, size, and whether the data is real or synthetic before any decision. diff --git a/curation/records/agribora_commodity_price_forecasting_challenge.md b/curation/records/agribora_commodity_price_forecasting_challenge.md index 7c7a3620..89e66d2a 100644 --- a/curation/records/agribora_commodity_price_forecasting_challenge.md +++ b/curation/records/agribora_commodity_price_forecasting_challenge.md @@ -5,8 +5,8 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Needs extensive data wrangling - Time-series (Forecasting) +- Needs extensive data wrangling tags: - Non-IID (Temporal) - AI-Filled (Verify) @@ -30,6 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): forecasting the weekly maize price of five Kenyan counties two weeks ahead, on a rolling leaderboard. +- Source: Zindi competition "agriBORA Commodity Price Forecasting Challenge", 2025-11-14 to 2025-12-28, 1,272 participants (Zindi API, read 2026-10-22), hosts agriBORA and Digital Africa. As published: agriBORA_maize_prices.csv ("weekly white maize prices from 2023/10 - 2025/10"), KAMIS maize prices 2021-2025/07 (raw and cleaned), the weeks 46-51 and final-week truth files. +- Use case: farmers time their sales from agriBORA's warehouses: a price forecast over a horizon. +- Metric: "MAE (50%)" and "RMSE (50%)" averaged (Zindi Evaluation page). +- Correction: the earlier AI note described a rainfall forecast; the source is a maize price forecast. The verdict is the same. +- Reject check (AI, 2026-10-22): stands. SEL-10 (Must) names a price over a horizon. Source: "predict average weekly prices of maize in the counties of Kiambu, Kirinyaga, Mombasa, Nairobi and Uasin-Gishu. At each prediction step, your model should generate forecasts for two consecutive weeks", scored on a rolling leaderboard. The use case (farmers timing sales) is itself a price forecast. https://zindi.africa/competitions/agribora-commodity-price-forecasting-challenge + Despite the 'commodity price' name, the record states the actual task is to forecast rainfall over the next 12/24 hours, generating two-week forecasts across a six-week horizon (Nov 2025-Jan 2026). That is explicit multi-step time-series forecasting, which is excluded, and the curators already flagged it as needing extensive wrangling with a temporal tag. The 50% MA / 50% RMSE metric confirms a forecasting setup. No further verification is needed to reject. --- diff --git a/curation/records/ai4d_lab_tanzania_tourism_classification_challenge.md b/curation/records/ai4d_lab_tanzania_tourism_classification_challenge.md index 06a629a8..fb1deb2f 100644 --- a/curation/records/ai4d_lab_tanzania_tourism_classification_challenge.md +++ b/curation/records/ai4d_lab_tanzania_tourism_classification_challenge.md @@ -4,6 +4,7 @@ name: ai4d-lab-tanzania-tourism-classification-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 4 tags: - Non-IID (Temporal) - AI-Filled (Verify) @@ -29,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: a 24,675-row NBS tourist-expenditure survey classified into six cost bins, with a stated actor and a log-loss leaderboard. +- Source: Zindi competition "AI4D Africa's Anglophone Research Lab Tanzania Tourism Classification Challenge", 2022-06-01 to 2022-07-01, 522 participants (Zindi API, read 2026-10-22); "The dataset describes 24,675 rows of up-to-date information on tourist expenditure collected by the National Bureau of Statistics (NBS) in Tanzania", from an exit survey at seven departure points. As published: Train.csv, Test.csv ("6,169 rows"), VariableDefinitions.csv; target six classes (Low, Lower, Normal, High, Higher, Highest Cost). An earlier cut of the same NBS survey was the Pycon Tanzania hackathon "Tanzania Tourism Prediction Challenge" (2020-12-12, 6,476 rows, regression on `total_cost`, MAE), the second link. +- Use case: "The model can be used by different tour operators and the Tanzania Tourism Board to automatically help tourists across the world estimate their expenditure before visiting Tanzania" (Zindi description); fit approximate: the survey is answered at departure, so trip-plan columns (party size, nights, package items) are known before the trip, while any spending or in-trip answers are not; regime IID (one survey cross-section, no date column known), so the `Non-IID (Temporal)` tag needs a check against the columns. +- Siblings and duplicates: `mmu_2022` (`TBD -> Yes`) is the same competition re-hosted on 2022-11-25 ("Tanzania Tourism Classification Challenge", same 24,675 rows, same log loss): a duplicate of this record (PROV-4); the 2020 regression cut has no record. External collections: none found. +- Papers and pages: the Zindi pages; NBS International Visitors' Exit Survey reports (not read). +- Metric: log loss over the six classes (Zindi Evaluation page: "The evaluation metric for this challenge is the Log Loss"); a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: columns answered after the trip (LEAK); the bins come from `total_cost` (whether the cut points are stated, and whether the regression framing is the better task); the 2020 6,476-row cut overlaps (PROV-6); the regime tag versus `Random (IID)`. +- Priority: gap 0 (business, multiclass and IID are not thin), use case + (tour operators and the Tourism Board named), source + (NBS survey, variable definitions), shape + (24,675 rows, mixed survey columns), risk 0. + Predict the range (bins) of expenditure a tourist spends in Tanzania; the record specifies using the 2022 multiclass version with ~24,675 rows. This is a genuine real-world tabular classification task with a clear, meaningful target and adequate size. The record carries a Temporal tag, so the split regime needs confirmation (it may simply be the year of collection rather than a true temporal dependency, in which case an IID split fits). A human must verify the row count, the number/definition of expenditure bins, and whether a temporal or random split is appropriate. --- diff --git a/curation/records/ai_babcock_competition.md b/curation/records/ai_babcock_competition.md index dc0d5eb1..838da114 100644 --- a/curation/records/ai_babcock_competition.md +++ b/curation/records/ai_babcock_competition.md @@ -3,9 +3,9 @@ unique_name: ai_babcock_competition name: ai-babcock-competition checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -26,4 +26,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-hosting of the Zimnat life-assurance lapse data first published in "The Zimnat Insurance Assurance Challenge" (Zindi, 2020), recorded as `the_zimnat_insurance_assurance_challenge`. +- Source: Zindi hackathon "AI+ Babcock Competition", 2022-05-14 to 2022-05-21, 5 participants, parent `umojahack-2022-intermediate-practice-challenge` (Zindi API); first publication: https://zindi.africa/competitions/the-zimnat-insurance-assurance-challenge (#ZindiWeekendz, 2020-05-22). As published (both pages): "The data describes 51,685 life assurance policies", files client_data.csv, policy_data.csv, payment_history.csv, train.csv; predict which policies "will lapse in 2020". +- Siblings and duplicates: `the_zimnat_insurance_assurance_challenge` (`TBD -> Yes`) is the original; the Data Description of this page is the 2020 text word for word; `zimnat_insurance_recommendation_challenge` and `zimnat_loan_default_predictio` are other Zimnat data. +- Metric: log loss (Zindi Evaluation page), the same as the original. +- Recommend: add https://zindi.africa/competitions/ai-babcock-competition to the original's `source_links`. +- Reject check (AI, 2026-10-22): stands. PROV-4. The Data Description and Evaluation pages match the 2020 Zimnat challenge character for character (ratio 1.0): "The data describes 51,685 life assurance policies ... will lapse in 2020". Same six files, log loss. The original the_zimnat_insurance_assurance_challenge is kept (TBD -> Yes). https://zindi.africa/competitions/ai-babcock-competition vs https://zindi.africa/competitions/the-zimnat-insurance-assurance-challenge + A Zindi 'ai-babcock-competition' with no comments, target, domain, or any descriptive metadata in the record. There is no basis to judge the task type, modality, size, or data provenance, and university-branded hackathons are frequently toy/educational datasets. The guidelines say not to reject solely for missing info. A human must open the source page to establish the target, modality, size, and whether the data is real before deciding. diff --git a/curation/records/ai_invasion_hackathon_2023.md b/curation/records/ai_invasion_hackathon_2023.md index aeb48d0d..99fd1220 100644 --- a/curation/records/ai_invasion_hackathon_2023.md +++ b/curation/records/ai_invasion_hackathon_2023.md @@ -3,9 +3,9 @@ unique_name: ai_invasion_hackathon_2023 name: ai-invasion-hackathon-2023 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-hosting of the "Great Motors" used-car price data first published in Zindi's "FREE AI Classes In Every City Hackathon" (2022), recorded as `free_ai_classes_in_every_city_hackathon`. +- Source: Zindi hackathon "AI Invasion Hackathon 2023" (subtitle "FUTO AI Hack"), 2023-05-08 to 2023-05-13, 19 participants, Nigeria; the Zindi API names its parent `free-ai-classes-in-every-city-hackathon` (2022-05-28, 690 participants). As published (both pages): "There are ~7000 entries in Train and ~2000 entries in Test"; predict the price "Amount (Million Naira)" from Location, Maker, Model, Year, Colour, Type, Distance. +- Siblings and duplicates: `free_ai_classes_in_every_city_hackathon` (`TBD -> Yes`) is the parent; `free_ai_classes_in_every_city_hackathon_2023` and `dsn_free_ai_classes_in_every_city_2025` are the same series (not opened here). +- Metric: Root Mean Squared Error (Zindi Evaluation page), as in the parent. +- Recommend: add https://zindi.africa/competitions/ai-invasion-hackathon-2023 to the parent's `source_links`. +- Reject check (AI, 2026-10-22): stands. PROV-4. The Zindi API names the parent free-ai-classes-in-every-city-hackathon (2022). Data Description and Evaluation are identical (ratio 1.0): "~7000 entries in Train and ~2000 entries in Test ... predict the price". Same files. The parent record free_ai_classes_in_every_city_hackathon is kept (TBD -> Yes). https://zindi.africa/competitions/ai-invasion-hackathon-2023 + A 2023 Zindi 'ai-invasion-hackathon' with no descriptive content in the record. Nothing is known about the target, modality, domain, size, or data origin, and generic hackathons are often educational/toy challenges. Per guidelines, insufficient information is not by itself grounds for a hard No. A human must inspect the competition page to determine the actual predictive task, modality, and dataset size. diff --git a/curation/records/ai_national_summit_2024_competition.md b/curation/records/ai_national_summit_2024_competition.md index 84ee304a..69d9ef46 100644 --- a/curation/records/ai_national_summit_2024_competition.md +++ b/curation/records/ai_national_summit_2024_competition.md @@ -3,9 +3,9 @@ unique_name: ai_national_summit_2024_competition name: ai-national-summit-2024-competition checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-hosting of the cryptojacking detection data of "UmojaHack Africa 2023: Cryptojacking Detection Challenge (INTERMEDIATE)", recorded as `umojahack_africa_2023_intermediate_challenge`. +- Source: Zindi hackathon "AI National Summit 2024 Competition", 2024-06-08, Tunisia (IEEE INSAT), 47 participants; parent `umojahack-africa-2023-intermediate-challenge` (2023-03-18, 435 participants) per the Zindi API; the second link, "IndabaX Rwanda 2024 Hackathon" (2024-06-21), has the same parent and text. As published: "~4 000 recordings in test and ~9 000 readings in train", network- and host-based features of website activity, label cryptojacking or not; the data "was originally collected by researchers from Instituto Politecnico Nacional" (the first publication is that group's paper, a question for the parent record). +- Siblings and duplicates: `umojahack_africa_2023_intermediate_challenge` (`TBD -> 2nd Tier`) holds the parent; all three pages share the Data Description word for word. +- Metric: F1 on hard labels (Zindi Evaluation page), as in the parent. +- Recommend: move both links to the parent's `source_links`. +- Reject check (AI, 2026-10-22): stands. PROV-4. Both linked pages (AI National Summit 2024, IndabaX Rwanda 2024) have API parent umojahack-africa-2023-intermediate-challenge. Data Description and Evaluation are identical to the parent (ratio 1.0): "~4 000 recordings in test and ~9 000 readings in train". Same files, F1. The parent record umojahack_africa_2023_intermediate_challenge is kept (TBD -> 2nd Tier). https://zindi.africa/competitions/ai-national-summit-2024-competition + A 2024 Zindi summit/hackathon competition (also linked to an IndabaX Rwanda hackathon) with no comments or metadata describing the task. There is no information on the target, modality, domain, or size, and summit hackathons are commonly toy/educational. The guidelines advise against rejecting purely for lack of info. A human must open the source pages to establish the predictive task, modality, size, and provenance, and check whether the two linked competitions are duplicates. diff --git a/curation/records/air_quality_prediction_from_low_cost_iot_devices.md b/curation/records/air_quality_prediction_from_low_cost_iot_devices.md index fe752284..ebd4d863 100644 --- a/curation/records/air_quality_prediction_from_low_cost_iot_devices.md +++ b/curation/records/air_quality_prediction_from_low_cost_iot_devices.md @@ -4,14 +4,15 @@ name: air-quality-prediction-from-low-cost-iot-devices checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Missing source information -- Time-series (Forecasting) +- TBD tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2025' domain: environmental science & climate required_split: - '?' @@ -28,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a sensor-calibration regression (CO2 from low-cost sensor readings against a reference meter), not forecasting, but a short beginner hackathon with an undocumented company experiment and unknown size. +- Source: Zindi hackathon "Air Quality Prediction from Low-Cost IoT devices", hosted by Hackade (Nigeria), 2025-01-30 to 2025-02-02 (Zindi API, read 2026-10-22); "The dataset is sourced from an experiment conducted by Chemotronix. The company deployed three prototype devices - Alpha, Beta, and Charlie - to collect sensor readings alongside a high-accuracy reference meter." As published: Train.csv, test.csv, Variable Description.csv; "Training Data: Contains sensor readings and corresponding CO2 levels measured by the reference meter"; row and column counts not stated. +- Use case: Chemotronix "seeks to build machine learning models that can accurately map sensor readings to CO2 levels measured by reference meters" so it can "manufacture low-cost sensors that rival the performance of expensive reference meters": calibration at the time of a reading; fit direct; regime grouped by device (three devices only) or temporal if the readings carry time stamps (not known without the data). +- Siblings and duplicates: AirQo calibration records `airqo_low_cost_air_quality_monitor_calibration_challenge`, `airqo_low_cost_air_quality_monitor_calibration_challenge_2` (PM2.5, other data) and the forecast records `airqo_ugandan_air_quality_forecast_challenge`, `airqo_african_air_quality_prediction_challenge`: same kind of task, different source; no duplicate found. External collections: none found. +- Papers and pages: the Zindi page only; no paper or Chemotronix data documentation found. +- Metric: Root Mean Squared Error (Zindi Evaluation page); a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: a stream of readings (`Time-series (Regression)` if timestamps exist; SPLIT-7 when none survived); three devices make a grouped split coarse; size unknown (TP-5); a public/private 70-30 test split hints the test may be a time block. The old markers `Missing source information` (the page names Chemotronix and the setup) and `Time-series (Forecasting)` (the target is the concurrent reference reading, not a future value) are dropped. +- Priority: gap 0, use case + (Chemotronix's calibration decision, stated), source 0 (a hackathon page, no paper), shape 0 (unknown size), risk - (stream of readings, three devices). + A Zindi challenge to predict air quality from low-cost IoT sensor devices, with no descriptive comments. Such tasks are usually either sensor calibration/regression on tabular sensor readings (potentially viable) or temporal forecasting of pollutant levels (excluded), and they are inherently time-stamped so a temporal split is likely. The modality and exact target are unverified from the record. A human must determine whether the task is a fixed calibration/estimation (acceptable) or a forecast over a horizon, plus the table structure and size. diff --git a/curation/records/airbnb.md b/curation/records/airbnb.md index 8d17f590..dd2465f8 100644 --- a/curation/records/airbnb.md +++ b/curation/records/airbnb.md @@ -6,6 +6,7 @@ checked_by: - Mustafa - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 tags: - Free Text (Sentences) - AI-Filled (Verify) @@ -27,6 +28,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** recommend `TBD -> 2nd Tier` over the current `TBD -> Yes`, priority 2: a listing-price regression is possible, but the source defines no task, the curators rejected the sibling `melbourne_airbnb_open_data` for that reason, and the asking price is the host's choice with price-derived columns beside it; the curator has the call. +- Source: Inside Airbnb (insideairbnb.com, independent scrapes of Airbnb listings per city); the Kaggle copy "Seattle Airbnb Open Data" (airbnb/seattle, CC0, Kaggle dataset id 393) "is part of Airbnb Inside, and the original source can be found here" (Kaggle metadata, read 2026-10-22). As published: listings.csv (16.9 MB), calendar.csv (36.7 MB, listing id, price and availability per day), reviews.csv (36.6 MB, free-text comments); listing and calendar join on the listing id; row and column counts not verified here. +- Use case: none in the source: the Kaggle "Inspiration" asks for neighbourhood vibes, busy seasons and trends (exploration). A price suggestion for a new host at listing time is the candidate (the same framing as `mercari_price_suggestion`); fit approximate, since the target is the host's asking price, not a realised booking price; regime IID on one city snapshot, grouped by host when several listings share one, temporal when snapshots are combined. +- Siblings and duplicates: `melbourne_airbnb_open_data` (`No`, `No Good Target / Scientific Discovery`, Lennart and Mustafa: "One of many https://insideairbnb.com/ datasets ... no predictive task connected so far"); this record's own human verdict was the same `No` before the AI re-opened it (git history, 2026-07-27). Any Inside Airbnb city is the same source at another place (PROV-6). External collections: TexTabBench (the record's tag; the task it uses was not checked). +- Papers and pages: Inside Airbnb data pages; the Kaggle dataset page. +- Metric: none found (the Kaggle page and Inside Airbnb define no task), so the fallback (rmse; a log target if the price is shipped log-scaled, MET-3). +- Risks for /add-dataset: price-derived columns (`weekly_price`, `monthly_price`, `cleaning_fee`, calendar prices) and post-listing columns (review counts and scores, availability) leak or are unknown at listing time (LEAK); free text dominates part of the signal; the target is a choice of the host (USE). Consistency: re-opening this record but not `melbourne_airbnb_open_data` treats two cuts of one source differently; settle both together. +- Priority: gap 0, use case 0 (written by the curation, not the source), source + (Inside Airbnb documents its scrape), shape + (thousands of listings, mixed types and text), risk - (price-derived and post-listing columns, a host-set target). + Re-opened for review (was `No`, reason `No Good Target / Scientific Discovery`). That reject only considered the **superhost** label, which is rule-based / deterministic — a poor target. But the insideairbnb / Kaggle listings carry a **nightly price** column, which supports a genuine **listing-price regression** task: predict a listing's price from its attributes (room type, location, capacity, amenities, review counts, free-text description, …). This is essentially the price-suggestion model Airbnb's own platform would build to recommend a price to a host — a *regression* task, **not** an out-of-scope ranking / RecSys task despite the informal 'price recommender' framing. To build / verify: construct the price target (strip currency symbols; consider **log-scaling** — nightly prices are heavy-tailed); confirm enough rows remain after cleaning; decide the split (likely IID on a single city snapshot, but watch for host-level **grouping** and for **temporal** structure if multiple insideairbnb snapshots are combined); and check that no column trivially leaks the price. Removed the `No Good Target` marker (the price target resolves it) and set `problem_type: Regression` on that basis. Airbnb listings in Seattle (more to get from the reference) diff --git a/curation/records/airlines_depdelay_10m.md b/curation/records/airlines_depdelay_10m.md index d7cb7d2f..59616115 100644 --- a/curation/records/airlines_depdelay_10m.md +++ b/curation/records/airlines_depdelay_10m.md @@ -4,6 +4,7 @@ name: Airlines_DepDelay_10M checked_by: - AI (UNVERIFIED) suggestion: 'Yes' +priority: 2 tags: - Non-IID (Temporal) - AI-Filled (Verify) @@ -26,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the `Yes` stays (it was written by the AI pass of 2026-06-24 from the workbook's "Temporal Tabular", commit bf95b671, so the curator may want to confirm it); priority 2. Recommend `TBD -> 2nd Tier` or `Duplicate` of the BTS flight-delay source already covered by aeolus; the curator has the call. +- Source: US DOT BTS on-time performance data (transtats.bts.gov); the OpenML upload 42728 (2020-11-18, no creator, no citation) says: "This is the first 10 million rows (and a subset of the columns) of this CSV file" of the raw data "years 1987-2013"; as published: 10,000,000 rows x 10 columns, target `DepDelay` ("in seconds" per the page), features Month, DayofMonth, DayOfWeek, CRSDepTime, CRSArrTime, UniqueCarrier, Origin, Dest, Distance; no missing values; one table. +- Use case: an airline or airport predicting a scheduled flight's departure delay before departure; the source names no actor or decision; fit approximate; regime temporal (a delay as a fixed task, SEL-11), but no Year column ships, and the first 10M rows of a file starting in October 1987 span about two years, so the order within a month is not recoverable across years (SPLIT-7). +- Siblings and duplicates: `airlines` (OpenML 1169, Data Expo 2009 / BTS, TBD -> Yes, CC "Requires temporal split ... only 7 features"); `aeolus` (`Yes`, shipped as `aeolus_1m`, BTS 2016-2024 with weather; its notes found the delay minutes a poor regression target: R² 0.01-0.02); `pnwflights14`. All end at the BTS on-time table, so this is a third cut of one source (PROV-6) unless it adds a distinct task; the Lennart CC of 2026-10-12 asks exactly this. External collections: OpenML (used in large-scale AutoML/GBDT benchmarks of the h2o airline data). +- Papers and pages: ; the 2009 ASA Data Expo for the 1987-2008 file. +- Metric: none found (the OpenML page states no task metric; BTS publishes no prediction task), so the fallback `rmse`. +- Risks for /add-dataset: no year column (SPLIT-7, SPLIT-8); the minutes target is dominated by a long tail (aeolus notes); 1987-1989 schedule data is outdated next to aeolus (`Outdated` worth considering); duplicate by source (PROV-6). +- Priority: gap - (domain Other, regression, temporal not thin), use case - (none named), source + (BTS), shape + (10M rows, mixed types), risk - (duplicate by source, no year). + The well-known OpenML 'Airlines_DepDelay_10M' dataset from US BTS flight records, predicting departure delay (regression) over ~10M rows. This is a large, genuine real-world tabular task already imported from the TabArena workbook with a verdict of 'Temporal Tabular' and multiple curators (Lennart, Andrej) agreeing a temporal split is required. It is clearly representative tabular ML with a meaningful target. A human should confirm it is not an unwanted duplicate of the smaller 'airlines' dataset and that the temporal split is correctly applied. --- diff --git a/curation/records/airqo_african_air_quality_prediction_challenge.md b/curation/records/airqo_african_air_quality_prediction_challenge.md index a4654294..e37bb8a1 100644 --- a/curation/records/airqo_african_air_quality_prediction_challenge.md +++ b/curation/records/airqo_african_air_quality_prediction_challenge.md @@ -3,21 +3,24 @@ unique_name: airqo_african_air_quality_prediction_challenge name: airqo-african-air-quality-prediction-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: TBD -> Yes +priority: 4 decision_markers: -- Time-series (Forecasting) - Wrong Domain / Source Modality tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2024' domain: environmental science & climate required_split: -- '?' +- Grouped (NON-IID) problem_type: Regression +original_data_state: One Table source_links: - https://zindi.africa/competitions/airqo-african-air-quality-prediction-challenge/data +- https://zindi.africa/competitions/airqo-african-air-quality-prediction-challenge source_row: 883 needs_review: - ai_unverified @@ -28,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: same-day PM2.5 estimation from Sentinel-5P products for cities without ground monitors, deployed by AirQo, with a train/test split by city. +- Source: Zindi competition by AirQo (Makerere University) with Mozilla Foundation, 2024-03-15 to 2024-06-16 ($3,000, 1,042 participants); as published: Train.csv and Test.csv (row count not stated on the page), target `pm2_5` from ground observations, features Sentinel-5P L3 products "extracted from Google Earth Engine" (SO2, CO, NO2, HCHO, UV aerosol index, O3, cloud), with city and date; one table. +- Use case: AirQo estimates PM2.5 "for eight cities in seven African countries" where ground networks are thin, and "the models you create will be deployed through AirQo's user-friendly digital platform"; fit direct; regime grouped: "Train set contains the cities, Kampala, Lagos, Nairobi and Bujumbura. The test contains the cities Kisumu, Gulu, Accra and Yaoundé" (deploy to unmonitored cities). The old `Time-series (Forecasting)` marker is refuted: the target is the same-day level, not a future value (SEL-10 does not apply), so it was dropped. +- Siblings and duplicates: `urban_air_pollution_challenge` (Zindi, Sentinel-5P + weather to PM2.5, TBD -> 2nd Tier), `layerai_air_quality_prediction_challenge` and `zindi_weekendz_layerai_air_quality_prediction_challenge` (TBD -> Yes), the other AirQo records (calibration, forecast); different cities and years, so no duplicate by provenance, but a curator may want one Sentinel-5P PM2.5 task, not several. External collections: none found. +- Papers and pages: ; no paper found. +- Metric: RMSE, the Zindi evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"); deliberate (leaderboard, MET-4); inputs none. +- Risks for /add-dataset: the inputs are satellite retrievals (SEL-13 names remote-sensing spectra as image-derived; these are retrieved gas columns, closer to SEL-15 measurements, so `Wrong Domain / Source Modality` stays as a concern for the curator); many missing retrievals under cloud; only 8 cities, so a grouped split by city gives few groups per fold (SPLIT); check whether the date and the city coordinates are needed as features. +- Priority: gap + (grouped regime thin), use case + (AirQo names the deployment), source + (AirQo's data page lists every product), shape + (several thousand rows likely, many numeric features, grouped), risk - (modality question, few groups). + Estimate PM2.5 levels from satellite observations (AirQo). Reliance on satellite/remote-sensing inputs suggests features derived from imagery rather than a naturally tabular table, and air-quality estimation from satellite data is often spatio-temporal, raising both modality and forecasting concerns. The task could still be a fixed per-observation regression if the satellite signals are pre-extracted to tabular features. A human must verify whether the provided features are genuinely tabular (pre-extracted) versus raw rasters, and whether the target is a fixed estimate or a temporal forecast. --- diff --git a/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge.md b/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge.md index 079e677d..a32bd505 100644 --- a/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge.md +++ b/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge.md @@ -3,21 +3,24 @@ unique_name: airqo_low_cost_air_quality_monitor_calibration_challenge name: airqo-low-cost-air-quality-monitor-calibration-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier -decision_markers: -- Missing source information -- Time-series (Forecasting) +suggestion: TBD -> Yes +priority: 4 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: environmental science & climate required_split: - '?' problem_type: Regression +original_data_state: One Table source_links: - https://zindi.africa/competitions/airqo-low-cost-air-quality-monitor-calibration-challenge/data +- https://zindi.africa/competitions/airqo-low-cost-air-quality-monitor-calibration-challenge +- https://zindi.africa/competitions/airqo-air-sensor-calibration-challenge +- https://doi.org/10.1002/ail2.76 source_row: 906 needs_review: - ai_unverified @@ -28,4 +31,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: calibrate AirQo's low-cost PM2.5 monitors to collocated reference monitors in Kampala, a fixed same-hour regression that AirQo uses on its network. +- Source: Zindi competition by AirQo (Makerere University), 2021-04-30 to 2021-06-06 ($1,000, 318 participants), "orignally launch as a hackathon at DataFest Kampala" (`airqo-air-sensor-calibration-challenge`, 2021-04-29/30, the same data and text); as published: Train.csv and Test.csv, hourly rows at three sites (US Embassy, Makerere, Nakawa, "approx 4:2:1") over about two years; target `ref_pm2_5` (reference monitor); features two low-cost PM2.5 and two PM10 sensors, temperature, humidity, `created_at`, `site`, and site metadata (latitude, longitude, altitude, terrain, greenness, distance to a major road), constant per site; one table. +- Use case: AirQo transforms low-cost readings "as accurately as possible to the reference value" so its network is shared "as reference grade"; fit direct; regime temporal or grouped by site: the readings are an hourly stream, the target is the same hour (calibration, not a forecast, so the old `Time-series (Forecasting)` and `Missing source information` markers are refuted and dropped). +- Siblings and duplicates: `airqo_low_cost_air_quality_monitor_calibration_challenge_2` is the DataFest hackathon of the same data (proposed `Duplicate`); `air_quality_prediction_from_low_cost_iot_devices` (TBD -> 2nd Tier, other data); `faulty_air_quality_sensor_challenge` (AirQo, a different task). External collections: none found. +- Papers and pages: ; Adong, Bainomugisha, Okure, Sserunjogi (2022), "Applying machine learning for large scale field calibration of low-cost PM2.5 and PM10 air pollution sensors", Applied AI Letters, doi:10.1002/ail2.76 (AirQo; random forest cut PM2.5 RMSE from 18.6 to 7.2 µg/m³ against BAM references, cross-unit and cross-site validation; not confirmed to be the same rows). +- Metric: RMSE, the Zindi evaluation page ("The evaluation metric for this challenge is the Root Mean Squared Error"); deliberate (leaderboard, MET-4); inputs none. +- Risks for /add-dataset: only three sites, so a grouped split has three groups (a temporal split is the likelier protocol, SPLIT); the site metadata is constant per site (an identifier in disguise); the raw sensor reading may carry most of the signal (TP-4 probe; the paper's 18.6 to 7.2 suggests not); the Zindi test rows may hold no target, so the train file alone may be the data. +- Priority: gap - (environmental, regression, temporal not thin), use case + (AirQo names it), source + (AirQo data page with variable definitions, a paper), shape + (thousands of hourly rows, a temporal structure), risk 0. + An AirQo challenge to calibrate low-cost air-quality monitors (no comments in the record). Sensor-calibration tasks predict a reference pollutant value from raw low-cost sensor readings, which can be a legitimate fixed tabular regression, but the data is time-stamped and may be framed as forecasting or require a temporal split. The exact target, table structure, and size are unverified. A human must confirm whether the task is a fixed calibration regression (acceptable) versus a temporal forecast, and assess the post-cleaning row count. diff --git a/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge_2.md b/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge_2.md index 83f92d02..3082beac 100644 --- a/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge_2.md +++ b/curation/records/airqo_low_cost_air_quality_monitor_calibration_challenge_2.md @@ -3,7 +3,9 @@ unique_name: airqo_low_cost_air_quality_monitor_calibration_challenge_2 name: AirQo Low-Cost Air Quality Monitor Calibration Challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Duplicate tags: - AI-Filled (Verify) collections: @@ -26,6 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the DataFest Kampala preliminary hackathon (https://zindi.africa/competitions/airqo-air-sensor-calibration-challenge, 2021-04-29 to 2021-04-30, 63 participants) of the same AirQo data as `airqo_low_cost_air_quality_monitor_calibration_challenge` (2021-04-30 to 2021-06-06). +- Source: AirQo (Makerere University) on Zindi; the two pages carry the same description, data description, variable definitions, files (Train.csv, Test.csv, SampleSubmission.csv) and metric (RMSE). The hackathon page: "This is a preliminary hackathon, all participants and the Zindi community are invited to continue with the ongoing competition until 6 June"; the competition page: "This challenge was orignally launch as a hackathon at DataFest Kampala. It is now open to the whole Zindi community." +- Siblings and duplicates: kept as `airqo_low_cost_air_quality_monitor_calibration_challenge` (TBD -> Yes, priority 4), whose record now lists this link too; the hackathon opened a day earlier, so it is the first publication by date (PROV-8), but the competition is the public release of the same data and the record the team will curate. +- Metric: RMSE (Zindi evaluation page), as for the kept record. +- Reject check (AI, 2026-10-22): stands. PROV-4. Same title, files and metric. The Data Description is identical (ratio 1.0, 4,594 chars). The hackathon page: "This is a preliminary hackathon, all participants and the Zindi community are invited to continue with the ongoing competition until 6 June". airqo_low_cost_air_quality_monitor_calibration_challenge (TBD -> Yes) already lists this link. https://zindi.africa/competitions/airqo-air-sensor-calibration-challenge + Zindi AirQo sensor-calibration challenge: learn a regression mapping from low-cost air-quality sensor readings (plus context) to a reference PM2.5 value. This is a genuine real-world tabular regression task (sensor calibration), not a forecast of future values over a horizon, so it is in scope despite 'air quality' often implying time series. Likely needs a temporal split because measurements are collocated time series. A human must verify the feature set is tabular (not raw spectra/time windows), the number of usable rows after cleaning, and that the target is a continuous reference PM2.5. --- diff --git a/curation/records/airqo_ugandan_air_quality_forecast_challenge.md b/curation/records/airqo_ugandan_air_quality_forecast_challenge.md index a3c7583d..efaf8101 100644 --- a/curation/records/airqo_ugandan_air_quality_forecast_challenge.md +++ b/curation/records/airqo_ugandan_air_quality_forecast_challenge.md @@ -3,7 +3,8 @@ unique_name: airqo_ugandan_air_quality_forecast_challenge name: airqo-ugandan-air-quality-forecast-challenge checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Time-series (Forecasting) tags: @@ -28,4 +29,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-10) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Time-series (Forecasting)`, SEL-10): AirQo's 2020 Zindi challenge asks for the PM2.5 level 24 hours after a 5-day window of hourly readings, a future value of the series at five Kampala sites. +- Source: Zindi competition by AirQo (Makerere University) with the ASAP project (University of Birmingham), 2020-03-14 to 2020-05-31 ($5,000, 917 participants); as published: Train/Test as CSV and pickles, each row a sensor's 121-hour series of weather readings (temp, precip, wind_dir, wind_spd, atmos_press) and the PM2.5 target, with site metadata (airqo_metadata.csv: altitude, parish area, aspect, road distances); no dates or times. +- Source's words: "The objective of this challenge is to accurately forecast air quality (as measured by PM2.5 µ/m3) for each hour of the coming 25 hours across five locations in Kampala Uganda" and "predict the air quality level at exactly 24 hours after a 5-day series of hourly weather data readings". +- Doubt to record: the inputs are weather series, not past PM2.5 ("you may be given weather indicators (but no air quality data)"), so a TP-10 reading is possible, but the task is a fixed-horizon forecast of a series by its own words, and the rows are series windows, not a table (SEL-10, SEL-12). +- Siblings: the other AirQo records (calibration TBD -> Yes; African prediction TBD -> Yes). +- Metric: RMSE (Zindi evaluation page). +- Reject check (AI, 2026-10-22): does not stand. SEL-10 holds only if the task is a future value of the target's own series, and the source contradicts itself on that. The Data page: "predict the air quality level at exactly 24 hours after a 5-day series of hourly weather data readings ... you may be given weather indicators (but no air quality data) ... you are not given the date or time". So each row is a fixed window of exogenous weather plus static site features, with no lagged target. Zindi data_type is ["Structured","Tabular"]. This is the TP-10 case of SEL-11, and it needs the data to decide. https://zindi.africa/competitions/airqo-ugandan-air-quality-forecast-challenge +- For /add-dataset: TBD -> 2nd Tier, priority 2. The add stage must: (1) open Train.csv/Train.p and check whether the 121-hour windows hold past PM2.5. If they do, the verdict is No with SEL-10. (2) Run TP-10: a model on the weather and site features against each site's naive PM2.5 profile, and check whether the top solutions used any lagged target. (3) Check SEL-12: tabular aggregates of the windows against a sequence model. (4) Check splits: there are no timestamps (SPLIT-7/SPLIT-8), and overlapping windows from one sensor leak across a random split. + Zindi AirQo Ugandan air-quality FORECAST challenge: the task is to forecast future PM2.5/air-quality values over a horizon. Forecasting future values is explicitly excluded from the benchmark scope. The 'forecast' in the name is the decisive signal. A human could double-check whether any framing reduces it to a fixed predictive task, but as posed it is time-series forecasting and should be excluded. diff --git a/curation/records/akeed_restaurant_recommendation_challenge.md b/curation/records/akeed_restaurant_recommendation_challenge.md index 602ad58c..31f1d4a0 100644 --- a/curation/records/akeed_restaurant_recommendation_challenge.md +++ b/curation/records/akeed_restaurant_recommendation_challenge.md @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Out-of-scope Task (CTR/RecSys/Ranking)`, SEL-9): Akeed's Zindi challenge asks for a recommendation engine: which of 100 vendors each customer location will order from. +- Source: Zindi competition by Akeed (Oman food delivery app) with UK Oman Digital Hub and PhazeRo, 2020-05-18 to 2020-08-16 ($3,000, 1,428 participants), launched by the #ZindiWeekendz hackathon of 2020-05-15/17 (the record's second link, same data); as published: train/test customers and locations, orders.csv, vendors.csv; the submission is every `CID X LOC_NUM X VENDOR` pair with a 0/1 target. +- Source's words: "The objective of this competition is to build a recommendation engine to predict what restaurants customers are most likely to order from given the customer location, restaurant information, and the customer order history". +- Siblings: `farm_to_feed_shopping_basket_recommendation_challenge` (a plain `Yes` by a curator; a recommendation task too, worth a consistency look). +- Metric: F1 on the pair labels (Zindi evaluation page). +- Reject check (AI, 2026-10-22): stands. SEL-9 (Must) is out even when served as classification. Source: "The objective of this competition is to build a recommendation engine to predict what restaurants customers are most likely to order from". Every CID x LOC_NUM x VENDOR pair is labelled 0/1. https://zindi.africa/competitions/akeed-restaurant-recommendation-challenge. Consistency flag for the curator: farm_to_feed_shopping_basket_recommendation_challenge is a plain Yes (Andrej), which contradicts this rule. Its record already holds an AI No recommendation under SEL-9 that has not been settled. + Zindi Akeed restaurant RECOMMENDATION challenge: predict which restaurants/vendors a customer will order from, a recommender-system / ranking task. Recommendation/RecSys is explicitly out of scope. The data is also a multi-table customer/order/vendor structure typical of recsys. Excluded; a human need only confirm it is not secretly a simple binary order-prediction reframed as tabular. diff --git a/curation/records/ambassador_skill_up_hackathon.md b/curation/records/ambassador_skill_up_hackathon.md index ffd7152e..0c6c0228 100644 --- a/curation/records/ambassador_skill_up_hackathon.md +++ b/curation/records/ambassador_skill_up_hackathon.md @@ -3,10 +3,9 @@ unique_name: ambassador_skill_up_hackathon name: ambassador-skill-up-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +26,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a Zindi Ambassador skill-up hackathon (2023-05-26 to 2023-05-29, 28 participants, prize "Knowledge") that reuses the SFC PAYGo Solar Credit Repayment Competition (2021) data and text. +- Source: the 2021 competition, recorded as `sfc_paygo_solar_credit_repayment_competition` (TBD -> Yes) and `sfc_paygo_solar_credit_repayment_hackathon` (TBD -> Yes). Both pages have the subtitle "Can you predict PAYGo solar customer payments?", the same data description ("explore and cleanse a dataset consisting of over ~37000 PAYGo SHS contracts ... expected future payments for n months ahead (for this competition n=6)"), the same files (VariablesDefinition.txt, Train.csv, Test.csv, metadata.csv, StarterNotebook.ipynb) and the same submission example (`ID_0026TMW x m1 1700`). +- Note for the kept record: the task predicts the next six months of payments per contract, which /add-dataset must check against SEL-10 and TP-10. +- Metric: RMSE (Zindi evaluation page), as in 2021. +- Reject check (AI, 2026-10-22): stands. PROV-4. Same subtitle ("Can you predict PAYGo solar customer payments?"), same seven files, same Evaluation (ratio 0.998) and the same submission example. The Data Description repeats the 2021 text: "over ~37000 PAYGo SHS contracts ... n months ahead (for this competition n=6)". The original sfc_paygo_solar_credit_repayment_competition is kept (TBD -> 2nd Tier), and that record carries the SEL-10/TP-10 question. https://zindi.africa/competitions/ambassador-skill-up-hackathon + Generic Zindi 'ambassador skill-up hackathon' with no comments, no described target, and no problem framing in the record. This is most likely a small community/learning hackathon rather than a substantive real-world dataset, but there is too little information to reject outright. A human must open the Zindi page to determine the actual predictive target, modality, size, and whether the data is genuine real-world tabular data; defaulting to 2nd Tier pending that inspection. diff --git a/curation/records/amini_soil_prediction_challenge.md b/curation/records/amini_soil_prediction_challenge.md index 7fb46d50..2ebc9dd1 100644 --- a/curation/records/amini_soil_prediction_challenge.md +++ b/curation/records/amini_soil_prediction_challenge.md @@ -4,13 +4,16 @@ name: amini-soil-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 decision_markers: - Needs extensive data wrangling tags: - AI-Filled (Verify) +- Multi-target collections: - New (BeyondArena) original_source: Zindi +year: '2025' domain: environmental science & climate required_split: - '?' @@ -18,6 +21,8 @@ problem_type: Regression original_data_state: Other source_links: - https://zindi.africa/competitions/amini-soil-prediction-challenge/data +- https://zindi.africa/competitions/amini-soil-prediction-challenge +- https://www.zindi.africa/blog/first-place-solution-amini-soil-prediction-a-hundred-heads-are-better-than-one source_row: 882 needs_review: - ai_unverified @@ -28,6 +33,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: predict 11 soil nutrients from environmental features for fertilizer recommendations to farmers; the soil samples' origin is unclear and the release is many tables. +- Source: Zindi competition by Amini (amini.ai, a data company), 2025-04-02 to 2025-06-23 ($7,000, 1,076 participants); as published: Train.csv/Test.csv with "measured soil chemical properties (in ppm)", "environmental and spatial features" and bulk density from "a set of soil from 2019 ... collected from farms across Africa" at 20 cm depth; Gap_Train.csv/Gap_Test.csv; satellite tables to join by field (LANDSAT8, MODIS MOD09GA/MOD11A1/MOD13Q1/MOD16A2/MCD43A4, Sentinel-1, Sentinel-2); row counts not stated. +- Use case: Amini gives farmers "precise recommendations, helping them optimize fertilizer use" from scalable soil estimates instead of a lab test per field; fit direct for the nutrient levels; regime IID or grouped by site/region (spatial; to check). +- Siblings and duplicates: `africa_soil_property_prediction_challenge` (AfSIS Kaggle, TBD -> 2nd Tier), `africa_soil_profiles_database`, `wosis_soil_profiles`, `lucas_topsoil`, `open_soil_data`; the 2019 African farm samples may come from AfSIS or iSDA (not stated): a possible duplicate to trace (PROV-4). External collections: none found. +- Papers and pages: ; first-place write-up on the Zindi blog (an ensemble of many models). +- Metric: RMSE on the nutrient gap in kg/ha (Zindi evaluation page: "The evaluation metric for this competition is Root Mean Squared Error"), where the gap is a published deterministic formula of the predicted ppm: "Available (kg/ha) = ppm x soil depth (cm) x bulk density (g/cm³) x 0.1", "Gap (kg/ha) = Required - Available"; deliberate (leaderboard, MET-4); inputs bulk density and the per-nutrient requirement, which ship. The gap is a known transform of the nutrient (SEL-5: the task is the nutrient level; the formula belongs to the metric). +- Risks for /add-dataset: 11 targets (`Multi-target`; pick one or ship several, CLEAN-18); the satellite tables are remote-sensing spectra (SEL-13 leans them out; the soil and environmental columns may suffice); joins by PID; provenance of the soil samples (PROV-1, PROV-4); a spatial grouping to decide. +- Priority: gap - (environmental, regression, IID not thin), use case + (Amini names it), source - (a company page; the samples' origin is not stated), shape + (many features, likely thousands of fields), risk - (joins, multi-target, possible duplicate). + Zindi Amini soil-prediction challenge: predict availability of 11 essential soil nutrients and nutrient gaps for maize yield, i.e. a multi-target regression problem. This is a genuine real-world tabular(ish) regression task in agronomy/environmental science, already flagged as needing extensive data wrangling. The main risks are (a) inputs may be soil spectral/sensor data closer to a non-tabular modality, and (b) multi-output handling. A human must verify the input features are tabular, choose a single representative target (or confirm multi-output is acceptable), and confirm adequate cleaned size. --- diff --git a/curation/records/assistments.md b/curation/records/assistments.md index be7239fa..4739fa48 100644 --- a/curation/records/assistments.md +++ b/curation/records/assistments.md @@ -4,6 +4,7 @@ name: ASSISTments checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 3 decision_markers: - No Good Target / Scientific Discovery - Time-series (Classification) @@ -24,6 +25,7 @@ source_links: - https://www.kaggle.com/datasets/nicolaswattiez/skillbuilder-data-2009-2010 - https://new.assistments.org/ - https://tableshift.org/datasets.html#assistments +- https://sites.google.com/site/assistmentsdata/datasets/2012-13-school-data-with-affect source_row: 853 needs_review: - ai_unverified @@ -34,6 +36,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 3: TableShift defines a target (next answer correct, held-out schools), but it is a knowledge-tracing log where most per-attempt columns are known only after the answer, and the curator's CC calls the task unclear. +- Source: ASSISTments (Heffernan's group, WPI), the 2012-13 school-year data with affect predictions, on the ASSISTments data site; as published: one problem log per row (user, problem, assignment, skill, start/end time, `correct`, `attempt_count`, `hint_count`, `first_action`, `ms_first_response`, averaged affect predictions); TableShift reports 2,667,776 rows. The Kaggle link in the record is the 2009-2010 skill-builder set, a different year: TableShift says "We use the open-source ASSISTments 2012-2013 dataset". +- Use case: the tutor (or teacher) deciding whether a student has mastered a skill or needs another problem; TableShift: "the process of deploying ASSISTments at a new school"; fit approximate (the source's own decision is a mastery rule, sequence models are the field's tool); regime grouped by school (TableShift: about 700 training schools, 10 held out) or temporal by student history. +- Siblings and duplicates: none in `curation/records/`; external collections: TableShift (`assistments`), knowledge-tracing benchmarks (ASSIST2012 in DKT, SAKT and pyKT). +- Papers and pages: Feng, Heffernan, Koedinger (2009), UMUAI 19:243-266 (the platform); Wang, Heffernan, Heffernan (2015), LAK (the affect detectors); Pardos et al. (2013), LAK (the record's reference); Gardner, Popovic, Schmidt (2023), TableShift, NeurIPS D&B. +- Metric: none deliberate found; TableShift reports accuracy (in-domain and out-of-domain) for every task, a habit (MET-6), and knowledge tracing reports AUC; so the fallback `roc_auc`. +- Risks for /add-dataset: `attempt_count`, `hint_count`, `bottom_hint`, `first_action`, `ms_first_response`, `end_time` describe the same attempt and leak `correct` (LEAK); the affect columns are detector outputs over the same clip; the target needs binarising (the site: "1 = correct, <1 = Incorrect"); 2.7M rows of a sequence (`Time-series (Classification)` stays as a concern); the old `No Good Target / Scientific Discovery` marker stays until the curator settles the task. +- Priority: gap + (education and grouped regime thin), use case - (approximate), source + (ASSISTments documents the columns), shape + (2.7M rows, mixed types, grouped), risk - (leaking per-attempt columns, unclear target). + ASSISTments skill-builder data (2009-2010), used in TableShift/Gardner for knowledge-tracing / affect-to-outcome prediction. The record itself flags an 'unclear predictive task' and uncertainty over whether grouped, temporal, or custom split is correct, and existing tags mark it 2nd Tier / Scientific Discovery. It is a real education dataset but the target definition and split regime are genuinely ambiguous (student-grouped, sequence/time-ordered logs). A human must pin down the exact target (e.g. end-of-year outcome vs next-correct), the correct split, and possibly recreate the table before it can be a Yes. --- diff --git a/curation/records/atf_2023_hackathon.md b/curation/records/atf_2023_hackathon.md index 3c8ff897..b2f99069 100644 --- a/curation/records/atf_2023_hackathon.md +++ b/curation/records/atf_2023_hackathon.md @@ -3,10 +3,9 @@ unique_name: atf_2023_hackathon name: atf-2023-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,4 +27,9 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): an Africa Tech Festival 2023 one-day hackathon (2023-11-15, 500 Zindi points, 0 participants listed) that reruns the Womxn in Big Data South Africa: Female-Headed Households challenge (Zindi, 2019-11-25 to 2020-02-24). +- Source: the 2019 challenge, recorded as `womxn_in_big_data_south_africa_female_headed_households_in_south_africa` (TBD -> 2nd Tier); `be_a_trailblazer` and `international_womens_day_challenge` carry the same target text and are likely re-runs too. The ATF page opens with "Tutorial: Womxn in Big Data South Africa: Female-Headed Households in South Africa Challenge" and describes the same data: wards aggregated from the 2011 census, target the share of households "both female-headed and earn an annual income that is below R19,600", 2,822 train and 1,013 test wards; the same files and the same submission example (`21001001: Ward 1 67.895`). +- Metric: RMSE (Zindi evaluation page), as in 2019. +- Reject check (AI, 2026-10-22): stands. PROV-4. The Zindi API names the parent womxn-in-big-data-south-africa-female-headed-households-in-south-africa. The Description opens "Tutorial: Womxn in Big Data South Africa: Female-Headed Households in South Africa Challenge". Same five files, RMSE, "2,822 wards in the train set ... 1,013 wards in the test set". The parent record is kept (TBD -> 2nd Tier). https://zindi.africa/competitions/atf-2023-hackathon + Generic Zindi 'ATF 2023 hackathon' record with no comments, target, or problem description. Too little information to assess the predictive task, modality, or size from the record alone. A human must open the Zindi competition page to identify the target, domain, and data type; defaulting to 2nd Tier pending inspection rather than rejecting on missing info alone. diff --git a/curation/records/atl_ocean_transect.md b/curation/records/atl_ocean_transect.md index b97551ef..7a0def15 100644 --- a/curation/records/atl_ocean_transect.md +++ b/curation/records/atl_ocean_transect.md @@ -24,6 +24,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): an OTU abundance table from a study of marine bacterial diversity patterns; no predictive task in the source and no target in the record. +- Source: Milici et al., "Low diversity of planktonic bacteria in the tropical ocean", Scientific Reports 6:19054, 2016 (doi 10.1038/srep19054); as published: 16S V5-V6 OTU table and station metadata in the SI, 26 stations x 5 depths x 3 size fractions = 359 samples. +- Use case: none stated; the paper's aim is a discovery ("revealed a clear biogeographic pattern with a double inverted latitudinal gradient"). A framing "predict an environmental variable from the community" would be ours, not the source's (SEL-8), and the variables (temperature, depth) are measured directly at sampling. +- Siblings and duplicates: the microbiome group of records with "metadata source / otu table source" notes (australia, bedford_basin_v4v5, bedford_basin_v6v8, bog_lakes, boso_fish, wastewater_treatment: AI `No`; eu_lakes_allfeatures, eu_lakes_allsamples, nz_springs, ports, subseafloor_arc, subseafloor_arc_2, tara_oceans: AI `TBD -> 2nd Tier`; none verified by a curator). The collection that gathered them is not identified; the closest ML use (Sperlea et al. 2021, Mol Ecol, on EU lakes) quantifies "covariation", also a discovery framing. +- Metric: none (no task). +- Doubt for the reject-checker: environmental-DNA monitoring does predict water-quality indicators from metabarcoding (Bell et al. 2026, bioRxiv, Baltic Sea), so a target could be defined for some siblings; nothing ties this table to such a task. +- Reject check (AI, 2026-10-22): stands. The microbiome family does have a collection: Zschaubitz, ..., Sperlea 2025, 'A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets', CSBJ (doi 10.1016/j.csbj.2025.04.017, https://pmc.ncbi.nlm.nih.gov/articles/PMC12049816/), Table 1, with targets fixed per dataset; but its Methods say 'The choice of target variable was based on an expectation that there is a correlation': chosen to benchmark feature selection, not for a decision, and the first source is a discovery study (SEL-8). Here the target is sampling depth (regression and classification, 113 samples), which is recorded when the sample is taken; no actor predicts it. First source Milici et al. 2016 (https://www.nature.com/articles/srep19054): 'revealed a clear biogeographic pattern'. Correct the block: a target exists in the collection, but it serves no decision. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + An Atlantic Ocean transect microbiome study (26 stations, 5 depths, 3 size fractions) with an OTU abundance table from a paper's supplementary information. This is a microbial-ecology OTU matrix for scientific discovery, not a real-world predictive ML task with a clear target, and it is extremely small (already tagged 'Tiny Data'). High-dimensional OTU tables with tens of samples are not representative of tabular ML and lack a well-posed target. Excluded; a human could confirm sample count but the small size and discovery framing are decisive. **Update (policy):** removed the "Too Small" marker — size is not a rejection reason (decided post-Data Foundry); this record stays No on its other ground(s). diff --git a/curation/records/australia.md b/curation/records/australia.md index b1ca5306..6d9047be 100644 --- a/curation/records/australia.md +++ b/curation/records/australia.md @@ -6,7 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative tags: - Many features - AI-Filled (Verify) @@ -25,6 +24,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): an OTU abundance table cut from the Australian Microbiome monitoring data; the source publishes sequence data for monitoring, with no predictive task, and the record names no target. +- Source: Brown et al., "Systematic, continental scale temporal monitoring of marine pelagic microbiota by the Australian Marine Microbial Biodiversity Initiative", Scientific Data 5:180130, 2018 (doi 10.1038/sdata.2018.130); the data from Bioplatforms Australia (https://data.bioplatforms.com/bpa/otu, accessed 04.10.22 per the record), subset 27f519r bacteria, marine, coastal/pelagic/surface water. +- Use case: none stated: "provides methodologically standardized, continental scale, temporal phylogenetic amplicon sequencing data"; a target (an oceanographic variable) would be ours (SEL-8). Samples are monthly to seasonal at fixed sites, so any task would be grouped or temporal. +- Siblings and duplicates: the same microbiome group as atl_ocean_transect (see there); none verified by a curator. +- Metric: none (no task). +- The old `Not Representative` marker is dropped: it is not the reason; the missing task is. +- Doubt for the reject-checker: eDNA monitoring can predict water-quality indicators (Bell et al. 2026, bioRxiv), so a target could be defined; the source does not. +- Reject check (AI, 2026-10-22): stands. The microbiome family does have a collection: Zschaubitz, ..., Sperlea 2025, 'A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets', CSBJ (doi 10.1016/j.csbj.2025.04.017, https://pmc.ncbi.nlm.nih.gov/articles/PMC12049816/), Table 1, with targets fixed per dataset; but its Methods say 'The choice of target variable was based on an expectation that there is a correlation': chosen to benchmark feature selection, not for a decision, and the first source is a discovery study (SEL-8). Here the target is phosphate (2,620 samples); the collection's intro gives only the general 'bioindicators of environmental states or levels of potential pollution', and AMMBI measures phosphate with each sample (Brown et al. 2018, https://www.nature.com/articles/sdata2018130). This is the weakest of the three: a curator could argue for eDNA nutrient monitoring. Correct the block: a target exists in the collection. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + A subset of Australian Microbiome Initiative 16S amplicon (27f519r bacteria, marine) sequencing data with an OTU abundance table, tagged 'Many features'. This is microbiome OTU data for ecological/scientific discovery, not a published real-world predictive classification/regression task, and OTU matrices (huge sparse feature space, no clear target) are not representative of tabular ML. There is no well-defined predictive target in the record. Excluded; a human would need to invent a target to make it usable, which is out of scope. --- diff --git a/curation/records/autoinland_vehicle_insurance_claim_challenge.md b/curation/records/autoinland_vehicle_insurance_claim_challenge.md index 4c79aeba..3a087cda 100644 --- a/curation/records/autoinland_vehicle_insurance_claim_challenge.md +++ b/curation/records/autoinland_vehicle_insurance_claim_challenge.md @@ -4,11 +4,13 @@ name: autoinland-vehicle-insurance-claim-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: insurance required_split: - Temporal (NON-IID) @@ -30,6 +32,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: a real claim-in-3-months task from an insurer, thin domain, but likely the same data as the AXA Mansard hackathon of 2020. +- Source: Zindi, "AutoInland Vehicle Insurance Claim Challenge", 2021-03-26 to 2021-06-27 (AutoInland, "an Insurance company in west Africa with a focus on Auto Insurance", Nigeria); as published: "~12,000 policies sold by AutoInland for car insurance. Information on the car type, make, customer age and start of policy are in the data"; Train.csv (target), Test.csv, VariableDefinitions.csv, NigerianStateNames.csv (lookup). Columns and the class balance not checked (the data needs a Zindi login). +- Use case: the insurer, at the customer's first transaction, flags who will "submit a claim within 3 months from their first transaction", to "anticipate future demands in terms of claims request volume" and "better manage their resources" (Zindi Description); fit direct; regime temporal (policy start dates ship; a later cohort of policies is what the model meets), to confirm on the dates. +- Siblings and duplicates: `axa_vehicle_insurance_claim_challenge` (UmojaHack Nigeria, 2020-11-28: the Description and Data Description are word for word the same with "AXA Mansard" for "AutoInland", ~17,000 policies) is probably the first publication of the same data, and AutoInland a pseudonymised, smaller re-release; PROV-4/PROV-5 need a row and column comparison of both releases before one is marked `Duplicate` (keep the one that is first and complete, PROV-8). Re-runs listed in `source_links` are Zindi children of this competition (the API's `parent`): the-smart-cube-hackathon (2022), women-in-data-2022, indabax-ghana-autoinland-vehicle-insurance-claim (2021), vehicle-insurance-claim-hackathon (2021), all with the same text and files: copies, not separate datasets. Other insurance records (allstate_claims_severity, fremtpl2, porto_seguro, monthly_insurance_claim_prediction_challenge) are different data. External collections: none found. +- Papers and pages: no paper; the competition page (https://zindi.africa/competitions/autoinland-vehicle-insurance-claim-challenge, read through the Zindi API); top solutions not read. +- Metric: F1 on hard 0/1 labels, the Zindi Evaluation page ("The error metric for this competition is the F1 score"); deliberate (a leaderboard metric, MET-4); the threshold is the participant's, not fixed by the source (MET-6, MET-7: decide at /add-dataset between F1 with a threshold rule and the fallback `roc_auc`); inputs none. +- Risks for /add-dataset: Test.csv lacks "the target-related columns", so check Train.csv for claim-side columns and for `Policy End Date`-like fields known only later (LEAK rules); the policy-to-customer relation (several policies per customer would need grouping); NigerianStateNames is a lookup join; the duplicate question above (PROV-4). +- Priority: gap + (insurance is thin), use case + (the source names the insurer and the planning decision), source 0 (a competition page and a variable list, no codebook or paper), shape + (~12k rows, mixed categorical, numeric and date columns), risk - (likely duplicate of the AXA release, leak check on target-related columns). + + Zindi AutoInland challenge: predict whether a vehicle-insurance customer will submit a claim within the next 3 months, a clear real-world binary classification task with a defined target. Insurance claim prediction on policyholder records is a representative tabular ML problem. Because the target is a future-window event, a temporal split is likely the right protocol. A human must verify the cleaned row count, class balance (claims are typically rare), and that features are policyholder/vehicle attributes rather than leakage from the claim itself. --- diff --git a/curation/records/autonomous_shopper_prediction_by_cape_ai.md b/curation/records/autonomous_shopper_prediction_by_cape_ai.md index 599b22a4..4df2bca7 100644 --- a/curation/records/autonomous_shopper_prediction_by_cape_ai.md +++ b/curation/records/autonomous_shopper_prediction_by_cape_ai.md @@ -3,9 +3,9 @@ unique_name: autonomous_shopper_prediction_by_cape_ai name: Autonomous Shopper Prediction by Cape AI checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD +- Image tags: - AI-Filled (Verify) collections: @@ -27,6 +27,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Image`, SEL-13): pose keypoints extracted from shop camera video, classified as 30-frame sequences ("take" or not); video-derived kinematics are a vision task even as numeric columns. +- Source: Zindi, "Autonomous Shopper Prediction" (IndabaX South Africa 2021, Cape AI / Moonshop), private hackathon 2021-09-03 to 2021-11-21; as published: Train.csv with one row per frame (66 keypoint coordinates from MediaPipe BlazePose GHUM 3D, [0.0, 0.0] for missing), frames grouped in 30 under one ID, Train_Target.csv with one label per sequence, Data_Dictionary.csv. +- Use case: Moonshop's camera-based checkout decides whether a shopper took a product; the decision is real, but the data is video recast as keypoints, and the source frames it as "human action recognition" and "time series prediction", solved with sequence models. +- Siblings and duplicates: none in curation/records/; no external collection. +- Metric: AUC on the predicted probability (Zindi Evaluation page). +- Also fits `Time-series (Classification)` (a sequence of readings classified as a whole). +- Reject check (AI, 2026-10-22): stands. SEL-13 names 'video-derived kinematics'. Zindi (https://zindi.africa/competitions/indabax-south-africa-2021): 'a stream of 30 video frames ... 2D Pose Estimation model (MediaPipe ... BlazePose GHUM 3D) over each frame'. SEL-14 does not apply: the inputs are a model's pose keypoints, not a human reading that tabular models use in practice. + Zindi/IndabaX 'Autonomous Shopper Prediction' (Cape AI): predict whether a customer has taken a product from the shelf, i.e. a binary classification for autonomous/cashier-less retail. The concern is the underlying data could be derived from camera/sensor (computer-vision) feeds rather than genuine tabular features, which would make it not representative of tabular ML. The target is clear, but the modality is uncertain. A human must verify the features are tabular sensor/aggregate signals (not raw video/image) and check the cleaned size and split. --- diff --git a/curation/records/awesome_public_datasets.md b/curation/records/awesome_public_datasets.md index 3348c384..e6e42009 100644 --- a/curation/records/awesome_public_datasets.md +++ b/curation/records/awesome_public_datasets.md @@ -5,8 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Missing source information -- Not Representative +- No Good Target / Scientific Discovery tags: - AI-Filled (Verify) domain: Other @@ -24,6 +23,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): a curated list of links to public data sources, not a dataset; there is no table, no target and no task to benchmark. +- Source: awesomedata/awesome-public-datasets on GitHub (incubated at OMNILab, Shanghai Jiao Tong University), generated from the apd-core metadata; as published: a README of topic-sorted links. +- Use case: none; a dataset found through the list gets its own record from its own source (PROV-8). +- The old markers `Missing source information` (the source is known) and `Not Representative` are dropped. +- Metric: none. +- Reject check (AI, 2026-10-22): stands. Not a dataset. README (https://github.com/awesomedata/awesome-public-datasets): 'This is a list of topic-centric public data sources'. It has no table and no target. + This is the 'awesome-public-datasets' GitHub repository, a curated LIST of datasets and data sources, not a dataset with a predictive task. There is no table, target, or task to benchmark. It was likely captured as a backlog/source pointer. Excluded as not an actual dataset; no human verification needed beyond confirming it is the index repo. --- diff --git a/curation/records/axa_vehicle_insurance_claim_challenge.md b/curation/records/axa_vehicle_insurance_claim_challenge.md index 9cb878fa..8800a7e4 100644 --- a/curation/records/axa_vehicle_insurance_claim_challenge.md +++ b/curation/records/axa_vehicle_insurance_claim_challenge.md @@ -4,11 +4,13 @@ name: AXA Vehicle Insurance Claim Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2020' domain: insurance required_split: - Temporal (NON-IID) @@ -26,6 +28,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: the first release (2020) of the claim-in-3-months task that the AutoInland challenge re-ran; real insurer, thin domain. +- Source: Zindi, "UmojaHack Nigeria: AXA Vehicle Insurance Claim Challenge", one-day private hackathon of 2020-11-28 (UmojaHack Africa with AXA Mansard, Data Science Nigeria, H2O.ai, AI6), open to university students in Nigeria; as published: "~17,000 policies sold by AXA for car insurance. Information on the car type, make, customer age and start of policy are in the data"; Train.csv, Test.csv, VariableDescription.csv. Columns and class balance not checked (Zindi login; the hackathon was restricted to its participants, so access is a question for the curator). +- Use case: AXA Mansard, at the customer's first transaction, predicts a claim "within 3 months from their first transaction" to "anticipate future demands in terms of claims request volume" and "streamline financial planning" (Zindi Description); fit direct; regime temporal (policy start dates ship), to confirm on the dates. +- Siblings and duplicates: `autoinland_vehicle_insurance_claim_challenge` (2021) has the same Description and Data Description word for word with "AutoInland" for "AXA Mansard" and ~12,000 policies: probably a pseudonymised subset of this release. If the rows match, this record is the first publication (PROV-8) and the AutoInland one (with its four Zindi re-runs) the `Duplicate`; confirm on the data first (PROV-5). External collections: none found. +- Papers and pages: no paper; https://zindi.africa/competitions/umojahack-nigeria (read through the Zindi API). +- Metric: F1 on hard 0/1 labels, the Zindi Evaluation page ("The error metric for this competition is the F1 score"); deliberate (leaderboard, MET-4); threshold not fixed by the source (MET-6, MET-7); inputs none. +- Risks for /add-dataset: data access (private hackathon); Test.csv lacks "the target-related columns", so check for columns known only after the claim window (LEAK rules); several policies per customer (grouping); the duplicate question with AutoInland. +- Priority: gap + (insurance is thin), use case + (the source names the insurer and the decision), source 0 (competition page and a variable list only), shape + (~17k rows, mixed types with dates), risk - (overlap with AutoInland, possible leak columns, restricted access). + + Zindi AXA/UmojaHack challenge: predict whether a client will submit a vehicle-insurance claim in the next 3 months, a clear real-world binary classification with a defined target. Insurance claim prediction on policyholder records is representative tabular ML. The future-window target suggests a temporal split. A human must verify cleaned size, class imbalance (rare claims), absence of target leakage, and whether this overlaps/duplicates the AutoInland claim challenge. --- diff --git a/curation/records/barbados_ai_fraud_detection_hackathon.md b/curation/records/barbados_ai_fraud_detection_hackathon.md index d3d5bdd5..ae819299 100644 --- a/curation/records/barbados_ai_fraud_detection_hackathon.md +++ b/curation/records/barbados_ai_fraud_detection_hackathon.md @@ -3,9 +3,9 @@ unique_name: barbados_ai_fraud_detection_hackathon name: barbados-ai-fraud-detection-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-run of the Xente fraud data, recorded as `xente_fraud_detection_challenge`. +- Source: Zindi, "Barbados AI Fraud Detection Hackathon", one day, 2025-05-10; the Zindi API gives its `parent` as `xente-fraud-dection-hackathon` (2020), itself a copy of the Xente Fraud Detection Challenge (2019-05-20, Uganda, ~140,000 transactions), which `xente_fraud_detection_challenge` lists in its `source_links`. +- As published: "anonymized transaction data", "a sample of approximately 140,000 transactions that occurred between 15 November 2018 and 15 March 2019": the same sentence, size and dates as the Xente Data Description; the Barbados framing is new, the data is not (PROV-5: confirmed by the parent link and the identical description). +- Siblings and duplicates: `xente_fraud_detection_challenge` (AI `TBD -> Yes`, unverified) carries the task; its first publication is the 2019 Xente challenge (PROV-8). +- Metric: F1 (Zindi Evaluation page), the same as Xente's. +- Reject check (AI, 2026-10-22): stands. Zindi API: parent = xente-fraud-dection-hackathon, whose parent is xente-fraud-detection-challenge (2019). The files are byte-identical in size: Train 17,071,505 / Test 7,951,537 / SampleSubmission 1,045,033 bytes in both; Variable_Definitions 977 in both. The Data Description has the same '140,000 transactions ... 15 November 2018 and 15 March 2019' sentence. The task is kept in xente_fraud_detection_challenge (PROV-4, PROV-8). + Zindi Barbados AI fraud-detection hackathon with no comments in the record, but the name strongly implies a transaction/fraud binary classification task, which is a representative real-world tabular ML problem. Fraud datasets typically have a clear binary target and tabular transaction features. The lack of detail means this is provisional. A human must open the Zindi page to confirm the target, feature modality, cleaned size, class imbalance, and the appropriate (likely temporal) split. diff --git a/curation/records/be_a_trailblazer.md b/curation/records/be_a_trailblazer.md index 73ec89e4..9818cdd7 100644 --- a/curation/records/be_a_trailblazer.md +++ b/curation/records/be_a_trailblazer.md @@ -3,9 +3,9 @@ unique_name: be_a_trailblazer name: be-a-trailblazer checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,6 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-run of the female-headed households data, recorded as `womxn_in_big_data_south_africa_female_headed_households_in_south_africa`. +- Source: Zindi, "Trailblazers Qualification Challenge" (slug be-a-trailblazer), 2022-02-04 to 2022-05-01, a career-programme qualifier; the Zindi API gives its `parent` as `womxn-in-big-data-south-africa-female-headed-households-in-south-africa` (2019-11-25, Women in Big Data South Africa and Microsoft). +- As published: the same Description and Data Description as the 2019 challenge: the percentage of households per ward "that are both female-headed and earn an annual income that is below R19,600", aggregated from the 2011 census, "2,822 wards in the train set and ... 1,013 wards in the test set"; only the HERE GIS paragraph is replaced by "GIS platforms and software may not be used" (PROV-5: confirmed by the parent link and the identical text and counts). +- Siblings and duplicates: the parent record (AI `TBD -> 2nd Tier`, unverified) and `international_womens_day_challenge` (the same target text; probably another re-run, to check there) carry the task; its first publication is the 2019 challenge (PROV-8). +- Metric: RMSE (Zindi Evaluation page), as in the parent. +- Reject check (AI, 2026-10-22): stands. Zindi API: parent = womxn-in-big-data-south-africa-female-headed-households-in-south-africa (2019). The files are byte-identical in size: Train.csv 2,049,620 / Test.csv 1,113,085 / SampleSubmission 20,866 / variable_descriptions 4,393. The task is kept in the womxn record (PROV-4). + Zindi 'Be a Trailblazer' challenge: predict the percentage of households per ward that are female-headed and living below an income threshold, i.e. a regression on aggregated census/ward-level socioeconomic features. The target is an aggregate proportion, which can be a valid regression task but raises representativeness questions (few rows = number of wards, ecological/aggregated data). A human must verify the number of wards (rows), the feature set (census-derived), and whether the sample size is adequate; aggregated-geography targets can be too small or not representative. --- diff --git a/curation/records/be_a_trailblazer_kenya.md b/curation/records/be_a_trailblazer_kenya.md index daf0d8f4..a91dbaaa 100644 --- a/curation/records/be_a_trailblazer_kenya.md +++ b/curation/records/be_a_trailblazer_kenya.md @@ -3,9 +3,9 @@ unique_name: be_a_trailblazer_kenya name: be-a-trailblazer-kenya checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -31,4 +31,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): re-runs of the Urban Air Pollution data, recorded as `urban_air_pollution_challenge`; not the ward-level task of `be_a_trailblazer` that the old AI paragraph assumed. +- Source: Zindi, "Trailblazers Kenyan Qualification Challenge" and its siblings (Nigerian, South African, Open to All), all 2022-06-24 to 2022-09-25; the Zindi API chains them (kenya -> parent nigeria -> parent south-africa), and their Description and Data Description repeat the #ZindiWeekendz Urban Air Pollution Challenge (2020-04-10) word for word. +- As published: daily PM2.5 per city as the target, with GFS weather and Sentinel 5P L3 pollutant columns, "the last three months, spanning hundreds of cities"; submission keyed 'Place_ID X Date' (PROV-5: confirmed by the identical text and submission format). +- Siblings and duplicates: `urban_air_pollution_challenge` (AI `TBD -> 2nd Tier`, `Time-series (Forecasting)` as a concern) carries the task, first published in 2020 (PROV-8); `urban_air_pollution_challenge_daust_students` is another copy (AI `No`, `Duplicate`). +- Metric: RMSE (Zindi Evaluation page), as in the original. +- The domain on this record (`social science`) is wrong for this data (environmental science & climate); moot under `No`. +- Reject check (AI, 2026-10-22): stands. The Zindi parent chain (kenya -> nigeria -> south-africa) ends without a parent link, but the files are byte-identical to urban-air-pollution-challenge (2020): Train 37,182,338 / Test 19,024,059 / SampleSubmission 371,151 bytes. The Data Description and Evaluation (RMSE, 'Place_ID X Date') are the same text. The task is kept in urban_air_pollution_challenge (PROV-4, PROV-8). + A regional variant (Kenya plus open/South Africa/Nigeria editions) of the 'Be a Trailblazer' challenge, with no comments in the record but presumably the same ward-level socioeconomic prediction (e.g. female-headed / below-income-threshold proportions). Same concerns as the parent challenge: aggregated geographic targets, possibly few rows, and likely overlap/duplication across the regional editions. A human must confirm the exact target, row count, feature set, and whether these editions are duplicates that should be consolidated. diff --git a/curation/records/bedford_basin_v4v5.md b/curation/records/bedford_basin_v4v5.md index 21869cf6..3d0b9ff8 100644 --- a/curation/records/bedford_basin_v4v5.md +++ b/curation/records/bedford_basin_v4v5.md @@ -8,7 +8,6 @@ data_foundry_status: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative tags: - '?' - AI-Filled (Verify) @@ -27,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the Bedford Basin V4-V5 16S ASV table is published for a descriptive study of phytoplankton community dynamics, not for a prediction; no target is named. +- Source: Robicheau, Tolman, Bertrand and LaRoche, "Highly-resolved interannual phytoplankton community dynamics of the coastal Northwest Atlantic", ISME Communications 2, 38 (2022), doi 10.1038/s43705-022-00119-2; weekly samples at 1, 5, 10 and 60 m of the Bedford Basin time-series station, 16S V4-V5 2014-2017, flow cytometry 2014-2018; ASV tables in the supplementary data; metadata from the Bedford Basin Monitoring Program. +- Use case: none; the paper characterises seasonal and interannual shifts, indicator taxa and networks (indicator species analysis, NMDS, envfit, CoNet), and fits no model to predict anything; any target (depth, season, temperature from the ASVs) would be our own (USE-1, USE-2). +- Siblings and duplicates: `bedford_basin_v6v8` is the V6-V8 amplicon of the same samples (a cut of one dataset, PROV-6); the other microbiome records of the same import (`bog_lakes`, `boso_fish`, `tara_oceans`, `eu_lakes_*`, `subseafloor_*`, `nz_springs`, `ports`, `australia`, `atl_ocean_transect`, `wastewater_treatment`) are all AI-screened, no human verdict. The record's `DF: Suspended` status is kept. +- Metric: none (no prediction in the paper). +- Reject check (AI, 2026-10-22): stands. SEL-3 (Must): the source is a descriptive study ("high-frequency DNA sampling allows for the detection of shifting baselines in phytoplankton communities", Robicheau et al. 2022, https://academic.oup.com/ismecommun/article/7461061); indicator species, NMDS, networks, no model and no target; any target (depth, season) would be ours. Same call as all 13 sibling microbiome records in this rescreen. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + Bedford Basin Monitoring Program microbiome data (V4V5 amplicon region) with an OTU table from a paper's supplementary information; the record is already marked 'DF: Suspended' and tagged '?'. This is microbial-ecology OTU data for scientific monitoring/discovery, lacking a well-defined real-world predictive target and not representative of tabular ML (sparse high-dimensional OTU matrix). The suspended status and discovery framing are decisive. Excluded; a human can confirm the suspension rationale. --- diff --git a/curation/records/bedford_basin_v6v8.md b/curation/records/bedford_basin_v6v8.md index ad046e76..d34f89b7 100644 --- a/curation/records/bedford_basin_v6v8.md +++ b/curation/records/bedford_basin_v6v8.md @@ -8,7 +8,6 @@ data_foundry_status: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative tags: - '?' - AI-Filled (Verify) @@ -27,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the V6-V8 16S table of the same Bedford Basin study as `bedford_basin_v4v5`, published for a descriptive study of community dynamics, with no target named. +- Source: Robicheau et al., ISME Communications 2022, doi 10.1038/s43705-022-00119-2 (SI table of the V6-V8 amplicon, which the paper also used for the Scotian Shelf samples); metadata from the Bedford Basin Monitoring Program. +- Use case: none; the study describes seasonal and interannual shifts and indicator taxa, without a prediction (USE-1, USE-2). +- Siblings and duplicates: a second amplicon of the same samples as `bedford_basin_v4v5` (PROV-6; it would be a duplicate even if a target were defined). `DF: Suspended` kept. +- Metric: none. +- Reject check (AI, 2026-10-22): stands. SEL-3 as bedford_basin_v4v5 (same paper, https://academic.oup.com/ismecommun/article/7461061); also the second primer set on the same weekly samples, so a cut without a distinct task (PROV-6). +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + Same Bedford Basin Monitoring Program microbiome dataset as the V4V5 record but for the V6V8 amplicon region (OTU table from supplementary information); also marked 'DF: Suspended' and tagged '?'. As an OTU abundance matrix for scientific monitoring/discovery it has no clear predictive target and is not representative of tabular ML. Excluded on the same grounds as its V4V5 counterpart (and it is effectively a sister table). A human can confirm the suspension and that V4V5/V6V8 are not meant to be merged. --- diff --git a/curation/records/bemtpl97.md b/curation/records/bemtpl97.md index 8b31ffd0..2a8acf95 100644 --- a/curation/records/bemtpl97.md +++ b/curation/records/bemtpl97.md @@ -4,8 +4,10 @@ name: beMTPL97 checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 4 tags: - AI-Filled (Verify) +original_source: Other year: '1997' domain: insurance required_split: @@ -14,6 +16,7 @@ problem_type: Regression original_data_state: One Table source_links: - https://dutangc.github.io/CASdatasets/reference/beMTPL97.html +- https://ideas.repec.org/a/taf/sactxx/v2018y2018i8p681-705.html source_row: 1045 needs_review: - ai_unverified @@ -24,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: a documented Belgian MTPL portfolio for claim frequency or severity, the insurer's pricing task, and insurance is thin. +- Source: CASdatasets R package (Dutang and Charpentier), `beMTPL97` page: "Unknown insurer", 163,212 unique policyholders, 18 columns (`id`, `expo`, `claim`, `nclaims`, `amount`, `average`, `coverage`, `ageph`, `sex`, `bm`, `power`, `agec`, `fuel`, `use`, `fleet`, `postcode`, `long`, `lat`); the page cites Henckaerts, Antonio, Clijsters and Verbelen (2018), "A data driven binning strategy for the construction of insurance tariff classes", Scandinavian Actuarial Journal 2018(8), as the reference for the data. One table. The `year` 1997 is the portfolio year the name suggests; the page does not state it (first publication of the data is about 2018). +- Use case: an insurer sets the premium of a policy at underwriting from the rating factors (Henckaerts et al.: GAMs on the number of claims and on severity, then GLM tariff classes); fit direct for the frequency target (`nclaims` with `expo` as exposure); regime IID, one row per policyholder, no date column. +- Siblings and duplicates: `fremtpl2` (French MTPL, CASdatasets, a different portfolio, so no duplicate, PROV-7); no other Belgian MTPL record. External collections: CASdatasets; used in Deprez, Verbeke and Verdonck (arXiv 2605.22892, 2026, TabPFN for insurance pricing on two public MTPL sets, which ones not confirmed from the abstract). +- Papers and pages: https://dutangc.github.io/CASdatasets/reference/beMTPL97.html; Henckaerts et al. 2018 (https://ideas.repec.org/a/taf/sactxx/v2018y2018i8p681-705.html, open working paper KU Leuven AFI 583471); CASdatasets manual https://cas.uqam.ca/pub/web/CASdatasets-manual.pdf. +- Metric: none found as one ranked criterion (the abstract of Henckaerts et al. reports model fit of GAMs and GLMs; the full text was not read; the CASdatasets page states none), so the fallback (rmse on the claim count) unless the full text fixes Poisson deviance; a deviance would need `expo` as an input, which ships (MET-1, MET-5). +- Risks for /add-dataset: choose one target (`nclaims` per exposure, or `amount` / `average` for severity on claimants); `claim`, `amount`, `average` are framings of the outcome and must be dropped as features (LEAK, USE-3); `postcode` with `long`/`lat` is spatial and may call for a grouped check; `expo` as weight or offset needs a decision. +- Priority: gap + (insurance thin), use case + (pricing named by the paper), source + (CASdatasets page and a paper), shape + (163k rows, mixed types, spatial), risk - (target and exposure handling to settle). + beMTPL97 from the CASdatasets actuarial collection: a Belgian motor third-party liability portfolio used for claim-frequency (and severity) modelling, a classic real-world tabular insurance task with policyholder/vehicle features and a count/exposure target. It is a genuine, well-known actuarial dataset, representative of tabular ML, typically modelled with an IID split. Considerations: the natural target is claim COUNT with exposure offset (Poisson), which a human should map to a supported problem type (regression, or binary has-claim), and confirm it is not a duplicate of other CASdatasets MTPL tables already in the benchmark. --- diff --git a/curation/records/ble_rssi_dataset_for_indoor_localization.md b/curation/records/ble_rssi_dataset_for_indoor_localization.md index b9498ba7..000ef40f 100644 --- a/curation/records/ble_rssi_dataset_for_indoor_localization.md +++ b/curation/records/ble_rssi_dataset_for_indoor_localization.md @@ -4,18 +4,20 @@ name: BLE_RSSI_dataset_for_Indoor_localization checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 1 tags: - Non-IID (Temporal) - AI-Filled (Verify) original_source: UCI -year: '2019' +year: '2018' domain: technology & internet required_split: -- '?' +- Temporal (NON-IID) problem_type: Multiclass Classification original_data_state: One Table source_links: - https://doi.org/10.24432/C5B62T +- https://archive.ics.uci.edu/dataset/435/ble+rssi+dataset+for+indoor+localization+and+navigation needs_review: - ai_unverified type_adapter_id: curation-record-v1 @@ -25,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 1: a real BLE fingerprinting table with the workbook's temporal call, but only 1,420 labelled readings spread over a grid of locations. +- Source: UCI dataset 435, donated 2018-01-24 by Mehdi Mohammadi and Ala Al-Fuqaha (Western Michigan University), CC BY 4.0; as published: `iBeacon_RSSI_Labeled.csv` with 1,420 rows and `iBeacon_RSSI_Unlabeled.csv` with 5,191 rows, columns `location`, `date` and 13 RSSI columns `b3001`-`b3013` (-200 for out of range); target `location`, a grid cell such as `A01` (10 x 10 ft cells), on the first floor of Waldo Library, read with an iPhone 6S. The record's `year` 2019 was AI-filled; UCI gives 2018 (citation 2017), corrected to 2018. +- Use case: a navigation app places a phone in a building from the beacons it hears; the source names "indoor localization and navigation" but no actor or decision; fit approximate; regime temporal, as the workbook reviewers (Lennart, Andrej) decided, because readings carry timestamps and were taken in sessions (neighbouring readings in time are near copies). +- Siblings and duplicates: `ujiindoorloc_mag` (`No` by Lennart, a magnetic-field time series with its own split, a different task); `mobile_c36_oversampling` mentions indoor localisation, not this data. No duplicate found. External collections: none found. +- Papers and pages: https://archive.ics.uci.edu/dataset/435; Mohammadi et al., "Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services" (arXiv 1810.04118, IEEE IoT Journal 2018), the creators' paper; later uses re-label into four zones (arXiv 2402.11433). +- Metric: none found as a single ranked criterion (UCI page states none; the creators' paper reports localisation accuracy and distance error among several, not read in full), so the fallback (log_loss or roc_auc for multiclass) (MET-1, MET-5). +- Risks for /add-dataset: the number of grid cells against 1,420 rows (likely `Too Small` per class, TP-5; a zone re-labelling would be our own target, USE-5); the unlabeled file is unusable; `date` order must drive the split (SPLIT-6). +- Priority: gap - (technology, temporal not thin), use case - (no actor named), source + (UCI with the creators' paper), shape - (1,420 rows, 13 features, many classes), risk - (Too Small). + UCI BLE RSSI indoor-localization dataset: predict location from Bluetooth Low Energy RSSI fingerprints at fixed reference points. It is a genuine real-world tabular task (RSSI values per beacon as features), already given a TabArena curation verdict of 'Temporal Tabular' with multiple reviewers agreeing on a temporal split. Representative of tabular ML and ethically unambiguous. A human must confirm the exact target formulation (discrete location/zone classification vs continuous coordinates), the cleaned size, and that the noted spatial+temporal structure is handled by the temporal split. --- diff --git a/curation/records/bloodsai_blood_spectroscopy_classification_challenge.md b/curation/records/bloodsai_blood_spectroscopy_classification_challenge.md index d26e7ba3..812d1ded 100644 --- a/curation/records/bloodsai_blood_spectroscopy_classification_challenge.md +++ b/curation/records/bloodsai_blood_spectroscopy_classification_challenge.md @@ -4,6 +4,7 @@ name: bloodsai-blood-spectroscopy-classification-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - TBD tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: medical & healthcare required_split: - Random (IID) @@ -28,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real non-invasive blood-test task, but raw NIR spectral bins and 60 shuffled repeat scans per sample with no sample id. +- Source: Zindi competition with bloods.ai, 2021-10-19 to 2022-02-13, $7,500; as published: `Train.csv` / `Test.csv` (and `_trimmed` versions), 170 `absorbance` columns (NIR, 900-1700 nm per the collection page), `temperature`, `humidity`, `Id`, three targets `hdl_cholesterol_human`, `cholesterol_ldl_human`, `hemoglobin(hgb)_human`, each low / ok / high; row count not seen (page could be read through the API; data needs a login). +- Use case: a person scans a fingertip with a NIR device and gets a blood-level class sent to a doctor (the source names patients, home devices and doctors); fit approximate, the label is a lab test paired to the scan; regime grouped by sample, but the source says "Each blood sample is scanned 60 times ... We have shuffled the measurements, so you won't know which measurements are from the same sample", so no group id ships. +- Siblings and duplicates: none found (soil spectral records `open_soil_spectral_library`, `africa_soil_property_prediction_challenge` are other domains). External collections: none found. +- Papers and pages: https://zindi.africa/competitions/bloodsai-blood-spectroscopy-classification-challenge (Description, Data Collection & Exporting Procedures, Data Description, Evaluation); a bloods.ai US patent is linked from the data page. +- Metric: Accuracy over the three targets (Zindi Evaluation page: "The evaluation metric for this challenge is Accuracy"), a leaderboard metric (MET-4), argmax, no threshold to fix (MET-6); inputs none beyond the class. +- Risks for /add-dataset: repeats of one sample across folds without a group id (a leak a random split cannot avoid; recovering the groups would be our own reconstruction, P4, P5); raw spectral bins lean out (SEL-15, weighed); three targets, pick one (CLEAN-18); `problem_type` Multiclass holds; `required_split` Random (IID) (AI-filled) is contradicted by the repeats and kept only until the group question is settled. +- Priority: gap - (medical not thin), use case + (actor and decision named by the host), source + (host documents collection and calibration), shape - (170 spectral bins, size unknown), risk - (hidden repeat groups). + Zindi BloodS.ai challenge to classify blood samples from spectroscopy measurements. Spectroscopy yields high-dimensional spectral intensity vectors (wavelength channels), which sit on the boundary between tabular ML and signal/spectral data where the feature ordering carries meaning, so it may not be fully representative of generic tabular ML. The task is a genuine real-world predictive classification with a clear target, which is in scope, but representativeness is the open question. A human must inspect the actual feature set (number/nature of spectral channels), sample count after cleaning, and whether standard tabular models are competitive here versus signal-specific methods. diff --git a/curation/records/bog_lakes.md b/curation/records/bog_lakes.md index 4a3a03d0..189153b2 100644 --- a/curation/records/bog_lakes.md +++ b/curation/records/bog_lakes.md @@ -6,8 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative -- Time-series (Regression) tags: - Many features - AI-Filled (Verify) @@ -26,6 +24,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the North Temperate Lakes bog-lake 16S OTU table (R package OTUtable) is published to describe bacterial community composition over five years, not for a prediction; no target is named. +- Source: Linz, Crary, Shade, Owens, Gilbert, Knight and McMahon, "Bacterial Community Composition and Dynamics Spanning Five Years in Freshwater Bog Lakes", mSphere 2017, doi 10.1128/mSphere.00169-17 (PMC5489657); 1,387 16S amplicon samples from eight lakes and two thermal layers near Minocqua, Wisconsin; data and scripts in the R package OTUtable (https://amlinz.r-universe.dev/OTUtable) and the McMahon Lab GitHub repository. +- Use case: none; the paper reports year-to-year shifts, a core community and lake-specific communities; any target (lake, layer, year from the OTUs) would be our own and partly a look back at the sampling design (USE-1, USE-2). +- Siblings and duplicates: the other microbiome records of the same import (all AI-screened). External collections: none found. +- Metric: none. +- Reject check (AI, 2026-10-22): stands. SEL-3: Linz et al. 2017 (mSphere, https://pmc.ncbi.nlm.nih.gov/articles/PMC5489657) describe composition and a core community ("Community composition shifted from year to year ... we detected a core community"); no prediction or target in the source; lake/layer/year as target would replay the sampling design. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + Multiyear 16S rRNA microbiome amplicon (OTU) sequencing of eight bog lakes near Minocqua, Wisconsin, joined from an OTU table and a metadata table. This is microbial-ecology scientific-discovery data: an OTU table is an extremely high-dimensional, sparse compositional matrix with no canonical supervised predictive target, and the multiyear sampling makes it a time series. It does not map to a representative real-world tabular classification/regression task. A human could confirm there is no clear predictive target, but on the available description this should be rejected. --- diff --git a/curation/records/boso_fish.md b/curation/records/boso_fish.md index 894b6149..c7a73142 100644 --- a/curation/records/boso_fish.md +++ b/curation/records/boso_fish.md @@ -8,8 +8,6 @@ data_foundry_status: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative -- Time-series (Regression) tags: - '?' - AI-Filled (Verify) @@ -28,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the Boso Peninsula fish eDNA metabarcoding series is published to infer fish-fish interaction strengths and their temperature sensitivity, not for a prediction; no target is named. +- Source: Ushio, Sado, Fukuchi, Sasano, Masuda, Osada and Miya, "Temperature sensitivity of the interspecific interaction strength of coastal marine fish communities", eLife 12, RP85795 (2023), doi 10.7554/eLife.85795; 550 seawater samples, 11 sites, twice a month, 2017-08 to 2019-08, 1,130 MOTUs (top 50 species analysed), covariates water temperature, salinity, wave height, tide; data on Zenodo 7865959, reads in DDBJ DRA014111. +- Use case: none; the study uses information-theoretic causality and the MDR S-map on the species series; a target (a species' abundance next sample) would be forecasting a series (SEL-10) and anything else our own (USE-1, USE-2). +- Siblings and duplicates: the other microbiome records of the same import (all AI-screened). `DF: Suspended` kept. +- Metric: none. +- Reject check (AI, 2026-10-22): stands. SEL-3: Ushio et al. 2023 (https://elifesciences.org/articles/85795) infer interaction strengths ("We detected fish-fish interactions as information flow between eDNA time series"); the quantity explained is an estimated interaction strength, forecasting is named only as future work; a next-sample abundance target would also be SEL-10. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + Fish environmental-DNA (eDNA) metabarcoding from seawater sampled twice monthly at 11 coastal sites over two years on the Boso Peninsula, Japan. Like other amplicon/OTU datasets this is a high-dimensional sparse compositional metabarcoding table from an ecology study with no canonical supervised target, and it is already flagged 'DF: Suspended'. It is a temporal scientific-discovery dataset, not a representative tabular predictive task. Reject; a human need only confirm the absence of a usable supervised target. --- diff --git a/curation/records/busara_mental_health_prediction_challenge.md b/curation/records/busara_mental_health_prediction_challenge.md index 70a3a91c..cbcf1977 100644 --- a/curation/records/busara_mental_health_prediction_challenge.md +++ b/curation/records/busara_mental_health_prediction_challenge.md @@ -4,6 +4,7 @@ name: busara-mental-health-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: - TBD tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2018' domain: social science required_split: - Random (IID) @@ -28,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: a depression-screening task on a household survey with a named actor (clinics, NGOs, community health volunteers), in thin social science. +- Source: Zindi competition hosted by the Busara Center for Behavioral Economics, 2018-11-17 to 2019-01-28; data "from a 2015 study conducted by the Busara Center in rural Siaya County ... more than 70 features, including information about household composition, economic activity, financial flows and health", target `depressed` from "a depression screening tool" ("an epidemiological rather than clinical measure"); one table keyed by `surveyid`; row count not seen (data needs a login). +- Use case: "mental health providers such as local clinics, NGOs or community health volunteers" reach out to those at risk (the source's words); fit direct; regime IID, one respondent per row, one survey wave. +- Siblings and duplicates: no other Busara or Siaya record found. External collections: none found. +- Papers and pages: https://zindi.africa/competitions/busara-mental-health-prediction-challenge (Description, Data Description, Evaluation); the 2015 Busara study itself was not traced to a paper. +- Metric: error rate, "the percentage of survey respondents for whom you predict the 'depressed' variable incorrectly" (Zindi Evaluation page), a leaderboard metric (MET-4) on hard labels; the source fixes no threshold beyond the 0/1 submission (MET-6), inputs none. +- Risks for /add-dataset: check that no item of the screening tool is among the features (SEL-5) and that the health columns are not post-screening (USE-3); the label is a screening score from the same interview, which SEL-7 allows because respondents do not know it; Zindi's rule "Your models should not use any of the metadata provided" names columns to drop; size near the floor. +- Priority: gap + (social science thin), use case + (actor named by the host), source - (only the Zindi page; the study is not published), shape - (row count unknown, likely near 1,000), risk 0. + Zindi Busara challenge to predict mental-health (depression) status from household survey/socioeconomic data collected in Kenya. This is a genuine real-world predictive classification task on structured survey features with a clear binary target, which fits the IID tabular criteria well. The main risks are sample size after cleaning and possible heavy missingness/many sparse survey columns. A human should verify the row count, the exact target definition, the proportion of usable (non-text) features, and confirm there are no ethical concerns given the sensitive mental-health target. diff --git a/curation/records/carbon_dioxide_prediction_challenge.md b/curation/records/carbon_dioxide_prediction_challenge.md index 424bfd0d..a829b20a 100644 --- a/curation/records/carbon_dioxide_prediction_challenge.md +++ b/curation/records/carbon_dioxide_prediction_challenge.md @@ -4,6 +4,7 @@ name: carbon-dioxide-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - TBD tags: @@ -11,9 +12,10 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: environmental science & climate required_split: -- '?' +- Grouped (NON-IID) problem_type: Regression original_data_state: One Table source_links: @@ -32,4 +34,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: Sentinel-5P pollutant columns per location-week against a modelled emission inventory; a spatial split, but the target is a model product. +- Source: first run as the Ernst & Young Carbon Prediction Competition on Zindi (2023-02-23 to 2023-05-01); the same data and pages were reused in UmojaHack Africa 2023 (beginner, 2023-03), IndabaX Zambia 2023, Data Journey Nairobi (2023-06) and the 2024 Carbon Dioxide Prediction Challenge (child of the UmojaHack page): one dataset, five competition pages, all in `source_links`. As published: `Train.csv` / `Test.csv`, IDs `ID_LAT_LON_YEAR_WEEK`, about 800 locations in 20 areas of South Africa (train 361 locations, test 137), weekly 2019-01 to 2022-11, seven Sentinel-5P products (SO2, CO, NO2, HCHO, UV aerosol index, O3, cloud) with sub-features, target `emission` "collected from GRACED and EDGAR". +- Use case: governments and others "estimate carbon emission levels across Africa, even in places where on-the-ground monitoring is not possible" (the source's words); fit approximate, because the target is itself a gridded inventory estimate (GRACED, EDGAR) that already covers those places; regime grouped by location (the source: "The test set is from a different province to the train set"), with a weekly time axis. +- Siblings and duplicates: the five Zindi pages are one dataset (kept in one record); `zindi_weekendz_layerai_air_quality_prediction_challenge` and `urban_air_pollution_challenge` use Sentinel-5P for other targets (AI-screened, no human verdict). The Kaggle Playground S3E20 (Rwanda CO2) uses a similar Sentinel-5P setup on other locations (not checked). External collections: none found. +- Papers and pages: https://zindi.africa/competitions/ey-carbon-prediction-hackathon (first), https://zindi.africa/competitions/carbon-dioxide-prediction-challenge. +- Metric: rmse (every page's Evaluation: "The error metric for this competition is the Root Mean Squared Error"), a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the target is a model output (GRACED/EDGAR), which may make it a smooth per-location profile (TP-10 naive-profile check, SEL-16 question whether a modelled target is real); Sentinel-5P L3 values are retrieved column densities (SEL-15 measurement rather than SEL-13, to confirm); lat/lon in the ID; which of the five releases to take. +- Priority: gap + (grouped regime thin), use case - (fit approximate), source - (Zindi pages only), shape + (about 80k location-weeks, ~70 features, grouped and temporal), risk - (modelled target). + Zindi CO2/carbon-emissions prediction challenge (several linked competition variants, including an EY carbon hackathon). Such challenges frequently derive features from satellite products (e.g. Sentinel-5P) over geographic locations and time, which raises both a remote-sensing/geospatial modality concern and a possible spatio-temporal/forecasting framing rather than a clean IID tabular regression. The underlying task could still be a fixed regression on engineered tabular features, so it is not an automatic reject. A human must determine which competition variant is intended, whether the features are tabular engineered covariates versus raw imagery, and whether the split is IID, temporal, or spatial. diff --git a/curation/records/cardiac_arrhythmias_prediction_challenge_by_lifeq.md b/curation/records/cardiac_arrhythmias_prediction_challenge_by_lifeq.md index 59a24850..b6cde7ba 100644 --- a/curation/records/cardiac_arrhythmias_prediction_challenge_by_lifeq.md +++ b/curation/records/cardiac_arrhythmias_prediction_challenge_by_lifeq.md @@ -4,19 +4,22 @@ name: Cardiac Arrhythmias Prediction Challenge by LifeQ checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Time-series (Classification) - TBD +- Needs extensive data wrangling tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: medical & healthcare required_split: - '?' problem_type: Binary Classification -original_data_state: One Table +original_data_state: Other source_links: - https://zindi.africa/competitions/cardiac-arrhythmias-prediction-challenge/leaderboard source_row: 1007 @@ -29,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: raw RR-interval series per night (515 sessions), a time-series classification that needs feature extraction before it is a table. +- Source: Zindi private hackathon for IndabaX South Africa, hosted by LifeQ, 2021-11-12 to 2021-11-22 (secret code required); as published: one CSV per sleep study (`rr_xxxxxx.csv`, RR intervals in ms with relative timestamps, from 200 Hz ECG in a sleep laboratory, awake periods included), `training_target.csv` with "expertly hand-labelled arrhythmia class labels"; 349 training and 166 test sessions. Not one table: `original_data_state` changed from One Table (AI-filled) to Other. +- Use case: a wearable warns a person of chronic atrial fibrillation or another arrhythmia from a normal night's sleep "and allow for early treatment" (the source's words); fit approximate (polysomnography ECG, not a wearable); regime IID over sessions, unless one person has several sessions (not stated). +- Siblings and duplicates: `arrhythmia` (UCI, ECG features, a different dataset); none other found. External collections: none found. +- Papers and pages: https://zindi.africa/competitions/cardiac-arrhythmias-prediction-challenge (Description, Data Description, Evaluation). +- Metric: Accuracy on the leaderboard (Zindi Evaluation page), but "Your 2 selected submissions will be scored against a different error metric by the LifeQ team. Prizes will be distributed according to this private validation", which is not named; so the leaderboard's accuracy is a weak choice and the fallback is likely (MET-4, MET-10). +- Risks for /add-dataset: a table has to be built from the series (HRV features of our own, CLEAN; `Time-series (Classification)`, SEL-12 judged per dataset: RR-interval features with tabular models are standard for AF detection, so not a clear out); 349 labelled sessions with an unknown class balance (TP-5); access through a private hackathon. +- Priority: gap - (medical not thin), use case + (actor and decision named by the host), source - (only the Zindi page), shape - (515 series, no table), risk - (series to featurise). + Zindi/LifeQ challenge to predict cardiac arrhythmias from sleep data. 'Sleep data' from a wearables company is typically physiological time-series (PPG/heart-rate/actigraphy signals), so the raw modality is likely sequential signal data rather than representative tabular features, and per-night arrhythmia detection is closer to time-series classification. If the provided data are aggregated per-record summary features it could be salvageable as tabular. A human must inspect whether the released features are raw signals/windows or engineered per-subject tabular aggregates, and the sample size. --- diff --git a/curation/records/cardiovascular_disease_dataset.md b/curation/records/cardiovascular_disease_dataset.md index 06a07c3b..66cd07ed 100644 --- a/curation/records/cardiovascular_disease_dataset.md +++ b/curation/records/cardiovascular_disease_dataset.md @@ -32,6 +32,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curators' `No` (CC lines, TabArena Reject) rests on `Missing source information`; the source can be traced, so that reason does not hold, while the data-quality doubts stay open. Counter-argument below; the curator has the call. +- Source: the Kaggle upload (sulianova, file `cardio_train.csv` dated 2019-10-17, licence "unknown") says only "70 000 records of patients data, 11 features + target" and "All of the dataset values were collected at the moment of medical examination". The same columns (age in days, height, weight, gender, ap_hi, ap_lo, cholesterol 1-3, gluc 1-3, smoke, alco, active, cardio) are the data of the Mail.Ru ML Boot Camp championship of 15 June to 15 July 2017, organised "together with Insilico Medicine" (VK/Mail.Ru blog on Habr, habr.com/en/articles/330960): "Датасет сформирован из 100.000 реальных клинических анализов" (built from 100,000 real clinical tests), "Тренировочная выборка состоит из 70 тысяч результатов" (the training set is 70 thousand), and 20 thousand in a hidden test set. The Kaggle file matches the competition's training set in size and columns; OpenML 45547 is a copy of the Kaggle file. +- Use case: screening for cardiovascular disease from a routine check-up ("по результатам классического врачебного осмотра", from a classic doctor's examination, Habr post on the 3rd-place solution, habr.com/post/335188); fit approximate: who examined the patients and how the label was set are not documented. Regime IID. +- Metric: the competition scored log loss ("метрикой является логарифмическая функция потерь", Habr 330960), a leaderboard metric (MET-4). +- Siblings and duplicates: `heart_disease_*`, `framingham_heart_study_fhs` are other cohorts, not duplicates. External collections: TabSTAR, OpenML 45547. +- Data quality: the Kaggle discussion the CC cites lists impossible blood pressures (negative or thousands for ap_hi/ap_lo) and heights; these are entry errors that can be made missing (CLEAN rules), and the organiser's statement of real clinical tests argues against generated data (PROV-3), though the clinic and the label definition stay unknown. +- **Counter-argument (AI):** `Missing source information` (PROV-1) needs a source that cannot be traced; here the first publication is a named competition by Mail.Ru and Insilico Medicine with its size, split and metric stated. What stays open is the data's documentation (which clinic, how `cardio` was diagnosed, PROV-2) and the outlier values. Suggest re-reading the verdict on those grounds (`Data Quality Issue` if the curator judges the damage irreparable, otherwise a `TBD -> 2nd Tier` with the competition as the source); the curator has the call. + Popular Kaggle/OpenML cardiovascular-disease dataset (predict presence of cardiovascular disease from clinical/examination features). Although widely used, the record and curation notes flag that the data source is unknown, the data may be synthetic, and there are documented quality issues (outliers, implausible values), and it is already in the TabArena Reject collection. Missing provenance plus credible suspicion of synthetic/dirty data make it unsuitable. A human could re-check the discussion threads, but the existing markers (Data Quality Issue, Missing source information) support rejection. --- diff --git a/curation/records/census_income.md b/curation/records/census_income.md index 678d5923..c5337503 100644 --- a/curation/records/census_income.md +++ b/curation/records/census_income.md @@ -3,10 +3,10 @@ unique_name: census_income name: Census-Income checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: +- No Good Target / Scientific Discovery - Outdated -- Duplicate tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -31,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the same income-above-50K task as `adult`, which the curator rejected because no actor decides on it; census survey data binarised by the donor. +- Source: UCI "Census-Income (KDD)" (dataset 117), donated 2000-03-06 (record year 1995 is the survey year; the first publication is 2000, PROV-9): "weighted census data extracted from the 1994 and 1995 current population surveys conducted by the U.S. Census Bureau"; 299,285 rows (199,523 train, 99,762 test), 41 demographic and employment variables; income "binned at $50K" to make a binary problem, from total person income, unlike Adult's. +- Use case: none named by the source; the CC on this record: "Task is as in adult - this can be seen as a larger version of adult"; the curator on `adult`: "Not a meaningful task ... label is wrong, higher/lower 50k is not relevant in such settings". Regime IID. +- Siblings and duplicates: `adult` (`No`, curator CC, TabArena Reject; today's AI assessment there also notes this record is another CPS extract, not a copy). `Duplicate` does not fit (different survey years, rows and variables; PROV-4) and should be dropped; `Outdated` stays (1994-95 data; Folktables ACSIncome is the modern rebuild, no record yet). Already in the TabArena Reject collection. +- Papers and pages: the UCI page lists two GAN papers (Xu & Veeramachaneni 2018) that used it, no task paper. +- Metric: none found on the UCI page, so the fallback would apply (MET-1, MET-11). +- Risks noted for the record: the instance weight "should *not* be used in the classifiers" (UCI). +- Reject check (AI, 2026-10-22): stands. SEL-3 by treat-alike: the record's own CC says 'Task is as in adult - this can be seen as a larger version of adult', and the curators' CC on adult rejects that task ('label is wrong, higher/lower 50k is not relevant in such settings'); UCI 117 bins total person income 'at $50K' from 1994-95 CPS (https://archive.ics.uci.edu/dataset/117/census+income+kdd). No other use case in the source. Agree to drop Duplicate (different CPS extract, PROV-4) and keep Outdated. + KDD Census-Income (1994/1995 Current Population Survey), ~300k rows and 41 demographic/employment variables, predicting whether income exceeds a threshold; effectively a larger version of the Adult dataset. It is a genuine, well-defined real-world tabular binary classification, but it is outdated (1990s) and substantially overlaps Adult, and it already sits in the TabArena Reject collection; note the instance-weight column must be excluded to avoid leakage. These overlap/outdated concerns argue against first-tier inclusion rather than against validity. A human should decide on Adult redundancy, confirm the instance-weight handling, and consider whether a newer Census/CPS extract is preferable. --- diff --git a/curation/records/cern_electron_collision_data.md b/curation/records/cern_electron_collision_data.md index 6f78863a..de35115b 100644 --- a/curation/records/cern_electron_collision_data.md +++ b/curation/records/cern_electron_collision_data.md @@ -6,7 +6,7 @@ checked_by: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative +- Trivial tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -28,6 +28,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3; `Trivial`, SEL-5): a CMS outreach sample of real dielectron events with no label; the only target-like column, the invariant mass, is computed from the other columns. +- Source: CERN Open Data Portal record 304, Thomas McCauley, published 2014 (data 2010), CC0, doi 10.7483/OPENDATA.CMS.PCSW.AHVG: "100k dielectron events in the invariant mass range 2-110 GeV for use in outreach and education", "not suitable for a full physics analysis". Columns: Run, Event, and per electron E, px, py, pz, pt, eta, phi, Q; M, "The invariant mass of two electrons (GeV)". Kaggle copy by fedesoriano (2020) repeats the CERN text and defines no task. +- Use case: none; there is no outcome or annotation per event, and M = sqrt((E1+E2)^2 - |p1+p2|^2) is the definition of the invariant mass from the row's own four-vectors, so a regression on M is a formula (SEL-5, TP-4). Events were also selected on M (2-110 GeV). +- Siblings and duplicates: `cern_proton_collision_dataset` (same uploader, same verdict proposed); `higgs`, `hepmass`, `susy` are simulations (Lennart `No` on `higgs`, SEL-16) and a different case: this is real data without a label. The human note "Unsure if the same problem as Higgs": it is not simulated, but it has no target. +- Metric: none (no task is defined anywhere). +- Physics is a thin domain; the gap does not create a target. +- Reject check (AI, 2026-10-22): stands. CERN record 304 (https://opendata.cern.ch/record/304): '100k dielectron events ... for use in outreach and education', 'not suitable for a full physics analysis'; no outcome or annotation per event, and M is the invariant mass of the row's own two four-vectors (SEL-3, SEL-5). + CERN Open Data dielectron collision events: each row gives the kinematic measurements (energy, momentum components, etc.) of two electrons. There is no natural supervised predictive target; the commonly used 'target' (invariant mass) is a deterministic algebraic function of the input columns, which would be trivial/leaky, and the dataset is fundamentally physics scientific-discovery data rather than a representative tabular ML task. A human could confirm no meaningful non-deterministic target exists, but it should be rejected as scientific-discovery. --- diff --git a/curation/records/cern_proton_collision_dataset.md b/curation/records/cern_proton_collision_dataset.md index 069e1a94..1c0190ce 100644 --- a/curation/records/cern_proton_collision_dataset.md +++ b/curation/records/cern_proton_collision_dataset.md @@ -6,7 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -28,4 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the example output of a CMS analysis tool that "computes the razor variables MR and Rsq" for real MultiJet events; no label and no task, and the computed variables are functions of the row's megajets. +- Source: CERN Open Data Portal record 554, Javier Duarte, 2015 (data 2010 RunB, parent /MultiJet/Run2010B-Apr21ReReco-v1/AOD; good-run list record 1000), CC0: "Example CSV output file for SUSYBSMAnalysis-RazorFilter", "computes the razor variables MR and Rsq, used in supersymmetric particle searches", 21,726 events; the CSV "can be used in online applications" (a histogram tool). Columns Run, Lumi, Event, MR, Rsq, the four-vectors of two megajets, HT, MET, nJets, nBJets. Kaggle copy by fedesoriano (2020, "CERN Proton Collision Dataset") repeats it, no task. The record's second link (record 1000 / 553) is the good-run list and parent dataset, not this file. +- Use case: none; real collision data has no truth label, and MR and Rsq are kinematic functions of the megajet four-vectors and MET (Phys. Rev. D 90, 112001), so a regression on them is close to a formula (SEL-5). +- Siblings and duplicates: `cern_electron_collision_data` (same uploader, same verdict proposed); `higgs`, `susy`, `hepmass` are simulations, a different case. +- Metric: none (no task is defined anywhere). +- Physics is a thin domain; the gap does not create a target. +- Reject check (AI, 2026-10-22): stands. CERN record 554 (https://opendata.cern.ch/record/554): 'Example CSV output file for SUSYBSMAnalysis-RazorFilter' that 'computes the razor variables MR and Rsq'; CSV meant for the online histogram tool. Real collision data with no label; MR/Rsq are kinematic functions of the row (SEL-3, SEL-5). + CERN Open Data multijet (proton collision) primary dataset of measured jet/event kinematics. As with the electron-collision data, this is particle-physics scientific-discovery measurement data with no canonical supervised predictive target, and any derived target risks being a deterministic function of the kinematic inputs. It is not a representative real-world tabular classification/regression task. A human should confirm the absence of a usable target before rejecting, but the description supports a No. diff --git a/curation/records/chicago_traffic_crashes.md b/curation/records/chicago_traffic_crashes.md index 2354aa78..0f5ff309 100644 --- a/curation/records/chicago_traffic_crashes.md +++ b/curation/records/chicago_traffic_crashes.md @@ -4,6 +4,7 @@ name: Chicago Traffic Crashes - Crashes (CPD E-Crash) checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) - Non-IID (Temporal) @@ -29,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: Chicago Police crash reports (E-Crash) on the city portal, 1.1M crashes with the officer's injury classification, a thin domain; the target is ours to pick and many fields are recorded after the crash. +- Source: City of Chicago Data Portal, "Traffic Crashes - Crashes" (85ca-t3if), published by the City of Chicago, portal entry created 2017-10: "Data are shown as is from the electronic crash reporting system (E-Crash) at CPD"; "citywide data are not available until September 2017"; "About half of all crash reports, mostly minor crashes, are self-reported at the police district"; fields follow the IDOT Traffic Crash Report SR1050 (manual linked from the page). As published: one table, one row per crash, API count on 2026-10-22: 1,106,184 rows; sibling tables Vehicles and People join on CRASH_RECORD_ID. Targets on offer: MOST_SEVERE_INJURY ("Most severe injury sustained by any person involved in the crash") or CRASH_TYPE ("Injury and/or Tow Due to Crash or No Injury / Drive Away"). +- Use case: no actor named by the source; the closest decision is dispatch or triage from the reported scene (location, time, road, weather, first crash type), or a city safety team prioritising locations; fit approximate. Regime temporal (SPLIT-6): yearly drift and the 2017 citywide roll-out break. +- Siblings and duplicates: `nyc_motor_vehicle_collisions` (AI `TBD -> Yes`, priority 3, same family), `nhtsa_crss_crash_severity` (AI `TBD -> Yes`, national sample), `us_accidents` (Lennart `No`: its Severity is a traffic-delay code; this record was opened as its injury counterpart). No duplicate found. External collections: none found. +- Papers and pages: the portal page and the SR1050 manual; Kaggle copies and notebooks, no competition or paper that defines a task. +- Metric: none found (the portal page, the SR1050 manual, no competition, no defining paper), so the fallback (`roc_auc` for injury vs none, `log_loss` for the injury classes) (MET-1, MET-8, MET-11). +- Risks for /add-dataset: INJURIES_*, DAMAGE, CRASH_TYPE (when the injury is the target), REPORT_TYPE, DATE_POLICE_NOTIFIED, PHOTOS_TAKEN_I, STATEMENTS_TAKEN_I are recorded after the crash and may encode the outcome (LEAK-10); PRIM/SEC_CONTRIBUTORY_CAUSE are the officer's judgement after the fact; self-reported desk reports are mostly minor crashes, so REPORT_TYPE nearly gives the class; records are amended later; pick one target and one regime (USE-5, SPLIT-6). +- Priority: gap + (social science thin, as for `nyc_motor_vehicle_collisions`), use case 0 (no actor named by the source), source + (the City documents the table and the form), shape + (1.1M rows, mixed categorical and numeric, temporal), risk - (post-crash fields, target to choose). + Chicago Data Portal, Chicago Police Department E-Crash system: "Data are shown as is from the electronic crash reporting system (E-Crash) at CPD, excluding any personally identifiable information"; citywide from September 2017, some districts from 2015. Socrata API on 2026-09-23: 1,096,581 crashes, 2013-03 to 2026-09-22. MOST_SEVERE_INJURY: no indication 935,239; nonincapacitating 88,254; reported not evident 51,886; incapacitating 17,675; fatal 1,142; missing 2,385. CRASH_TYPE: "INJURY AND / OR TOW DUE TO CRASH" 297,837 vs "NO INJURY / DRIVE AWAY" 798,744. Rich scene descriptors: posted speed limit, traffic control device, weather, lighting, first crash type, trafficway type, road surface, road defect, primary/secondary contributory cause, lat/lon, hit-and-run, intersection flag. Task: MOST_SEVERE_INJURY (5 classes, heavy imbalance; collapse to injury vs none, 15%) or CRASH_TYPE (27% positive, but it mixes injury with tow-away). Drop outcome fields: DAMAGE, INJURIES_*, CRASH_TYPE when MOST_SEVERE_INJURY is the target, and the report/date-police-notified stamps. Best documented of the city registers; sibling Vehicles / People tables exist for a join. Temporal split, with the 2017 citywide roll-out as a reporting break. Data inspected only through API aggregates. diff --git a/curation/records/childline_kenya_call_volume_prediction_challenge.md b/curation/records/childline_kenya_call_volume_prediction_challenge.md index 64166150..155ebade 100644 --- a/curation/records/childline_kenya_call_volume_prediction_challenge.md +++ b/curation/records/childline_kenya_call_volume_prediction_challenge.md @@ -6,6 +6,7 @@ checked_by: suggestion: 'No' decision_markers: - Time-series (Forecasting) +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,6 +29,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10; `Duplicate`, PROV-4): an hourly call-count forecast over a fixed future window, and a private re-run of `mtoto_news_childline_kenya_call_volume_prediction_challenge`. +- Source: Zindi, "Zindi Mentorship #3: Childline Kenya Call Volume Prediction Challenge", 2020-08-04 to 2020-08-26, a private hackathon for the mentorship cohort (19 participants); the Zindi API names its parent competition `mtoto-news-childline-kenya-call-volume-prediction-challenge` (2019, 272 participants), problem type "Forecast". As published: train.csv, every call from 1 Jan to 12 Jul 2016 (over 135,000 calls with case fields), plus public holidays, Nairobi school dates and weather; the submission is `time_index` (YYYYMMDDHH) and `calls`. +- Use case: Childline staffs its call centre ("manage its resources and staff their call center more efficiently"); the prediction is the future values of one series, which is forecasting (SEL-10), not a fixed task with a temporal split (SEL-11). +- Siblings and duplicates: `mtoto_news_childline_kenya_call_volume_prediction_challenge` (AI `No`, `Time-series (Forecasting)`) is the original competition on the same data; this record is its re-run. +- Metric: "The evaluation metric for this challenge is the Root Mean Squared Error" (Zindi evaluation page, read through the Zindi API). +- A per-call classification (case priority, category) would be a task the source did not set (SEL-8). +- Reject check (AI, 2026-10-22): stands. Zindi API (api.zindi.world/v1/competitions/zindi-mentorship-3-childline-kenya-call-volume-prediction-challenge): 'create a forecast model to predict the number of incoming calls that Childline Kenya will receive per hour per day', RMSE, 19 participants, parent mtoto-news-childline-kenya-call-volume-prediction-challenge (272 participants, already No). Future values of one series (SEL-10) and a re-run of the parent (PROV-4). + Zindi challenge to forecast the number of calls a child-protection hotline will receive each hour of each day. This is explicitly forecasting future call volumes over a time horizon, which is out of scope for the benchmark per the time-series-forecasting exclusion. There is no fixed predictive task that merely needs a temporal split here. Reject; no further verification needed beyond confirming the forecasting framing, which the record already states. --- diff --git a/curation/records/chscase_foot.md b/curation/records/chscase_foot.md index 5925368c..09801579 100644 --- a/curation/records/chscase_foot.md +++ b/curation/records/chscase_foot.md @@ -4,8 +4,11 @@ name: chscase_foot checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: +- No Good Target / Scientific Discovery - Too Small +- Outdated tags: - Tiny Data - AI-Filled (Verify) @@ -26,6 +29,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the proposed fast reject (SEL-3 (with the curators' workbook verdict; TP-5 not measured)) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`No Good Target / Scientific Discovery`, SEL-3): the source names no target, OpenML's default target is the season year of a mis-parsed table of NFL games, and the curators already judged it too small and too thin on information; this restores the workbook verdict that an AI policy note had turned into `TBD -> 2nd Tier`. +- Source: file `foot.dat` of the StatLib `chscase` collection from Chatterjee, Handcock & Simonoff, "A Casebook for a First Course in Statistics and Data Analysis" (Wiley, 1995); OpenML 703 (uploaded 2014): "attribute names were generated automatically since there was no information in the data itself", "CLASSTYPE: numeric / CLASSINDEX: none specific"; default target `col_6`. As published on OpenML: 526 rows x 6 columns; preview rows read `BUF__MIA,-1,0,0,1,89`: col_1 a game (visitor__home NFL teams), col_2 a mix of -1/0/1 and team codes (the parse is shifted on some rows, e.g. `GB`, `KC`, `NE` alone in col_1), col_6 = 89, the season. +- Use case: none recorded; the casebook case is a teaching example on football games (likely point spreads), and the OpenML target is a constant-like year, not an outcome (SEL-3). +- Siblings and duplicates: `chscase_geyser1` (Lennart `No`: "outdated, trivial and not useful for our purpose"), same casebook. The workbook import on this record: "TabArena curation verdict: Tiny data", "Lennart: too small", "Andrej: Insufficient information". +- Metric: none (no task is defined). +- `Too Small` stays as the curators' marker (TP-5 not measured here); `Outdated` added (1989-era teaching data). +- Reject check (AI, 2026-10-22): does not stand. The SEL-3 reason rests on OpenML 703's mis-parse (default target col_6 = season 89/90/91). The source names a predictive question: the casebook table of contents lists the foot case as 'Betting on professional football -- can you beat the bookies?' (http://www.stern.nyu.edu/~jsimonof/Casebook/guide.html, categorical-data chapter), and the StatLib file (http://lib.stat.cmu.edu/datasets/chscase, foot.dat) has columns visitor, home, a -1/0/1 column (264/15/247, plausibly the result against the spread), three 0/1-ish columns, week 1-17 and season 89-91. A bettor's decision is a use case SEL-2 can weigh; the curators' 'too small' is no reject reason before processing (record's own policy note), 'insufficient information' is partly answered by the case title. +- For /add-dataset: TBD -> 2nd Tier, priority 1. Add stage must: get the casebook case text (Stern Casebook pages, Updates/Data folders) to name every column and confirm col_2 is the result against the spread; check whether the spread itself is in the file (without it the task is near-noise by market design, TP-4/TP-2); measure TP-5 on 526 games; weigh Outdated (1989-91) and a market-efficiency (scientific-discovery) reading of the case. + Small statistics-textbook casebook dataset ('chscase_foot') with only 526 samples and limited task information. The prior TabArena curation verdict was 'Tiny data, too small for 3-fold CV', and multiple reviewers (Lennart, Andrej) flagged it as too small or insufficiently informative. 526 rows is below the practical threshold for the benchmark's CV protocol. Reject; the size and existing verdicts are clear. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/cic_ids2017.md b/curation/records/cic_ids2017.md index 5fab5777..420d6233 100644 --- a/curation/records/cic_ids2017.md +++ b/curation/records/cic_ids2017.md @@ -6,7 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - AHDS (Artifical/Handmade/Deterministic/Simulated) -- Data Quality Issue tags: - AI-Filled (Verify) - Non-IID (Temporal) @@ -31,6 +30,14 @@ type_adapter_id: curation-record-v1 ⚠️ **AI-FILLED — UNVERIFIED**: triaged by an AI (2026-08-12), no human has verified it. Check the verdict and the fields before relying on them. +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16): a testbed capture whose benign side is generated from abstract user profiles, the method and the call of its sibling `intrusion_detection` (CSE-CIC-IDS2018, Lennart `No`). +- Source: Canadian Institute for Cybersecurity, UNB (ids-2017 page), Sharafaldin, Habibi Lashkari & Ghorbani, ICISSP 2018; capture Monday 3 July to Friday 7 July 2017; page: "we built the abstract behaviour of 25 users based on the HTTP, HTTPS, FTP, SSH, and email protocols" with the B-Profile system "to profile the abstract behavior of human interactions"; a testbed of a user profiling agent and 12 victim machines against an attack network; labels by "the time stamp, source, and destination IPs". Versions: CICFlowMeter CSVs (with the tool's documented defects), the BCCC NTLFlowLyzer re-extraction (2025, a profiling system, "not a detection system"), and the corrected labels of Engelen et al. (WTMC 2021). +- Use case: a network operator flags malicious flows; the use case is real, but the benign traffic it would be scored against is scripted (SEL-16, PROV-3), as decided for the 2018 capture. +- Siblings and duplicates: `intrusion_detection` (Lennart `No`: AHDS, profiling not detection), `unsw_nb15` (AI `No`, AHDS, traffic generator), `kddcup99` (`No`); `ctu_13` (AI `TBD -> Yes`, priority 4) is the counter-case with real captured background traffic. External collections: many IDS benchmarks; not in our collections. +- Metric: the paper reports per-classifier precision, recall and F1 (several, MET-5), so none deliberate. +- `Data Quality Issue` should go: the label and flow errors are documented and a corrected release exists (Engelen et al.), so the damage is repairable (PROV-13); the verdict rests on SEL-16. +- Reject check (AI, 2026-10-22): stands. SEL-16 by treat-alike: the CIC page (https://www.unb.ca/cic/datasets/ids-2017.html) says 'we built the abstract behaviour of 25 users' with the B-Profile system for the benign traffic, the same testbed method as CSE-CIC-IDS2018, which Lennart rejected with the AHDS marker (intrusion_detection, checked_by Lennart, CC 2026-08-12). The corrected WTMC 2021 labels keep the same scripted benign traffic. Agree to drop Data Quality Issue (repairable). + The 2017 sibling capture of [[intrusion_detection]] (CSE-CIC-IDS2018), generated by the same CIC profile-based method, so the same criterion 4B call applies: the benign side is traffic generated from abstract user profiles in a testbed, not observed traffic. Recorded because the family was otherwise only half covered. Three versions exist and they disagree. The original CIC CSVs come from CICFlowMeter, whose own authors list 16 defects they later fixed — incorrect flow creation, empty flow lists for attack pcaps, PSH-flag and Down/Up-ratio bugs, ICMP mishandling, negative IAT values (Shafi et al., Computers & Security 148:104160, 2025, Sec. 4.1.1, p. 8). The BCCC re-release re-extracts ~300 features from the raw pcaps with NTLFlowLyzer, but that paper builds a profiling system, "not a detection system" (Sec. 6.1, p. 12), so it ships no task; its Table 2 also shows how thin the attack classes are (SQL injection 21 -> 24 flows, Heartbleed 11 -> 12). The independent audit (Engelen et al., WTMC 2021; Liu et al., CNS 2022) publishes separately corrected labels. diff --git a/curation/records/clinical_trials.md b/curation/records/clinical_trials.md index a5549258..260a8767 100644 --- a/curation/records/clinical_trials.md +++ b/curation/records/clinical_trials.md @@ -4,6 +4,7 @@ name: Clinical Trials checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - No Good Target (yet) - TBD @@ -31,4 +32,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a descriptive extract of ClinicalTrials.gov published "to provide a bird's-eye view of the clinical trial landscape", with no target; a trial-termination task would need its own build from the registry. +- Source: Kaggle upload by thedevastator (2022-11-25, one file `AERO-BirdsEye-Data.csv`, 9.7 MB, about 13K trials per its subtitle), credited to Aero Data Lab, whose page (the second link) now shows an expired Squarespace site. The Kaggle text: "The purpose of this dataset is to provide a bird's-eye view of the clinical trial landscape"; columns NCT, Sponsor, Title, Summary, Start Year, Start Month, Phase, Enrollment, Status (Recruiting ... Completed, Suspended, Terminated), Condition; "Research Ideas": "Identify patterns in clinical trials", "Understand how different sponsors fund clinical trials". The trials themselves come from ClinicalTrials.gov (NCT numbers), the first publisher of the records (PROV-8). +- Use case: none named; a framing that exists in the literature is predicting whether a trial terminates early from what is known at registration (a sponsor or a funder deciding on a trial), but the table mixes ongoing trials (Recruiting, Active) with finished ones and keeps only a few registration fields; fit approximate at best. Regime temporal by start date if a target is defined. +- Siblings and duplicates: `covid_trials` (Alex, Mustafa, Lennart `No`, `No Good Target / Scientific Discovery`: another ClinicalTrials.gov extract where no meaningful predictive task was seen), `trec_2022_clinical_trials_dataset` (retrieval, a different task). Not a duplicate of either. External collections: none found. +- Papers and pages: the Kaggle page only; no paper or competition defines a task on this extract. +- Metric: none found (the Kaggle description, the dead Aero Data Lab page), so the fallback if a target is ever defined (MET-1, MET-11). +- Risks for /add-dataset: no target (SEL-3; `No Good Target (yet)`); Status is a snapshot at download, so a termination label for recent trials is unmatured (SPLIT-9); Title and Summary are free text (SEL-12); the publisher's page is gone, so a build should start from ClinicalTrials.gov or AACT instead (PROV-8, PROV-11). The closest precedent `covid_trials` went to `No`; treat like cases alike unless a real termination task is built. +- Priority: gap 0 (medical and temporal not thin), use case 0 (none named), source 0 (third-party extract, the publisher's page is dead), shape + (about 13K rows, mixed types), risk - (no target, unmatured status, text columns). + A Kaggle 'overview of clinical trials' table (aggregated research-landscape data), already tagged 2nd Tier / Scientific Discovery and Non-IID (Temporal). It reads as a descriptive registry/overview table rather than a dataset built around a clear predictive target, which risks falling into the non-predictive/scientific-discovery exclusion. It could be reframed as a predictive task (e.g. predicting trial outcome/completion) if a sensible target and adequate clean rows exist. A human must define whether a real predictive target is available, the row count after cleaning, and confirm the temporal split regime. diff --git a/curation/records/comex_day_1.md b/curation/records/comex_day_1.md index 2ab0ed0d..b4ba5749 100644 --- a/curation/records/comex_day_1.md +++ b/curation/records/comex_day_1.md @@ -3,10 +3,9 @@ unique_name: comex_day_1 name: comex-day-1 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -29,4 +28,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): "COMEX Vehicle Insurance Claim Challenge", a one-day private Zindi re-run of `autoinland_vehicle_insurance_claim_challenge` on the same AutoInland policies. +- Source: Zindi competition `comex-day-1` (read through the Zindi API; the web page refused the connection): title "COMEX Vehicle Insurance Claim Challenge", 22 May 2023, 09:30 to 18:30, Oman, "This challenge is COMEX participants", 3 participants; restricted, secret code required. Its description, data description and files (Train.csv, Test.csv, VariableDefinitions.csv, NigerianStateNames.csv, starter notebooks) are word for word those of the AutoInland Vehicle Insurance Claim Challenge (Zindi, 2021-03-26): "The data describes ~12,000 policies sold by AutoInland for car insurance"; target: "if a customer will submit a claim within 3 months from their first transaction"; metric F1 in both. +- Siblings and duplicates: `autoinland_vehicle_insurance_claim_challenge` (AI `TBD -> Yes`, which already lists four other re-runs: the-smart-cube-hackathon, women-in-data-2022, indabax-ghana-autoinland-vehicle-insurance-claim, vehicle-insurance-claim-hackathon) is the first publication; add the comex-day-1 link there. `axa_vehicle_insurance_claim_challenge` (UmojaHack Nigeria) may be the same data again; check it when AutoInland is added. +- The old markers `Missing source information` and `TBD` go: the source is AutoInland via Zindi, and the record's domain (finance) would be insurance. +- Reject check (AI, 2026-10-22): stands. Zindi API (api.zindi.world/v1/competitions/comex-day-1): 'COMEX Vehicle Insurance Claim Challenge', 22 May 2023, 3 participants, 'This challenge is COMEX participants.'; Data Description 'The data describes ~12,000 policies sold by AutoInland for car insurance', same file list (incl. NigerianStateNames.csv) and error_metric f1 as autoinland-vehicle-insurance-claim-challenge (2021). Same source, same task (PROV-4); keep autoinland_vehicle_insurance_claim_challenge (TBD -> Yes). + Zindi 'comex-day-1' competition with no comments or description in the record, so there is too little information to judge confidently. The name suggests a COMEX commodity-exchange financial dataset, which would raise a real risk that the task is price prediction/forecasting (out of scope) rather than a fixed IID tabular classification/regression. Per guidance I do not reject solely for sparse info, but the likely financial-price framing keeps it below first tier. A human must read the Zindi competition page to determine the actual target, whether it is forecasting versus a fixed task, the feature types, and the data size. diff --git a/curation/records/confidence.md b/curation/records/confidence.md index 6441fc98..c8c72947 100644 --- a/curation/records/confidence.md +++ b/curation/records/confidence.md @@ -4,19 +4,21 @@ name: confidence checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small tags: - AI-Filled (Verify) -original_source: Github +original_source: Other year: '?' domain: Other required_split: - '?' -problem_type: Regression +problem_type: Multiclass Classification original_data_state: One Table source_links: - https://github.com/EpistasisLab/pmlb/blob/master/datasets/confidence/metadata.yaml +- https://www.openml.org/d/468 needs_review: - ai_unverified type_adapter_id: curation-record-v1 @@ -26,6 +28,22 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict (Tiny data; Lennart and Andrej: too small) stands, priority 1; +recommend `No` with `Too Small` and `No Good Target / Scientific Discovery` (TP-5, SEL-3); the curator has the call. +- Source: StatLib, consumer-confidence items given by Gordon Bechtel (Univ. of Florida) with permission of the Conference + Board and the Michigan Survey Research Center, "1992 DATA" (OpenML 468, "Source: Unknown - Date unknown"); PMLB copies + it. As published: 72 rows x 3 numeric features `P`, `N`, `O` (share of pessimistic, neutral, optimistic answers) and + the target `ITEM`, which of six survey questions (business conditions, jobs, family income, 6 or 12 months ahead) the + row is; 12 rows per item. OpenML 1015 is a binarised version (60/12). The record's old `Regression` was wrong: + PMLB and OpenML serve a 6-class classification (`problem_type` corrected, it was AI-filled). +- Use case: none. Telling which survey question produced a month's answer shares is no decision anyone makes; the table + is a survey summary (SEL-3). Regime would be IID at best; the 12 rows per item are likely the months of 1992 (temporal). +- Siblings and duplicates: none in `curation/records/` besides the PMLB/OpenML copies named here. +- Papers and pages: no paper found; OpenML 468 description, PMLB metadata (empty description). +- Metric: none found (OpenML, PMLB, StatLib text); the fallback. +- Risks for /add-dataset: 72 rows, 12 per class (TP-5); no target a decision fits (SEL-3). +- Priority: gap 0, use case -, source - (anonymous StatLib note), shape - (72 x 3). + PMLB 'confidence' dataset with only 72 samples. Both the prior TabArena verdict and the two named reviewers (Lennart, Andrej) flagged it as too small, and 72 rows is far below what the benchmark's CV protocol can support. There is no path to making this representative at that size. Reject on size; no further verification needed. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/cps88wages.md b/curation/records/cps88wages.md index 23e11163..874bf956 100644 --- a/curation/records/cps88wages.md +++ b/curation/records/cps88wages.md @@ -4,6 +4,7 @@ name: cps88wages checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 4 decision_markers: - Outdated tags: @@ -12,6 +13,7 @@ tags: collections: - TabArena Reject - TabSTAR +original_source: Other year: '1988' domain: social science required_split: @@ -31,6 +33,24 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 4: a real, documented CPS wage table, but built for +econometric specification testing, not for a decision (the curator's CC line and the TabArena reject point the same way). +- Source: March 1988 Current Population Survey (US Census Bureau), sample drawn by Bierens & Ginther (Empirical Economics + 26, 2001) for testing quantile-regression models; distributed as `CPS1988` in the R package AER and as OpenML 44984. + As published: 28,155 men aged 18-70 x 7 columns: `wage` (target, deflated to 1992 dollars), `education`, `experience` + (potential), `ethnicity` (cauc/afam), `smsa`, `region`, `parttime`. One table. +- Use case: no actor named by the source; the paper estimates a (median) wage equation, a returns-to-schooling question + (USE-2). A use case the curation could write (an HR salary benchmark) would be approximate at best and uses 1988 data. + Regime IID (one cross-section, one survey month). +- Siblings and duplicates: `census_income` (CPS 1994/95, income threshold, also `TabArena Reject`, TBD -> 2nd Tier) is a + different survey year and task; no duplicate of this extract found. External collections: TabArena Reject, TabSTAR. +- Papers and pages: Bierens & Ginther 2001; OpenML 44984 ("The goal is to estimate the wage using information about + working individuals"); the original page http://www.personal.psu.edu/hxb11/MEDIAN.HTM (offline). +- Metric: none found as a prediction metric (OpenML page, AER docs; the paper tests model specifications), so the + fallback (RMSE on log wage would be the econometric habit). +- Risks for /add-dataset: few features (6), `ethnicity` as a sensitive input (the CC line); `Outdated` stays. +- Priority: gap + (social science thin), use case -, source + (paper, CPS documentation), shape + (28k rows, mixed types). + CPS 1988 wages dataset: predict (log) wage from census/survey demographic features. It is a genuine real-world tabular regression with a clear target, but it is outdated (1988) and is already in the TabArena Reject and TabSTAR collections, with a curation note questioning whether predicting wage from sensitive demographic features is desirable. These outdatedness and sensitivity concerns argue against first-tier inclusion rather than against task validity. A human should weigh the ethical/sensitive-feature angle, the age of the data, and whether a fresher CPS extract is preferable, plus confirm the row count. --- diff --git a/curation/records/credit_card_dataset_for_clustering.md b/curation/records/credit_card_dataset_for_clustering.md index a13e2c72..6af8aa83 100644 --- a/curation/records/credit_card_dataset_for_clustering.md +++ b/curation/records/credit_card_dataset_for_clustering.md @@ -5,7 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- No Good Target (yet) +- No Good Target / Scientific Discovery - Missing source information tags: - 2nd Tier / Scientfic Discovery @@ -28,6 +28,20 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, `Missing source information`; SEL-3, PROV-1): the source publishes a +segmentation case with no predictive target, from an untraced "sample Dataset". +- Source: Kaggle arjunbhasin2013/ccdata ("Credit Card Dataset for Clustering"), CC0; OpenML 43618 is a copy. As + published: about 9,000 active card holders x 18 behavioural variables over 6 months (`BALANCE`, `PURCHASES`, + `CASH_ADVANCE`, `CREDIT_LIMIT`, `PAYMENTS`, `PRCFULLPAYMENT`, `TENURE`, ...), one table, no label. +- Use case: "This case requires to develop a customer segmentation to define marketing strategy": clustering, no actor + deciding on a predicted outcome (SEL-3). A target could be carved out (`PRCFULLPAYMENT`, `CREDIT_LIMIT`), but it + would be our construction from features of the same window, not the source's task. +- Provenance: the page names no bank, institution or paper ("The sample Dataset summarizes ..."); no upstream found + (PROV-1). The old note "unclear if real data" stays open. +- Siblings: `customer_segmentation_data` (`No`, AHDS, no target) is the same pattern; external collection TabSTAR. +- Metric: none (no predictive task). +- Reject check (AI, 2026-10-22): stands. SEL-3 (SEL-8, original task): the source's own task is unsupervised, "This case requires to develop a customer segmentation to define marketing strategy" (https://www.kaggle.com/datasets/arjunbhasin2013/ccdata); no label ships. PROV-1 is weaker under the amended PROV-2 (the first upload is a source to point to), but SEL-3 decides alone. + Kaggle 'Credit Card Dataset for Clustering' of credit-card customer behavior features, explicitly published for unsupervised clustering with no supervised predictive target, and the record notes it is 'not a predictive task, unclear if real data'. Lacking a real classification/regression target and with questionable provenance, it does not satisfy the predictive-task and uniqueness/quality criteria. A human could check whether any sensible supervised target could be carved out, but as published it should be rejected. --- diff --git a/curation/records/credrails_banking_transaction_categoriser_challenge.md b/curation/records/credrails_banking_transaction_categoriser_challenge.md index 3b5e8a32..8a943056 100644 --- a/curation/records/credrails_banking_transaction_categoriser_challenge.md +++ b/curation/records/credrails_banking_transaction_categoriser_challenge.md @@ -4,6 +4,7 @@ name: credrails-banking-transaction-categoriser-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - NLP (Text) - TBD @@ -12,6 +13,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: finance required_split: - Random (IID) @@ -19,6 +21,7 @@ problem_type: Multiclass Classification original_data_state: One Table source_links: - https://zindi.africa/competitions/credrails-banking-transaction-categoriser-challenge/data +- https://zindi.africa/competitions/credrails-banking-transaction-categoriser-challenge source_row: 995 needs_review: - ai_unverified @@ -29,6 +32,28 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real categorisation need, but the data is an +unnamed "public data source", the labels are not explained, and the signal is most likely the description text. +- Source: Zindi #ZindiWeekendz hackathon for Credrails (open-banking API), 2-4 September 2022. Data page: "This data is + from a public data source but is very similar to the data we usually see. It is 80 000 transactions over a 7-year + period"; columns named include `DATE` and `BALANCE AMT`; 21 target classes (category/subcategory). The column names + match the anonymous Kaggle upload "Bank Transaction Data" (Indian bank statements; `TRANSACTION DETAILS`, `WITHDRAWAL + AMT`, `DEPOSIT AMT`, `BALANCE AMT`), which has no categories, so Credrails probably added the labels (unverified: the + Zindi files need a login). +- Use case: Credrails "categorise any transaction on our platform" for customer analytics: actor the API provider, at + ingestion of a statement; fit approximate (public statements, not Credrails' own). Regime: temporal by `DATE` (7 years) + or grouped by account; IID would mix one account's recurring payees across folds. +- Siblings and duplicates: `alvin_smart_money_management_classification_challenge` (purchase categories, `No`, Too Small, + Lennart 2026-10-12) is a different source; no duplicate found. +- Papers and pages: Zindi competition page (description, data, evaluation read through the Zindi API); no paper. +- Metric: multiclass log loss over 21 classes, Zindi evaluation page ("The evaluation metric for this challenge is Log + Loss. Note there are 21 classes"); deliberate (leaderboard, MET-4); inputs none. +- Risks for /add-dataset: labels likely assigned by keyword rules on the description (SEL-5, `Trivial`) and the task is + then text matching (SEL-12, `NLP (Text)`); the upstream source is anonymous (PROV-2); `ID` is built from `DATE` and + `BALANCE AMT` (drop); balance columns leak the row order within an account. +- Priority: gap 0, use case + (Credrails names the decision), source - (unnamed public source), shape + (80k rows, + amounts, dates, text), risk - (text-driven, rule-made labels). + Zindi/Credrails challenge to categorise banking transactions for an API. Transaction categorisation is typically driven by free-text transaction descriptions/merchant strings, which makes it primarily an NLP (text-classification) task where text models tend to dominate, rather than a representative numeric/categorical tabular task. It could be partly tabular if amount, channel, and other structured fields carry most of the signal. A human must inspect the feature set to see whether the predictive signal is mostly in text fields versus structured columns, plus the number of target categories and the data size. --- diff --git a/curation/records/crop_type_classification_challenge.md b/curation/records/crop_type_classification_challenge.md index c2e05ccb..4d8f7dea 100644 --- a/curation/records/crop_type_classification_challenge.md +++ b/curation/records/crop_type_classification_challenge.md @@ -3,10 +3,9 @@ unique_name: crop_type_classification_challenge name: Crop Type Classification Challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Wrong Domain / Source Modality -- TBD +- Image tags: - AI-Filled (Verify) collections: @@ -29,6 +28,21 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Image`, SEL-13): every feature is a statistic of Sentinel-2 pixel values per field, a remote-sensing spectrum +that serves to recognise what the image shows (the satimage pattern). +- Source: Zindi "GeoAI Hack 2022 Crop Type Classification Challenge", a private hackathon for Tunisian participants (22-23 + April 2022), by IEEE Sup'Com GRSS and AGEOS with InstaDeep; field labels from an AGEOS ground survey in Kairouan, + parcels digitised from VHR imagery. As published: Train.csv (13.6 MB) and Test.csv; features `timestepT_BAND_AGG` for + 24 shuffled dates (Nov 2020 - Apr 2021), 10 bands (B02-B12), 5 aggregations (mean, median, std, max, min); 13 crop + classes ('olive+cereals', 'wheat', 'greenhouses', ...). +- Use case: crop-type mapping for agricultural policy (Zindi description); the prediction is made from imagery, and + remote-sensing models (pixel and time-series classifiers) are the field's tool (SEL-12, SEL-13). +- Siblings: `geoai_challenge_for_cropland_mapping_in_dry_environments` (TBD -> Yes, Andrej) is a counter-precedent of + the same family; `opengeoai_2023_hack` (TBD -> 2nd Tier), `telangana_crop_health_challenge` (TBD -> Yes, WIP). + The curator may want one call for the satellite-feature Zindi challenges. +- Metric: log loss (Zindi: "The evaluation metric for this challenge is Log Loss"), deliberate (MET-4). +- Reject check (AI, 2026-10-22): stands. SEL-13 lists remote-sensing spectra: "AGG is the aggregation that was applied on the pixels values per field" over 24 Sentinel-2 dates x 10 bands (Zindi data page, https://api.zindi.world/v1/competitions/geoai-hack-2022); curator precedents crowdsourced_mapping (Landsat NDVI series -> land cover, Lennart and Andrej: Image), satimage, satellite, oil_spill. For the curator (consistency, not a refutation): geoai_challenge_for_cropland_mapping_in_dry_environments is TBD -> Yes (Andrej) on a similar satellite-feature setup; one call for the family. + Zindi GeoAI-Hack-2022 crop-type classification challenge. Crop-type mapping challenges are usually built on satellite remote-sensing data (e.g. Sentinel-2 multispectral pixels, often as time series of band reflectances), which is a geospatial/image modality and frequently spatio-temporal rather than a clean IID tabular task. If the released data are per-field tabular spectral/temporal aggregates with a categorical crop label it could be usable as tabular classification. A human must determine whether the features are raw imagery/time-series bands versus engineered tabular features, the label set, the size, and the appropriate (likely spatial) split. --- diff --git a/curation/records/cryptocurrency_closing_price_prediction.md b/curation/records/cryptocurrency_closing_price_prediction.md index 32ab4e99..9314a22e 100644 --- a/curation/records/cryptocurrency_closing_price_prediction.md +++ b/curation/records/cryptocurrency_closing_price_prediction.md @@ -29,6 +29,21 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): the organisers frame the task as forecasting future prices of one +hourly series. +- Source: Zindi "Cryptocurrency Closing Price Prediction" by Think.iT (25 June - 19 September 2021; a parent + "cryptocurrency-closing-price-prediction-challenge" closed April 2021); re-run as a learning hackathon `kabarak-2022` + (Developer Student Club Kabarak University, Nov-Dec 2022) with the same files (Train.csv 3.4 MB, Test.csv 1.6 MB). + As published: hourly rows from 1 March 2020 to 1 March 2021, `asset_id`, open/high/low/volume, market cap and + about 40 social-media counts and scores (tweets, Reddit, sentiment, `price_score` from a moving average); target `close`. +- Use case: investment decisions on a volatile asset (Zindi); the target is the next values of a price series (SEL-10), + not a fixed task that only needs a temporal split (SEL-11). +- Siblings: `g_research_crypto_forecasting` (`Yes`, Lennart) predicts residualised 15-minute returns across 14 assets, + a fixed target over a cross-section; this record has the raw price level of one series. + `the_ultimate_hacking_challenge_by_uct_cryptocurrency_and_ai_society` is a different challenge. +- Metric: RMSE (Zindi: "The error metric for this competition is the Root Mean Squared Error"). +- Reject check (AI, 2026-10-22): stands. SEL-10: the source frames forecasting of one series, "we would like to build a model that forecasts a cryptocurrency's price ... We will predict future cryptocurrency prices", hourly data 2020-03-01 to 2021-03-01 (Zindi description via https://api.zindi.world/v1/competitions/cryptocurrency-closing-price-prediction). Unlike g_research_crypto_forecasting (residualised returns across 14 coins, TP-10 passed) this is the raw price level of a cryptocurrency, so no cross-entity panel for TP-10 to rescue. + Zindi challenge to predict the closing price of a cryptocurrency. Predicting a future closing price is financial time-series forecasting over a horizon, which is explicitly out of scope for the benchmark. There is no fixed, non-temporal predictive task here that would merely need a temporal split. Reject; the forecasting framing is clear from the title and the record's description. --- diff --git a/curation/records/ctu_13.md b/curation/records/ctu_13.md index b2d170ae..94569996 100644 --- a/curation/records/ctu_13.md +++ b/curation/records/ctu_13.md @@ -3,7 +3,8 @@ unique_name: ctu_13 name: CTU-13 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: TBD -> Yes +priority: 4 decision_markers: - Needs extensive data wrangling tags: @@ -20,6 +21,7 @@ original_data_state: Database (or multiple to-be-joined tables) source_links: - https://www.stratosphereips.org/datasets-ctu13 - https://www.kaggle.com/datasets/dhoogla/ctu13 +- https://doi.org/10.1016/j.cose.2014.05.011 needs_review: - ai_unverified type_adapter_id: curation-record-v1 @@ -29,6 +31,28 @@ type_adapter_id: curation-record-v1 ⚠️ **AI-FILLED — UNVERIFIED**: triaged by an AI (2026-08-12), no human has verified it. Check the verdict and the fields before relying on them. +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: the one network-traffic candidate with real captured +background traffic and a detection paper, and a natural grouped split by scenario; the labels need checking first. +- Source: Stratosphere Lab, CTU Prague, captured 2011, published with Garcia, Grill, Stiborek & Zunino, "An empirical + comparison of botnet detection methods", Computers & Security 45:100-123, 2014. Page: "The goal of the dataset was to + have a large capture of real botnet traffic mixed with normal traffic and background traffic"; 13 scenarios, each + one malware sample; labels Background, Botnet, C&C Channels, Normal ("we manually analyzed and label each scenario"); + bidirectional Argus NetFlows with labels; full pcaps not released; "These unidirectional NetFlows should not be used". +- Use case: a network operator flags infected hosts from flow records (the paper's detection task); fit direct. + Regime grouped by scenario (an unseen botnet), temporal within a scenario. +- Siblings and duplicates: `intrusion_detection` (`No`, Lennart: BCCC profiling, labels by attacker IP), `kddcup99`, + `unsw_nb15`, `cic_ids2017` (`No`, AHDS: generated traffic); CTU-13 differs by its real background traffic. Kaggle + mirror dhoogla/ctu13 is a re-export, not a separate source. +- Papers and pages: Garcia et al. 2014 (BClus, CAMNEP, BotHunter compared); widely reused in botnet-detection papers. +- Metric: the paper defines its own comparison methodology and "a novel error metric" for botnet detection that scores + per IP address over time windows (stratosphereips.org publication page; definition not read in full); deliberate + (MET-5); inputs the source IP and the flow start time, which ship in the NetFlows. To read in the paper before adding. +- Risks for /add-dataset: `Background` means unknown, not benign (drop or treat as unlabelled); labels assigned by + infected-host IP, and the same infected IPs recur across scenarios, so IP addresses must not be features (LEAK); + 13 scenarios make few groups; very large files. +- Priority: gap + (Grouped thin), use case +, source + (paper, lab page), shape + (millions of flows, mixed types), + risk - (label-by-IP shortcut, unlabelled background). + Thirteen botnet captures ('scenarios') taken at CTU University, Czech Republic, in 2011; per the canonical page the goal was "a large capture of **real** botnet traffic mixed with normal traffic and background traffic", with a specific malware sample executed per scenario. That makes it the one candidate in the network-IDS family whose background traffic is genuinely observed rather than profile-generated, so criterion 4B does not auto-fail it the way it does [[intrusion_detection]], [[kddcup99]] and [[unsw_nb15]] — hence 2nd Tier rather than No. **Must be checked before any work.** (1) The three-way label is Botnet / Normal / **Background**, and `Background` means *ground truth unknown*, not benign — most flows are Background, so the usable labelled subset may be small. (2) How labels were assigned: believed to be from the known infected-host IPs, which would be the same label-by-IP shortcut as the BCCC release — unverified, the source is Garcia et al., "An empirical comparison of botnet detection methods", Computers & Security 45:100-123, 2014, which I have not read. (3) The page states the unidirectional NetFlows "should not be used because they were outperformed by our second analysis" using bidirectional NetFlows — take the bidirectional ones. Generalising to an unseen botnet means a grouped split by scenario; within a scenario the flows are also temporal. diff --git a/curation/records/customer_personality_analysis.md b/curation/records/customer_personality_analysis.md index d6630230..381a30f6 100644 --- a/curation/records/customer_personality_analysis.md +++ b/curation/records/customer_personality_analysis.md @@ -3,9 +3,9 @@ unique_name: customer_personality_analysis name: Customer_Personality_Analysis checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- No Good Target / Scientific Discovery +- AHDS (Artifical/Handmade/Deterministic/Simulated) - Missing source information tags: - 2nd Tier / Scientfic Discovery @@ -30,6 +30,22 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16, PROV-3): the case the data comes from calls it a +mock sample of a hypothetical company. +- Source: the Kaggle upload imakash3011/customer-personality-analysis (2021; "provided by Dr. Omar Romero-Hernandez"; + "Target: Need to perform clustering") re-uploads the iFood CRM Data Analyst Case (hiring case of the iFood Brain team, + on GitHub since May 2020: `ml_project1_data.csv`, `iFood Data Analyst Case.pdf`). As published: 2,240 customers x 29 + columns (birth year, education, marital status, income, spend per product category over 2 years, purchases per + channel, `AcceptedCmp1`-`5`, `Complain`, `Z_CostContact`, `Z_Revenue` (constants 3 and 11), target `Response`). +- The case text: "you are presented a sample dataset, that mocks metainformation on the customer and on iFood campaign + interactions with that customer", and its company is hypothetical ("Consider a well-established company operating in + the retail food sector ... wines, rare meat products, exotic fruits"), not iFood (a food-delivery app). +- Use case, had the data been real: the CMO targets the sixth campaign at likely buyers; pilot of 2,240 random + customers, cost 6,720 MU, revenue 3,674 MU (3 MU per contact, 11 MU per sale), a cost matrix (MET-7). +- Siblings: `customer_personality_analysis_2` is the same Kaggle URL. The CC line ("source is not entirely clear ... + rather not use this task") points the same way. +- Reject check (AI, 2026-10-22): stands. SEL-16/PROV-3 by a source statement: the iFood case the upload copies says "you are presented a sample dataset, that mocks metainformation on the customer and on iFood campaign interactions" and sets a hypothetical company (https://github.com/nailson/ifood-data-business-analyst-test); the earlier CC line also advises against the task. + Well-known Kaggle 'Customer Personality Analysis' marketing dataset (~2240 rows, demographics, spend, campaign-response columns). It was published primarily as a segmentation/clustering exercise rather than a clean supervised task; the only natural target is the binary 'Response' (accepted last campaign), but as the curator notes the collection timing is unclear, raising leakage/temporal concerns. Source provenance is also fuzzy (single Kaggle re-upload, no original company). Maps weakly to criteria 1-2 and questionably to criterion 3. A human should verify whether 'Response' is a leak-free target, the true source, and the post-cleaning size before promoting; tentatively 2nd Tier. --- diff --git a/curation/records/customer_personality_analysis_2.md b/curation/records/customer_personality_analysis_2.md index d716f017..c5c4fd73 100644 --- a/curation/records/customer_personality_analysis_2.md +++ b/curation/records/customer_personality_analysis_2.md @@ -5,8 +5,8 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Missing source information - Duplicate +- AHDS (Artifical/Handmade/Deterministic/Simulated) tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -30,4 +30,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4; `AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16): the record points at +https://www.kaggle.com/datasets/imakash3011/customer-personality-analysis, the same upload as +`customer_personality_analysis` (whose link adds only `/data`); both end at the iFood CRM Data Analyst Case, whose text +calls the data a mock sample ("a sample dataset, that mocks metainformation on the customer"). Its verdict follows +`customer_personality_analysis`. +- Reject check (AI, 2026-10-22): stands. PROV-4: the same Kaggle upload as customer_personality_analysis (https://www.kaggle.com/datasets/imakash3011/customer-personality-analysis, the other record adds only /data); its verdict follows the parent. + This record points at the exact same Kaggle dataset URL (imakash3011/customer-personality-analysis) as 'customer_personality_analysis' and shares the same name, so it is a duplicate entry of that candidate. The same caveats apply: clustering-oriented data, unclear source, questionable supervised target. It should be deduplicated against the primary record rather than triaged independently. A human should confirm it is the same upload and merge/reject it. diff --git a/curation/records/customer_retention_in_the_media_industry_datathon.md b/curation/records/customer_retention_in_the_media_industry_datathon.md index 99c3a602..1aad67ad 100644 --- a/curation/records/customer_retention_in_the_media_industry_datathon.md +++ b/curation/records/customer_retention_in_the_media_industry_datathon.md @@ -4,8 +4,12 @@ name: customer-retention-in-the-media-industry-datathon checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 +decision_markers: +- Needs extensive data wrangling tags: - AI-Filled (Verify) +original_source: Kaggle domain: business & marketing required_split: - Temporal (NON-IID) @@ -15,6 +19,7 @@ original_data_state: Database (or multiple to-be-joined tables) source_links: - https://www.kaggle.com/competitions/customer-retention-in-the-media-industry-datathon/overview - https://www.kaggle.com/competitions/customer-retention-datathon-amsterdam-edition +- https://www.kaggle.com/competitions/kkbox-churn-prediction-challenge source_row: 1043 needs_review: - ai_unverified @@ -25,4 +30,25 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: the files look like a re-use of KKBox's WSDM Cup 2018 churn +data, a strong real churn task; the record should probably take KKBox as its source (PROV-8). +- Source: two Kaggle community datathons (June 2024 and an Amsterdam edition, October 2024, "using Google Cloud + Platform"), the same five files each: `members.csv` (427,921,437 bytes), `transactions.csv`, `user_logs.csv`, + `train_data.csv`, `kaggle_test_data.csv`. These names and the members size match KKBox's Churn Prediction Challenge + (Kaggle, WSDM Cup, launched 2017-09-18: `members_v3`, `transactions`, `user_logs`, `train`), so the first publication + is most likely KKBox 2017 (unverified: the datathon pages render no text and its files need an accepted entry). +- Use case: KKBox's retention team predicts whether a subscriber renews within 30 days after the subscription expires + (KKBox page; labeller script `WSDMChurnLabeller.scala` published); fit direct. Regime temporal (KKBox labels users + by expiry month: train February, test March 2017). +- Siblings and duplicates: no KKBox record in `curation/records/`; the two datathons share one file set (one record is + right). External collections: none found. +- Papers and pages: Kaggle kkbox-churn-prediction-challenge (WSDM 2018 Cup); solutions on GitHub (e.g. WSDM_2018). +- Metric: the datathons score ROC AUC (Kaggle API `evaluationMetric`), KKBox scored log loss (Kaggle API). The + original task's metric is log loss (MET-1, MET-4); inputs none. +- Risks for /add-dataset: heavy joins (user_logs 1.4 GB, transactions with dates after the cut-off must be filtered: + LEAK), a label leak was reported in KKBox v1 and the labels were re-issued (train_v2), and the datathon may have relabelled or + subsampled; confirm the source before choosing which files to use. +- Priority: gap 0, use case +, source + (KKBox, WSDM Cup), shape + (millions of users, temporal), risk - (joins, + known label leak in v1). + Kaggle datathon (media-industry customer retention / churn), a genuine real-world predictive task that is a natural binary churn classification. Churn challenges typically supply multiple tables (subscription, activity, demographics) needing a join, and a temporal split is often the intended protocol. This fits the in-scope tabular criteria well, but I have not verified the dataset is downloadable, its size, the exact target, or whether the two linked competitions share data. A human must confirm data availability, the precise target column, table layout, and split regime. diff --git a/curation/records/customer_segmentation_data.md b/curation/records/customer_segmentation_data.md index bfc2737c..a96b434a 100644 --- a/curation/records/customer_segmentation_data.md +++ b/curation/records/customer_segmentation_data.md @@ -7,7 +7,6 @@ suggestion: 'No' decision_markers: - Missing source information - AHDS (Artifical/Handmade/Deterministic/Simulated) -- No Good Target / Scientific Discovery tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -28,6 +27,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Missing source information`, `AHDS (Artifical/Handmade/Deterministic/Simulated)`; PROV-1, PROV-3, SEL-16): an anonymous Kaggle upload with no traceable origin, and its values look generated. +- Source: Kaggle `ravalsmit/customer-segmentation-data` (file created 2024-03-11, CC0), the only link. The description gives no origin: "This dataset provides comprehensive customer data suitable for segmentation analysis. It includes anonymized demographic, transactional, and behavioral attributes" (Kaggle API metadata); no company, paper, survey or collection named; no other first source found (web search on the file name and columns). +- As published (preview, 2026-10-22): 53,503 rows, 20 columns (`Customer ID`, `Age`, `Gender`, `Marital Status`, `Education Level`, `Geographic Information`, `Occupation`, `Income Level`, `Behavioral Data`, `Purchase History`, `Interactions with Customer Service`, `Insurance Products Owned`, `Coverage Amount`, `Premium Amount`, `Policy Type`, `Customer Preferences`, `Preferred Communication Channel`, `Preferred Contact Time`, `Preferred Language`, `Segmentation Group`). +- Generated (PROV-3): Indian states in `Geographic Information` with German (27%) and French (25%) as the most common `Preferred Language`; placeholder values (`Behavioral Data` = "policy5", `Insurance Products Owned` = "policy2"); `Policy Type` has the same distribution in every `Segmentation Group` (Group 33-35%, Business 25-27%, Individual 16-18%); 40,259 distinct `Customer ID` for 53,503 rows; the curator's note: "distributions seem very weird and too equal". +- Target: `Segmentation Group` (5 segments, 16-26%) is a segment label of unknown origin, not an observed outcome (SEL-3, SEL-4 if it came from clustering). +- Siblings and duplicates: not checked further (no origin to trace). +- Reject check (AI, 2026-10-22): stands. PROV-1 (Must), with PROV-3/SEL-16 as support. The Kaggle API description is the whole source: "This dataset provides comprehensive customer data suitable for segmentation analysis. It includes anonymized demographic, transactional, and behavioral attributes", plus marketing copy; no company, survey, paper or first uploader behind it, and no upstream found (https://www.kaggle.com/datasets/ravalsmit/customer-segmentation-data, metadata re-read 2026-10-22). The preview (Indian states with German/French as top languages, placeholder 'policy5' values) and the curator's note "distributions seem very weird and too equal" back the AHDS marker. + Kaggle 'Customer Segmentation Data' upload with no documented provenance; the curator's note ('distributions seem very weird and too equal') strongly suggests synthetic/handmade data. Segmentation is fundamentally a clustering exercise with no clear supervised target, so it fails criterion 3 (real predictive task) and likely criterion 4B (artificial data). Confidence in rejection is moderate-high. A human should confirm the data is synthetic and that no leak-free supervised target exists before final rejection. --- diff --git a/curation/records/dare_to_women_tech_shecures_ai_for_diabetes_prediction.md b/curation/records/dare_to_women_tech_shecures_ai_for_diabetes_prediction.md index 273dce2e..7f04429c 100644 --- a/curation/records/dare_to_women_tech_shecures_ai_for_diabetes_prediction.md +++ b/curation/records/dare_to_women_tech_shecures_ai_for_diabetes_prediction.md @@ -3,7 +3,8 @@ unique_name: dare_to_women_tech_shecures_ai_for_diabetes_prediction name: dare-to-women-tech-shecures-ai-for-diabetes-prediction checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 tags: - AI-Filled (Verify) collections: @@ -27,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a 10-day community hackathon whose page says nothing about where the health data comes from; the task (type 2 against gestational diabetes) is unclear as a decision. +- Source: Zindi hackathon "Dare to women Tech - SheCures: AI for Diabetes Prediction" (2025-04-14 to 2025-04-24, secret code required, Data Sharing CC-BY SA 4.0), run by the Dare to be Women Tech community for International Girls in ICT Day. As published: `Train.csv`, `Test.csv`, `VariableDefinitions.csv`; the Data Description page is empty, so rows, columns and the origin of the data are unknown (Zindi needs a login; read through the Zindi API, https://zindi.africa/competitions/dare-to-women-tech-shecures-ai-for-diabetes-prediction). +- Use case: "distinguishing between different types of diabetes (such as Type 2 and Gestational Diabetes) using health data", because "many cases are diagnosed late—or incorrectly" (Description page); no actor or moment named; fit approximate at best (pregnancy status separates gestational diabetes by definition, so the target may be near-trivial); regime IID (one row per patient assumed). +- Siblings and duplicates: no record of this competition (grep: shecures, dare-to-women); the diabetes records (`diabetes_prediction_dataset`, `pima_indians_diabetes_database`, `early_stage_diabetes_risk_prediction`, `diabetes_130_us`, ...) are candidates for the upstream, since community hackathons often re-use a Kaggle table: to check against the variable list (PROV-4). External collections: none found. +- Papers and pages: the competition page only; no paper, no top solutions found. +- Metric: accuracy, "The error metric for this challenge is Accuracy" (Evaluation page), the leaderboard metric (MET-4), hard labels; inputs none. +- Risks for /add-dataset: the origin of the data (PROV-1, PROV-3: a generated or re-uploaded table is likely), the target's meaning (USE-5), a pregnancy column that settles the class (SEL-5, TP-4). +- Priority: gap - (medical, binary, IID), use case - (no actor or decision), source - (no data description), shape - (unknown), risk - (unknown origin, unclear target): floor 1. + Zindi hackathon for diabetes prediction, a plausible real-world medical binary-classification task on tabular clinical/demographic features. The record carries no description, so size, feature set, and whether the data is a re-upload of a known (possibly synthetic) diabetes set are unknown. Many 'diabetes prediction' Kaggle/competition sets turn out to be synthetic or duplicates of the Pima set, so provenance is the key risk. A human must verify the data is real and original, its size, the target, and that there are no ethical/synthetic issues before confirming Yes. diff --git a/curation/records/data_science_zw_2022_fire_prediction.md b/curation/records/data_science_zw_2022_fire_prediction.md index 455f73ab..9b0c5fc3 100644 --- a/curation/records/data_science_zw_2022_fire_prediction.md +++ b/curation/records/data_science_zw_2022_fire_prediction.md @@ -4,9 +4,9 @@ name: data-science-zw-2022-fire-prediction checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Time-series (Regression) -- Time-series (Forecasting) tags: - AI-Filled (Verify) collections: @@ -33,6 +33,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: monthly burned area per grid cell in Zimbabwe with covariates, a 2022 rerun of the 2020 "Fighting Fire with Data" data; a forecasting-like panel that needs TP-10 before it is in scope. +- Source: first released in Zindi's "Fighting Fire with Data Hackathon" (2020-06-27, https://zindi.africa/competitions/fighting-fire-with-data-hackathon), reused for CMU-Africa (2020-10-31), this record's "Fire Extent Prediction Challenge" (Data Science Zimbabwe and Repliteq AI, 2022-11-23 to 2022-11-27, Zimbabwean residents only) and the 2024 "Wildfire Prediction Challenge" (slug `predict-fire-extent`); all four pages share the text, the files and the metric. So `year` 2022 is the rerun; the data was first published in 2020 (left as is, metadata is filled only where empty). As published: `Train.csv` (monthly rows per cell, 2001 to 2013), `Test.csv` (2014 to 2016, no target), `variable_definitions.csv`; target `burn_area`, "the percentage of the area that has been burned in a given month"; covariates from Google Earth Engine (TerraClimate climate, MODIS MCD12Q1 land cover, GPW population density, GMTED2010 elevation, TRMM precipitation) that "extends into the test period". The page says both 533 areas over Zimbabwe and "3821 area squares across the DRC": a copy error to check. +- Use case: "Figuring out the dynamics that influence where and when these fires occur ... predicting how these dynamics will play out in the future, under different climatic conditions" (Description); no actor or decision named; fit approximate (fire management planning, not stated); regime temporal (train to 2013, test 2014 to 2016, as the competition did), with cells repeated in both. +- Siblings and duplicates: `predict_fire_extent` and `wildfire_prediction_challenge` (both `AI (UNVERIFIED)`) point at reruns of the same data: one record should carry it and the others become `Duplicate` (PROV-4); this record already lists the 2020 first release. `forest_fires` is a different dataset (Portugal). External collections: none found. +- Papers and pages: the four Zindi pages (read through the Zindi API); no paper. +- Metric: RMSE, "The error metric for this competition is the Root Mean Squared Error" on the proportion 0 to 1 (Evaluation page of every run), the leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: a panel of future values of one series per cell, so SEL-10 against SEL-11: TP-10 (does a model on the covariates beat each cell's monthly profile?); cell id and lat/lon carry the history (LEAK, SPLIT); the 533 against 3821 cells. +- Priority: gap - (environmental, regression, temporal not thin), use case - (no actor), source + (the page names every covariate source), shape + (about 533 cells x some 190 months from 2000-2001 to 2016, a temporal structure), risk - (forecasting-like target, SEL-10). The old `Time-series (Forecasting)` marker is dropped as a verdict marker: forecasting is not settled from the page and waits for TP-10; `Time-series (Regression)` stays as the concern. + Zindi Zimbabwe competition to predict burned area, a regression target. Burned-area prediction is usually spatio-temporal (satellite/weather over time and locations), so this risks being a forecasting/time-series task (excluded) rather than a fixed tabular regression; it could be acceptable if it is a per-record regression needing only a temporal/grouped split. The record has no feature/structure detail, so the protocol is undetermined. A human must inspect whether the task is a horizon forecast (reject) or a fixed predictive regression with a temporal/spatial split, plus size and feature modality. --- diff --git a/curation/records/data_scientist_salary.md b/curation/records/data_scientist_salary.md index 325d448d..08f21ded 100644 --- a/curation/records/data_scientist_salary.md +++ b/curation/records/data_scientist_salary.md @@ -4,6 +4,7 @@ name: data_scientist_salary checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Data Quality Issue - NLP (Text) @@ -31,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real 2018-2019 MachineHack task (salary band of an Indian data-science job posting), but every feature is a string and the job description is the main signal, so it sits at the edge of SEL-12. +- Source: MachineHack hackathon "Predict The Data Scientists Salary In India" (2018-2019; "a classification hackathon", Analytics India Magazine, 2019-01-24, https://analyticsindiamag.com/deep-tech/machinehack-winners-how-industry-mavericks-predicted-data-scientists-salary-and-won-big); used in Shi et al. 2021, "Benchmarking multimodal AutoML for tabular data with text fields" (arXiv:2111.02705), and uploaded to OpenML 46664 from that paper (2025-02-19). As published on OpenML: 15,841 rows (the competition train file had 19,802), 6 string features (`experience`, `job_description`, `job_desig`, `job_type`, `key_skills`, `location`), target `salary` with 6 bands, majority class 22.8%, 15,520 missing values. +- Use case: estimate the salary band of a posting from its text, skills and location (a job seeker or recruiter, not named by the source); fit approximate; regime IID (no posting date ships). +- Siblings and duplicates: no other record of this data (grep: 46664, salary in india). External collections: Shi et al. 2021 multimodal benchmark, TabSTAR. +- Papers and pages: the MachineHack page (not readable without login), Shi et al. 2021. +- Metric: none found (the MachineHack evaluation page could not be read; the AIM article names none; Shi et al. report accuracy for all classification sets, a habit), so the fallback `log_loss` (MET-8). +- Risks for /add-dataset: text-dominated (SEL-12, `NLP (Text)`): judge whether tabular models are competitive; the curator's note that descriptions are cut after the first blurb (as published, so a note rather than repairable damage, CLEAN); `experience` as a range string to parse; salary bands are ordered (a regression on band midpoints is not what the source asked). +- Priority: gap - (technology, multiclass, IID not thin), use case - (no actor named), source + (benchmark paper and OpenML description), shape + (about 16k rows, 6 features), risk - (text-dominated). + MachineHack 'Predict the Data Scientist's Salary in India' hackathon (OpenML id 46664), framing salary as a multiclass classification rather than regression. The curator flags that the free-text job-description fields are truncated to the first blurb, a data-quality defect, and that the task is dominated by string preprocessing, pushing it toward an NLP-heavy rather than representative tabular problem. It is a real-ish task but with notable caveats. A human should verify how much signal survives the text truncation, the post-cleaning size, and whether it is representative tabular ML or text-dominated; 2nd Tier for now. --- diff --git a/curation/records/data_wave_competition_lab.md b/curation/records/data_wave_competition_lab.md index 0bbad47f..7fad5804 100644 --- a/curation/records/data_wave_competition_lab.md +++ b/curation/records/data_wave_competition_lab.md @@ -3,10 +3,9 @@ unique_name: data_wave_competition_lab name: data-wave-competition-lab checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +26,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`; PROV-4, PROV-6): a one-day rerun of Ghana's Indigenous Intel Challenge on the same Smart Indigenous Weather App data and task, recorded as `ghanas_indigenous_intel` (No (Retired), Andrej; Too Small, Data Quality Issue). +- Source: Zindi hackathon "Data Wave - Competition Lab" (2025-11-29, 08:00 to 14:45 UTC, secret code required, 500 Zindi points; read through the Zindi API, https://zindi.africa/competitions/data-wave-competition-lab). Its data files include `ghana-indigenous-intel-challenge20250815-4756-1bz0qx9.zip`, the archive of the parent challenge (https://zindi.africa/competitions/ghana-indigenous-intel-challenge, started 2025-08-14), next to `train.csv`, `test.csv`, `SampleSubmission.csv` and `Indigenous_Weather_Forecasting-Starter_Notebook.ipynb`. +- Same data and task: "The data for this challenge was collected using the Smart Indigenous Weather App ... Data was collected from local farmers in the Pra River Basin of Ghana"; "Your task is to build a classification model that predicts the type of rainfall—heavy, moderate, or small—expected in the next 12 to 24 hours, based solely on indigenous ecological indicators submitted by trained farmers" (Description and Data Description pages), hosted by RAIL at KNUST: the parent challenge's description, data and target. +- Metric: F1 Score (Evaluation page). The page says the variables "were and continue to be collected", so the rerun may carry more rows; added rows of the same collection and task are a version, not a distinct task (PROV-6). If it does carry many more farmer-days, that bears on the sibling's `Too Small`, not on a second record. +- Siblings: `ghanas_indigenous_intel` (CC 2026-10-01, Lennart: retired for Too Small and a likely censoring bias in the rain-gauge label; "Revisit if a cleaner or larger release appears"). A later rerun exists too (`ghana-indigenous-intel-challenge-cameroon-community-hack`, 2026-10-26). +- Reject check (AI, 2026-10-22): stands. PROV-4/PROV-6 (Must), source confirmed (PROV-5). The Zindi API names ghana-indigenous-intel-challenge as this hackathon's parent, and every data file is byte-identical in size to the parent's (train.csv 870,530; test.csv 202,185; SampleSubmission.csv 60,111; the parent's zip ghana-indigenous-intel-challenge20250815-4756-1bz0qx9.zip 186,984), with the same description, data description and F1 metric (https://api.zindi.world/v1/competitions/data-wave-competition-lab vs .../ghana-indigenous-intel-challenge). Same data, same task: a duplicate of ghanas_indigenous_intel (No (Retired)). The screener's open question about added rows is settled: there are none. + Zindi 'data-wave-competition-lab' with no description, comments, or metadata in the record, so the task type, target, domain, size, and modality are all unknown. Per the instructions, too little information is not by itself grounds for rejection. The only signal is that it is a Zindi competition (often genuine African real-world tabular tasks, but also sometimes image/forecasting). A human must open the competition page to determine the task, target, data modality, size, and split before any verdict. diff --git a/curation/records/dataset_from_a_novel_classification_algorithm_for_customer_churn_prediction_based_on_hybrid_ensemble_fusion_model.md b/curation/records/dataset_from_a_novel_classification_algorithm_for_customer_churn_prediction_based_on_hybrid_ensemble_fusion_model.md index b68c76d0..ddd0d22a 100644 --- a/curation/records/dataset_from_a_novel_classification_algorithm_for_customer_churn_prediction_based_on_hybrid_ensemble_fusion_model.md +++ b/curation/records/dataset_from_a_novel_classification_algorithm_for_customer_churn_prediction_based_on_hybrid_ensemble_fusion_model.md @@ -3,7 +3,8 @@ unique_name: dataset_from_a_novel_classification_algorithm_for_customer_churn_pr name: Dataset from A novel classification algorithm for customer churn prediction based on hybrid Ensemble-Fusion model checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Data Requested tags: @@ -29,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a private company churn table (2015-2022) that the paper barely describes; the data is "available on request" and was requested on 07/03/2026. +- Source: He and Ding (2024), "A novel classification algorithm for customer churn prediction based on hybrid Ensemble-Fusion model", Scientific Reports (doi:10.1038/s41598-024-71168-x, PMC11364882). The data: "a private dataset of the customer production line system of the Company from 2015 to 2022 is used, where 80% of the data is used for training and 20% of the data is used for testing" (Experiments); "Data is available on request from the author (Chenggang He)" (Data availability). Rows, columns, the target's definition and the industry are not stated. +- Use case: "facilitate the enterprises to take actions quickly and efficiently to retain customers", predicting "whether customers will churn in the following months" (Introduction, Fig. 2 text); actor a company's retention team, fit direct if the data matches; regime likely temporal (seven years of records, churn in the following months), not the paper's random 80/20; `required_split` left as is until the data is seen. +- Siblings and duplicates: the other churn-paper records (`dataset_from_feature_selection_based_dynamic_transfer_ensemble_model_for_customer_churn_prediction`, `dataset_from_research_model_of_churn_prediction_based_on_customer_segmentation_and_misclassification_cost_in_the_context_of_big_data`) are other companies' data; a private table cannot duplicate a public churn set. External collections: none. +- Papers and pages: the paper; its predecessor He et al. 2021 (ICTAI) on an "intelligent machine learning system" may use the same data. +- Metric: accuracy, AUC and F1 reported side by side, none named as the criterion, so the fallback `roc_auc` (MET-5, MET-8). +- Risks for /add-dataset: no access (`Data Requested`, no answer recorded); unknown features and churn definition (USE-5); the paper says the collected data had "no labels to mark customer churn" before cleaning, so the label is the authors' construction. +- Priority: gap - (business, binary), use case + (retention named), source - (the paper does not describe the table), shape - (unknown), risk 0. + Dataset underlying a 2024 Nature Scientific Reports paper on customer churn prediction via a hybrid ensemble-fusion model; churn is a standard real-world tabular binary-classification task. The record notes the data was formally requested from the authors on 07/03/2026, so it is not yet in hand and may be one of the common public telecom-churn sets (potential duplicate risk). It maps well to the in-scope criteria if the data is original and obtainable. A human must confirm the data arrives, verify it is not a re-upload of an existing TabArena churn set, and check size/target. --- diff --git a/curation/records/dataset_from_feature_selection_based_dynamic_transfer_ensemble_model_for_customer_churn_prediction.md b/curation/records/dataset_from_feature_selection_based_dynamic_transfer_ensemble_model_for_customer_churn_prediction.md index feae60e5..286c9691 100644 --- a/curation/records/dataset_from_feature_selection_based_dynamic_transfer_ensemble_model_for_customer_churn_prediction.md +++ b/curation/records/dataset_from_feature_selection_based_dynamic_transfer_ensemble_model_for_customer_churn_prediction.md @@ -3,7 +3,8 @@ unique_name: dataset_from_feature_selection_based_dynamic_transfer_ensemble_mode name: Dataset from Feature-selection-based dynamic transfer ensemble model for customer churn prediction checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Data Requested - Duplicate @@ -20,6 +21,7 @@ problem_type: Binary Classification original_data_state: One Table source_links: - https://link.springer.com/article/10.1007/s10115-013-0722-y +- '# AI note 2026-10-22: OpenML 46444 is credit_risk_china (bank personal credit risk, 27,522 rows, target five_categories), not China-Churn' - https://www.openml.org/search?type=data&id=46444 source_row: 797 needs_review: @@ -31,6 +33,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the paper's first dataset is a duplicate (curator's note) and its second, "China-Churn", is not public; the OpenML link the record found is a different dataset, so the old `No` rests only on dataset (1). +- Source: Xiao, Xiao, Huang, Liu and Wang (2014 online, 2015 print), "Feature-selection-based dynamic transfer ensemble model for customer churn prediction", Knowledge and Information Systems 43, 29-51 (doi:10.1007/s10115-013-0722-y, closed access; abstract: two churn datasets). Dataset (1) is the churn set already in TabArena (curator's note); dataset (2) "China-Churn" was requested on 07/03/2026. +- The OpenML link: OpenML 46444 is `credit_risk_china`, "Multi-classification assessment of bank (china) personal credit risk based on multi-source information fusion" (27,522 rows, 27 features, target `five_categories`, 92.7% majority), uploaded by "AutoML Benchmark" with the citation "Metaverse Financial Transactions Dataset". It is not China-Churn and not churn at all; the `TabSTAR` collection tag likely comes from that link. A credit-risk candidate in its own right (no record found for it), with its own provenance question. +- Use case: a telecom operator's retention campaign (transfer between customer groups); fit direct if China-Churn arrives; regime unknown. +- Siblings and duplicates: the TabArena churn dataset (curator's note); the other churn-paper records are other data. External collections: none for China-Churn. +- Papers and pages: the KAIS paper (full text not read: closed access). +- Metric: not checked (closed access), so the fallback `roc_auc` until read (MET-1, MET-8). +- Risks for /add-dataset: no access to China-Churn; whether it is a cut of a known telecom set (PROV-4). +- Priority: gap -, use case + (retention), source - (closed paper, data not described where we could read), shape - (unknown), risk - (dataset (1) duplicate, wrong link): floor 1. `Duplicate` stays for dataset (1). + Two-part record for a churn-prediction paper: the first linked dataset is explicitly noted as a duplicate of the churn dataset already in TabArena, and the second (a 'China-Churn' / OpenML id 46444 set) was data-requested with uncertainty about whether it is the same or distinct. Given the explicit duplicate flag on the primary dataset, this should be rejected as a duplicate; only if the China-Churn set proves genuinely distinct and original would a separate Yes be warranted. A human must determine whether the OpenML 46444 set is a distinct, non-duplicate source; if so it could be reconsidered. --- diff --git a/curation/records/dataset_from_research_model_of_churn_prediction_based_on_customer_segmentation_and_misclassification_cost_in_the_context_of_big_data.md b/curation/records/dataset_from_research_model_of_churn_prediction_based_on_customer_segmentation_and_misclassification_cost_in_the_context_of_big_data.md index df43af54..3c0198da 100644 --- a/curation/records/dataset_from_research_model_of_churn_prediction_based_on_customer_segmentation_and_misclassification_cost_in_the_context_of_big_data.md +++ b/curation/records/dataset_from_research_model_of_churn_prediction_based_on_customer_segmentation_and_misclassification_cost_in_the_context_of_big_data.md @@ -3,7 +3,8 @@ unique_name: dataset_from_research_model_of_churn_prediction_based_on_customer_s name: Dataset from Research Model of Churn Prediction Based on Customer Segmentation and Misclassification Cost in the Context of Big Data checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 3 decision_markers: - Data Requested tags: @@ -20,6 +21,7 @@ problem_type: Binary Classification original_data_state: One Table source_links: - http://dx.doi.org/10.4236/jcc.2015.36009 +- https://www.scirp.org/journal/PaperDownload.aspx?paperID=56907 source_row: 796 needs_review: - ai_unverified @@ -30,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 3: a large real Chinese telecom churn snapshot (about 1M customers, 25 attributes, 2.8% churn) with a clear retention use case, but private: no data availability statement, requested on 07/03/2026. +- Source: Liu and Zhuang (2015), "Research Model of Churn Prediction Based on Customer Segmentation and Misclassification Cost in the Context of Big Data", Journal of Computer and Communications 3, 87-93 (doi:10.4236/jcc.2015.36009, open access, CC BY). Sec. 3.1: "we select customer consumption listings from November 2014 to January 2015 and churn data for February 2015 as our model data. This data contains a total of 1,048,575 records and 25 customer attributes like customer ID, costs, traffic, voice package information etc."; Sec. 3.2: 1,016,610 records after removing abnormal ones, "including 27,925 customer churn records" (2.8%), 19 attributes kept. +- Use case: "Companies are able to previously recognize the consumers who are ... move toward a competitor by churn prediction, then, take appropriate marketing campaigns to maintain and retain these customers" (Sec. 1); actor the operator's retention team, moment end of January 2015, decision whom to target; fit direct; regime IID (one snapshot, one row per customer). +- Siblings and duplicates: no other record of this data; the other churn-paper records are other companies. External collections: none. +- Papers and pages: the paper only. +- Metric: "accuracy and coverage" (coverage is the churn recall) on a 3:1 resampled set with an unstated misclassification cost (Sec. 3.3), threshold metrics without a fixed threshold or cost (MET-6), so the fallback `roc_auc` (MET-8). +- Risks for /add-dataset: no access (`Data Requested`); 1,048,575 is the Excel row limit (2^20 - 1), so the table the authors had was likely cut at export: a selective sample (PROV-6, `Not Representative`) to ask about; the K-means segment is the authors' construction (not a feature to ship as given). +- Priority: gap - (business, binary, IID), use case + (retention named, direct), source + (the paper documents window, size and churn rate), shape + (1M rows, 25 attributes), risk - (the Excel cut). + Dataset behind a 2015 JCC paper on churn prediction using customer segmentation and misclassification cost; churn is an in-scope real-world tabular binary-classification task. The data was requested from the authors on 07/03/2026 and is not yet available, and older churn papers frequently use the widely-circulated public telecom-churn data (duplicate risk). It plausibly fits the criteria pending data delivery. A human must confirm the data is obtainable, original (not a re-upload of an existing TabArena/UCI churn set), and check its size and target. --- diff --git a/curation/records/dataset_from_uncertainty_guided_model_selection_for_tabular_foundation_models_in_biomolecule_efficacy_prediction.md b/curation/records/dataset_from_uncertainty_guided_model_selection_for_tabular_foundation_models_in_biomolecule_efficacy_prediction.md index 3d1136d3..0154ae76 100644 --- a/curation/records/dataset_from_uncertainty_guided_model_selection_for_tabular_foundation_models_in_biomolecule_efficacy_prediction.md +++ b/curation/records/dataset_from_uncertainty_guided_model_selection_for_tabular_foundation_models_in_biomolecule_efficacy_prediction.md @@ -4,6 +4,7 @@ name: Dataset from Uncertainty-Guided Model Selection for Tabular Foundation Mod checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 4 decision_markers: - Data Requested tags: @@ -20,6 +21,7 @@ problem_type: Regression original_data_state: One Table source_links: - https://arxiv.org/abs/2510.02476 +- https://doi.org/10.1038/nbt1118 source_row: 865 needs_review: - ai_unverified @@ -30,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: siRNA knockdown efficacy on the Huesken et al. (2005) library, 2,361 siRNAs over 29 target mRNAs, a grouped regression in a thin domain; the paper's two patent-derived sets are not public. +- Source: the efficacy data was first published by Huesken et al. (2005), "Design of a genome-wide siRNA library using an artificial neural network", Nature Biotechnology (doi:10.1038/nbt1118, supplementary data); this record's paper, Li, McCarthy, Zhang and Young (2025), arXiv:2510.02476, re-uses it ("We divided the Huesken dataset (2361 data points) into 29 subsets, each corresponding to a different mRNA target") and adds Target1 (patents of institutions A, B, C: 295, 366, 9 points) and Target2 (a patent of institution D: 252 points), with no data availability statement. So `year` 2025 is the re-use; the data dates from 2005 (left as is, metadata is filled only where empty). Features in the paper: 574 per siRNA (361 one-hot positions of a 57-nt window, 189 trimer counts, 24 thermodynamic values); target the knockdown efficacy. +- Use case: a researcher designing siRNAs for a new gene picks the candidates to synthesise and test (Huesken: a genome-wide library designed by a network); fit direct; regime grouped by target mRNA (a new gene at prediction time; the paper used 5-fold CV on Huesken and few-shot on new targets). +- Siblings and duplicates: no record of Huesken or siRNA data (grep: huesken, sirna, knockdown). External collections: none known; Huesken is a standard set of siRNA efficacy predictors. +- Papers and pages: Huesken et al. 2005; Li et al. 2025 (TabPFN on sequence features beats specialised siRNA predictors, their claim). +- Metric: MAE and Pearson correlation reported side by side, correlation emphasised in the conclusion but not named as the criterion (MET-5); Huesken's own evaluation not read; so the fallback `rmse` (MET-8) unless the curator takes Pearson. +- Risks for /add-dataset: sequence data (SEL-12): one-hot and k-mer features where sequence models may win (the paper argues tabular wins); the features are a design choice of ours, to be derived from the sequences as published; the group column (target mRNA) must ship; the `New IID` tag contradicts the grouped regime; Target1 and Target2 not available (`Data Requested`, 14/03/2026). +- Priority: gap + (biology & life sciences and grouped are thin), use case + (siRNA design), source + (Nature Biotechnology paper), shape + (2,361 rows, grouped structure), risk - (sequence modality, SEL-12). + Dataset(s) from a 2025 arXiv paper on tabular foundation models for biomolecule efficacy prediction; this is a genuine tabular predictive task (efficacy regression/classification on molecular descriptor features) and explicitly relevant to tabular ML. The record notes the data was preprocessed from prior work, was requested on 14/03/2026, and could possibly be reconstructed manually. Risks: the features may be derived/transformed descriptors, and a grouped (by scaffold/assay) split may be required. A human must verify data availability, that the features are not lossy/irreversible transforms, the exact target, and the appropriate split. --- diff --git a/curation/records/deep_learning_indabax_guinee_2024_hackathon.md b/curation/records/deep_learning_indabax_guinee_2024_hackathon.md index c4d33db1..89e71faa 100644 --- a/curation/records/deep_learning_indabax_guinee_2024_hackathon.md +++ b/curation/records/deep_learning_indabax_guinee_2024_hackathon.md @@ -3,10 +3,9 @@ unique_name: deep_learning_indabax_guinee_2024_hackathon name: deep-learning-indabax-guinee-2024-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,4 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-run of the UmojaHack Africa 2023 Carbon Dioxide Prediction Challenge (BEGINNER), whose data `carbon_dioxide_prediction_challenge` already carries. +- Source: Zindi "Deep Learning IndabaX Guinee 2024 Hackathon", 2-4 November 2024 (https://zindi.africa/competitions/deep-learning-indabax-guinee-2024-hackathon); the Zindi API names its parent `umojahack-africa-2023-beginner-challenge` (18 March 2023), and both ship the same files (Train.csv 93,563,905 bytes, Test.csv 35,229,807, SampleSubmission.csv 725,308, the same "EY_Carbon_Prediction_StarterNotebook.ipynb") and the same data description word for word. +- As published: weekly Sentinel-5P pollutant columns (SO2, CO, NO2, HCHO, aerosol index, O3, cloud) from January 2019 to November 2022 for about 500 locations in South Africa; target CO2 emissions from GRACED and EDGAR; "The test set is from a different province to the train set". +- Siblings and duplicates: `carbon_dioxide_prediction_challenge` lists https://zindi.africa/competitions/umojahack-africa-2023-beginner-challenge among its links, so the first publication is recorded there; that record decides the task (its own open questions: remote-sensing modality, spatio-temporal framing). The `data_type` "Structured" and the deep-learning name do not matter for the verdict. +- Metric: RMSE (Zindi Evaluation page), the same as the parent. +- For the main session: the year of first publication is 2023 (the parent), not 2024. +- Reject check (AI, 2026-10-22): stands. Zindi API (api.zindi.world/v1/competitions/deep-learning-indabax-guinee-2024-hackathon) names parent umojahack-africa-2023-beginner-challenge; Train.csv 93,563,905 and Test.csv 35,229,807 bytes are identical to the parent's; carbon_dioxide_prediction_challenge already lists the parent link. PROV-4 with PROV-5 confirmed. + Zindi 'Deep Learning IndabaX Guinee 2024' hackathon with no description or metadata in the record. The 'deep learning' framing of an IndabaX hackathon raises the likelihood that the data is image/text/audio rather than tabular, which would make it out of scope, but this is unconfirmed. Too little information to decide, and insufficient info alone is not grounds for rejection. A human must open the competition to determine the data modality (tabular vs image/text), the task and target, size, and split. diff --git a/curation/records/deepflow_ai_hackflow.md b/curation/records/deepflow_ai_hackflow.md index 2b2e649b..01e4724d 100644 --- a/curation/records/deepflow_ai_hackflow.md +++ b/curation/records/deepflow_ai_hackflow.md @@ -4,17 +4,20 @@ name: deepflow-ai-hackflow checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- TBD +- AHDS (Artifical/Handmade/Deterministic/Simulated) - Missing source information tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2023' required_split: - '?' -problem_type: TBD +problem_type: Regression +original_data_state: One Table source_links: - https://zindi.africa/competitions/deepflow-ai-hackflow/data source_row: 974 @@ -27,4 +30,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a building-energy regression whose columns read like the inputs of a parametric energy simulation, with no named origin; likely `No` under SEL-16 or PROV-1 once the data is seen, but the page does not say it is simulated. +- Source: Zindi hackathon "DeepFlow - AI HackFlow", Tunisia, 6-7 May 2023 (https://zindi.africa/competitions/deepflow-ai-hackflow); as published: "78 000 entries", Train.csv (30.3 MB) and Test.csv (12.2 MB), target "Operational Energy"; columns `building`, `File` ("a file name for the building shape"), setpoints, cooling and heating COP, WWR, thermal resistances (wall, roof, floors, windows), infiltration, boiler efficiency, internal mass, permeability, occupancy, `Town`, and "random load measures"; IDs like `Building_10005_Town_2`. No data provider is named; the licence is CC-BY SA 4.0 (rules page). +- Use case: the page names designers optimising a building "before building construction" (early-stage energy prediction); that is a surrogate for a simulation, not a measured outcome; regime grouped by `Town` would be the natural check if the data were real. +- Siblings and duplicates: none in `curation/records/` (`nyc_building_energy_and_water_data_disclosure_for_local_law_84_2022_present` is measured NYC benchmarking data, another source). External collections: none found; two web searches on the column names found no upstream dataset. +- Papers and pages: none found. +- Metric: RMSE, Zindi Evaluation page ("The error metric for this competition is the Root Mean Squared Error"); a leaderboard choice (MET-4); inputs none. +- Risks for /add-dataset: PROV-3 / SEL-16 (a design-parameter grid with a "building shape" file per row and "random load measures" points at an EnergyPlus-style parametric study: check the value grids, duplicates per parameter and whether the target is a deterministic function of the inputs); PROV-1 (no provider named); `EUI` is energy per floor area and with `Total Floors Area` may give the target away (LEAK). +- Metadata filled from the page: problem type Regression, year 2023, one table. +- Priority: gap 0, use case 0 (design-stage surrogate), source 0 (no provider, no codebook beyond the page), shape + (78,000 rows, about 30 columns), risk - (likely simulated, EUI leak): 1. + Zindi 'deepflow-ai-hackflow' competition with no description, comments, or metadata in the record, so task type, target, domain, modality, and size are all unknown. The 'deep ... flow' naming gives no reliable signal and Zindi competitions span tabular, image, and forecasting tasks. Insufficient information to assess, and that alone is not a rejection reason. A human must inspect the competition page to determine modality, task, target, size, and split. diff --git a/curation/records/delivery_rider_response_challenge_by_galsenai.md b/curation/records/delivery_rider_response_challenge_by_galsenai.md index b4e5bd9b..3d7172d1 100644 --- a/curation/records/delivery_rider_response_challenge_by_galsenai.md +++ b/curation/records/delivery_rider_response_challenge_by_galsenai.md @@ -3,7 +3,8 @@ unique_name: delivery_rider_response_challenge_by_galsenai name: Delivery Rider Response Challenge by GalsenAI checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Out-of-scope Task (CTR/RecSys/Ranking) tags: @@ -13,10 +14,11 @@ collections: original_source: Zindi year: '2021' domain: business & marketing -problem_type: Other +problem_type: Multiclass Classification original_data_state: Database (or multiple to-be-joined tables) source_links: - https://zindi.africa/competitions/indabax-senegal-2021 +- https://zindi.africa/competitions/umojahack-africa-2021-2-sendy-challenge-intermediate source_row: 1004 needs_review: - ai_unverified @@ -27,6 +29,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real dispatch-response task (accept, decline or ignore) from Sendy's platform; the old reject as a ranking task rests on a reading, so the curator decides whether an offer-acceptance label falls under SEL-9. +- Source: first published as Zindi "UmojaHack Africa 2021 #2: Sendy - Delivery Rider Response Challenge (INTERMEDIATE)", 27-28 March 2021 (https://zindi.africa/competitions/umojahack-africa-2021-2-sendy-challenge-intermediate); this record's IndabaX Senegal hackathon (GalsenAI, 23-29 August 2021) is a re-run with the same files (Train.csv 28,686,419 bytes, Test.csv 12,096,330, Riders.csv 49,652 in both Zindi APIs). As published: "a subset of over 200,000 dispatches", "only includes direct orders (i.e. Sendy 'express' orders) with bikes in Nairobi", "fully anonymized while preserving the distribution"; Train/Test (one row per dispatch: order, client, rider ids, rider_amount, licence and carrier flags, pickup, drop-off and rider coordinates at dispatch, `dispatch_day` (day of month), `dispatch_day_of_week`, `dispatch_time`) joined to Riders.csv (rider information, not described on the page) on `rider_id`; target 0 ignore, 1 decline, 2 accept. +- Use case: Sendy's dispatch system, at the moment an order is sent to a rider, picks the rider most likely to accept ("Picking the best rider to service the order will improve the experience of the customer"); fit direct for the response label; regime grouped (by order, several dispatches each, or by rider) or temporal; the times carry day of month and weekday but no month or year, so a temporal split may not be possible (SPLIT-7); Zindi's own split is not described. +- Siblings and duplicates: no record for the UmojaHack 2021 #2 original, so this record is the dataset's only one (add the first-publication link, done); `sendy_logistics_challenge` and `sendy_logistics_challenge_2` are Sendy's 2019 ETA regression, other data and another task (PROV-7). External collections: none found. +- Papers and pages: no paper; the two Zindi pages only; top solutions not read. +- Metric: accuracy on hard labels, Zindi Evaluation page ("The evaluation metric for this challenge is Accuracy ... Where ignore is 0, decline is 1 and accept is 2"); a leaderboard choice (MET-4); inputs none (MET-1, MET-5, MET-6). +- Risks for /add-dataset: SEL-9 (an offer-acceptance label used to rank riders is close to a click-through task; the old marker stays as that concern); Riders.csv may hold per-rider aggregates (counts, ratings, acceptance rates) computed over the whole period, which would leak the target (LEAK; the page does not describe the file); "anonymized while preserving the distribution" may mean perturbed values (PROV-3); a join (Needs extensive data wrangling is not warranted for one key). +- Problem type changed from `Other` (AI-filled) to `Multiclass Classification`: the source names three classes. +- Priority: gap 0 (business & marketing, multiclass, temporal not thin), use case + (Sendy's dispatch decision, named by the source), source 0 (competition page only, anonymised subset), shape + (tens of thousands of dispatches, ids, coordinates, times and a rider table), risk - (SEL-9 doubt, rider-metric leak): 2. + Zindi/GalsenAI IndabaX Senegal 2021 challenge whose stated goal is to predict the best delivery rider for an order placed via the Sendy logistics company. Matching/selecting the best rider for an order is a ranking/recommendation-style task, which is explicitly excluded under criterion 3 (no recommendation/ranking). Even if it could be reframed as a per-rider accept/decline classification, the framing as described is out of scope. A human could confirm whether the actual target is a simple binary rider-response label (which might rescue it), but as described it is a reject. --- diff --git a/curation/records/diabetes_prediction_dataset.md b/curation/records/diabetes_prediction_dataset.md index 16c6277c..3eaf1420 100644 --- a/curation/records/diabetes_prediction_dataset.md +++ b/curation/records/diabetes_prediction_dataset.md @@ -5,10 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Ethical Issue -- Data Quality Issue - Missing source information -- AHDS (Artifical/Handmade/Deterministic/Simulated) tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -29,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Missing source information`, PROV-1): an anonymous Kaggle upload whose only provenance is a generic paragraph about EHRs; no provider, institution, paper or first upload upstream is named, and the curator's CC below already calls the source missing and the data likely fake. +- Source: Kaggle "Diabetes prediction dataset" by user iammustafatz (https://www.kaggle.com/datasets/iammustafatz/diabetes-prediction-dataset, dataset id 3102947, year of first version not checked), licence "copyright-authors"; as published: one table with gender, age, hypertension, heart_disease, smoking_history, bmi, HbA1c_level, blood_glucose_level and the binary `diabetes`. +- What the uploader says about the source (Kaggle metadata "sources"): "EHRs were collected from multiple healthcare providers and aggregated into a single dataset. The data was then cleaned and preprocessed" — no provider, country, period or ethics approval, and no reply naming one. +- PROV-2 does not keep it: there is no source to point to beyond the uploader, and the CC line reports a Kaggle discussion (588830) arguing the data is wrong or fake; if the data is generated it is also a PROV-3 case (`AHDS`), not measured here. +- Previous markers `Ethical Issue`, `Data Quality Issue` and `AHDS` are not decided by the source's words: proposed to drop `Ethical Issue` (no objection by subjects or creators is stated, SEL-1); `AHDS` and `Data Quality Issue` stay only if the reject-checker or a measurement backs them (PROV-3 tests not run). +- Siblings: many diabetes records (`cdc_diabetes_health_indicators`, `pima_indians_diabetes_database`, `early_stage_diabetes_risk_prediction`, `nhanes_undiagnosed_diabetes`, ...) have named sources; none is this table's upstream as far as checked by name. +- Reject check (AI, 2026-10-22): stands. PROV-1 (no upstream beyond the anonymous uploader) and PROV-2 cannot rescue it, because the data fails PROV-3 when measured (checker download, 100,000 rows): HbA1c_level takes only 18 values and blood_glucose_level only 18, each level near-uniform (~7,500 rows), and the label is tied to the levels: every row with HbA1c >= 6.8 or glucose >= 200 is diabetic, and every diabetic row has HbA1c >= 5.7 and glucose >= 126 (label-linked marks, PROV-3 tests 5/7); corr(HbA1c, glucose) only 0.17. Add `AHDS` back with the measurement; drop `Ethical Issue`. https://www.kaggle.com/datasets/iammustafatz/diabetes-prediction-dataset + Popular Kaggle 'Diabetes Prediction Dataset' (iammustafatz) with no documented source; the curator and a linked Kaggle discussion indicate the data is likely fake/fabricated, and it is flagged for ethical and data-quality issues. This fails criterion 1 (unique real source), criterion 4B (artificial data), and likely criterion 5 (ethical). Confidence in rejection is high. A human need only confirm the fabricated-data discussion thread before finalizing the reject. --- diff --git a/curation/records/digital_green_crop_yield_estimate_challenge.md b/curation/records/digital_green_crop_yield_estimate_challenge.md index 51bb5f30..95b6a2a6 100644 --- a/curation/records/digital_green_crop_yield_estimate_challenge.md +++ b/curation/records/digital_green_crop_yield_estimate_challenge.md @@ -4,6 +4,7 @@ name: digital-green-crop-yield-estimate-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - No Good Target / Scientific Discovery tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: environmental science & climate required_split: - Grouped (NON-IID) @@ -29,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real survey of Indian smallholder farms with a yield target, but the survey records the season after harvest, so the use case is a look back unless the post-harvest columns go; the curator's note below calls it predictive discovery. +- Source: Zindi "Digital Green Crop Yield Estimate Challenge", 4 September 2023, data collected by Digital Green, supported by GIZ's FAIR Forward (https://zindi.africa/competitions/digital-green-crop-yield-estimate-challenge); as published: "more than 5000 data points, each having more than 40 features", "collected through a survey conducted across multiple districts in India", Train.csv (1.6 MB, about 75%), Test.csv, VariableDescription.csv; target `Yield` per acre ("rice or wheat" in the description, rice in the data description). +- Use case: the page says yield predictions "empower smallholder farmers to make informed planting and resource allocation decisions"; that needs only what is known at planting, while the survey (from memory of the files, to verify) holds harvest and threshing dates, harvest method and residue columns: fit approximate; regime grouped by district or block would mirror a new area, else IID; temporal if the sowing dates span several seasons. +- Siblings and duplicates: none found in `curation/records/` (no other Digital Green record). External collections: none found. +- Papers and pages: no paper; competition page and variable description only; top solutions not read (the forum reports extreme yield outliers, unverified). +- Metric: RMSE, Zindi Evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"); a leaderboard choice (MET-4); inputs none. +- Risks for /add-dataset: post-harvest columns (LEAK: drop to a planting-time task, USE-1); outliers or unit errors in `Yield` and `Acre` (CLEAN); SEL-3 doubt raised by the curator's note. +- Priority: gap 0 (environmental science & climate, regression, IID not thin), use case 0 (approximate), source + (Digital Green named, a codebook ships), shape + (about 5,000 farms, 40+ mixed columns), risk - (post-harvest columns, outliers): 2. + Zindi 'Digital Green Crop Yield Estimate Challenge' to predict per-acre rice/wheat yield in India from farm survey data, a continuous regression target. It is a plausible real-world tabular regression, but the curator frames it as survey-based 'predictive discovery' and yield prediction can blur into agronomic estimation; a grouped split (by farm/region) may be needed. It maps to the criteria reasonably but with caveats about target quality and representativeness. A human must verify the data is one tabular file (not satellite imagery), the size, the exact yield target, and the appropriate split; 2nd Tier for now. --- diff --git a/curation/records/dli_ghana_2023.md b/curation/records/dli_ghana_2023.md index 3bed2084..244c9ec2 100644 --- a/curation/records/dli_ghana_2023.md +++ b/curation/records/dli_ghana_2023.md @@ -3,10 +3,9 @@ unique_name: dli_ghana_2023 name: dli-ghana-2023 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD -- Missing source information +- Wrong Domain / Source Modality tags: - AI-Filled (Verify) collections: @@ -28,4 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Wrong Domain / Source Modality`, SEL-12): each row is a raw 1,000-base-pair DNA string, the label whether its middle 200 bp overlaps an enhancer; the competition is built around a DNA language model's embeddings, so sequence models are the tool, not tabular ones. +- Source: Zindi "Unveiling Cassava's Secrets: AI Hackathon for Accelerating African Agrigenomics" (slug dli-ghana-2023), InstaDeep, from 5 September 2023 (https://zindi.africa/competitions/dli-ghana-2023); as published: Train.csv with `ID`, `Sequence` (1,000 letters A/C/G/T), `Chromosome`, `Region`, `Target`; 13,225 training sequences (6,464 positive), 5,668 test, split by chromosome; plus 1,500-dimensional mean-pooled AgroNT embeddings (Train_embeddings.npy, Test_embeddings.npy). +- Why not vectorised-and-fine (SEL-12's exception): the only non-sequence columns are the chromosome and the region; the embeddings are a pre-trained model's hidden states, and the page frames the task as NLP on DNA ("natural language processing (NLP) and its concept of attention have successfully been applied to similar challenges in genomics"). +- Siblings and duplicates: none in `curation/records/` (no cassava or AgroNT record). +- Metric: accuracy (Zindi Evaluation page). +- Previous markers `TBD` and `Missing source information` dropped: the source is known (InstaDeep, the Zindi page). +- Reject check (AI, 2026-10-22): stands. Zindi lists data_type ['Unstructured'] and type_of_problem ['Classification', 'Natural Language Processing']; each row is "a 1000-base pair (bp) DNA sequence ... represented as a string of 1000 letters", and the only vectors are a DNA language model's embeddings (https://zindi.africa/competitions/dli-ghana-2023). SEL-12, consistent with the AI's No on instadeep_enzyme/kinase and snakes_and_sequences. + Zindi 'dli-ghana-2023' competition with no description, comments, or metadata in the record, so the task, target, domain, modality, and size are all unknown. The 'DLI' (deep learning indaba) framing slightly raises the chance of an image/text task, but this is unverified. Insufficient information to assess, which on its own is not a rejection. A human must open the competition page to determine modality, task, target, size, and split. diff --git a/curation/records/dlix_botswana.md b/curation/records/dlix_botswana.md index 1280538f..cc07dc9c 100644 --- a/curation/records/dlix_botswana.md +++ b/curation/records/dlix_botswana.md @@ -3,10 +3,9 @@ unique_name: dlix_botswana name: dlix-botswana checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +26,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-run of the Income Prediction Challenge for Azubian, which `income_prediction_challenge_for_azubian` already records. +- Source: Zindi "DLIx Botswana Challenge" (Deep Learning IndabaX Botswana 2025), from 31 July 2025 (https://zindi.africa/competitions/dlix-botswana); the Zindi API names its parent `income-prediction-challenge-for-azubian` (7 November 2022), and both ship the same files (Train.csv 68,209,849 bytes, Test.csv 28,207,389, VariableDefinitions.csv 1,674, SampleSubmission.csv 1,257,026) and the same text ("~200 000 individuals in train and ~100 000 individuals in the test file", "predict if an individual earns above 50 000 of a specific currency"). +- Siblings and duplicates: `income_prediction_challenge_for_azubian` (TBD -> Yes, AI unverified; links the Azubian page and umojahack-india-2022) carries this dataset; it is the record to screen. Whether that data is itself a re-upload of a census extract is its question, not this record's. +- Metric: F1 (Zindi), the same as the parent. +- Previous markers `TBD` and `Missing source information` replaced by `Duplicate`. +- Reject check (AI, 2026-10-22): stands. Zindi API (api.zindi.world/v1/competitions/dlix-botswana) names parent income-prediction-challenge-for-azubian; Train.csv 68,209,849 and Test.csv 28,207,389 bytes are identical to the parent's; income_prediction_challenge_for_azubian carries it (its own parent is umojahack-india-2022, for that record's year). PROV-4/PROV-5. + Zindi 'dlix-botswana' competition with no description or metadata in the record; task, target, domain, data modality, and size are all unknown. The 'DLIX' (Deep Learning IndabaX) naming raises the possibility of a non-tabular image/text task, but that is unconfirmed. Too little information to decide, and that alone is not grounds for rejection. A human must inspect the competition to establish modality, task, target, size, and split. diff --git a/curation/records/dont_overfit_by_uofk_fmsi.md b/curation/records/dont_overfit_by_uofk_fmsi.md index a445b089..d41db4c1 100644 --- a/curation/records/dont_overfit_by_uofk_fmsi.md +++ b/curation/records/dont_overfit_by_uofk_fmsi.md @@ -3,10 +3,9 @@ unique_name: dont_overfit_by_uofk_fmsi name: dont-overfit-by-uofk-fmsi checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Too Small -- AHDS (Artifical/Handmade/Deterministic/Simulated) +- No Good Target / Scientific Discovery tags: - AI-Filled (Verify) collections: @@ -28,4 +27,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-7): the target is a respondent's own answer in the World Bank's Global Findex 2017 survey ("if you were in an emergency and needed to make a payment within the next month, can you?"), asked in the same interview as the feature answers Q1-Q28; nobody predicts what the respondent states, and no provider holds these answers for its clients. +- Source: first published as Zindi "UmojaHack Africa 2021 #3: Financial Resilience Challenge (BEGINNER)", 27-28 March 2021 (https://zindi.africa/competitions/umojahack-africa-2021-3-financial-health-challenge-beginner); this record's "Don't Overfit! By UofK FMSI" (from 9 March 2023, https://zindi.africa/competitions/dont-overfit-by-uofk-fmsi) is a re-run with the same files (Train.csv 15,557,583 bytes, Test.csv 6,576,155, VariableDefinitions.csv 2,017, the same R starter notebook). The rows are Global Findex 2017 (Demirgüç-Kunt et al. 2018, World Bank, WLD_2017_FINDEX_v02_M), cited on the page. +- As published: "~100 000" train and "~45 000" test "survey responses from around Africa and the world"; columns Q1-Q28 coded "1 - Yes, 2 - No, 3 - Don't Know 4 - refused to answer"; binary target. +- Use case claimed by the page: "help financial services providers including insurance companies and banks tailor their services"; a provider does not hold Findex answers, so the fit is a look back at survey answers (SEL-3, SEL-7; precedents cdc_diabetes_health_indicators, and the curator's `No` on the FinScope survey `dataorg_financial_health_prediction_challenge`, 2026-10-22). +- The old markers `Too Small` and `AHDS` are refuted: real World Bank survey answers, about 145,000 rows; the "Don't Overfit" name is the hackathon's title, not the Kaggle challenge. +- Siblings: `ix_mobile_banking_prediction_challenge` (another Findex-style survey answer as target) was screened today as `TBD -> 2nd Tier`, priority 1, with the same SEL-7 doubt; the two should be decided alike. No record carries the UmojaHack 2021 #3 original. +- Metric: ROC AUC (Zindi: "Do not set thresholds ... Zindi will need the raw probabilities"). +- If the reject-checker refutes it: `TBD -> 2nd Tier`, priority 1 (gap 0, use case 0, source + (World Bank Findex), shape +, risk - (survey-answer target)). +- Reject check (AI, 2026-10-22): stands. The target is a Global Findex 2017 respondent's own answer in the same interview: "if you were in an emergency and needed to make a payment within the next month, can you?" (https://zindi.africa/competitions/dont-overfit-by-uofk-fmsi). That is SEL-7 word for word (a label the respondent already knows). Same files as the UmojaHack 2021 #3 parent (Train 15,557,583 / Test 6,576,155 bytes, same in the API), so no other record would be lost. Consistency: ix_mobile_banking_prediction_challenge (TBD -> 2nd Tier) has the same SEL-7 shape and should be aligned with this one; the curator's 2026-10-22 No on dataorg_financial_health_prediction_challenge (survey data, unclear target) points the same way. + Zindi 'Don't Overfit' competition (UofK FMSI), modeled on the well-known Kaggle 'Don't Overfit!' challenges, which deliberately provide a tiny training set with many anonymized continuous features to stress-test overfitting. Such datasets are typically very small and often artificial/anonymized, putting them at risk under criterion 4B (artificial) and 'Too Small'. They can still be interesting tabular benchmarks but are unrepresentative edge cases. A human must verify the training-set size, whether features are real or synthetic/anonymized, and the target before any promotion; 2nd Tier at most. diff --git a/curation/records/drug_consumption.md b/curation/records/drug_consumption.md index 0494b26e..3e540c56 100644 --- a/curation/records/drug_consumption.md +++ b/curation/records/drug_consumption.md @@ -4,9 +4,9 @@ name: drug_consumption checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 3 decision_markers: - No Good Target / Scientific Discovery -- Ethical Issue tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -30,6 +30,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 3: a documented survey with a paper, but the labels are the respondents' own reports of drug use in the same questionnaire as the features (SEL-7 doubt) and the target is a choice among 18 drugs; the CC note below calls it a scientific-discovery task. +- Source: Fehrman, Muhammad, Mirkes, Egan, Gorban, "The Five Factor Model of personality and evaluation of drug consumption risk" (arXiv 1506.06297, 2015; Springer 2017); UCI "Drug Consumption (Quantified)", donated 2016-10-16 (https://doi.org/10.24432/C5TC7S); as published: 1,885 respondents x 12 features (NEO-FFI-R, BIS-11, ImpSS scores, age, gender, education, country, ethnicity, all quantified to reals) and 19 drug labels with seven classes each ("Never Used" to "Used in Last Day"), including the decoy drug Semer. +- Use case: UCI states the purpose as "Evaluation of risk to be drug consumer for each drug"; no actor or decision is named (a clinician screening risk from personality tests is our guess), and every label is self-reported in the same anonymous survey (SEL-7 doubt; cdc_diabetes_health_indicators precedent): fit approximate at best; regime IID (one row per respondent). +- Siblings and duplicates: none in `curation/records/`; collection tag `TabArena Reject` (the TabArena study excluded it); the CC note says the TALENT benchmark's target is unclear. +- Papers and pages: the paper above (user/non-user splits by time window, several classifiers, sensitivity and specificity reported; not re-read here). +- Metric: none verified as one deliberate criterion (the paper reports sensitivity and specificity per drug and classifier; MET-5, MET-6 not checked against the text), so the fallback (`roc_auc` for a binary user/non-user target, `log_loss` for the seven classes). +- Risks for /add-dataset: which drug and which binarisation (USE-5); the quantified categorical columns should be decoded back to categories (CLEAN; the CC note); 1,885 rows with rare classes for some drugs (TP-5). +- `Ethical Issue` removed: no source states that the subjects or creators object to ML use (SEL-1); the survey is anonymous. +- Priority: gap + (social science is thin), use case 0 (no actor), source + (paper and UCI page), shape + (1,885 rows, 12 features), risk - (self-reported label, target choice): 3. + UCI 'Drug Consumption (quantified)' dataset: ~1885 respondents with Big-Five personality scores plus demographics, used to predict use of each of ~18 drugs (multiple binary/ordinal targets). It is genuine real-world tabular data with a defensible predictive target (per-drug usage), but it was framed as a scientific-discovery study of personality-drug links rather than a single benchmark task, and it is already tagged TabArena Reject / 2nd Tier. The choice of target is ambiguous (18 candidates) and drug-use prediction from personality raises mild ethical/representativeness concerns. A human must pick the actual target used, confirm sample size after dropping near-constant drugs, and weigh the ethical framing before promoting it. --- diff --git a/curation/records/dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria.md b/curation/records/dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria.md index 5ba9cfdc..f0fd69c5 100644 --- a/curation/records/dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria.md +++ b/curation/records/dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria.md @@ -4,11 +4,13 @@ name: DSN AI Bootcamp Qualification Hackathon by Data Science Nigeria checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2020' domain: finance required_split: - Random (IID) @@ -26,6 +28,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: a credit-default classification at application with 80,000 rows and documented credit-bureau-style fields, but the lender is a scenario name and some fields describe post-default collections. +- Source: Zindi "DSN AI Bootcamp Qualification Hackathon", Data Science Nigeria, from 9 September 2020 (https://zindi.africa/competitions/dsn-ai-bootcamp-qualification-hackathon); as published: "a structure dataset pull from the Kowope Mart's database", train 56,000 and test 24,000 rows; `Applicant_ID`, `form_field1` to `form_field50` (risk scores, default severity, credit limits and utilisation, income, property value, tenures, missed payments, a financial stress index, `form_field47` the product applied for, Charge or Lending; `form_field48` and `form_field49` "Undefined Variable"), and `default_status`. +- Use case: Kowope Mart and its partner bank want "a system that profiles customers who are worthy of the card with minimum if not zero risk of defaulting": the card issuer decides at application; fit direct if the fields are known at application; regime IID (one row per applicant, no dates). +- Siblings and duplicates: `microsoft_x_data_science_nigeria_2024_ai_bootcamp_qualification_hackathon`, `dsn_free_ai_classes_in_every_city_2025` and `dsn_trail_ooo` are other DSN hackathons (not compared by files here); no other record carries Kowope Mart. External collections: none found. +- Papers and pages: competition page only; no paper. +- Metric: ROC AUC, Zindi Evaluation page ("The error metric is Area Under the Curve"); a leaderboard choice (MET-4); inputs none. +- Risks for /add-dataset: `form_field11` ("The amount of dues collected post-default") and the default-severity fields may describe the outcome itself (LEAK, check against the target); whether Kowope Mart and DSBank are real or a teaching scenario over another lender's data (PROV-2); the data description also says "You must predict growth_stage for the corresponding images", a copy-paste slip. +- Priority: gap 0 (finance, binary, IID not thin), use case + (card issuer at application, named by the source), source 0 (page only, provenance of the rows unclear), shape + (80,000 rows, 50 columns), risk - (post-default fields, unclear provenance): 2. + Zindi DSN AI Bootcamp qualification hackathon: predict which customers will default on a loan, using a structured table pulled from Kowope Mart's database. This is a clear real-world binary credit-default classification task, a strong fit for tabular ML in the finance domain. The split is plausibly IID unless an account/customer or application-date field forces a grouped/temporal protocol. A human must download the data to confirm row count, the exact target column, and whether a temporal/grouped split is warranted; data access on Zindi may require competition registration. --- diff --git a/curation/records/dsn_free_ai_classes_in_every_city_2025.md b/curation/records/dsn_free_ai_classes_in_every_city_2025.md index 46b160af..d8acbbec 100644 --- a/curation/records/dsn_free_ai_classes_in_every_city_2025.md +++ b/curation/records/dsn_free_ai_classes_in_every_city_2025.md @@ -3,7 +3,8 @@ unique_name: dsn_free_ai_classes_in_every_city_2025 name: dsn-free-ai-classes-in-every-city-2025 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Missing source information tags: @@ -14,7 +15,7 @@ original_source: Zindi year: '2025' required_split: - '?' -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/dsn-free-ai-classes-in-every-city-2025/data source_row: 945 @@ -27,4 +28,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a 2025 re-run of DSN's 2020 "Excellent Store Challenge" (same three files, same byte sizes), whose columns map one to one onto the Big Mart sales practice data under new names; the store chain is a teaching scenario. +- Source: first published as the Zindi hackathon "DSN Pre-Bootcamp Hackathon: The Excellent Store Challenge" by Data Science Nigeria, 8-22 Aug 2020 (https://zindi.africa/hackathons/dsn-pre-bootcamp-hackathon-the-excellent-store-challenge); this record's competition (2-29 Jun 2025) names it as its parent in Zindi's API, and its train.csv (651,526 bytes), test.csv (432,867) and SampleSubmission.csv (84,802) match the 2020 files byte for byte in size. The record's `year` 2025 is the re-run, PROV-9 would give 2020. As published: one table per product and store, `Item_ID, Item_Weight, Item_Sugar_Content, Item_Visibility, Item_Type, Item_Price, Store_ID, Store_Start_Year, Store_Size, Store_Location_Type, Store_Type, Item_Store_ID`, target `Item_Store_Returns` ("profit returns on the product in the particular store"); about 1,500 products in 10 stores. Rows not counted (login, secret code). +- Use case: the page's CEO of "Impact Stores" (2020: "Babatunji Stores") wants to know which products and store types return more profit before expanding; a fictional scenario of a training bootcamp, fit approximate at best; regime unclear (IID over item-store pairs, or grouped by store for the expansion question, with only 10 stores). +- Siblings and duplicates: no record of the 2020 hackathon or of Big Mart Sales; the Big Mart practice problem (Analytics Vidhya, "2013 sales data for 1559 products across 10 stores", target `Item_Outlet_Sales`) has the same columns (Item_Fat_Content -> Item_Sugar_Content, Item_MRP -> Item_Price, Outlet_* -> Store_*); if confirmed, this is Big Mart renamed and its target relabelled from sales to "profit returns" (PROV-3, PROV-4). Big Mart itself has no documented first source. +- Papers and pages: no paper; the Zindi tabs (read through the page API); no public solution read. +- Metric: RMSE ("The evaluation metric for this challenge is Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: confirm or refute the Big Mart match on the rows (PROV-3, a reworked table is `Data Quality Issue`; PROV-4); the target's meaning; the split (store-grouped with 10 stores is thin). +- Priority: gap 0 (business & marketing), use case 0 (a teaching scenario), source - (a likely renamed copy of an undocumented practice set), shape + (mixed types, item x store structure), risk - (PROV-3). + + Zindi 'DSN Free AI Classes in Every City 2025' competition, but the record carries no description, comments, or metadata beyond the link. Data Science Nigeria competitions are typically genuine tabular supervised tasks, so the prior is favorable, but there is too little information here to judge target, modality, size, or split. Per the heuristics, lack of record detail alone is not grounds for rejection. A human must open the competition page/data to determine the task type, target, problem domain, and appropriate split. diff --git a/curation/records/dsn_trail_ooo.md b/curation/records/dsn_trail_ooo.md index c2c96382..d85ed079 100644 --- a/curation/records/dsn_trail_ooo.md +++ b/curation/records/dsn_trail_ooo.md @@ -3,9 +3,9 @@ unique_name: dsn_trail_ooo name: dsn-trail-ooo checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -26,4 +26,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): "TRAIL OOU X DSN DATA SCIENCE TRAINING" (Zindi, 2 Apr - 3 May 2024, https://zindi.africa/competitions/dsn-trail-ooo, 44 participants, run by Olabisi Onabanjo University's TRAIL initiative with Data Science Nigeria) re-uses the data of the "DSN AI Bootcamp Qualification Hackathon" (Zindi, 9 Sep - 3 Oct 2020, https://zindi.africa/competitions/dsn-ai-bootcamp-qualification-hackathon), recorded as `dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria` (`TBD -> Yes`). +- Source: Zindi's competition API names the 2020 hackathon as this competition's `parent`; both pages carry the same Kowope Mart / DSBank description and the same Data Description ("The data sample has been split into train and test with a volume of 56,000 and 24,000", `form_field1` ... `form_field50`, target `default_status`), and the same files: Train.csv 18,065,184 bytes, Test.csv 7,668,359, SampleSubmission.csv 378,110 in both. +- Metric: both score Area Under the Curve ("The error metric is Area Under the Curve", Evaluation tab). +- The task stays in the benchmark through the 2020 record; nothing here adds a distinct task (PROV-6). +- Reject check (AI, 2026-10-22): stands. Zindi API: parent = dsn-ai-bootcamp-qualification-hackathon (2020). The files are byte-identical in size: Train 18,065,184 / Test 7,668,359 / SampleSubmission 378,110. The task is kept in dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria (PROV-4). + Zindi 'dsn-trail-ooo' competition with an empty record (only the link is present). The name is opaque and there is no description of the task, target, or data. Data Science Nigeria / Zindi competitions are usually legitimate tabular predictive tasks, so it should not be rejected solely for missing detail. A human must inspect the competition page and data to establish whether it is a real tabular classification/regression task, its size, target, and split regime. diff --git a/curation/records/earthquake.md b/curation/records/earthquake.md index 30ad7db7..90919c23 100644 --- a/curation/records/earthquake.md +++ b/curation/records/earthquake.md @@ -7,7 +7,6 @@ suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery - Time-series (Forecasting) -- Duplicate tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -32,6 +31,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3; `Time-series (Forecasting)`, SEL-10): a catalogue of every earthquake recorded 1990-2023 with no predictive target; the uploader offers it for "analyzing spatial and temporal distribution patterns" and to "forecast future seismic events". +- Source: Kaggle upload "All the Earthquakes Dataset : from 1990-2023" by alessandrolobello (Aug 2023, https://www.kaggle.com/datasets/alessandrolobello/the-ultimate-earthquake-dataset-from-1990-2023), "The data are from https://www.usgs.gov/" (the USGS ComCat catalogue is the first source, PROV-8). As published: one CSV of about 3 million events (494.8 MB), columns time, place, status, tsunami, significance, data_type, magnitudo, state, longitude, latitude, depth, date; no target named. +- Use case: none stated beyond research, public awareness and forecasting ("develop predictive models to forecast future seismic events more accurately"); predicting one catalogue column from the others (magnitude from depth and place, `significance`, which USGS computes from magnitude, felt reports and damage estimates) fits no decision; forecasting future events is SEL-10. +- Siblings and duplicates: `quake` (Simonoff 1996, 2,178 earthquakes 1964-1986, OpenML 550; `No`, Scientific Discovery: "Lennart: ... not a predictive task originally and not a real task either", which named this Kaggle upload as its "newer version"). Not a row duplicate of `quake` (other years), so the old `Duplicate` marker is dropped. The record's second link (stern.nyu.edu/SOR/SmoothMeth) is the book of `quake`, not of this upload. External collections: TabSTAR (as recorded; the target it used not checked). +- Metric: none found (Kaggle description). +- The human note on this record ("Unsure if a real task, or how it could be used. However, we can still take a look at it") leaves it open; `quake`'s verdict is the precedent with the same facts. +- Reject check (AI, 2026-10-22): stands. The Kaggle upload of the USGS catalogue names no target; its only stated uses are pattern analysis and forecasting ('develop predictive models to forecast future seismic events', https://www.kaggle.com/datasets/alessandrolobello/the-ultimate-earthquake-dataset-from-1990-2023). The precedent quake was rejected by Lennart on the same facts: 'not a predictive task originally and not a real task either'. A TabSTAR target, if there is one, would be a relabel under SEL-8 (not checked). + A Kaggle compilation of global earthquakes 1990-2023 (date, location, depth, magnitude). It has no well-defined supervised target: magnitude/occurrence prediction is a temporal forecasting / scientific-discovery problem rather than a fixed IID tabular task, and it is already tagged TabArena Reject / 2nd Tier. The record also notes it relates to OpenML d/550, suggesting overlap with existing data. Excluded as a forecasting / no-good-target scientific catalogue; a human need only confirm there is no extractable fixed classification/regression target before final rejection. --- diff --git a/curation/records/economic_well_being_prediction_challenge.md b/curation/records/economic_well_being_prediction_challenge.md index 3bd47e87..546e8633 100644 --- a/curation/records/economic_well_being_prediction_challenge.md +++ b/curation/records/economic_well_being_prediction_challenge.md @@ -4,16 +4,16 @@ name: economic-well-being-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes -decision_markers: -- Missing source information +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: social science required_split: -- '?' +- Grouped (NON-IID) problem_type: Regression source_links: - https://zindi.africa/competitions/economic-well-being-prediction-challenge/data @@ -27,4 +27,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: survey-cluster wealth index predicted from Africa-wide geospatial layers for countries not in training; a known poverty-mapping task with a grouped (country) split; the inputs are remote-sensing products, the one scope question. +- Source: first published as the Zindi "AIMS Data Science Hackathon" (Microsoft, for AIMS and CMU Africa students, 25 Jan 2021), re-opened publicly as "Economic Well-Being Prediction Challenge" (16 Apr - 15 Aug 2021, https://zindi.africa/competitions/economic-well-being-prediction-challenge); same text and the same Train.csv (5,829,604 bytes), Test.csv and SampleSubmission.csv. As published: one row per survey cluster; target "a derived 'wealth index' based on responses in various household surveys ... on a scale from 0 to 1"; features: GHSL built-up fractions by epoch, water surface, population density (5 km radius), Copernicus land-cover fractions (crops, urban, permanent and seasonal water), nighttime lights, distance to capital and to shoreline, urban or rural. Rows not counted (login); the survey programme is not named (likely DHS). +- Use case: "Granular information on economic well-being is extremely useful for governments, policy makers, and NGOs ... A successful model could be useful for filling in the gaps between the more expensive surveys" (Description): an agency estimates wealth where no recent survey exists; fit direct; regime grouped because "you must correctly predict the cluster-level estimated wealth measures found from surveys in 7 different countries not covered in the training data" (only Train.csv has labels: group by country within it). +- Siblings and duplicates: no other record of the AIMS hackathon or of DHS wealth mapping found (`grep` for wealth, poverty, well-being, aims); the old `Missing source information` marker is refuted (the competition documents every layer). External collections: none found. +- Papers and pages: the competition tabs (read through Zindi's API); the task follows the satellite poverty-mapping literature (Jean et al. 2016, Yeh et al. 2020), not checked as a source of these rows. +- Metric: RMSE ("The evaluation metric for this competition is Root Mean Squared Error", Evaluation tab); MET-4; inputs: a country column for the grouped split (to check that it ships; the API listed the AIMS parent's metric as MAE while its page says RMSE). +- Risks for /add-dataset: SEL-13 (land-cover fractions and night lights are classified satellite products aggregated to 5 km, a layer of abstraction above spectra; argue it from the columns); whether Train.csv carries country and survey year; clusters near each other across surveys (spatial leakage within a country). +- Priority: gap + (social science and the grouped regime are thin), use case + (the source names the actors and the gap-filling decision), source 0 (the competition documents the layers, not the surveys), shape + (thousands of clusters in 18 countries, grouped), risk - (SEL-13 question). + + Zindi 'Economic Well-Being Prediction Challenge'; the record has no body, but the title strongly implies predicting a household/regional wealth or well-being index, typically a regression task over survey and/or geospatial covariates. This is a plausible real-world tabular task in the social-science domain. There is a risk the features are derived from satellite imagery (which would lean non-tabular) or require a grouped/spatial split. A human must check the feature set (tabular vs. imagery-derived), the exact continuous target, sample size, and whether a grouped/spatial split is needed. diff --git a/curation/records/eicu_collaborative_research_database.md b/curation/records/eicu_collaborative_research_database.md index c91bfd85..9de53778 100644 --- a/curation/records/eicu_collaborative_research_database.md +++ b/curation/records/eicu_collaborative_research_database.md @@ -4,9 +4,9 @@ name: eICU Collaborative Research Database checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 3 decision_markers: - Needs extensive data wrangling -- No Good Target / Scientific Discovery tags: - 2nd Tier / Scientfic Discovery - '?' @@ -30,4 +30,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 3: a large, well-documented multi-centre ICU database with real outcomes (hospital and ICU mortality, length of stay), but no task as published: one has to be defined and the tables joined; the old `No Good Target / Scientific Discovery` marker is refuted and dropped. +- Source: the eICU Collaborative Research Database by MIT LCP and Philips (PhysioNet, https://physionet.org/content/eicu-crd/2.0/, v2.0 of 15 Apr 2019; Pollard et al. 2018, Scientific Data 5:180178, "a freely available multi-center database for critical care research"). As published: a relational database of over 200,000 ICU unit stays of over 139,000 patients in 208 US hospitals (335 units), 2014-2015: patient, vital signs, labs, medications, diagnoses, care plan, APACHE IVa components and results; credentialed access (CITI training and a data use agreement; no criterion, EV-4). `year` left empty: the first release before v2.0 not checked. +- Use case: ICU mortality or length of stay at 24 h after admission, the APACHE IV setting (an intensivist's severity score for triage and for benchmarking units); the source names no decision ("secondary analysis of electronic health records"), so the use case is ours; fit approximate; regime grouped by hospital (a model trained on some hospitals used at a new one), patients repeat across stays. +- Siblings and duplicates: `mimic_iv` (`TBD -> 2nd Tier`, same kind of credentialed database), `icu_length_of_stay_icu_mortality` (MIMIC-III via TableShift, Lennart: `TBD -> Yes`), `sepsis_prediction` (PhysioNet 2019, other hospitals): no duplicate. External collections: YAIB (van de Water et al. 2024) and Sheikhalishahi et al. 2020 (PLOS ONE) define eICU tasks (mortality at 24 h, length of stay, AKI, sepsis); not read in detail here. +- Papers and pages: Pollard et al. 2018; the public eicu-code repository of MIT-LCP on GitHub; the benchmarks above. +- Metric: none for the database (PhysioNet page, Pollard et al. 2018); a derived benchmark's metric would be the source if we take its task (YAIB reports AUROC and AUPRC side by side: fallback, MET-5); otherwise the fallback (`roc_auc` / `log_loss`, `rmse`). Inputs: a hospital id for the grouped split (ships, `hospitalid`). +- Risks for /add-dataset: the task and the cohort have to be defined (first stay per patient, 24 h window); features charted after the prediction point and the APACHE predictions themselves (LEAK); heavy joins; size under the 500k cap is fine. +- Priority: gap + (grouped regime is thin), use case 0 (written by us), source + (Scientific Data paper, PhysioNet), shape + (200k stays, mixed types, hospital groups), risk - (heavy joins, a task to define). + + The eICU Collaborative Research Database (PhysioNet) is a large multi-center ICU relational database with many tables (patients, vitals, labs, diagnoses), not a single curated predictive table. Real tabular tasks (e.g., ICU mortality, length-of-stay) can be derived from it, but doing so requires substantial wrangling, cohort definition, and a patient-grouped split, and it is credentialed-access (data-use agreement). It is already flagged 2nd Tier / '?'. A human must decide on a specific extracted task and target, handle the credentialing/access, and define the join and grouping before this could be promoted. diff --git a/curation/records/elusage.md b/curation/records/elusage.md index 312c57bb..2a2162bc 100644 --- a/curation/records/elusage.md +++ b/curation/records/elusage.md @@ -25,6 +25,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict stands: "Tiny data", "Lennart: too small", "Andrej: too small"; recommend restoring `No` with `Too Small` (and noting `No Good Target / Scientific Discovery`); the curator has the call. The current `TBD -> 2nd Tier` came from an AI "policy" note, not from a curator. +- Source: OpenML 228 (uploaded 2014), "Dataset from Smoothing Methods in Statistics (ftp stat.cmu.edu/datasets) Simonoff, J.S. (1996)", the same book as `quake` (Lennart: `No`, Scientific Discovery). As published: 55 rows, 3 columns, target `average_electricity_usage` (monthly household electricity use against month and average temperature), no missing values. +- Use case: none stated; a textbook smoothing example. A utility forecasting its load would use a series (SEL-10), not 55 monthly rows; fit none. +- Siblings and duplicates: `quake` (same book, `No`); no copy found. External collections: OpenML only. +- Metric: none found (OpenML description, the book's citation); the fallback would be `rmse`. +- Risks: TP-5 (55 rows, about 18 per test fold), no task (SEL-3); `year` `?` would be 1996 (PROV-9). +- No priority set: the human verdict is a reject. + + OpenML 'elusage' is a tiny dataset of 55 samples (electricity usage vs. month/temperature). The record already records a TabArena verdict of 'Tiny data' with two reviewers (Lennart, Andrej) independently calling it too small for 3-fold CV. It is far below any reasonable size threshold for a stable benchmark task. Excluded as Too Small; nothing further to verify. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/ensi_junior_entreprise.md b/curation/records/ensi_junior_entreprise.md index d40c3408..e9c3d417 100644 --- a/curation/records/ensi_junior_entreprise.md +++ b/curation/records/ensi_junior_entreprise.md @@ -4,13 +4,15 @@ name: ensi-junior-entreprise checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Needs extensive data wrangling +- Time-series (Regression) tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: environmental science & climate required_split: - Grouped (NON-IID) @@ -28,6 +30,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: daily temperature per weather station from only the date, a station id and normalised coordinates; a spatial interpolation of a reading stream with three inputs, from a one-day student hackathon that does not name its data source. +- Source: Zindi private hackathon "ENSI Junior Entreprise" (Hack'prise event of the ENSI student association with Infor Tunisia, 14 Jul 2021, https://zindi.africa/competitions/ensi-junior-entreprise); as published: Train.csv "1,449,100 rows and 5 columns" (`Date, LOC_ID, X, Y, TEMP`), Test.csv 391,661 rows without `TEMP`; "3 years of data, for each day"; values such as 79.8 in September and 15.2 in January suggest degrees Fahrenheit (a US station network?); the provider is not named (PROV-2). +- Use case: the page motivates weather for retail demand, but the task is "predict weather temperature at any location given by its X and Y coordinates": no actor acts on a station's past temperature; fit approximate at best; regime: grouped by station if the test holds new locations, temporal if new dates; the page does not say which (the record's `Grouped (NON-IID)` is an AI guess, kept). +- Siblings and duplicates: none found in `curation/records/`; external collections: none. +- Papers and pages: the competition tabs (read through Zindi's API); no paper, no solution read. +- Metric: RMSE ("The error metric for this challenge is Root Mean Squared Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: three inputs (date, X, Y) plus an id: the task is a smoother over space and season (TP-4 risk); a station's other days in training make it near-trivial under a random split; the unnamed origin (PROV-2); `Needs extensive data wrangling` dropped (one table as published). +- Priority: gap 0 (environmental science), use case 0, source 0 (a student hackathon, data origin unnamed), shape 0 (1.4M rows but three features), risk - (thin features, unclear split), floored at 1. + + Zindi ENSI Junior Entreprise competition: predict weather temperature at a location given its X/Y coordinates. This is essentially a spatial-interpolation regression task whose covariates may be little more than coordinates, which risks being thin/not representative of general tabular ML and would need a spatial (grouped) split to avoid leakage. It is a real predictive target, so not an outright reject. A human must inspect the actual feature set (whether richer covariates exist beyond coordinates), the sample size, and the spatial split requirement before deciding. --- diff --git a/curation/records/era.md b/curation/records/era.md index ed99d2c6..9dadd5e1 100644 --- a/curation/records/era.md +++ b/curation/records/era.md @@ -6,7 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- AHDS (Artifical/Handmade/Deterministic/Simulated) tags: - 2nd Tier / Scientfic Discovery - Non-IID (Grouped) @@ -30,6 +29,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): a decision-making experiment run in an MBA course to find which candidate qualities matter; the label is a student's subjective tendency to accept a candidate, which no hiring decision acts on as a prediction target. +- Source: OpenML 1030 "ERA" (Employee Rejection\Acceptance), donated by Arie Ben-David (Holon Academic Institute of Technology), owner Yoav Ganzah (Tel Aviv University), in Ben-David's WEKA ordinal sets with ESL, LEV and SWD; OpenML cites Dobrska, Wang and Blackburn 2012 (an ordinal-regression method paper). As published: 1,000 rows, 4 ordinal inputs ("past experience, verbal skills, etc."), ordinal output `out1`, no missing values; no year of collection. +- Use case: none: "The ERA data set was originally gathered during an academic decision-making experiment aiming at determining which are the most important qualities of candidates for a certain type of jobs. Unlike the ESL data set (enclosed) which was collected from expert recruiters, this data set was collected during a MBA academic course" (OpenML description); a study of the raters, not a decision; the CC line adds that the judges likely repeat across rows (an unknown grouping). +- Siblings and duplicates: `esl` (the same donor's expert-recruiter set; Lennart: `No`, `Trivial`, `Missing source information`); no copy found. External collections: TabArena Reject (the TabArena v0.1 curation). +- Metric: none found (OpenML; the cited paper evaluates an ordinal method on many sets). +- The old `AHDS` marker is dropped: nothing in the source says the candidate profiles were generated, though they likely were hypothetical vignettes (not checked). +- Reject check (AI, 2026-10-22): stands. The source states a discovery purpose and a non-expert label (OpenML 1030, https://www.openml.org/d/1030): 'gathered during an academic decision-making experiment aiming at determining which are the most important qualities of candidates ... Unlike the ESL data set ... collected from expert recruiters, this data set was collected during a MBA academic course'. That fails SEL-3's 'observed outcome or an expert annotation'. The curator's CC line on the record points the same way. + OpenML 'ERA' (Employee Rejection/Acceptance) is a small ordinal-classification dataset of human judgements collected during an MBA course, originating from Arie Ben-David's WEKA sample sets. The comment notes it is grouped judgement data where it is unclear whether judges recur, making it a subjective scientific-discovery / human-rating exercise rather than a real-world predictive task, and it is already a TabArena Reject. Excluded on no-good-target / not-representative grounds; a human need only confirm the grouping ambiguity and lack of a genuine predictive target. --- diff --git a/curation/records/eswatini_inflation_forecasting_challenge.md b/curation/records/eswatini_inflation_forecasting_challenge.md index bd21f7d0..c4c07b27 100644 --- a/curation/records/eswatini_inflation_forecasting_challenge.md +++ b/curation/records/eswatini_inflation_forecasting_challenge.md @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): "Your task is to predict Eswatini's headline inflation two months ahead, using the 13 years of historical data provided" (Description). +- Source: Zindi "Eswatini Inflation Forecasting Challenge" ("Can you nowcast Eswatini headline inflation?", AI Eswatini with the Central Bank of Eswatini, 15 Sep - 15 Oct 2025, https://zindi.africa/competitions/eswatini-inflation-forecasting-challenge). As published: CPI_GDP_data.xlsx with "13 years of monthly headline inflation data" and "20 years of annual GDP data"; the submission has two rows, `September_headline` and `October_headline`. +- Use case: the central bank's inflation forecast ("The top three solutions will be delivered to the Central Bank of Eswatini"); a single national series over a horizon, the forecasting task SEL-10 excludes; about 156 monthly values, no panel for TP-10. +- Siblings and duplicates: none found. External collections: none. +- Metric: RMSE on a rolling leaderboard "rescored ... using the official September value ... the official October value" (Evaluation tab). +- Reject check (AI, 2026-10-22): stands. A single national series forecast over a horizon (SEL-10). Zindi (https://zindi.africa/competitions/eswatini-inflation-forecasting-challenge): 'predict Eswatini's headline inflation two months ahead, using the 13 years of historical data provided'. Two submission rows, no panel, so SEL-11/TP-10 cannot apply. + Zindi 'Eswatini Inflation Forecasting Challenge' is, by its name, a time-series forecasting task predicting future inflation values over a horizon. Forecasting future values is explicitly out of scope for the benchmark. Excluded as Time-series (Forecasting); a human need only confirm it is genuine horizon forecasting rather than a fixed predictive task that merely needs a temporal split. diff --git a/curation/records/eu_lakes_allfeatures.md b/curation/records/eu_lakes_allfeatures.md index 48500051..6c2d9179 100644 --- a/curation/records/eu_lakes_allfeatures.md +++ b/curation/records/eu_lakes_allfeatures.md @@ -3,10 +3,9 @@ unique_name: eu_lakes_allfeatures name: eu_lakes_allfeatures checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Too Small -- Not Representative +- No Good Target / Scientific Discovery tags: - Tiny Data - Many features @@ -29,6 +28,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): a microbial-ecology survey table: the study relates bacterial abundances to lake chemistry and geography and predicts nothing, and no target is defined for this record. +- Source: the August 2012 sampling of European lakes by Jens Boenigk's group: metadata from Molecular Ecology (doi:10.1111/mec.15872), OTU table "from J. Boenigk"; the record's reference, Nuy, Hoetzinger, Hahn, Beisser and Boenigk 2020, "Ecological Differentiation in Two Major Freshwater Bacterial Taxa Along Environmental Gradients", Frontiers in Microbiology 11:154: "Sampling of 255 European freshwater lakes was conducted in August 2012", one sample per lake. As published: an OTU (16S V2-V3) abundance matrix plus pH, temperature, conductivity, lake area, elevation, latitude and longitude. +- Use case: none: the paper's responses are OTU and subcluster abundances and diversity indices, explained by the abiotic factors through variation partitioning and rank correlations; no machine learning, no prediction. A target of our own (lake pH from the community) would be a task the source never posed (SEL-8), on about 255 rows with thousands of OTU columns. +- Siblings and duplicates: `eu_lakes_allsamples` (the same sampling, another cut; `TBD -> 2nd Tier`, AI), and the other environmental OTU records of the same import (`bog_lakes` `No` by an AI, `tara_oceans`, `nz_springs`, `bedford_basin_*`, `subseafloor_arc*`), to be treated alike. External collections: none found. +- Metric: none found (the paper fits no model). +- Reject check (AI, 2026-10-22): stands. The microbiome family does have a collection: Zschaubitz, ..., Sperlea 2025, 'A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets', CSBJ (doi 10.1016/j.csbj.2025.04.017, https://pmc.ncbi.nlm.nih.gov/articles/PMC12049816/), Table 1, with targets fixed per dataset; but its Methods say 'The choice of target variable was based on an expectation that there is a correlation': chosen to benchmark feature selection, not for a decision, and the first source is a discovery study (SEL-8). Here the target is Mg (86 samples; eu_lakes_allsamples uses pH, 232 samples), measured in the same lake survey; Nuy et al. 2020 (https://www.frontiersin.org/article/10.3389/fmicb.2020.00154/full) fits no predictive model. Treat it the same as eu_lakes_allsamples, nz_springs, ports, subseafloor_*, tara_oceans (currently AI TBD -> 2nd Tier): same collection, same reasoning. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + A 2012 sampling of European lakes with a microbiome OTU (operational taxonomic unit) abundance table plus metadata. It is already tagged 'Tiny Data' and 'Many features', i.e. a wide, low-sample-count omics matrix that is poorly representative of typical tabular ML and likely too small for stable CV. The intended supervised target is unclear from the record (some lake/sample property to be regressed or classified). A human must identify a concrete target, confirm the (small) sample count and feature count, and judge whether the high-dimensional microbiome data is suitable at all. --- diff --git a/curation/records/eu_lakes_allsamples.md b/curation/records/eu_lakes_allsamples.md index 36aa11f0..9514c292 100644 --- a/curation/records/eu_lakes_allsamples.md +++ b/curation/records/eu_lakes_allsamples.md @@ -3,10 +3,9 @@ unique_name: eu_lakes_allsamples name: eu_lakes_allsamples checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Too Small -- Not Representative +- No Good Target / Scientific Discovery - Duplicate tags: - Tiny Data @@ -30,6 +29,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, `Duplicate`; SEL-3, PROV-6): an OTU table of European lakes published for describing community structure, with no predictive target, and the all-samples cut of the same data as `eu_lakes_allfeatures`. +- Source: Boenigk et al., "Geographic distance and mountain ranges structure freshwater protist communities on a European scale", Metabarcoding and Metagenomics 2 (2018), doi 10.3897/mbmg.2.21519: "eukaryotic ribosomal amplicon diversity from 218 European freshwater lakes sampled in August 2012 to assess the effect of mountain ranges as biogeographic barriers". Metadata from Sperlea et al., Molecular Ecology 2021, doi 10.1111/mec.15872, which presents "a methodological framework to quantify the covariation of the microbial community of a habitat and environmental variables" (more than twenty physicochemical and geographic variables, used to identify bioindicators). OTU table from J. Boenigk (not public as a file). As published: about 218 lakes x thousands of OTUs; no target named. +- Use case: none in the sources; the ML in Sperlea et al. measures explained variance per environmental variable (discovery), not a decision at a prediction point (SEL-3, USE-2). +- Siblings and duplicates: `eu_lakes_allfeatures` is the same study and OTU table (another cut, PROV-6); the other OTU records of this list (`bog_lakes`, `bedford_basin_v4v5`, `bedford_basin_v6v8`, `atl_ocean_transect`) were proposed `No` on the same ground in this re-screen. +- Metric: none (no task in the source). +- Reject check (AI, 2026-10-22): stands. SEL-3 holds on its own: all three papers on this survey are discovery studies. The record's own reference (doi 10.1111/1462-2920.14992, Environmental Microbiology 2020) says 'we address the relative importance of these different factors in shaping protists and bacterial communities on a European scale', and Sperlea et al. (https://doi.org/10.1111/mec.15872) 'quantify the covariation of the microbial community ... and environmental variables'. Its mention of bioindicators 'for lake water monitoring schemes' is about picking indicator taxa, not running a model to make a decision. The PROV-6 cut was not verified (the two records cite different papers), but the reject does not depend on it. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + Companion to eu_lakes_allfeatures: the same 2012 European-lakes microbiome OTU study, here the 'all samples' variant, again tagged 'Tiny Data' and 'Many features'. As a wide, low-sample omics table it is poorly representative of mainstream tabular ML, likely too small, and overlaps heavily with the sibling record (potential duplication of source data). The supervised target is unspecified. A human must define the target, confirm size/dimensionality, and decide whether to keep this or the features variant (or neither). --- diff --git a/curation/records/exchange_rate_forecasting_challenge.md b/curation/records/exchange_rate_forecasting_challenge.md index 02966cb3..a3d87c08 100644 --- a/curation/records/exchange_rate_forecasting_challenge.md +++ b/curation/records/exchange_rate_forecasting_challenge.md @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): the source asks for the future values of one exchange-rate series over a two-week horizon. +- Source: Zindi "Exchange Rate Forecasting Challenge" (IndabaX Nigeria with Miva University), 2024-05-10 to 05-29, 81 participants; one file `Competition_data.csv` of dated closing rates (submission columns `Date`, `Close`). +- Task: "build a time series or machine learning model that will accurately forecast what the naira rate against the dollar will be in the next 2 weeks, ranging from May 22, 2024, to June 4, 2024"; "This is a real-time forecasting challenge". +- Metric: Zindi's Evaluation page, Root Mean Squared Error. +- Siblings: other forecasting re-screens (`cryptocurrency_closing_price_prediction`, `airqo_ugandan_air_quality_forecast_challenge`) went the same way. +- Reject check (AI, 2026-10-22): stands. Zindi API (https://zindi.africa/competitions/exchange-rate-forecasting-challenge): 'forecast what the naira rate against the dollar will be in the next 2 weeks, ranging from May 22, 2024, to June 4, 2024' and 'This is a real-time forecasting challenge'. The only data file is Competition_data.csv and the submission has Date and Close, so this is one series forecast over a horizon (SEL-10). + Zindi 'Exchange Rate Forecasting Challenge' is, by name, a financial time-series forecasting task predicting future exchange-rate values over a horizon, which is explicitly excluded from the benchmark. Excluded as Time-series (Forecasting); a human need only confirm it is true horizon forecasting rather than a fixed predictive task needing only a temporal split. diff --git a/curation/records/exploreml_funaab_hackathon_nigerian_cars_price_prediction.md b/curation/records/exploreml_funaab_hackathon_nigerian_cars_price_prediction.md index 45e20c9d..e0788a58 100644 --- a/curation/records/exploreml_funaab_hackathon_nigerian_cars_price_prediction.md +++ b/curation/records/exploreml_funaab_hackathon_nigerian_cars_price_prediction.md @@ -3,12 +3,14 @@ unique_name: exploreml_funaab_hackathon_nigerian_cars_price_prediction name: exploreml-funaab-hackathon-nigerian-cars-price-prediction checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: business & marketing required_split: - Random (IID) @@ -26,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: an in-scope used-car price task, but a 16-participant campus hackathon with no stated data origin, and the benchmark already has several used-car price records. +- Source: Zindi "ExploreML FUNAAB Hackathon: Nigerian Cars Price Prediction" (Data Science Nigeria FUNAAB with GDSC FUNAAB), 2023-06-02 to 06-09, 16 participants. As published: Train, Test and `Variable_Description.csv`; features listed as "Make, Model, Year of Manufacture, Colour, Fuel, Transmission, Condition, Mileage, Engine Size, Selling Condition, Bought Condition"; target the price in Naira; the Data page says only "The data for this challenge is a tabular dataset". Rows not stated. +- Use case: the source: "This solution can help individuals and companies when trying to know the value of their vehicles before buying or selling on the market". Fit approximate (listing prices, not sale prices, is likely); regime IID unless listing dates ship. +- Siblings and duplicates: no other record of these Nigerian listings; the same task on other markets: `used_cars_24`, `used_cars_benz_italy`, `used_cars_dataset_cardekho`, `used_cars_pakistan`, `used_cars_saudi_arabia`, `cars`. The upstream (a scrape of a Nigerian listings site such as Cars45, Jiji or Autochek, or a Kaggle upload) was not found (PROV-2). +- Papers and pages: Zindi description and evaluation pages (read through the Zindi API); a 2025 University of Lagos paper on Nigerian car prices from online platforms (jer.unilag.edu.ng/article/view/2884) uses other data as far as its page shows. +- Metric: Zindi's Evaluation page: "The evaluation metric for this competition is Root Mean Squared Error" on the Naira price; a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the data's origin (PROV-2) and size; duplicates of a public scrape; `Selling Condition` and `Bought Condition` meaning; price outliers and currency scale (MET-3). +- Priority: gap 0 (business not thin), use case + (the source names individuals and companies valuing a car before buying or selling), source 0 (no data documentation), shape 0 (size unknown, about 11 mixed features), risk - (unknown origin, possible re-upload). + Zindi ExploreML/FUNAAB hackathon: predict the price of Nigerian used cars. This is a clear, genuine real-world tabular regression task (price prediction from vehicle attributes such as make, model, year, mileage), a good fit for the benchmark in the business domain, and an IID split is appropriate. The main risks are smallish hackathon size and listing-date leakage. A human must download the data to confirm the row count, the continuous price target, the feature set, and that no temporal/grouped split is required. diff --git a/curation/records/facebook_live_sellers_in_thailand.md b/curation/records/facebook_live_sellers_in_thailand.md index 0e49adc7..0348a2b6 100644 --- a/curation/records/facebook_live_sellers_in_thailand.md +++ b/curation/records/facebook_live_sellers_in_thailand.md @@ -29,6 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line, `TabArena Reject`) stands: the source publishes the data for describing engagement patterns, with no predictive target. +- Source: UCI dataset 488, donated 2019-04-21, 7,051 posts x 11 columns from the Facebook pages of 10 Thai fashion and cosmetics retailers (status type, publish time, counts of reactions, comments, shares, likes, loves, wows, hahas, sads, angrys); introduced by N. Dehouche, "Dataset on usage and engagement patterns for Facebook Live sellers in Thailand", Data in Brief, 2020. UCI lists the associated task as "Clustering" and names no target. +- Use case: none in the source; the CC line's only conceivable task (status type and date to an engagement count) would be engagement prediction, close to SEL-9 and outside what the data was published for (SEL-3, SEL-8). Regime would be temporal (`status_published`). +- Siblings and duplicates: `facebook_comment_volume` is a different dataset (comment-volume regression on other pages). External collections: none checked beyond UCI. +- Metric: none (no task in the source). +- Markers: `No Good Target / Scientific Discovery` fits UCI's "Clustering"; `Out-of-scope Task (CTR/RecSys/Ranking)` rests on a hypothetical framing, not on the source, and could be dropped; the curator has the call. + UCI dataset of Facebook Live-seller posts (video/photo/status/link) with reaction/engagement counts, published for scientific discovery of engagement patterns rather than a defined predictive task. As the comment notes, the only conceivable target would be mapping status type and date to engagement features, which is essentially an engagement/CTR-style descriptive problem, not a clean tabular classification/regression. It is already a TabArena Reject. Excluded on no-good-target / out-of-scope grounds; a human need only confirm no usable predictive target exists. --- diff --git a/curation/records/fair_forward.md b/curation/records/fair_forward.md index 6b154461..cf78beab 100644 --- a/curation/records/fair_forward.md +++ b/curation/records/fair_forward.md @@ -3,9 +3,9 @@ unique_name: fair_forward name: fair-forward checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) required_split: @@ -23,4 +23,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the record holds the same single link as `row_876`, the FAIR Forward catalog page; that record keeps the pointer for browsing. +- Source: https://fair-forward.github.io/datasets/, "FAIR Forward - Catalog of Open Source AI Datasets, Models & AI Systems", "Reusable AI building blocks for global challenges across agriculture, language technology, climate action, energy, and more"; GitHub repo Fair-Forward/datasets: "A catalog that links to the various datasets and use-cases created throughout our programme." A catalog, not one dataset: no rows, target or task. +- Siblings: `row_876` (same URL, note "Several interesting datasets"). +- Next step: candidate tabular datasets from the catalog get records of their own (via `row_876`). +- Reject check (AI, 2026-10-22): stands. fair_forward.md and row_876.md both list https://fair-forward.github.io/datasets/ as their only source link (PROV-4). The page is a catalog ('A catalog that links to the various datasets and use-cases created throughout our programme'), not a dataset. row_876 keeps the pointer. + The record is only a link to a 'fair-forward' datasets landing page with no description, comments, or metadata. 'FAIR Forward' is an open-data initiative that likely hosts a collection of multiple datasets rather than a single curated tabular task, so it is unclear whether any individual dataset, target, modality, or size is benchmark-suitable. There is too little information to decide and it should not be rejected on that basis alone. A human must open the page, enumerate the available datasets, and pick/evaluate any concrete tabular predictive task. diff --git a/curation/records/fair_higgs_boson_decay.md b/curation/records/fair_higgs_boson_decay.md index 0f08d7fd..5afc739c 100644 --- a/curation/records/fair_higgs_boson_decay.md +++ b/curation/records/fair_higgs_boson_decay.md @@ -5,8 +5,8 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Wrong Domain / Source Modality - AHDS (Artifical/Handmade/Deterministic/Simulated) +- Wrong Domain / Source Modality tags: - AI-Filled (Verify) original_source: Website @@ -27,6 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, `Wrong Domain / Source Modality`; SEL-16, SEL-12): an open simulated CMS dataset of jets, simulated physics with its own benchmarks, whose informative content is variable-length particle lists. +- Source: Chen et al., "A FAIR and AI-ready Higgs boson decay dataset", arXiv 2108.02214 (2021), Scientific Data 9:31 (2022): "an open simulated dataset produced by the CMS Collaboration at the CERN Large Hadron Collider. This dataset consists of Higgs boson decays and quark and gluon background", on the CERN Open Data Portal (ROOT and HDF5). The paper is a FAIR-ness assessment guide; the data serves H(bb) jet tagging with particle-cloud and graph models. +- Precedent: `higgs` (CC 2026-09-22, Lennart: `No`, simulated physics with dedicated benchmarks, criterion 4B; that comment names "the FAIR Higgs decay set" among the field's own benchmarks). Related: `higgsml_2014` and `fair_universe_higgsml_uncertainty`, proposed `No` on the same ground in this re-screen. +- Modality: per-jet lists of particle-flow candidates, tracks and secondary vertices of variable length (SEL-12), as the old note quoted from the paper; not re-checked in the full text. +- Reject check (AI, 2026-10-22): stands. This follows the higgs precedent directly: Lennart's CC of 2026-09-22 in curation/records/higgs.md names 'the FAIR Higgs decay set' among the field's own benchmarks and rejects simulated physics with dedicated benchmarks. The source calls it 'an open simulated dataset produced by the CMS Collaboration' (https://arxiv.org/abs/2108.02214), so SEL-16 holds. The point about variable-length particle lists (SEL-12) comes from the old note and is not needed for the reject. + Chen et al., Scientific Data 9:31 (2022). CMS open *simulation* on the CERN Open Data Portal, released in ROOT and HDF5, CC0. One record is a jet: "the dataset consists of 3.9 million H(bb) jets and 1.9 million QCD jets". Branches are per-jet scalars plus variable-length lists of particle-flow candidates, tracks and secondary vertices ("there may be a variable number per jet"), built for jet tagging with particle-cloud / graph models. Not a tabular task: the informative content is the variable-length constituent lists (source modality), so `Wrong Domain / Source Modality`, and it is simulated (criterion 4B). Per-jet scalar summaries could be tabulated but would recreate a weaker version of a benchmark that belongs to jet-tagging models. Data not inspected. diff --git a/curation/records/fair_universe_higgsml_uncertainty.md b/curation/records/fair_universe_higgsml_uncertainty.md index 4b32b892..a84879fc 100644 --- a/curation/records/fair_universe_higgsml_uncertainty.md +++ b/curation/records/fair_universe_higgsml_uncertainty.md @@ -3,7 +3,7 @@ unique_name: fair_universe_higgsml_uncertainty name: FAIR Universe HiggsML Uncertainty Challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - AHDS (Artifical/Handmade/Deterministic/Simulated) tags: @@ -29,6 +29,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16): simulated proton-proton collisions in a dedicated HEP benchmark, whose task is an uncertainty interval, not a classification. +- Source: Benato, Bhimji et al., "FAIR Universe HiggsML Uncertainty Dataset and Competition", arXiv 2410.02867 (2024): "Each instance represents a simulated proton-proton collision as observed at CERN's Large Hadron Collider"; "a tabular dataset of 28 features and 280 million instances"; "Participants were required to compute and report confidence intervals for a parameter of interest regarding the Higgs boson". Zenodo record 15131565, GitHub FAIR-Universe/FAIR_Universe_dataset. +- Precedent: `higgs` (CC 2026-09-22, Lennart: `No`; it names FAIR Universe as one of "the field's own benchmarks"); `higgsml_2014` proposed `No` in this re-screen on the same ground. +- A Label classification reframing would drop the weights and the systematics that are the point of the dataset (SEL-8). +- Reject check (AI, 2026-10-22): stands. higgs precedent: the same CC in curation/records/higgs.md lists 'FAIR Universe HiggsML Uncertainty 2024-25' as one of the field's own benchmarks. arXiv 2410.02867: 'Each instance represents a simulated proton-proton collision' and the task is to 'compute and report confidence intervals for a parameter of interest'. That is SEL-16, and a classification reframing would also break SEL-8. + NeurIPS 2024 competition dataset (Bhimji et al., arXiv 2410.02867), the HEP community's successor to HiggsML 2014. 13 TeV pp collisions, Pythia 8.2 + Delphes 3.5.0 with an ATLAS-like detector; H -> tau tau signal vs Z -> tau tau, diboson and t tbar (DetailedLabel). 28 tabular features (13 `PRI_*`, 15 `DER_*`) plus Weight, Label, DetailedLabel; 280M training events, 120M held-out; public zip 15.1 GB on Zenodo, CC BY 4.0 (2025-04-03). Native task is not classification: participants return a 68.27% confidence interval on the Higgs signal strength mu for pseudo-experiments whose data are shifted by six nuisance parameters (systematics); paper Sec. 3.2 on the limits ("known unknowns" only). A Label-classification reframing would discard weights and systematics, i.e. the point of the dataset, and the data are simulated (criterion 4B). Listed in the HEP-ML living review Datasets section. Data not inspected. diff --git a/curation/records/faulty_air_quality_sensor_challenge.md b/curation/records/faulty_air_quality_sensor_challenge.md index 524e82dd..978634ae 100644 --- a/curation/records/faulty_air_quality_sensor_challenge.md +++ b/curation/records/faulty_air_quality_sensor_challenge.md @@ -4,6 +4,7 @@ name: Faulty Air Quality Sensor Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 tags: - AI-Filled (Verify) collections: @@ -27,6 +28,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: AirQo names the actor and decision (flag a faulty device for repair), 400k readings, but how the fault label was made is not documented. +- Source: Zindi "UmojaHack Africa 2022: Faulty Air Quality Sensor Challenge (BEGINNER)", 2022-03-19 to 03-20, 719 participants, data from AirQo (Makerere University) low-cost PM sensors deployed across Uganda. As published: train "approximately 300 000 readings", test "approximately 100 000", with missing values; target an offset fault per reading (0/1; the Data page says "multi-class", the Description and the submission format say binary); one table each. +- Use case: the source states it: "The model can be used by AirQo to automatically flag a device that is returning faulty data", "faulty devices need to be identified, isolated and fixed or replaced". Fit direct. Regime likely temporal or grouped by device ("identify a device has an off set fault or not, regardless of the device"): readings are time-stamped per device, so an IID split of readings would share a device's fault episode across folds; to be settled on the columns. +- Siblings and duplicates: other AirQo records (`airqo_african_air_quality_prediction_challenge`, `airqo_low_cost_air_quality_monitor_calibration_challenge`, `airqo_ugandan_air_quality_forecast_challenge`) are different tasks on AirQo data (prediction, calibration, forecasting), not duplicates. Same event, other tasks: `african_snake_antivenom_binding_challenge`, `monthly_insurance_claim_prediction_challenge`. External collections: none found. +- Papers and pages: Zindi description, evaluation and data pages (read through the Zindi API); no paper or codebook for this data found; column names not confirmed (no public notebook found). +- Metric: Zindi's Evaluation page: "The error metric for this competition is Accuracy" (API `straight_accuracy`); a leaderboard metric on hard labels (MET-4), kept; inputs none. +- Risks for /add-dataset: how the offset fault was labelled (an injected offset, or a rule on the difference between a device's two PM sensors, would make it `Trivial` or `AHDS`, SEL-5, SEL-16); whether a device ID and timestamp ship, for the split (SPLIT); time-series classification of a signal (SEL-12) if the task needs windows; Zindi login needed. +- Priority: gap 0 (industry not thin; environment not thin), use case + (AirQo names the decision), source 0 (no documentation of the data beyond the page), shape + (400k rows, temporal structure), risk - (label origin unknown). + Zindi UmojaHack 2022 beginner challenge: classify whether an air-quality sensor/device is faulty from its output data. This is a plausible real-world binary classification task in the industry/IoT domain. The key risk is that the sensor outputs are time-stamped readings, which could make this a per-device grouped or temporal problem (or even a time-series classification) rather than IID, and beginner-track data may be small. A human must inspect the data to confirm the feature structure, target balance, dataset size, and whether a grouped/temporal split is required. --- diff --git a/curation/records/financial_inclusion_in_africa_for_botswana.md b/curation/records/financial_inclusion_in_africa_for_botswana.md index 73c791f3..d7aee21e 100644 --- a/curation/records/financial_inclusion_in_africa_for_botswana.md +++ b/curation/records/financial_inclusion_in_africa_for_botswana.md @@ -4,19 +4,20 @@ name: financial-inclusion-in-africa-for-botswana checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes -decision_markers: -- Duplicate +priority: 2 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: finance required_split: - Random (IID) problem_type: Binary Classification original_data_state: One Table source_links: +- https://zindi.africa/competitions/financial-inclusion-in-africa - https://zindi.africa/competitions/financial-inclusion-in-africa-for-botswana/data - https://zindi.africa/competitions/zindi-mentorship-1-financial-inclusion-in-africa - https://zindi.africa/competitions/nust-2022 @@ -36,6 +37,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: a real 33.6k-person survey task, but the label is the respondent's own account status from the same interview (SEL-7 doubt). +- Source: the Zindi knowledge competition "Zindi Financial Inclusion in Africa" (opened 2019-07-29, 10,679 participants); the Botswana hackathon (2023-10-21 to 10-29, WiMLDS Gaborone) is a re-run with the same description, files and metric (Zindi API, parent `financial-inclusion-in-africa`). As published: "approximately 33,600 individuals" from FinAccess Kenya 2018, FinScope Rwanda 2016, Tanzania 2017, Uganda 2018, demographic and household columns plus `VariableDefinitions.csv`; target `bank_account` (Yes/No); one table, Train/Test 70/30; the ID is `uniqueid x country` (uniqueid alone is not unique). +- Use case: the source wants a model "to predict which individuals are most likely to have or use a bank account", for "an indication of the state of financial inclusion"; no actor or decision is named, so a bank or NGO targeting the unbanked is our reading. Fit approximate; regime IID (cross-sectional survey snapshot; four countries, a grouped-by-country split is not what the task asks). +- Siblings and duplicates: this record is the only one for this data; the nine Zindi links in `source_links` are re-runs of the same files, now led by the 2019 original. The old `Duplicate` marker is dropped: no other record holds the dataset (this one should stand for it). Related but distinct: `ix_mobile_banking_prediction_challenge`, `indabax_tanzania_mobile_banking_prediction_challenge` (mobile-banking use, other data), `mobile_money_and_financial_inclusion_in_tanzania_challenge` (FinScope Tanzania 2017 alone; check for overlap of rows with the Tanzania part here, PROV-6). External collections: none found. +- Papers and pages: Zindi description, evaluation and data pages (read through the Zindi API); FinScope and FinAccess survey reports are the upstream documentation; no paper on the competition data found. +- Metric: Zindi's Evaluation page: "The evaluation metric for this challenge is Mean Absolute error, where 1 indicates that the individual does have a bank account and 0 indicates that they do not"; a leaderboard metric (MET-4), on hard 0/1 submissions it equals the error rate; inputs none. +- Risks for /add-dataset: SEL-7, the respondent knows whether they have an account and answers in the same interview as the features, so the task needs a use case where the account status of non-surveyed people is predicted from demographics; FinScope sampling weights not shipped (whether the release is a weighted sample is unknown); check the overlap with the Tanzania records (PROV-6); columns such as `cellphone_access` are answers from the same interview, fine as features if the use case holds. +- Priority: gap 0 (finance not thin), use case 0 (no actor or decision named), source + (national FinScope/FinAccess surveys, Zindi variable definitions), shape + (33.6k rows, mixed categorical features), risk - (SEL-7 label known to the respondent). + Zindi 'Financial Inclusion in Africa' task: predict whether an individual has/uses a bank account from demographic and household survey features, a genuine real-world binary classification problem in finance/social science. It is a strong tabular task, but the many listed source links show the same Financial Inclusion in Africa dataset reused across numerous Zindi competitions, so this 'Botswana' variant may be a subset or re-upload of the canonical FSD/FinScope survey data and risks duplication with other candidates. A human must confirm whether the Botswana data is distinct, check the survey-weighting/sampling concerns, the row count, and the target column. --- diff --git a/curation/records/firm_teacher_clave_direction_classification.md b/curation/records/firm_teacher_clave_direction_classification.md index a164fdab..952a1e5c 100644 --- a/curation/records/firm_teacher_clave_direction_classification.md +++ b/curation/records/firm_teacher_clave_direction_classification.md @@ -27,6 +27,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the `No` rests on the TabArena workbook verdict ("Audio"); the AI agrees with `No`, on other grounds: the inputs are constructed rhythm patterns, not a sample from a real distribution, and no decision uses the prediction. +- Source: UCI 324, donated 2015-04-23 by M. Vurkac (dataset citation 2011), from his Portland State dissertation; 10,800 rows, "16 binary inputs and one 'four-bit' one-hot classification output"; each input is "the substantial presence (0, absence) of an onset (note start) in a certain time window" of one 4/4 bar in 16th notes; classes neutral, reverse clave, forward clave, incoherent, "determined through the music-theoretic/ethnomusicological portion of the my dissertation studies ... based on both double-blind listening tests and informal interviews with with four professional master-musicians". +- Use case: none found; the dissertation studies prestructured neural networks on a symbolic rhythm problem. No actor or decision (USE-1). +- Not audio: the rows are symbolic onset patterns, so `Wrong Domain / Source Modality` (SEL-12) does not fit well. The stronger grounds: the 10,800 rows are a selection of the 65,536 possible 16-bit patterns, labelled by the "firm teacher" (the codified expert rules; the "firm" against a fuzzy teacher reading is from the title, not confirmed in the text), so the data is handmade and the label close to deterministic (`AHDS`, SEL-16, PROV-3), which the record already carries. +- Siblings: none (grep clave). Metric: none found (UCI page; the ISSS paper on RA models of onset interactions compares modelling criteria, not a task metric). +- Recommendation: keep `No`; the curator has the call. Not checked: the dissertation's label section. + UCI 'Firm-Teacher Clave-Direction Classification' is a multi-target dataset of musical clave/rhythm patterns where the labels are assigned from human/expert musical rules rather than measured outcomes, making it a deterministic, handmade rule-encoding task. The TabArena workbook flagged it as an audio-domain problem and two reviewers (Lennart, Andrej) could not tell whether it is a real predictive task, with a possible temporal aspect. Because the targets are rule-derived/deterministic rather than a genuine real-world predictive signal, it is excluded as AHDS; a human could verify the label-generation process if reconsidering. --- diff --git a/curation/records/first_order_theorem_proving.md b/curation/records/first_order_theorem_proving.md index 9af3f02a..a429d0e9 100644 --- a/curation/records/first_order_theorem_proving.md +++ b/curation/records/first_order_theorem_proving.md @@ -26,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (`Outdated`, Lennart, 2026-07-27) stands; the research gives one argument for and one against it, and the open consistency check of 2026-10-19 is the place to settle it. +- Source: UCI 249 (donated 2013-04-16; Bridge, Holden and Paulson, dataset citation 2012) and OpenML 1475; "Given a theorem, predict which of five heuristics will give the fastest proof when used by a first-order prover. A sixth prediction declines to attempt a proof, should the theorem be too difficult"; 6,118 rows, static and dynamic features of the conjecture; paper: Bridge, Holden and Paulson, "Machine Learning for First-Order Theorem Proving: Learning to Select a Good Heuristic", Journal of Automated Reasoning 53 (2014). +- Use case: a prover picks its heuristic per conjecture before the run (algorithm selection); fit direct; regime IID over conjectures (TPTP problems; families of related problems may call for a grouped check). +- For `No`: the labels are tied to one prover version and five fixed heuristic settings of 2012; a selector trained on them serves that configuration only. +- Against `No`: `asp_potassco_classification` and `sat11_hand_algo_runtime` ship as `Yes` with the same kind of fixed solver portfolio, and `Outdated` has no rule (selection.md). A fact that sets this one apart is still to be named. +- Metric: the paper ranks selectors by theorems proved and time taken under the selection (read from the abstract), which needs the per-heuristic timings; the multiclass mirror on OpenML loses them, so the fallback (log loss) unless the timings ship in the UCI files (MET-1, MET-5). +- Siblings: `asp_potassco_classification`, `sat11_hand_algo_runtime` (ASlib, `Yes`); external collections TabSTAR (as recorded). Not checked: the UCI files for the timings; the paper's full text. + first-order-theorem-proving (OpenML d/1475) is an **algorithm-selection** dataset: each row is a first-order-logic conjecture described by static syntactic / derived features, and the target is which theorem-prover heuristic (under a time bound) proves it fastest — a genuine tabular **multiclass classification** task, *not* a robotics / RL problem. The prior `Robotics (RL)` and `Wrong Domain / Source Modality` markers were incorrect and have been removed. It is kept as **No** on `Outdated` grounds (confirmed by Lennart, 2026-07-27): the ASlib-style algorithm-selection framing predates modern RL / LLM-based proving and is a niche, dated benchmark. To verify: confirm the prover set / class count and whether the algorithm-selection framing is itself in-scope. CC: "Likely done with something like reinforcement learning or LLMs nowadays, but clearly an unsolved problem. Would need to read the paper; sounds like an algorithm selection problem too" diff --git a/curation/records/fossil_stock_forecasting_challenge.md b/curation/records/fossil_stock_forecasting_challenge.md index 4c031902..805c061a 100644 --- a/curation/records/fossil_stock_forecasting_challenge.md +++ b/curation/records/fossil_stock_forecasting_challenge.md @@ -30,6 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): Fossil's monthly sell-in per SKU, to be forecast four months ahead. +- Source: Zindi "Fossil Demand Forecasting Challenge" (2022-05-24 to 2022-08-28, $5,000) and its invite-only follow-up "Fossil Top 5 Challenge" (2022-10); Zindi problem type "Forecast". Data: "the number of items sold of each item (or SKU) each month", sales per major channel, beginning-of-period and on-hand inventory; target "sellin" four months ahead; Train.csv, Test.csv, DataDictionary.csv. +- Deciding words: "The objective of this challenge is to build a forecasting or other predictive model to determine as accurately as possible the demand for each individual product, four months into the future" and (Top 5) "Predictions are calculated on a monthly basis, pointing to the 4-month-out forecast ... the forecast has to be computed with the data available at the moment of the prediction" (https://zindi.africa/competitions/fossil-stock-forecasting-challenge, https://zindi.africa/competitions/fossil-top-5-challenge). +- Not SEL-11: the target is the future value of each SKU's own monthly series, not a fixed outcome of a unit; the submission IDs are SKU_month. +- Metric: MAE (Zindi Evaluation page; Fossil also reviewed ME, RMSE, MPE, MAPE). Siblings: none (grep fossil). +- Reject check (AI, 2026-10-22): stands. SEL-10 (Must). Zindi Description: "The objective of this challenge is to build a forecasting or other predictive model to determine as accurately as possible the demand for each individual product, four months into the future"; type_of_problem ["Forecast"]; submission ids are SKU_month (YOSHTLYNYOSHZZ_11_2021), so the target is each SKU's own monthly sell-in series over a horizon. Fossil also scores ME/RMSE/MPE/MAPE "of the quantity being forecast". No fixed per-unit outcome to rescue under SEL-11. https://zindi.africa/competitions/fossil-stock-forecasting-challenge + Zindi competition whose stated task is to predict demand for each individual product four months into the future. Forecasting future values over a horizon is explicitly out of scope for the benchmark. The data is per-product/per-time stock/sales records, i.e. a forecasting setup rather than a fixed predictive task that merely needs a temporal split. A human should confirm there is no reframable static target, but on the stated framing this is a clear forecasting exclusion. --- diff --git a/curation/records/free_ai_classes_in_every_city_hackathon.md b/curation/records/free_ai_classes_in_every_city_hackathon.md index a56ca6ac..2c4653e1 100644 --- a/curation/records/free_ai_classes_in_every_city_hackathon.md +++ b/curation/records/free_ai_classes_in_every_city_hackathon.md @@ -3,12 +3,16 @@ unique_name: free_ai_classes_in_every_city_hackathon name: free-ai-classes-in-every-city-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 2 +decision_markers: +- TBD tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: business & marketing required_split: - Random (IID) @@ -16,6 +20,7 @@ problem_type: Regression original_data_state: One Table source_links: - https://zindi.africa/competitions/free-ai-classes-in-every-city-hackathon/data +- https://zindi.africa/competitions/free-ai-classes-in-every-city-hackathon source_row: 990 needs_review: - ai_unverified @@ -26,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: real Nigerian used-car listings (7.2k rows) for a price regression, but the upstream listing site is unnamed and car-price siblings exist. +- Source: Zindi hackathon "FREE AI Classes In Every City Hackathon" (Data Science Nigeria, AI Invasion 2022, 2022-05-28 to 2022-07-03, access restricted, secret code); "There are ~7000 entries in Train and ~2000 entries in Test"; as published: Train 7,205 x 9 (`VehicleID`, `Location`, `Maker`, `Model`, `Year`, `Colour`, `Amount (Million Naira)`, `Type`, `Distance`), target `Amount (Million Naira)`, one table plus VariableDefinitions.csv (preview of a public GitHub copy of Train.csv). Three cities (Lagos, Abuja, Ibadan), 55 makers, 1,223 models, `Type` Foreign Used / Nigerian Used / Brand New, `Distance` 33% missing: the shape of scraped listings (the site is not named; PROV-2). +- Use case: a used-car platform ("Great Motors", the organiser's scenario) lists a car at a price from the mechanic's report; fit approximate (listing prices, a written scenario); regime IID, no listing date. +- Siblings and duplicates: `exploreml_funaab_hackathon_nigerian_cars_price_prediction` (DSN FUNAAB 2023, another column set: fuel, transmission, engine size, condition; a different table, no duplicate by the columns); other used-car price records (`used_cars_pakistan`, `used_cars_saudi_arabia`, `used_cars_24`, `used_cars_dataset_cardekho`, `used_cars_benz_italy`); `free_ai_classes_in_every_city_hackathon_2023` is a different task (house prices). External collections: none found. +- Papers and pages: https://zindi.africa/competitions/free-ai-classes-in-every-city-hackathon (Zindi API read 2026-10-22); public solution repositories on GitHub (top-solution notebooks), no paper. +- Metric: RMSE on `Amount (Million Naira)`, the Zindi Evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"); deliberate leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the scraping source and date are unknown (PROV-2); `Year` and `Distance` are strings with thousands separators; possible repeated listings of one car (PROV-4 within the table); heavy-tailed price (max 456 M Naira). +- Priority: gap 0 (business & marketing, IID not thin), use case 0 (organiser's scenario), source 0 (hackathon page only), shape + (7.2k rows, mixed types, high-cardinality model), risk 0. + Zindi hackathon dataset whose task is to predict car selling price, a standard real-world tabular regression problem with a clear target. Used-car price prediction is representative of tabular ML and a random split is appropriate. The main unknowns are dataset size after cleaning and whether the features are genuine (vs. a small/teaching hackathon set). Lean Yes pending verification of row count and that the data is not a trivial/synthetic re-upload. --- diff --git a/curation/records/free_ai_classes_in_every_city_hackathon_2023.md b/curation/records/free_ai_classes_in_every_city_hackathon_2023.md index 7ec1d8c8..ff42b3a5 100644 --- a/curation/records/free_ai_classes_in_every_city_hackathon_2023.md +++ b/curation/records/free_ai_classes_in_every_city_hackathon_2023.md @@ -3,9 +3,9 @@ unique_name: free_ai_classes_in_every_city_hackathon_2023 name: free-ai-classes-in-every-city-hackathon-2023 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: -- TBD +- AHDS (Artifical/Handmade/Deterministic/Simulated) tags: - AI-Filled (Verify) collections: @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16, PROV-3): the "Wazobia Real Estate" house-price table is generated: rows spread evenly over Nigeria's 36 states and nine house types, prices with fractions of a Naira, and about 13% missing values injected in every column. +- Source: Zindi hackathon "DSN and Microsoft Skills for Job" (slug free-ai-classes-in-every-city-hackathon-2023, Data Scientists Network with Microsoft, 2023-06-12 to 2023-08-06); "Can you accurately predict the prices of houses in Nigeria"; "There are ~7000 entries in Train and ~3000 entries in Test" (the data description still names the car task of 2022); files Housing_dataset_train.csv, Housing_dataset_test.csv; Wazobia Real Estate Limited is the scenario's company, no data source named. +- Deciding evidence (a preview of Housing_dataset_train.csv from a public participant repository, github.com/FelixFrankFelix/DSN-MICROSOFT-HACKATHON, not the source's words): 14,000 x 7 (`ID`, `loc`, `title`, `bedroom`, `bathroom`, `parking_space`, `price`); each of the 36 states holds 302 to 370 rows; nine titles hold 1,266 to 1,372 rows each (Cottage 450); `price` has three decimals (1149999.565) and only 0.08% whole numbers; every feature is missing in 12.3% to 12.9% of the rows; a five-bedroom detached duplex in Anambra lists at 2.41 M Naira (median 1.9 M), far below the market. +- Metric: RMSE (Zindi Evaluation page). +- If the checker finds this rests on a reading: `TBD -> 2nd Tier`, priority 1 (gap 0, use case 0, source 0, shape +, risk - for PROV-3). +- Reject check (AI, 2026-10-22): stands. SEL-16 / PROV-3 (Must), on measured marks, not page words (EV-2: a measurement outranks the page). I re-measured the public copy of Housing_dataset_train.csv (github.com/FelixFrankFelix/DSN-MICROSOFT-HACKATHON, Data/). It has 14,000 rows. Test 4 (marginals too even): 36 states with 302-370 rows each, including Lagos. Bedrooms 1-5 have about 1,740 rows each and 6-9 about 880 each, a step-uniform shape. Parking 1-4 have about 2,370 each and 5-6 about 1,335 each. Test 6 (structure the world would show): bathrooms are independent of bedrooms, so a 1-bedroom has 7 bathrooms as often as a 4-bedroom does. Prices have 3 decimals (0.08% whole numbers) and a mansion's median is 3.6M Naira. Missingness was injected: 12.3-13% per column, 7,731 rows have exactly one NaN and only 580 have two or more, and the column-wise missingness correlates at -0.10. A plain log-linear model on the complete rows gives R^2 0.93. That is several calibrated marks from at least two tests, which meets PROV-3 as amended 2026-10-21. The Zindi page names no data source: "Wazobia Real Estate Limited" is a scenario, and the Data Description still describes the 2022 car task. https://zindi.africa/competitions/free-ai-classes-in-every-city-hackathon-2023 + Zindi hackathon dataset from the same series as free_ai_classes_in_every_city_hackathon but with no comments or task description in the record. The sibling 2022/earlier edition was a car-price regression task, so this is plausibly a similar real tabular predictive problem, but the actual target is unknown. Do not reject for lack of info alone. A human must open the competition page to confirm the target, problem type, split regime, and dataset size before a final verdict. diff --git a/curation/records/fremtpl2.md b/curation/records/fremtpl2.md index 07e9302c..fb5f5781 100644 --- a/curation/records/fremtpl2.md +++ b/curation/records/fremtpl2.md @@ -3,9 +3,11 @@ unique_name: fremtpl2 name: freMTPL2 checked_by: - AI (UNVERIFIED) -suggestion: 'Yes' +suggestion: TBD -> Yes +priority: 4 tags: - AI-Filled (Verify) +original_source: Other domain: insurance required_split: - Random (IID) @@ -13,6 +15,9 @@ problem_type: Regression original_data_state: Database (or multiple to-be-joined tables) source_links: - https://www.kaggle.com/datasets/karansarpal/fremtpl2-french-motor-tpl-insurance-claims +- https://dutangc.github.io/CASdatasets/reference/freMTPL.html +- https://cas.uqam.ca/pub/web/CASdatasets-manual.pdf +- https://doi.org/10.2139/ssrn.3164764 source_row: 1044 needs_review: - ai_unverified @@ -23,6 +28,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: the documented French MTPL portfolio for claim frequency, an insurer's pricing task, and insurance is thin. The earlier plain `Yes` was set by the older AI pass (no curator name in `checked_by`), so it is put back to `TBD -> Yes` for the curator to confirm. +- Source: CASdatasets R package (Dutang and Charpentier), `freMTPL` page (https://dutangc.github.io/CASdatasets/reference/freMTPL.html): "risk features are collected for 677,991 motor third-part liability policies", "The period is approximately 2011-2013", source "Unknown private insurer". Two tables joined on `IDpol`: `freMTPL2freq` (677,991 x 12: `IDpol`, `ClaimNb`, `Exposure`, `Area`, `VehPower`, `VehAge`, `DrivAge`, `BonusMalus`, `VehBrand`, `VehGas`, `Density`, `Region`) and `freMTPL2sev` (26,444 claims: `IDpol`, `ClaimAmount`). The Kaggle link in `source_links` is a re-upload (karansarpal); the first publication is CASdatasets (the year it entered the package was not confirmed, so `year` stays empty). +- Use case: an insurer sets the premium of a motor TPL policy at underwriting from the rating factors (claim frequency per year of exposure, Noll, Salzmann and Wuthrich, "Case study: French motor third-party liability claims", SSRN 3164764); fit direct for the frequency target `ClaimNb` with `Exposure`; regime IID, one row per policy-period and no date, but see the risks. +- Siblings and duplicates: `bemtpl97` (Belgian MTPL, CASdatasets, a different portfolio: no duplicate, PROV-7); no other freMTPL record (grep over `curation/records/` for fremtpl, MTPL, CASdatasets, ClaimNb). External collections: CASdatasets, OpenML (freMTPL2freq / freMTPL2sev uploads); used in Deprez, Verbeke and Verdonck, "Is TabPFN the Silver Bullet for Insurance Pricing?" (arXiv 2605.22892, 2026: TabPFN vs GLM and XGBoost on two public MTPL sets, frequency and severity). +- Papers and pages: CASdatasets manual https://cas.uqam.ca/pub/web/CASdatasets-manual.pdf; Noll et al. (doi:10.2139/ssrn.3164764); Denuit, Charpentier and Trufin 2021 (doi:10.1016/j.insmatheco.2021.09.001); Miljkovic and Fernandez 2018 (doi:10.3390/risks6020057); Pocuca et al. 2020 (doi:10.1016/j.insmatheco.2020.06.004); arXiv 2605.22892. +- Metric: no competition; the actuarial case study (Noll et al.) compares frequency models by the out-of-sample Poisson deviance with `Exposure` (read from the abstract and the papers that follow it, the full text not read here); likely a deliberate choice (MET-5), inputs `Exposure` (ships). Severity would be a gamma deviance on `ClaimAmount`. To confirm in /add-dataset; else the fallback. +- Risks for /add-dataset: choose the target (`ClaimNb` frequency with `Exposure`, or severity from the join, one `_sev` dataset); `ClaimNb` is capped and `Exposure` above 1 is truncated in the case study's cleaning (check); identical risk profiles recur across rows (the actuarial literature builds a group id from equal features before splitting), so check a grouped split (SPLIT, LEAK); some `ClaimAmount` values are fixed IRSA-IDA convention amounts ("Some claim amounts of `freMTPL2sev` are fixed claim amounts based on the French IRSA-IDA claim convention"). +- Priority: gap + (insurance thin), use case + (pricing named by the case study), source + (CASdatasets page and papers), shape + (678k rows, mixed types), risk - (target, exposure and repeated profiles to settle). + freMTPL2 is the well-known French Motor Third-Party Liability insurance dataset from CASdatasets, a standard actuarial benchmark. The canonical predictive task is claim frequency modelling (Poisson/count regression of claim number against exposure and policy features), a genuine real-world tabular regression/count task that is widely used and adequately sized (hundreds of thousands of policies). It maps cleanly to the criteria: unique original source, real predictive target, representative tabular data, no ethical issue. Note it comes as freq + severity tables (freMTPL2freq / freMTPL2sev) so a human should confirm which table/target is used and how exposure is handled; an IID split is standard. --- diff --git a/curation/records/from_scores_to_seats_the_grad_school_ml_challenge.md b/curation/records/from_scores_to_seats_the_grad_school_ml_challenge.md index de493080..ed958944 100644 --- a/curation/records/from_scores_to_seats_the_grad_school_ml_challenge.md +++ b/curation/records/from_scores_to_seats_the_grad_school_ml_challenge.md @@ -3,7 +3,8 @@ unique_name: from_scores_to_seats_the_grad_school_ml_challenge name: from-scores-to-seats-the-grad-school-ml-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD tags: @@ -11,10 +12,11 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2025' domain: education required_split: - '?' -problem_type: TBD +problem_type: Binary Classification source_links: - https://zindi.africa/competitions/from-scores-to-seats-the-grad-school-ml-challenge/data source_row: 940 @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a two-day beginner hackathon on graduate-admission features whose data is most likely the Kaggle "Graduate Admission" set (500 rows, a self-estimated "Chance of Admit") enlarged or relabelled; the data could not be inspected. +- Source: Zindi hackathon "From Scores to Seats: The Grad School ML Challenge" by Hackade (Nigeria, 2025-04-04 to 2025-04-06, access restricted); "predict whether a student will be admitted into a graduate program based on their academic profile, including standardized test scores, university rating, CGPA, and research background"; files train.csv, test.csv, variable_description.csv; target `Admitted` (1 admitted, 0 not). Row count not stated (the sample IDs reach ID_004125, more than the 500 rows of the Kaggle set). +- Use case: an applicant or a university estimates admission from the profile; fit unclear, since the features match Acharya et al.'s Kaggle "Graduate Admission" data (GRE, TOEFL, University Rating, SOP, LOR, CGPA, Research), whose target is a 0-1 "Chance of Admit", not an observed admission decision; regime IID. +- Siblings and duplicates: no graduate-admission record (grep for admission, Acharya, chance of admit); `law_school_admission_bianry` and `turkish_university_admissions` are other admission tasks. If the data is the Kaggle set, the Kaggle upload is the first publication and this would be a `Duplicate` (PROV-4). +- Papers and pages: https://zindi.africa/competitions/from-scores-to-seats-the-grad-school-ml-challenge (Zindi API read 2026-10-22); Acharya, Armaan and Antony 2019, "A comparison of regression models for prediction of graduate admissions" (the Kaggle set); no copy of the Zindi files found on GitHub or the web. +- Metric: F1 on hard 0/1 labels, the Zindi Evaluation page ("The evaluation metric for this challenge is F1 Score"); leaderboard metric (MET-4), threshold left to the participant (MET-6); inputs none. +- Risks for /add-dataset: the provenance (a copy, a synthetic enlargement or a threshold of "Chance of Admit"; PROV-3, PROV-4, SEL-5); the target may not be an observed outcome (SEL-3); size unknown. +- Priority: gap + (education thin), use case 0, source 0 (a two-day hackathon page), shape 0 (size unknown), risk - (provenance and target). + Zindi competition with no comments; the name suggests predicting graduate-school admission/placement ('seats') from applicant scores, an education-domain tabular classification or regression task with a clear target. This would be representative real-world tabular ML if the data is genuine applicant records. Insufficient detail to confirm the exact target, size, and whether the data is synthetic/handmade. A human must inspect the competition data to confirm the target definition, problem type, and that it is not an artificial/generated set. diff --git a/curation/records/galsenais_fraud_detection_competition.md b/curation/records/galsenais_fraud_detection_competition.md index 66eafffc..c088d0cf 100644 --- a/curation/records/galsenais_fraud_detection_competition.md +++ b/curation/records/galsenais_fraud_detection_competition.md @@ -3,9 +3,9 @@ unique_name: galsenais_fraud_detection_competition name: galsenais-fraud-detection-competition checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a 2025 re-run of the Xente Fraud Detection Challenge on the same Xente transaction sample, recorded as `xente_fraud_detection_challenge`. +- Source: Zindi hackathon "GalsenAI's Fraud Detection Competition" (GalsenAI and AI Hub Senegal, 2025-03-19 to 2025-04-27, secret code); the Zindi API lists as its parent the 2020 "Xente Fraud Dection Hackathon", itself a re-run of the 2019 "Xente Fraud Detection Challenge". +- Deciding words: the data page repeats the 2019 challenge word for word: "Xente is an e-commerce and financial service app serving 10,000+ customers in Uganda. This dataset includes a sample of approximately 140,000 transactions that occurred between 15 November 2018 and 15 March 2019", with the same files (training.csv "Transactions from 15 November 2018 to 13 February 2019", test.csv "from 13 February 2019 to 14 March 2019", Xente_Variable_Definitions.csv) and the same subtitle "Accurately classify the fraudulent transactions from Xente's e-commerce platform" (https://zindi.africa/competitions/galsenais-fraud-detection-competition, https://zindi.africa/competitions/xente-fraud-detection-challenge). +- Canonical: `xente_fraud_detection_challenge` (`TBD -> Yes`, its `source_links` already list the 2020 hackathon and other re-runs); this link can be added there. +- Metric: F1 (the same as the 2019 challenge). +- Reject check (AI, 2026-10-22): stands. PROV-4/PROV-5 (Must), source confirmed. Zindi API: parent = xente-fraud-dection-hackathon (the 2020 re-run of the 2019 xente-fraud-detection-challenge). The data page repeats "a sample of approximately 140,000 transactions that occurred between 15 November 2018 and 15 March 2019". The file list and file descriptions are identical (training.csv "Transactions from 15 November 2018 to 13 February 2019", test.csv "from 13 February 2019 to 14 March 2019", Xente_Variable_Definitions.csv). It is the same fraud task with the same metric, so not distinct under PROV-7. Canonical: xente_fraud_detection_challenge (TBD -> Yes); add this link to its source_links. https://zindi.africa/competitions/galsenais-fraud-detection-competition + Zindi fraud-detection competition with no comments in the record. Fraud detection is a classic, highly representative tabular binary-classification task with a clear target (fraud vs. legitimate), so it is a plausible Yes. Unknowns include dataset size, whether a temporal/grouped split is required (transaction data often is), and whether the data is real or synthetically generated for the competition. A human should verify the target, the split regime, and data provenance before finalizing. diff --git a/curation/records/geoai_challenge_for_air_pollution_susceptibility_mapping.md b/curation/records/geoai_challenge_for_air_pollution_susceptibility_mapping.md index 6d3a4e83..3aefea46 100644 --- a/curation/records/geoai_challenge_for_air_pollution_susceptibility_mapping.md +++ b/curation/records/geoai_challenge_for_air_pollution_susceptibility_mapping.md @@ -3,22 +3,26 @@ unique_name: geoai_challenge_for_air_pollution_susceptibility_mapping name: geoai-challenge-for-air-pollution-susceptibility-mapping checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Time-series (Forecasting) +- Needs extensive data wrangling +- TBD tags: - Non-IID (Temporal) - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: environmental science & climate required_split: - Temporal (NON-IID) -problem_type: Regression +problem_type: Multiclass Classification original_data_state: Other source_links: - https://zindi.africa/competitions/geoai-challenge-for-air-pollution-susceptibility-mapping/discussions +- https://zindi.africa/competitions/geoai-challenge-for-air-pollution-susceptibility-mapping/data source_row: 891 needs_review: - ai_unverified @@ -29,6 +33,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a small ITU GeoAI challenge to map seasonal AQI classes of Milan in 2022 from station series and terrain layers; the target must be built from the pollutants and the scored unit is unclear. +- Source: Zindi competition "GeoAI Challenge for Air Pollution Susceptibility Mapping" (ITU AI for Good, 2023-07-21 to 2023-10-14, 35 on the leaderboard); "The challenge is to implement a machine learning method which can accurately estimate the pollution levels (AQI) of the city of Milan during 2022 ... the expected output are the seasonal (spring, summer, autumn, and winter) AQI susceptibility maps during 2022"; data: daily meteorological series 2016-2022, daily pollutant series 2016-2021 (NOx, SO2, CO, O3, PM2.5, PM10, benzene) at the stations, terrain, land cover and geology at the stations and on a 100 m grid; files Train.csv, Test.csv, four `meteo__2022.csv`, `air_pollution_Milan_Comune_topo_only.csv`. The AQI target is to be computed by the participant from the EEA guidelines ("must be used to prepare the data"). +- Use case: the city of Milan maps where pollution is likely high ("support decision-making to improve the resilience of the city"); fit approximate (no actor or decision named); regime temporal (train 2016-2021, score 2022) and spatial (interpolation between stations). +- Forecasting (SEL-10)? Zindi tags the problem "Forecast", but the 2022 meteorology is given and the output is a seasonal map, an estimation rather than future values of a series; not a clear reject from the source's words, so it stays with the doubt noted. +- Siblings and duplicates: `geoai_ground_level_no2_estimation_challenge` (same ITU GeoAI series, Lombardy and Veneto NO2 from satellite data, a different task); `geoai_challenge_for_cropland_mapping_in_dry_environments`; `no2` (OpenML Norwegian traffic NO2, unrelated). External collections: none found. +- Papers and pages: https://zindi.africa/competitions/geoai-challenge-for-air-pollution-susceptibility-mapping (Zindi API read 2026-10-22); the GEOAIR project of the University of Padova (https://www.cirgeo.unipd.it/geoair/) runs the GeoAI challenges. +- Metric: accuracy on the AQI class (1 to 5), the Zindi Evaluation page ("The error metric for this competition is Accuracy"); leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the target has to be built (AQI from up to five pollutants, by station type); few stations, so the task may be `Too Small` (TP-5); Test.csv rows and their unit (station-season or grid point) not checked; `problem_type` changed from Regression to Multiclass Classification (the source scores AQI classes by accuracy). +- Priority: gap 0, use case 0, source + (ITU and the GEOAIR project document the data), shape 0 (size unknown, likely small), risk - (target to build, unclear unit). + GeoAI Zindi challenge whose stated task is to predict the pollution levels (AQI) of Milan over a year, flagged as time-series in the record. This is forecasting air-quality values over a temporal horizon, which is explicitly out of scope. The data is also geospatial/satellite-derived, adding a non-tabular modality concern. On the stated framing this is a forecasting exclusion; a human need only confirm there is no static reframing before final rejection. --- diff --git a/curation/records/geoai_ground_level_no2_estimation_challenge.md b/curation/records/geoai_ground_level_no2_estimation_challenge.md index ed12dd77..cbfaa897 100644 --- a/curation/records/geoai_ground_level_no2_estimation_challenge.md +++ b/curation/records/geoai_ground_level_no2_estimation_challenge.md @@ -3,22 +3,23 @@ unique_name: geoai_ground_level_no2_estimation_challenge name: geoai-ground-level-no2-estimation-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier -decision_markers: -- Needs extensive data wrangling +suggestion: TBD -> Yes +priority: 3 tags: -- Non-IID (Temporal) +- Non-IID (Grouped) - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2024' domain: environmental science & climate required_split: -- Temporal (NON-IID) +- Grouped (NON-IID) problem_type: Regression -original_data_state: Other +original_data_state: One Table source_links: - https://zindi.africa/competitions/geoai-ground-level-no2-estimation-challenge +- https://zindi.africa/competitions/geoai-ground-level-no2-estimation-challenge/data source_row: 878 needs_review: - ai_unverified @@ -29,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: estimate daily ground-level NO2 at monitoring stations in Lombardy and Veneto from public satellite and weather data, scored on stations held out, a grouped task. +- Source: Zindi competition "GeoAI Ground-level NO2 Estimation Challenge" (ITU AI for Good with the GEOAIR project, 2024-05-22 to 2024-11-15, 370 on the leaderboard); "develop ML models to estimate surface NO2 concentrations using only public remote sensing data as predictor variables"; data: "Ground-truth data from air quality monitoring stations in the continental part of Lombardy and Veneto regions", Sentinel-5P TROPOMI NO2, CHIRPS precipitation and NOAA land surface temperature from Google Earth Engine; Train.csv, Test.csv, DataDescription.pdf. Rows not checked (Zindi login). +- Use case: an environmental agency estimates surface NO2 where no station measures it (exposure maps, monitoring); fit direct for estimation at unmonitored sites; regime grouped by station, because the source scores "in-situ data from selected ground stations not provided to participants" and states "The ground truth data used for evaluation are independent from the training data set" (`required_split` changed from Temporal to Grouped; a temporal component remains: models must hold "regardless of season"). +- Siblings and duplicates: `geoai_challenge_for_air_pollution_susceptibility_mapping` (same series, Milan AQI maps, a different task); `no2` (OpenML, Norwegian road traffic NO2, a different dataset). External collections: none found; related paper on dense NO2 estimation from sparse stations and satellite data (arXiv 2504.17039). +- Papers and pages: https://zindi.africa/competitions/geoai-ground-level-no2-estimation-challenge (Zindi API read 2026-10-22); discussions 21512, 22132, 23436 (an unofficial third place: features from longitude and latitude, three towns removed as outliers, ensembles); GEOAIR outputs https://www.cirgeo.unipd.it/geoair/index.php?id=outputs. +- Metric: RMSE on `GT_NO2`, the Zindi Evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"); leaderboard metric (MET-4); the jury also reported Pearson r and MAE. Inputs: none for the metric; the station id is needed for the grouped split (check it ships). +- Risks for /add-dataset: the satellite columns are retrieved physical quantities at the station point (TROPOMI NO2 column), which reads as a measurement (SEL-15) rather than image descriptors (SEL-13), to confirm; satellite gaps (clouds); whether the Zindi Test.csv holds other stations or other days of the same stations; coordinates and station identity as features under a grouped split (LEAK). +- Priority: gap + (grouped regime thin), use case 0 (the actor is not named by the source), source + (ITU, GEOAIR, a data description), shape + (daily station series over two regions, assumed thousands of rows), risk - (SEL-13 reading and the held-out design to confirm). + GeoAI Zindi challenge to estimate ground-level NO2 concentrations, tagged Non-IID (temporal). Unlike a pure horizon forecast this is an estimation/regression mapping task (predicting NO2 from satellite-column, meteorological and geospatial features), which could be a legitimate tabular regression if the inputs are tabularized. However it is geospatial/satellite-derived and may carry raster modality and heavy wrangling, and the split likely needs to be temporal or spatial. A human must inspect whether the features are genuinely tabular vs. raw raster, the target definition, and the appropriate split regime; place in 2nd tier pending that. diff --git a/curation/records/geographicaloriginalofmusic.md b/curation/records/geographicaloriginalofmusic.md index cebc908a..fbad6ff5 100644 --- a/curation/records/geographicaloriginalofmusic.md +++ b/curation/records/geographicaloriginalofmusic.md @@ -25,6 +25,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook's `No` (Lennart and Andrej: Audio) holds under SEL-12 and SEL-13; recommend adding `Duplicate` as well: this is the same UCI dataset as `geographical_origin_of_music` (Lennart, `No`), here under its PMLB name. The curator has the call. +- Source: UCI 315 "Geographical Original of Music", donated 2014-10-17, DOI 10.24432/C5VK5D; as published: 1,059 tracks of traditional and world music from 33 countries or areas, 68 MARSYAS timbre features (a second file adds chroma features), targets latitude and longitude of the capital of the country of origin. UCI: "The program MARSYAS was used to extract audio features from the wave files" and "All features were transformed to have a mean of 0, and a standard deviation of 1". +- Use case: none named; the source states the task ("predict the geographical origin of music") as a music-information-retrieval question, with no actor or decision; regime IID if it were kept. +- Siblings and duplicates: `geographical_origin_of_music` (OpenML 44965 and the same UCI DOI; Lennart: `No`, `Wrong Domain / Source Modality`, `Time-series (Classification)`), whose comment points to PMLB `4544_GeographicalOriginalofMusic`, the name of this record. External collections: PMLB; TabSTAR (on the sibling). +- Metric: not looked up (the verdict is `No`). +- Risks for /add-dataset: none to pass on; features computed from audio describe the recording's content (SEL-12, SEL-13), and they were scaled over all rows (LEAK-30). +- Priority: none (verdict `No`). + UCI 'Geographical Original of Music' dataset consisting of audio features extracted from 1059 wave files, with the goal of predicting geographic origin (lat/long). Prior TabArena curation and both reviewers (Lennart, Andrej) flagged it as Audio. It is a non-tabular (audio) modality recast as engineered features, where audio models would dominate, so it is not representative tabular ML. Confidently No on the audio-modality exclusion. --- diff --git a/curation/records/global_data_on_sustainable_energy_2000_2020.md b/curation/records/global_data_on_sustainable_energy_2000_2020.md index 01812396..a9be2253 100644 --- a/curation/records/global_data_on_sustainable_energy_2000_2020.md +++ b/curation/records/global_data_on_sustainable_energy_2000_2020.md @@ -3,9 +3,10 @@ unique_name: global_data_on_sustainable_energy_2000_2020 name: Global Data on Sustainable Energy (2000-2020) checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery +- Time-series (Forecasting) tags: - 2nd Tier / Scientfic Discovery - Non-IID (Temporal) @@ -29,4 +30,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, `Time-series (Forecasting)`; SEL-3, SEL-10): a country-year panel compiled for exploration; the source names no target, and the predictive uses it lists are forecasts of a series. +- Source: Kaggle `anshtanwar/global-data-on-sustainable-energy`, uploaded 2023-08-19 by Ansh Tanwar; one CSV (514 KB), one row per country and year 2000-2020 (about 176 countries, so about 3,600 rows) x 21 indicators (access to electricity, clean cooking fuels, renewable capacity per capita, financial flows, renewable share, electricity by source, energy per capita, energy intensity, CO2 per capita, GDP growth, GDP per capita, density, land area, latitude, longitude). The card names no upstream source; the indicators are World Bank / IEA / IRENA / Our World in Data series (PROV-8: the first publishers are those agencies, not this upload). +- The card: "Explore 20-year Insights on Sustainable Energy", "gain profound insights into global energy consumption patterns over time"; its "Potential Use cases" are "Energy Consumption Prediction: Predict future energy usage" and "Carbon Emission Forecasting: Forecast CO2 emissions", plus analysis and tracking items (SDG 7 tracking, equity analysis). +- Use case: none with an actor and a decision at a prediction point; any target picked from the indicators is an identity or near-identity of others (electricity from renewables vs low-carbon share vs renewables share). +- Siblings and duplicates: none among the records (checked by name and slug); `sustainable_development_goals_sdgs_text_classification_challenge` is unrelated. External collections: none found. +- If the checker refutes it: `TBD -> 2nd Tier`, priority 1 (gap -, use case -, source -, shape -; risk: targets that are sums or shares of other columns). +- Reject check (AI, 2026-10-22): stands. SEL-3: the card is an exploration compilation ("Explore 20-year Insights on Sustainable Energy") with no target, and its only predictive uses are SEL-10 forecasts ("Predict future energy usage", "Forecast CO2 emissions") (https://www.kaggle.com/datasets/anshtanwar/global-data-on-sustainable-energy); any target picked from the indicators would be ours, and several are shares/sums of others. + Kaggle country-year panel of sustainable-energy indicators (2000-2020), already tagged 2nd Tier / Scientific Discovery and Non-IID (temporal). It is an aggregated descriptive panel with no canonical predictive target, and it is small (roughly a couple thousand country-year rows), so it leans toward scientific-discovery rather than a genuine predictive task. One could contrive a target (e.g. renewable share, CO2), but this risks leakage among the indicators and is not clearly representative. A human must decide whether any non-leaky target and adequate size exist; keep in 2nd tier. diff --git a/curation/records/golf_play_dataset_extended.md b/curation/records/golf_play_dataset_extended.md index 05fcc8b1..19bcf6cd 100644 --- a/curation/records/golf_play_dataset_extended.md +++ b/curation/records/golf_play_dataset_extended.md @@ -29,6 +29,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the `No` (TabArena workbook reject, the CC above) holds: the uploader made the table for teaching, so it is handmade data (SEL-16, PROV-3), and no decision is named (USE-1). The curator has the call. +- Source: Kaggle `samybaladram/golf-play-extended`, uploaded 2023-11-03 by Samy Baladram, CC BY 4.0; the card: "designed to extend the classic golf dataset" and "especially useful for teaching because it offers many small datasets within it, each one created for a different learning purpose"; no origin of the rows is named. As published: `golf_dataset_long_format_with_text.csv` (1.6 MB) with Outlook, Temperature, Humidity, Windy, Play (target, Yes/No), ID, Date, Weekday, Holiday, Season, Crowded-ness, PlayTime-Hour and the text columns Review, EmailCampaign, MaintenanceTasks, plus mini files built for exercises (`golf_dataset_mini_dirty.csv`, `..._lotsofmissingvalues.csv`, `..._allcategorical.csv`). +- Use case: none from the source ("Learning With This Dataset" lists teaching skills); the CC's reading (did someone play on a given day) would need a temporal split, but nothing says the rows were observed. +- Siblings and duplicates: none among the records; the classic 14-row golf/weather table and its Kaggle re-uploads (`priy998/golf-play-dataset`, `davorbudimir/golf-play`) have no records. External collections: none found. +- Metric: none (no task). +- Risks: the tags `2nd Tier / Scientfic Discovery` and `Non-IID (Temporal)` do not fit a `No`; left for the curator. +- Priority: none (verdict `No`). + Extended version of the classic 'play golf' teaching dataset, already in the TabArena Reject collection. The author describes it as a teaching tool made of many small handmade datasets, not an actual predictive task. It is artificial/handmade/toy data and tiny, so it fails the representativeness criterion. Confidently No. --- diff --git a/curation/records/goodreads_computer_books.md b/curation/records/goodreads_computer_books.md index 4bbc9c8b..c3f76e58 100644 --- a/curation/records/goodreads_computer_books.md +++ b/curation/records/goodreads_computer_books.md @@ -19,6 +19,8 @@ required_split: problem_type: Other original_data_state: One Table source_links: +- https://www.kaggle.com/datasets/muneeralhajri/goodreadscomputerbooks +- '# not this dataset: the general Goodreads API books scrape (jealousleopard, 2019-2020), linked here before 2026-10-22' - https://www.kaggle.com/datasets/jealousleopard/goodreadsbooks source_row: 813 needs_review: @@ -30,6 +32,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the `No` (TabArena workbook reject; the CC asks "what to predict here?") holds: a scraped book listing with no predictive target the source names (SEL-3), close to `books_goodreads` (`No`). One correction: the old link was another dataset; the record's name and year match `muneeralhajri/goodreadscomputerbooks`, now first in `source_links`. The curator has the call. +- Source: Kaggle `muneeralhajri/goodreadscomputerbooks` ("Goodreads-Computer-Books"), uploaded 2021-01-29, CC0; collected from the Goodreads "Computer" category ("This data was entirely scraped via the Webdriver"); as published: one file `Book_Dataset.csv` (146 KB); the card: "The missing values are imputed in this data by the creator" (method not stated). The old link, `jealousleopard/goodreadsbooks`, is the general Goodreads API scrape of 2019-2020 (`books.csv`, 1.6 MB). +- Use case: none from the source ("find out what book you should be reading next", a recommendation idea, SEL-9); a rating would be the only numeric target, with no actor or moment; regime IID. +- Siblings and duplicates: `books_goodreads` (Lennart, Mustafa: `No`, `No Good Target / Scientific Discovery`, crawled data without a real task); `book_price_prediction` not compared. External collections: none found. +- Metric: none (no task). +- Risks: imputation by the uploader with an unknown method (PROV-3, PROV-13); scraped Goodreads ratings. +- Priority: none (verdict `No`). + Goodreads computer-books metadata table, already in TabArena Reject. The curation comment notes it is a book-recommendation toy task built from crawled metadata with no user-id (not personalizable) and no clear predictive target. It is recommendation-flavored / scientific-discovery crawled data without a genuine predictive label, so it fails the real-predictive-task and representativeness criteria. No. --- diff --git a/curation/records/grocery_store_forecasting_challenge_for_azubian.md b/curation/records/grocery_store_forecasting_challenge_for_azubian.md index 8cb284ef..a50c4b48 100644 --- a/curation/records/grocery_store_forecasting_challenge_for_azubian.md +++ b/curation/records/grocery_store_forecasting_challenge_for_azubian.md @@ -6,6 +6,7 @@ checked_by: suggestion: 'No' decision_markers: - Time-series (Forecasting) +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,4 +29,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10; `Duplicate`, PROV-4): weekly sales forecasting, and a re-run of `grocery_store_product_forecasting_challenge` on the same files. +- Source: Zindi "Grocery Store Forecasting Challenge For Azubian", a knowledge hackathon (Senegal cohort, 2022-11-07 to 2024-06-06, 25 participants, "As this is a learning challenge, aside from the rules in the Terms of Use, no other particular rules apply"); the API names `grocery-store-product-forecasting-challenge` (AI Hack, 2022-08) as its parent, and the files are the same (`train.csv` 88 MB, `test.csv` 2,766,947 bytes, `dates.csv`, `holidays.csv`, `stores.csv`). Read through the API; the web page refused the connection. +- Source's words: "Can you forecast the number of products sold per store per week for a local grocery store?"; `type_of_problem`: Forecast; metric RMSE. +- Use case: a store forecasting weekly sales (SEL-10). +- Siblings and duplicates: `grocery_store_product_forecasting_challenge` (the parent, also proposed `No`); the other Azubian re-runs follow the same pattern (`customer_churn_prediction_challenge_for_azubian` is `No`, `Duplicate` of `expresso_churn_prediction`). External collections: none found. +- Reject check (AI, 2026-10-22): stands. PROV-4: a learning re-run of grocery_store_product_forecasting_challenge; the Zindi API lists that competition as its parent, and description, data description and file list are word-for-word the same (https://api.zindi.world/v1/competitions/grocery-store-forecasting-challenge-for-azubian). The SEL-10 half is refuted on the parent (see there), but the duplicate holds whatever the parent's verdict. + Zindi 'grocery store forecasting' competition; the name indicates demand/sales forecasting for a grocery store. Forecasting future sales over a horizon is explicitly out of scope. The data is per-store/per-time sales records, a forecasting setup rather than a static predictive task. No, pending only a human confirmation that there is no reframable fixed target. diff --git a/curation/records/grocery_store_product_forecasting_challenge.md b/curation/records/grocery_store_product_forecasting_challenge.md index 8e1f5fbb..d2ce0742 100644 --- a/curation/records/grocery_store_product_forecasting_challenge.md +++ b/curation/records/grocery_store_product_forecasting_challenge.md @@ -3,7 +3,8 @@ unique_name: grocery_store_product_forecasting_challenge name: grocery-store-product-forecasting-challenge checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Time-series (Forecasting) tags: @@ -28,6 +29,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-10) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Time-series (Forecasting)`, SEL-10): the competition asks for future weekly sales per store, a forecast of a series. +- Source: Zindi "Grocery Store Product Forecasting Challenge", a hackathon of AI Hack (Tunisia), 2022-08-29 to 2022-08-30, 171 participants, access restricted; files `train.csv` (88 MB), `test.csv`, `dates.csv` (date features per period), `holidays.csv`, `stores.csv`, `SampleSubmission.csv`. Description and data pages did not load (zindi.africa now redirects to zindi.world, whose web page refused the connection); read through the API. +- Source's words: "Can you forecast the number of products sold per store per week for a local grocery store?"; `type_of_problem`: Forecast; metric RMSE. +- Use case: a store forecasting its weekly sales for stocking, the forecasting setting SEL-10 excludes; no fixed per-row task that TP-10 could rescue was found (the store-week panel with a date table is the series itself). +- Siblings and duplicates: `grocery_store_forecasting_challenge_for_azubian` is a later knowledge re-run of this competition (the API lists this one as its parent; the same files, `test.csv` 2,766,947 bytes in both). External collections: none found. +- Reject check (AI, 2026-10-22): does not stand. SEL-10 alone does not decide a store panel: rossmann_store_sales (store sales, given forecast horizon) ships as Yes (Andrej), and TP-10 (amended 2026-10-22) admits a per-entity panel when measured. Here: "54 different stores ... 33 different products", weekly target with promotions, transactions, holidays, "forecast the next 8 weeks for the same stores and same products" (https://api.zindi.world/v1/competitions/grocery-store-product-forecasting-challenge). Whether it passes TP-10 needs the data. +- For /add-dataset: TBD -> 2nd Tier, priority 2. Add stage: (1) PROV-8/PROV-4: 54 stores x 33 categories with onpromotion, transactions, holidays and stores tables match Corporacion Favorita's Kaggle 'Store Sales - Time Series Forecasting' (daily); confirm, and if so the source is Favorita (weekly aggregation by Zindi is a cut, PROV-6), with this record and store_sales_forecasting_challenge_for_beginners checked as duplicates; (2) TP-10 on the 8-week windows: GBM on point-in-time features vs each store-category's naive profile, and whether top solutions used target lags; (3) the log1p RMSE metric (MET-4). + Zindi competition whose stated task is to forecast the number of products sold per store per week for a local grocery store. This is sales forecasting over a temporal horizon, explicitly out of scope. The data is per-store weekly sales, a forecasting setup. Confidently No on the forecasting exclusion. --- diff --git a/curation/records/health_insurance.md b/curation/records/health_insurance.md index 1e135e2b..836c4656 100644 --- a/curation/records/health_insurance.md +++ b/curation/records/health_insurance.md @@ -32,6 +32,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the `No` (TabArena workbook reject, the CC above) holds: the source is an econometric estimate of a causal effect, not a prediction a decision uses (SEL-3, USE-1). Counter-point for the curator: social science is thin in the benchmark (1 accepted), but a thin domain does not make a task (the priority factors never override a Must). +- Source: Olson, C. A. (1998), "A comparison of parametric and semiparametric estimates of the effect of spousal health insurance coverage on weekly hours worked by wives", Journal of Applied Econometrics 13(5):543-565, on the March 1993 Current Population Survey; distributed as `HI` in the R package Ecdat; OpenML 44993 (`health_insurance`, v3, uploaded 2022-12-22): 22,272 married women x 12 features, target `whrswk` (hours worked per week by the wife), with `wght` a "sampling weight (should be ignored)". +- Use case: none; the paper asks how much spousal coverage changes wives' labour supply (an effect estimate); nobody decides on a predicted weekly hours figure. Regime would be IID (one cross-section). +- Siblings and duplicates: none among the records (checked by name, OpenML id, Olson, spousal). External collections: TabSTAR (as recorded). +- Metric: none for prediction (the paper compares estimators of an effect). +- Risks: `year` holds 1993, the survey year; by PROV-9 the first publication of this table is the 1998 paper and its data archive (not changed: a human-verdict record). +- Priority: none (verdict `No`). + OpenML dataset derived from the March 1993 CPS, used in an econometrics study estimating the effect of spousal health-insurance coverage on weekly hours worked by wives. The curation comment notes the target (predicting work hours of married wives in 1993) is outdated, not a genuine predictive task, and confounded by income. It is already in TabArena Reject. It fails on outdatedness and lack of a meaningful, non-confounded predictive target. No. --- diff --git a/curation/records/heart_statlog.md b/curation/records/heart_statlog.md index beefc252..2e482fc5 100644 --- a/curation/records/heart_statlog.md +++ b/curation/records/heart_statlog.md @@ -3,9 +3,9 @@ unique_name: heart_statlog name: heart_statlog checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Too Small +- Duplicate tags: - AI-Filled (Verify) domain: medical & healthcare @@ -22,6 +22,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-6): Statlog (Heart) is a 270-row cut of the Cleveland heart-disease data, which ships as `heart_disease_cleveland` (Lennart, `Yes`); the workbook's reject (Lennart, Andrej: sample size) points the same way, for another reason. +- Source: UCI 145 "Statlog (Heart)", DOI 10.24432/C57303, from the StatLog project; UCI: "This dataset is a heart disease database similar to a database already present in the repository (Heart Disease databases) but in a slightly different form"; `heart.doc`: "13 attributes (which have been extracted from a larger set of 75)", "Absence (1) or presence (2) of heart disease", "270 observations", and a cost matrix (a missed disease costs 5, a false alarm 1). +- Measured (2026-10-22, the two small UCI files): all 270 rows of `heart.dat` match a row of `processed.cleveland.data` (297 rows without missing values) on all 13 features, and all 270 labels agree (Statlog 2 = Cleveland num > 0). So it is a subset of `heart_disease_cleveland` with the binary target we already ship. +- Does the cost matrix make it a distinct task (PROV-7)? No: the same target, units and features; the matrix is a metric that `heart_disease_cleveland` could adopt (MET-6, MET-7), noted there if the curator wants it. +- On `Too Small`: 270 rows with 120 positives is not a handful per fold (TP-5); the old marker would not hold by itself. +- External collections: OpenML 53 (heart-statlog), PMLB, many benchmark suites. +- Reject check (AI, 2026-10-22): stands. PROV-6, by measurement (re-run 2026-10-22): all 270 rows of UCI heart.dat equal a row of processed.cleveland.data (297 complete rows) on all 13 features and all 270 labels agree (2 = num>0); UCI: "similar to a database already present in the repository ... in a slightly different form" (https://archive.ics.uci.edu/dataset/145/statlog+heart). heart_disease_cleveland ships (Yes). The cost matrix is a metric, not a distinct task (PROV-7 does not apply). + The Heart (Statlog) dataset is a classic UCI heart-disease classification set with only about 270 rows. Prior TabArena curation and both reviewers flagged it as tiny / sample-size. While the task itself (presence of heart disease) is a legitimate medical binary classification, the dataset is too small to be useful for the benchmark. No on the size criterion. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/hepmass.md b/curation/records/hepmass.md index d6a3af8d..fa9e8bad 100644 --- a/curation/records/hepmass.md +++ b/curation/records/hepmass.md @@ -3,9 +3,10 @@ unique_name: hepmass name: HEPMASS checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - AHDS (Artifical/Handmade/Deterministic/Simulated) +- Not Representative tags: - AI-Filled (Verify) - 2nd Tier / Scientfic Discovery @@ -30,6 +31,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, `Not Representative`; SEL-16): simulated collider events, the same group, pipeline and question as `higgs`, which the curator rejected. +- Source: UCI 347 "HEPMASS", donated 2016-01-27 (Whiteson group), with Baldi, Cranmer, Faucett, Sadowski, Whiteson, "Parameterized Machine Learning for High-Energy Physics" (arXiv 1601.07913). UCI: the signatures "are learned from Monte Carlo simulations of the collisions that produce these particles and the resulting decay products"; 10.5M events, three variants (`1000`, `not1000`, `all`), each 7M train / 3.5M test, "50% of the data is from a signal process, while 50% is from the background process", 27 normalized features plus a mass that for background rows is "selected randomly". +- Label: Monte Carlo truth of a hypothetical new particle, balanced by construction; the mass column is a hypothesis parameter, not a measurement. +- Precedent: `higgs` (Lennart, 2026-09-22: `No`, `AHDS (Artifical/Handmade/Deterministic/Simulated)`, `Not Representative`; simulation-vs-simulation separability on balanced classes is not the physics objective, and the field has its own benchmarks: HiggsML 2014, FAIR Universe). +- Siblings: `higgs` (`No`), `susy`, `higgs_tautau` (same group; open), `higgsml_2014`, `fair_universe_higgsml_uncertainty`, `fair_higgs_boson_decay`. +- Reject check (AI, 2026-10-22): stands. Same group, pipeline and setup as higgs (the curator's No, 2026-09-22; SEL-16 Cases "Higgs"): UCI 347, "learned from Monte Carlo simulations" and "50% of the data is from a signal process", balanced by construction, with a hypothesised-mass column (https://archive.ics.uci.edu/dataset/347/hepmass). No fact sets it apart from higgs. + Third UCI benchmark from the Whiteson group, same simulation family as `higgs` / `susy`. UCI page: 10.5M Monte Carlo events, "27 normalized features (22 low-level and 5 high-level)" plus a mass feature; 50% signal; three variants: `1000` (signal mass fixed at 1000, no mass feature), `not1000` (mass drawn from {500, 750, 1250, 1500}) and `all` ({500, ..., 1500}); background rows get a mass drawn at random from the same set; 7M train / 3.5M test per variant; donated 2016-01-27, CC BY 4.0. Associated paper per the UCI physics portal: Baldi, Cranmer, Faucett, Sadowski, Whiteson, "Parameterized Machine Learning for High-Energy Physics" (arXiv 1601.07913). Curation notes: the mass column is a *hypothesis parameter*, not a measurement (random noise for background by construction), so a plain classifier on `all` mixes five tasks; `1000` is the clean single-task variant. Features are "normalized" like `higgs` (global standardization, judged harmless there). Simulated physics with dedicated benchmarks (criterion 4B): verdict should follow `higgs`. Data not inspected. diff --git a/curation/records/higgs_tautau.md b/curation/records/higgs_tautau.md index 0d2eeeaa..7343d7b7 100644 --- a/curation/records/higgs_tautau.md +++ b/curation/records/higgs_tautau.md @@ -3,9 +3,10 @@ unique_name: higgs_tautau name: Higgs to Tau Tau (Baldi, Sadowski & Whiteson 2015) checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - AHDS (Artifical/Handmade/Deterministic/Simulated) +- Not Representative tags: - AI-Filled (Verify) - 2nd Tier / Scientfic Discovery @@ -28,6 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, `Not Representative`; SEL-16): two Monte Carlo files, signal and background, from the same group and pipeline as `higgs`, which the curator rejected. +- Source: UCI physics portal (mlphysics.ics.uci.edu/data/htautau/), files dated 2017-01-03, for Baldi, Sadowski & Whiteson, "Enhanced Higgs Boson to tau+ tau- Search with Deep Learning", Phys. Rev. Lett. 114, 111801 (2015). The readme: "The data has been produced using Monte Carlo simulations. The first 10 columns are the low level features, followed by the 15 high level features"; files `htautau.txt.gz` (signal, 3.8 GB) and `ztautau.txt.gz` (background, 3.7 GB); the label is which file a row comes from; the event count is not stated. +- Unlike `higgs`, the signal is the observed 125 GeV Higgs; the curator's reason for `higgs` (simulation-vs-simulation on a class balance set by construction, not the physics objective) applies unchanged. The field's benchmark for this very channel is HiggsML 2014 (`higgsml_2014`), with event weights and the AMS. +- Precedent: `higgs` (Lennart, 2026-09-22: `No`, `AHDS (Artifical/Handmade/Deterministic/Simulated)`, `Not Representative`; simulation-vs-simulation separability on balanced classes is not the physics objective, and the field has its own benchmarks: HiggsML 2014, FAIR Universe). +- Siblings: `higgs` (`No`), `susy`, `hepmass`, `higgsml_2014`, `fair_universe_higgsml_uncertainty`, `fair_higgs_boson_decay`. +- Reject check (AI, 2026-10-22): stands. readme: "The data has been produced using Monte Carlo simulations" (https://mlphysics.ics.uci.edu/data/htautau/readme.txt), two files whose label is the file. The curator's reason for higgs (simulation-vs-simulation separability without weights or AMS) applies unchanged; the real 125 GeV signal does not touch that reason. SEL-16 and the higgs precedent. + Dataset behind "Enhanced Higgs Boson to tau+ tau- Search with Deep Learning" (PRL 114, 111801, 2015), hosted only on the UCI physics portal (not UCI ML repo / OpenML). Readme: two gzipped CSVs, `htautau` (signal, 3.8 GB) and `ztautau` (background, 3.7 GB), "produced using Monte Carlo simulations"; "The first 10 columns are the low level features, followed by the 15 high level features" (lepton momenta, MET, jets; axial MET, scalar momentum sum, delta-phi/eta/R of the leptons, m_ll, MMC missing mass, sphericities, visible mass). Label = which file a row comes from. Event count and whether features are standardized are not in the readme; check on download. Same pipeline and curation question as `higgs` (simulated collider events, criterion 4B); the physics target here is the real 125 GeV Higgs rather than an exotic one. Verdict should follow `higgs`. Data not inspected. diff --git a/curation/records/higgsml_2014.md b/curation/records/higgsml_2014.md index 94fe7ab2..c1b81559 100644 --- a/curation/records/higgsml_2014.md +++ b/curation/records/higgsml_2014.md @@ -4,6 +4,7 @@ name: Higgs Boson Machine Learning Challenge (HiggsML, ATLAS / Kaggle 2014) checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - AHDS (Artifical/Handmade/Deterministic/Simulated) tags: @@ -29,6 +30,17 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-16 (PROV-3)) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16): ATLAS full-detector Monte Carlo events of the 2014 HiggsML challenge, simulated physics with its own benchmark. +- Source: ATLAS collaboration, CERN Open Data record 328 (2014, CC0): "built from official ATLAS full-detector simulation", "an extended version of the data provided for the Higgs Boson Machine Learning Challenge on Kaggle"; 818,238 events, 35 columns (EventId, 13 `DER_*`, 17 `PRI_*`, Weight, Label, KaggleSet, KaggleWeight); Kaggle used 250,000 training events; undefined values are -999.0. +- Use case: event selection for an H -> tau tau search; the label is Monte Carlo truth; regime IID within the simulation. +- Siblings and duplicates: `higgs` (Lennart, 2026-09-22: No, `AHDS`, `Not Representative`), `higgs_tautau` and `fair_universe_higgsml_uncertainty` (AI, 2nd Tier, `AHDS`), `fair_higgs_boson_decay` (AI, No). Not a duplicate of UCI `higgs` (other process, other simulation). +- Papers and pages: Adam-Bourdarios et al., "The Higgs boson machine learning challenge", JMLR W&CP 42 (2015); Kaggle `higgs-boson` (2014, 1,785 teams). +- Metric: AMS on the weighted selection (Kaggle evaluation, paper Sec. 4), deliberate; needs Weight, which ships; the paper, Sec. 2.2: "classification accuracy is a very poor measure of success in this case". +- Counterpoint for the checker: physics & astronomy has no accepted dataset (gap), and unlike `higgs` this release carries the physics metric (weights and AMS), which answers part of the curator's metric argument on `higgs`; the simulation itself (SEL-16, "Higgs" in its Cases) still decides. +- Reject check (AI, 2026-10-22): does not stand. The release is simulated ("built from official ATLAS full-detector simulation", https://opendata.cern.ch/record/328). But the curator's higgs No rested on the missing physics objective ("AUC on balanced classes, which is not the physics objective"), and named HiggsML 2014, with event weights and AMS, as the field's benchmark that encodes "exactly what this one omits". This dataset carries those weights and the AMS. The curator also wrote that training on simulation "is the real workflow". So the higgs precedent does not carry over as it stands. Whether SEL-16's "simulated physics that has its own benchmarks" excludes the field's benchmark itself is a reading for the curator, not a clear failure. Judged together with the other three: they share higgs's facts; this one differs on exactly the fact the curator named. +- For /add-dataset: TBD -> 2nd Tier, priority 2 (gap + physics & astronomy; source + CERN Open Data CC0; risk - SEL-16 reading). A weighing call for the curator first: is HiggsML 2014 in scope under SEL-16? If yes, the add stage must check: AMS on the weighted selection as the metric (MET-4; Weight as a scoring column, never a feature); use the KaggleSet split (t/b/v) or justify IID; -999.0 to missing; whether KaggleWeight or Weight is the scoring weight; and that `fair_universe_higgsml_uncertainty` is not the better successor version (TP-6). + The Kaggle HiggsML challenge (2014, 1785 teams), a *different* dataset from UCI `higgs`: ATLAS full detector simulation, H -> tau tau signal at 125 GeV vs Z -> tau tau, t tbar and W backgrounds. Full release on CERN Open Data record 328 (CC0): 818,238 events, 30 features (`DER_*` derived, `PRI_*` primary) plus EventId, Weight, Label, KaggleSet (t/b/v/u) and KaggleWeight; Kaggle used 250,000 training events. Undefined values are coded -999.0 (challenge doc, Appendix B: "their value is -999.0, which is outside the normal range of all variables") -> convert to NA. Why it is not a plain classification set (Adam-Bourdarios et al., JMLR W&CP 42): the simulation is enriched in signal while real S/B is ~2/1000, so every event carries an importance weight and the metric is the Approximate Median Significance (AMS) on the weighted selection; Sec. 2.2: "classification accuracy is a very poor measure of success in this case. Indeed, the AMS is not a function of classification accuracy." Weight must never be a feature. An unweighted AUC task on Label is possible but is a different, less physical task; still simulated (criterion 4B). This is what the old `higgs` comment about a "challenge with special evaluation/postprocessing" referred to. Kaggle discussion threads not read here (forum links in the write-up). Data not inspected. diff --git a/curation/records/hospital_ratings.md b/curation/records/hospital_ratings.md index 83cf5a20..18ef542b 100644 --- a/curation/records/hospital_ratings.md +++ b/curation/records/hospital_ratings.md @@ -4,9 +4,9 @@ name: Hospital ratings checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: -- Data Quality Issue -- No Good Target / Scientific Discovery +- No Good Target (yet) tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -20,6 +20,8 @@ original_data_state: Database (or multiple to-be-joined tables) source_links: - https://www.kaggle.com/datasets/center-for-medicare-and-medicaid/hospital-ratings - Medicare.gov +- '# origin named by the Kaggle card (CMS Hospital Compare, now the CMS Provider Data Catalog)' +- https://data.medicare.gov/data/hospital-compare source_row: 835 needs_review: - ai_unverified @@ -30,4 +32,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a CMS hospital listing published for comparison, not for a predictive task; the only candidate target is CMS's own composite star rating. +- Source: CMS Hospital Compare (Medicare.gov), the `Hospital General Information` file, mirrored on Kaggle by Kaggle staff (dataset 1743, CC0, file of 2019-09-20, 2.6 MB); the card: "official datasets used on the Medicare.gov Hospital Compare Website ... These data allow you to compare the quality of care at over 4,000 Medicare-certified hospitals". As published: one table, one row per hospital ("over 4,000", count not checked), 28 columns: ID, name, address, city, state, ZIP, county, phone, hospital type, ownership, emergency services, EHR meaningful use, `Hospital overall rating` (1-5 stars) and seven measure-group national comparisons (mortality, safety, readmission, patient experience, effectiveness, timeliness, imaging; each above / same / below national average) with footnotes. +- Use case: none stated; the card's "Inspiration" is finding the best hospital for a broken leg, a lookup. A predicted star rating has no actor: CMS computes the rating from the measure-group scores itself. A curated use case (predict a hospital's rating before CMS publishes it) would be written by us; fit approximate at best. Regime IID (one row per hospital, one snapshot). +- Siblings and duplicates: none among `curation/records/` (grep for hospital rating, CMS, Hospital Compare); external collections: none found. +- Papers and pages: the Kaggle card; CMS star-rating methodology (latent variable model over the measure groups, then clustering into stars) not read here. +- Metric: none found (Kaggle card, no task, no competition), so the fallback. +- Risks for /add-dataset: the target is a composite of the same seven measure groups the table ships as comparisons (SEL-5 when the rule is deterministic, a leak otherwise, LEAK checks); name, address and phone are identifiers; the release is old (2019 snapshot), a newer one from data.cms.gov exists. The old `Data Quality Issue` marker is dropped: nothing irreversible is shown, the concern is the target (SEL-3), now `No Good Target (yet)`. +- Priority: gap 0 (medical not thin), use case 0 (none stated), source + (CMS), shape + (over 4,000 rows, mixed categorical columns); `original_data_state` says Database but the Kaggle release is one file (2.6 MB), left as set, risk - (target computed from the shipped measure groups). + Kaggle dataset of CMS/Medicare.gov hospital ratings, already tagged 2nd Tier / Scientific Discovery. Predicting a hospital star rating is plausible as a tabular task, but the overall rating is computed deterministically from the same quality-measure columns, creating a strong leakage risk, and much of the file is administrative/descriptive metadata rather than predictive features. A human must determine whether a non-leaky target and adequate clean feature set exist (and how to join the multiple measure tables). Keep in 2nd tier pending that inspection. diff --git a/curation/records/hulkshare_recommendation_algorithm_challenge.md b/curation/records/hulkshare_recommendation_algorithm_challenge.md index d7fc217e..4224aa1b 100644 --- a/curation/records/hulkshare_recommendation_algorithm_challenge.md +++ b/curation/records/hulkshare_recommendation_algorithm_challenge.md @@ -3,10 +3,11 @@ unique_name: hulkshare_recommendation_algorithm_challenge name: hulkshare-recommendation-algorithm-challenge checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: -- Needs extensive data wrangling - Out-of-scope Task (CTR/RecSys/Ranking) +- Needs extensive data wrangling tags: - AI-Filled (Verify) collections: @@ -29,6 +30,17 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-9) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Out-of-scope Task (CTR/RecSys/Ranking)`, SEL-9): a recommendation challenge; the model scores songs so that "HulkShare [can] rank its songs and recommend the best content". +- Source: Zindi, Hulkshare Recommendation Algorithm Challenge (2022-02-03 to 2022-05-01, 516 participants), data by Hulkshare Limited. As published: Train.csv (10 MB) and Test.csv keyed by song `Vid` and `Session_id`, plus ~9 GB of per-session listening frames (each song cut into 1,000 frames; play, pause, skip, rewind) and a play-count file; target: whether the song has a high likes-to-views ratio. +- Use case: Zindi: "How does a platform recommend good content without input from a user"; the decision is the ranking of songs for recommendation; the Zindi problem type is "Collaborative Filtering". +- Siblings and duplicates: none among `curation/records/`; external collections: none found. +- Papers and pages: the Zindi page (via the API); winners' write-ups not read. +- Metric: RMSE on the target (Zindi Evaluation page), a leaderboard metric. +- Also: the release is raw session logs that need heavy aggregation (`Needs extensive data wrangling`). +- Reject check (AI, 2026-10-22): does not stand. The task is not user-item recommendation. The unit is one song (Vid), the target is that song's observed likes-to-views ratio, scored by RMSE, and the features are its aggregated listening sessions: "predicts whether the song has a high number of likes to views (listens) since the song was uploaded" (Zindi Data Description, https://zindi.africa/competitions/hulkshare-recommendation-algorithm-challenge). There are no user-item pairs and no ranking metric. Downstream ranking is the use of the prediction, as in many in-scope tasks, so SEL-9 rests on the competition's title and its 'Collaborative Filtering' tag, not on how the prediction is made. +- For /add-dataset: TBD -> 2nd Tier, priority 2. The add stage must check: (1) the time window. If likes and views are counted over the same 6 months as the frames, it may be a look-back (SEL-6) or a target leak through Num_sessions (views are the target's denominator, LEAK). (2) Whether the target is a continuous ratio or a 0/1 high-ratio flag. (3) The licence: Zindi rules say the data is 'sole property of Zindi and the competition host' with no redistribution (house rules on shipping). (4) The wrangling: 9 GB of per-session frames to aggregate per song (`Needs extensive data wrangling`). (5) Download access (login). + Zindi 'recommendation algorithm' challenge whose task is to predict whether a streamed song is liked from viewing patterns. This is a music-recommendation / implicit-feedback task, which is out of scope (RecSys/ranking). The record also notes gigabytes of raw data requiring heavy feature engineering. It fails the out-of-scope-task criterion and is impractical without extensive wrangling. No. --- diff --git a/curation/records/human_genome_annotation.md b/curation/records/human_genome_annotation.md index 4a06a2f1..82f35f8b 100644 --- a/curation/records/human_genome_annotation.md +++ b/curation/records/human_genome_annotation.md @@ -3,10 +3,9 @@ unique_name: human_genome_annotation name: Human genome annotation checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- TBD tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -29,4 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): a CSV conversion of the Ensembl GRCh38.92 genome annotation, published to "explore the structure within the annotations"; a reference listing with no target and no decision. +- Source: Ensembl release 92 GTF (2018), converted with the uploader's `gtf2csv` script and posted on Kaggle (dataset 31883, DbCL-1.0, one file `Homo_sapiens.GRCh38.92.csv`, 501 MB, 2019). As published: one row per annotated feature (gene, transcript, exon, CDS, UTR, ...) with chromosome, start, end, strand, source and the GTF attributes as columns. +- Use case: none; the card's "Inspiration" is visualization and analysis. Any label (feature type, biotype) is the annotation itself, read off coordinates and attributes, not an outcome someone predicts. +- Siblings and duplicates: none among `curation/records/`. +- Papers and pages: the Kaggle card; Ensembl release 92. +- Metric: none found (Kaggle card). +- Reject check (AI, 2026-10-22): stands. The Kaggle card's stated aim is "to encourage data scientists who are interested in genomics to explore the structure within the annotations", and the rows are Ensembl GRCh38.92 GTF features (https://www.kaggle.com/datasets/alfrandom/human-genome-annotation). It is a reference listing. Any label (feature type, biotype) is the annotation itself, and predicting it in practice is a sequence task. SEL-3. + Kaggle 'Human genome annotation' dataset, already tagged 2nd Tier / Scientific Discovery, with no task description in the record. Genome-annotation tables are typically descriptive feature/coordinate listings (gene/region annotations) without a canonical predictive target, leaning toward scientific discovery; there is also a risk the underlying signal is sequence-based rather than genuinely tabular. There is too little information to confirm a usable target or split. A human must inspect whether a real predictive label exists and whether the features are tabular; keep in 2nd tier. diff --git a/curation/records/ibm_skillsbuild_hydropower_climate_optimisation_challenge.md b/curation/records/ibm_skillsbuild_hydropower_climate_optimisation_challenge.md index c3aff957..9485cc00 100644 --- a/curation/records/ibm_skillsbuild_hydropower_climate_optimisation_challenge.md +++ b/curation/records/ibm_skillsbuild_hydropower_climate_optimisation_challenge.md @@ -4,8 +4,8 @@ name: ibm-skillsbuild-hydropower-climate-optimisation-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: -- Needs extensive data wrangling - Time-series (Forecasting) tags: - Non-IID (Temporal) @@ -30,6 +30,18 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-10) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Time-series (Forecasting)`, SEL-10): the task is to "forecast the total daily energy consumption (kWh) per data user for one month into the future" from 5-minute meter series. +- Source: Zindi with IBM SkillsBuild (2025-03-03 to 2025-04-14, 1,241 participants); data curated by CISNR, UET Peshawar, from micro-hydropower plants in the Kalam region of Pakistan. As published: Data.zip (166 MB) of 5-minute readings per consumer device and data user (voltage per phase, current, power factor, kWh) and Climate Data.zip (temperature, dew point, wind speed, precipitation); submission rows are user-days of the following month. +- Use case: load planning of an off-grid micro-hydropower plant; the prediction is future values of each user's consumption series over a one-month horizon. +- Siblings and duplicates: none among `curation/records/`. +- Papers and pages: the Zindi page; top solutions not read. +- Metric: RMSE on daily kWh (Zindi Evaluation page), a leaderboard metric. +- Counterpoint for the checker: the workbook note says "time-series, but the 1st place solution is tabular"; a tabular model on lagged or calendar features is still a forecast of the series, and SEL-11 / TP-10 would need the naive-profile test and top solutions without lagged targets, which cannot be shown from the source. +- Reject check (AI, 2026-10-22): does not stand. The source says it is a forecast ('forecast the total daily energy consumption (kWh) per data user for one month into the future', https://zindi.africa/competitions/ibm-skillsbuild-hydropower-climate-optimisation-challenge). But it is a panel of users, and the 1st-place write-up (https://zindi.africa/competitions/ibm-skillsbuild-hydropower-climate-optimisation-challenge/discussions/26290) uses LightGBM on daily climate, calendar, holiday and Ramadan features with 5-fold CV and names no lagged target. SEL-11/TP-10 can bring it back into scope, and only the naive-profile measurement on the data can settle that. +- For /add-dataset: TBD -> 2nd Tier, priority 4. Add stage: run TP-10 (a model on the features against each user's naive daily profile); check whether the test month comes with 5-minute voltage, current or power-factor readings (if so, kWh is close to V*I*PF: Trivial or LEAK); read the 1st-place notebook (discussion 26337) for lagged targets; temporal split by month. +- Priority set to 2 by the main session (the checker proposed 4), like the other refuted forecasting-like panels (grocery_store_product_forecasting_challenge, uber_movement_sanral_cape_town_challenge): the TP-10 risk takes a point off. + Zindi IBM SkillsBuild hydropower/climate challenge, flagged as time-series but with a note that the winning solution was tabular. The framing (predicting/optimizing hydropower output from climate inputs over time) leans toward forecasting, which is out of scope, yet the tabular winning approach suggests it might be reframable as a fixed predictive regression with a temporal split. It likely needs substantial wrangling of climate/operational data. A human must determine whether the target is a genuine static prediction vs. a horizon forecast and assess data size/quality; keep in 2nd tier pending that. --- diff --git a/curation/records/income_prediction_challenge_for_azubian.md b/curation/records/income_prediction_challenge_for_azubian.md index eecfb151..f2b92620 100644 --- a/curation/records/income_prediction_challenge_for_azubian.md +++ b/curation/records/income_prediction_challenge_for_azubian.md @@ -3,7 +3,10 @@ unique_name: income_prediction_challenge_for_azubian name: income-prediction-challenge-for-azubian checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 2 +decision_markers: +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a Zindi re-run of UmojaHack India 2022 whose columns are those of the UCI Census-Income (KDD) data, so very likely a renamed copy of `census_income`; not a fast reject because the source never names its origin. +- Source: Zindi, "Income Prediction Challenge For Azubian" (knowledge challenge, 2022-11-07 to 2024-06-06, Senegal); the Zindi API gives `umojahack-india-2022` (2022-10-15, private, Indian citizens) as its parent, and both list the same files byte for byte (Train.csv 68,209,849 B, Test.csv 28,207,389 B, VariableDefinitions.csv 1,674 B). As published: "There are ~200 000 individuals in train and ~100 000 individuals in the test file ... predict if an individual earns above 50 000 of a specific currency", target `income_above_limit`; "This data has been collected from a random population." +- Use case: Zindi: "reduce the cost and improve the accuracy of monitoring key population indicators such as income level in between census years" for policymakers; fit approximate (the rows look like 1994-95 US CPS records, not the population the page speaks of). Regime IID (one row per person, no time index). +- Siblings and duplicates: the column names used by public notebooks on this data (GitHub `LewyseK/Income-predictor` dashboard.py, Hugging Face Space `Faith-theAnalyst/income_FastAPI`: `worker_class`, `industry_code`, `industry_code_main`, `occupation_code_main`, `total_employed`, `household_summary`, `vet_benefit`, `tax_status`, `gains`, `losses`, `stocks_status`, `citizenship`, `mig_year`, `country_of_birth_mother`, `importance_of_record`) are the KDD Census-Income attributes renamed (class of worker, industry / major industry code, num persons worked for employer, veterans benefits, tax filer status, capital gains / losses, dividends from stocks, migration year, instance weight), and the ~200k / ~100k rows and the 50,000 threshold match its 199,523 / 99,762 rows; so very likely a duplicate of `census_income` (TBD -> 2nd Tier, `Outdated`) and close to `adult`. `dlix_botswana` uses the same `income_above_limit` submission format (check it in the same pass). Not to be confused with `predictive_insights_youth_income_prediction_challenge` (South African youth survey). +- Papers and pages: the Zindi pages above; no paper. The UCI Census-Income (KDD) page is the likely origin. +- Metric: F1 on the hard 0/1 label (Zindi Evaluation page: "The error metric for this competition is the F1 score"); a leaderboard metric, deliberate (MET-4), threshold not fixed by the source beyond submitting labels (MET-6); inputs none. +- Risks for /add-dataset: confirm the duplicate by rows against UCI census-income (needs a Zindi login or a mirror), then the verdict follows `census_income` (PROV-4, PROV-5); `importance_of_record` is the CPS instance weight and never a feature; the old note "not a re-upload of a known income dataset" is the open question. +- Priority: gap + (social science thin), use case 0 (approximate), source 0 (Zindi names no origin), shape + (~300k rows, ~40 mixed columns), risk - (likely duplicate). + Zindi competition to predict whether an individual earns above or below an income threshold, an adult-census-style binary classification on socioeconomic/demographic tabular features. This is a genuine real-world predictive tabular task with a clear target, mapping well to the IID classification criterion. Plausibly a Yes but the record carries no size/feature details. A human must verify row count after cleaning, the exact feature set, the actual target column, and that it is not a re-upload of a known income dataset. --- diff --git a/curation/records/indabax_guinea_2022.md b/curation/records/indabax_guinea_2022.md index 916aa658..8addb23c 100644 --- a/curation/records/indabax_guinea_2022.md +++ b/curation/records/indabax_guinea_2022.md @@ -3,7 +3,9 @@ unique_name: indabax_guinea_2022 name: indabax-guinea-2022/ checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,6 +30,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4, PROV-5): the Sapa "customer response" data of the DSN 2021 AI Bootcamp hackathon, which is the Kaggle `marketing_campaign` table (2,240 customers) with renamed columns; `marketing_campaign` ships as `Yes`. +- Source: Zindi, IndabaX Guinea 2022 Hackathon (2022-08-19, 4 hours, private); the Zindi API gives `ai-bootcamp-2021` (Data Science Nigeria 2021 AI Bootcamp Qualification Hackathon, 773 participants) as its parent, with the same files (Train.csv 160,318 B, Test.csv 67,526 B, SampleSubmission.csv 8,748 B). The "sapa.com" company and its CEO "Mr Echoke" are the hackathon's fictional framing. As published: "split into train and test with a volume of 1568 and 672"; 26 variables from `Year_of_Birth` to `User_Response`. +- Use case: respond / not respond to a marketing campaign; regime IID. +- Siblings and duplicates: the variables map one to one onto `marketing_campaign` (Kaggle rodsaldanha/arketing-campaign, Parr-Rud 2014; `Yes`, v2 folder `datasets/_dev/tabarena-v0pt2/marketing_campaign/`, 2,240 rows): Year_Birth, Education, Marital_Status, Income, Kidhome, Teenhome, Dt_Customer, Recency, NumDealsPurchases, NumWeb/Catalog/StorePurchases, Mnt* (wines, fruits, meat, fish, sweets, gold), NumWebVisitsMonth, AcceptedCmp1-5, Complain, Response; 1,568 + 672 = 2,240. The same table is under `customer_personality_analysis` and `customer_personality_analysis_2`. Precedent: `indabax_drc_2022` and `indabax_eswatini_2022` (Lennart, 2026-10-12, `Duplicate` on the parent competition and identical file sizes). +- Papers and pages: the two Zindi pages (via the API). +- Metric: F1 (Zindi Evaluation page). +- Reject check (AI, 2026-10-22): stands. Zindi API: parent 'ai-bootcamp-2021', and the files are byte-identical (Train.csv 160,318, Test.csv 67,526, SampleSubmission.csv 8,748). The data page's 1,568 + 672 = 2,240 rows and a one-to-one column mapping point to the shipped `marketing_campaign` (Yes). The Zindi re-run alone already makes it a PROV-4 duplicate (precedent indabax_drc_2022, Lennart 2026-10-12). https://zindi.africa/competitions/indabax-guinea-2022 + IndabaX Guinea 2022 provides a structured dataset (described as pulled from a CRM-style database) to predict customer response to marketing campaigns, a marketing-response binary classification on tabular features. This is a legitimate real-world predictive tabular task aligned with the IID criterion. Plausible Yes, though the record has no size/feature specifics and 'sapa.com database' provenance is vague. A human must verify final table size, the response target, feature provenance, and uniqueness. --- diff --git a/curation/records/indabax_hackathons.md b/curation/records/indabax_hackathons.md index 5e918c3b..b65538d5 100644 --- a/curation/records/indabax_hackathons.md +++ b/curation/records/indabax_hackathons.md @@ -4,8 +4,10 @@ name: indabax-hackathons checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Needs extensive data wrangling +- Duplicate tags: - AI-Filled (Verify) collections: @@ -39,4 +41,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: not one dataset but twelve IndabaX-type Zindi hackathons; most are re-runs of recorded competitions or out of scope, one (Tunisia 2025 Challenge 1) may deserve its own record. +- Source: the Zindi API for each linked competition (2022-2025, all private hackathons). As published, per link: Benin Republic 2024, cryptojacking detection, parent `indabax-rwanda-2024-hacktathon` (recorded under `ai_national_summit_2024_competition`); Madagascar 2023, data type NLP, accuracy; Mozambique 2023, weekly product sales per store, parent `grocery-store-product-forecasting-challenge` (record `grocery_store_product_forecasting_challenge`, a forecast, SEL-10); `la-detection-des-fraudes` 2023, Xente fraud, parent `indabax-benin-2022`, same training.csv (17,071,505 B) as `grand-hackathon-dindabax-togo-2023` (record `xente_fraud_detection_challenge`); Equatorial Guinea 2025, CO2 emissions, parent `umojahack-africa-2023-beginner-challenge` (record `carbon_dioxide_prediction_challenge`); Tunisia 2025 Challenge 1, LTE cell KPIs, multi-label anomaly flags ("Bad Congestion", "Very Low Throughput", ...), Hamming loss, Train.csv 687 MB, no parent; Tunisia 2025 Challenge 2, BLEU (text generation, SEL-12); South Africa 2025 with InstaDeep, interatomic potentials from .xyz files (molecular structures, not tabular); Isazi 2024, daily product sales volume "forecasts the future sales volumes" (SEL-10); Kenya Pwani 2024, coral images, data type Image (SEL-12); Togo 2022, Financial Inclusion in Africa re-run, parent `financial-inclusion-in-africa` (see `financial_inclusion_in_africa_for_botswana`). +- Use case: per competition; none for the bundle. Regime: per competition. +- Siblings and duplicates: listed above; `indabax_drc_2022` and `indabax_eswatini_2022` were set to `Duplicate` by the curator (2026-10-12) on the same kind of evidence (parent competition, identical file sizes). +- Papers and pages: the twelve Zindi pages; none read beyond the API text. +- Metric: per competition (F1, accuracy, RMSE, Hamming loss, BLEU); none for the bundle. +- Risks for /add-dataset: the record cannot be added as is. Recommend: split `indabax-tunisia-2025-challenge-1` into its own record (telecom KPIs, check whether it is a per-cell time series, `Time-series (Classification)`, and the 687 MB size), keep the duplicate links as notes on their canonical records, and retire this bundle as `No` + `Duplicate` once that is done; the curator decides. +- Priority: gap 0, use case 0, source 0 (a bundle), shape 0, risk - (mostly duplicates and out-of-scope tasks); floor 1. + This record is not a single dataset but an aggregation of roughly a dozen distinct IndabaX hackathon competitions (Benin, Madagascar, Mozambique, Equatorial Guinea, multiple Tunisia, South Africa, Togo, Isazi, Kenya, etc.), each with its own task, target, and modality. It cannot be triaged as one dataset and several entries may be fraud/marketing/classification while others could be out of scope. The right move is to split this into individual candidate records and assess each separately. A human must enumerate each linked competition, deduplicate against existing records, and decide per-task. diff --git a/curation/records/indabax_mali_2023.md b/curation/records/indabax_mali_2023.md index 0563554a..0f11cd6c 100644 --- a/curation/records/indabax_mali_2023.md +++ b/curation/records/indabax_mali_2023.md @@ -3,7 +3,9 @@ unique_name: indabax_mali_2023 name: indabax-mali-2023 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Duplicate tags: - AI-Filled (Verify) collections: @@ -25,4 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4, PROV-5): a private re-run of the STEG electricity and gas fraud challenge with the same files; duplicate of `fraud_detection_in_electricity_and_gas_consumption`. +- Source: Zindi, IndabaX Mali 2023 (2023-07-15 to 2023-07-17, RobotsMali and GDSC ENI-ABT, 12 participants). As published: "Les données fournies par STEG consistent en deux fichiers" (Client_train / Invoice_train, Client_test / Invoice_test), target fraud 1 / 0 per client; files train.zip 64,178,074 B, test.zip 27,696,276 B, SampleSubmission.csv 2,153,008 B. +- Use case: STEG flags clients for meter-fraud inspection; regime grouped by client in the canonical record. +- Siblings and duplicates: `fraud_detection_in_electricity_and_gas_consumption` (`Yes`, WIP (DF), Lennart; lists seven Zindi re-runs); `indabax_drc_2022` (Lennart, 2026-10-12: No, `Duplicate`, "the same STEG files (train.zip 61.2 MB, test.zip 26.4 MB, SampleSubmission.csv 2.1 MB)"); 64,178,074 B = 61.2 MiB and 27,696,276 B = 26.4 MiB. Same files as `indabax_nigeria_23`. +- Metric: AUC (Zindi Evaluation page: "l'erreur Area Under the Curve"). +- Note for the canonical record: add this link to its re-run list. +- Reject check (AI, 2026-10-22): stands. Zindi API: train.zip 64,178,074 B, test.zip 27,696,276 B and SampleSubmission.csv 2,153,008 B, byte for byte the files of fraud-detection-in-electricity-and-gas-consumption-challenge and indabax-drc-2022 (curator No, Duplicate, 2026-10-12); the page says 'Les données fournies par STEG consistent en deux fichiers'. https://zindi.africa/competitions/indabax-mali-2023 + IndabaX Mali 2023 is a Zindi educational hackathon with no comments or task description in the record, so the target, modality, and size are unknown. Most IndabaX hackathons are real-world tabular classification/regression tasks, so it should not be rejected on missing info alone. A human must open the Zindi page to determine the actual prediction task, modality (rule out image/text), size after cleaning, and split regime before a final verdict. diff --git a/curation/records/indabax_nigeria_23.md b/curation/records/indabax_nigeria_23.md index 80a4adda..96833460 100644 --- a/curation/records/indabax_nigeria_23.md +++ b/curation/records/indabax_nigeria_23.md @@ -3,7 +3,9 @@ unique_name: indabax_nigeria_23 name: indabax-nigeria-23 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Duplicate tags: - AI-Filled (Verify) collections: @@ -25,4 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4, PROV-5): IndabaX Nigeria 2023 re-runs the STEG electricity and gas fraud challenge with the same files; duplicate of `fraud_detection_in_electricity_and_gas_consumption`. +- Source: Zindi, "An application of a classification problem - Fraud Detection" (2023-06-01 to 2023-06-10, IndabaX Nigeria with Thomas Adewumi University, 67 participants). As published: "The data provided by STEG is composed of two files" (client data and billing history since 2005), target fraud 1 / 0; files train.zip 64,178,074 B, test.zip 27,696,276 B, SampleSubmission.csv 2,153,008 B. +- Use case: STEG flags clients for meter-fraud inspection. +- Siblings and duplicates: `fraud_detection_in_electricity_and_gas_consumption` (`Yes`, Lennart); `indabax_drc_2022` (Lennart, 2026-10-12: No, `Duplicate` on the same file sizes, 61.2 MB / 26.4 MB / 2.1 MB); same files as `indabax_mali_2023`. +- Metric: AUC (Zindi Evaluation page: "The metric used for this challenge is AUC"). +- Note for the canonical record: add this link to its re-run list. +- Reject check (AI, 2026-10-22): stands. Zindi API: the same three STEG files byte for byte (64,178,074 / 27,696,276 / 2,153,008 B) as the canonical STEG challenge and the indabax_drc_2022 precedent (curator No, Duplicate); 'The data provided by STEG is composed of two files'. https://zindi.africa/competitions/indabax-nigeria-23 + IndabaX Nigeria 2023 is a Zindi hackathon with no task description in the record; target, modality, and size are unknown. IndabaX hackathons are commonly genuine tabular prediction tasks, so missing info alone is not grounds to reject. A human must inspect the Zindi page for the actual target, confirm it is tabular (not image/text/NLP), and check size and split regime. diff --git a/curation/records/indabax_tanzania_mobile_banking_prediction_challenge.md b/curation/records/indabax_tanzania_mobile_banking_prediction_challenge.md index 70c113e7..64d9d4d7 100644 --- a/curation/records/indabax_tanzania_mobile_banking_prediction_challenge.md +++ b/curation/records/indabax_tanzania_mobile_banking_prediction_challenge.md @@ -3,7 +3,9 @@ unique_name: indabax_tanzania_mobile_banking_prediction_challenge name: indabax-tanzania-mobile-banking-prediction-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Duplicate tags: - AI-Filled (Verify) collections: @@ -26,6 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a private IndabaX Tanzania 2021 hackathon (21 to 23 June 2021) re-running the iX Mobile Banking Prediction Challenge (17 May to 13 June 2021). +- Source: https://zindi.africa/competitions/indabax-tanzania-mobile-banking-prediction-challenge; the Zindi API lists Train.csv 11,296,855 bytes, Test.csv 4,748,025 bytes, SampleSubmission.csv 604,211 bytes and VariableDefinitions.csv 2,231 bytes, byte for byte the sizes of `ix_mobile_banking_prediction_challenge`, with the same description ("predict if individuals across Africa and around the world use mobile or internet banking"), the same data description ("~100 000 and the test contains ~45 000 survey responses") and the same metric (AUC). +- Siblings and duplicates: the first publication is `ix_mobile_banking_prediction_challenge` (screened `TBD -> 2nd Tier`, 2026-10-22); this record ends at the same source (PROV-4, PROV-8). +- Metric: ROC AUC, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. Zindi API: parent 'ix-mobile-banking-prediction-challenge' (17 May to 13 Jun 2021, run first), with identical files: Train.csv 11,296,855, Test.csv 4,748,025, SampleSubmission.csv 604,211, VariableDefinitions.csv 2,231 B. https://zindi.africa/competitions/indabax-tanzania-mobile-banking-prediction-challenge + IndabaX Tanzania mobile-banking challenge asks to predict who uses mobile banking, a financial-inclusion binary classification on survey/demographic tabular features (FinScope-style). This is a genuine real-world predictive tabular task fitting the IID criterion. Plausible Yes, but mobile-banking/financial-inclusion data can be survey-derived and overlaps with other Zindi financial-inclusion challenges. A human must verify size, the target, whether it duplicates the financial-inclusion-in-Africa dataset, and that features are not aggregate/survey-only weighting issues. --- diff --git a/curation/records/indabax_tunisia_2023.md b/curation/records/indabax_tunisia_2023.md index 1da84150..4b6296df 100644 --- a/curation/records/indabax_tunisia_2023.md +++ b/curation/records/indabax_tunisia_2023.md @@ -3,7 +3,9 @@ unique_name: indabax_tunisia_2023 name: indabax-tunisia-2023 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Wrong Domain / Source Modality tags: - AI-Filled (Verify) collections: @@ -25,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Wrong Domain / Source Modality`, SEL-12): "Sequence-based predictor for the impact of mutation on protein stability", a 20-hour IndabaX Tunisia 2023 hackathon with InstaDeep (13 to 14 May 2023), predicting ddG of a single amino-acid mutation from protein sequences. +- Source: https://zindi.africa/competitions/indabax-tunisia-2023; as published: train.csv (54 MB), test.csv, columns ID, pdb_id, mutation (XnY), wt_seq, mut_seq, ddg; the page names the "Mega-scale dataset" as its origin (Tsuboyama et al., Nature 2023), so the first publication is not Zindi (PROV-8). +- Why out: the inputs are raw amino-acid sequences, where protein language and structure models are the tool; the page itself asks for "a Deep Learning algorithm that is trained on the amino acid sequences". The only tabular-looking columns (pdb_id, the mutation code) are identifiers of the sequence pair. +- Siblings: `instadeep_enzyme_classification_challenge`, `instadeep_kinase_classification_challenge` (sequence tasks of the same sponsor, proposed `No` on SEL-12 in the same batch). +- Metric: RMSE, Zindi Evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"). +- Reject check (AI, 2026-10-22): stands. The page asks for 'a Deep Learning algorithm that is trained on the amino acid sequences of wild-type (WT) proteins and mutant proteins'. The inputs are wt_seq and mut_seq (Mega-scale ddG data, Tsuboyama et al. 2023), where sequence and protein language models are the tool (SEL-12); the only other columns are identifiers. https://zindi.africa/competitions/indabax-tunisia-2023 + IndabaX Tunisia 2023 is a Zindi hackathon with no task description in the record, so target, modality, and size are unknown. IndabaX hackathons are usually real tabular prediction tasks, so it should not be rejected for missing info alone. A human must check the Zindi page for the actual target, confirm tabular modality, and verify size and split regime. diff --git a/curation/records/indabax_zimbabwe_2023.md b/curation/records/indabax_zimbabwe_2023.md index 81d236ae..4520a20e 100644 --- a/curation/records/indabax_zimbabwe_2023.md +++ b/curation/records/indabax_zimbabwe_2023.md @@ -3,7 +3,10 @@ unique_name: indabax_zimbabwe_2023 name: indabax-zimbabwe-2023 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Out-of-scope Task (CTR/RecSys/Ranking) +- Duplicate tags: - AI-Filled (Verify) collections: @@ -25,4 +28,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Out-of-scope Task (CTR/RecSys/Ranking)`, SEL-9; `Duplicate`, PROV-4): the IndabaX Zimbabwe 2023 hackathon (21 to 23 July 2023) is the "Insurance Recommendation Challenge", a re-run of the Zimnat Insurance Recommendation Challenge (Zindi, 1 July to 13 September 2020). +- Source: https://zindi.africa/competitions/indabax-zimbabwe-2023; as published: one row per customer with joining date, birth year, occupation, branch and 21 product columns; in Test "ONE product has been removed" and the task is "to predict the missing product" for Zimnat, which "wants an ML model ... to predict which kinds of insurance products to recommend to customers". +- Why out: a next-product recommendation task (SEL-9); the files match the 2020 competition byte for byte (Train.csv 2,424,810, Test.csv 832,462, SampleSubmission.csv 3,570,017 bytes in both Zindi APIs), so it also ends at the same source as `zimnat_insurance_recommendation_challenge` (AI-proposed `No`, same marker). +- Metric: log loss over every customer x product pair, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. Zindi API: Train.csv 2,424,810, Test.csv 832,462 and SampleSubmission.csv 3,570,017 B are identical to zimnat-insurance-recommendation-challenge (2020, tagged 'Collaborative Filtering'), so it is a PROV-4 duplicate whatever the SEL-9 call; the task is to predict 'the missing product' to recommend. https://zindi.africa/competitions/indabax-zimbabwe-2023 + IndabaX Zimbabwe 2023 is a Zindi hackathon with no comments describing the task; target, modality, and size are unknown. Such hackathons are typically genuine tabular prediction tasks, so missing info alone is not a rejection reason. A human must open the Zindi page to determine the target, confirm it is tabular, and check size and split regime. diff --git a/curation/records/indabax_zimbabwe_2023_nyaradzo_group.md b/curation/records/indabax_zimbabwe_2023_nyaradzo_group.md index 84009d93..5f01b4a7 100644 --- a/curation/records/indabax_zimbabwe_2023_nyaradzo_group.md +++ b/curation/records/indabax_zimbabwe_2023_nyaradzo_group.md @@ -3,7 +3,9 @@ unique_name: indabax_zimbabwe_2023_nyaradzo_group name: indabax-zimbabwe-2023-nyaradzo-group checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Time-series (Forecasting) tags: - AI-Filled (Verify) collections: @@ -26,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): "Nyaradzo Group - Predictive Analytics Challenge: Forecasting Human Resources utilization with seasonality", IndabaX Zimbabwe 2023 (22 to 25 July 2023). +- Source: https://zindi.africa/competitions/indabax-zimbabwe-2023-nyaradzo-group; as published: files received per branch in 2022 (total_files_for_the_year.csv, files_breakdown 2022.csv, file_ageing.csv, 2022 data.xlsx). +- Why out: the task is "to forecast the number of files received per WEEK for the first 6 months of the year 2023", future values of a series per branch (SEL-10); the file-ageing part is descriptive analytics. +- Metric: none; "there are no leaderboard scores for this competition, as evaluations will be completed by a panel of judges". +- The record's old domain `business & marketing` would rather be insurance (the sponsor is a funeral-insurance group); moot for a reject. +- Reject check (AI, 2026-10-22): stands. 'create a forecasting model to forecast the number of files received per WEEK for the first 6 months of the year 2023' from one year (2022) of branch counts; type_of_problem ['Forecast']; 'no leaderboard scores ... evaluations will be completed by a panel of judges'. There are no per-row features that would make it a fixed task. https://zindi.africa/competitions/indabax-zimbabwe-2023-nyaradzo-group + IndabaX Zimbabwe 2023 sponsored by the Nyaradzo Group (a Zimbabwean funeral/insurance services company) has no task description in the record, but the sponsor suggests a customer/business prediction task on tabular data. Missing info alone is not grounds to reject given the typical IndabaX format. A human must inspect the Zindi page for the actual target (e.g., customer behavior, payments, churn), confirm tabular modality, and verify size and split regime. diff --git a/curation/records/inegi_gcim_vegetation_mapping_challenge.md b/curation/records/inegi_gcim_vegetation_mapping_challenge.md index 549daaca..5af041f9 100644 --- a/curation/records/inegi_gcim_vegetation_mapping_challenge.md +++ b/curation/records/inegi_gcim_vegetation_mapping_challenge.md @@ -4,6 +4,7 @@ name: inegi-gcim-vegetation-mapping-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - No Good Target (yet) - Data Quality Issue @@ -12,6 +13,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2024' domain: environmental science & climate required_split: - '?' @@ -19,6 +21,7 @@ problem_type: Multiclass Classification original_data_state: One Table source_links: - https://zindi.africa/competitions/inegi-gcim-vegetation-mapping-challenge/data +- https://zindi.africa/competitions/inegi-gcim-vegetation-mapping-challenge source_row: 885 needs_review: - ai_unverified @@ -29,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real land-cover classification from INEGI with tabular per-point inputs, but the inputs are largely remote-sensing reflectance (SEL-13 boundary) and the labels are noisy by design. +- Source: Zindi / ITU "INEGI UN-GGIM Vegetation Mapping Challenge", 2 to 28 September 2024, data by INEGI, Mexico's national statistics and mapping agency (https://zindi.africa/competitions/inegi-gcim-vegetation-mapping-challenge); as published: Train.csv (about 1.1 MB) and Test.csv, per-record "values of remote sensing reflectance (year and dry season composites from Harmonized Landsat Sentinel imagery) as well as bioclimate data for precipitation and temperature, relief (elevation, slope, aspect, topographical position), canopy height", plus "a second set with the original noisy vegetation labels, and a third set with revised labels" (so several label sets, not a plain one table); 9 classes; one tile of about 150 x 150 km near Puerto Vallarta. +- Use case: INEGI cleans training labels for its deep-learning vegetation map, to replace "manual – visual interpretation"; the benchmark task would be per-point vegetation class from the covariates, an approximate fit to the stated label-cleaning goal; regime spatial (grouped) because points in one tile are spatially autocorrelated, the Zindi split is not described. +- Siblings and duplicates: `crowdsourced_mapping` (land cover from satellite time series) and `covertype` (cartographic covariates) are related tasks, not duplicates; no other INEGI record. External collections: none found. +- Papers and pages: competition page only; no paper or published solution found. +- Metric: multiclass log loss, Zindi Evaluation page ("The evaluation metric for this challenge is the Log Loss", submission Class_1 ... Class_9); a deliberate leaderboard choice (MET-4); inputs none; which label set the test is scored against (noisy or revised) is not stated (MET-1). +- Risks for /add-dataset: the reflectance bands are remote-sensing spectra, which SEL-13 counts as image-derived, mixed with many non-visual covariates (bioclimate, relief, canopy height) that may keep it tabular (SEL-15 vs SEL-13, a curator call); the target is noisy and the revised labels are "NOT exhaustively checked" (unclear target); coordinates and a spatial split need checking (SPLIT); row count unknown (Train.csv 1.1 MB suggests a few thousand rows). +- Priority: gap + (spatial grouped regime is thin), use case 0 (label cleaning, not the decision itself), source 0 (institutional but documented only on the competition page), shape 0 (size unknown), risk - (noisy target, SEL-13 doubt) -> 1 + 1 = 2. + The stated task is a label-cleaning / weak-supervision meta-task: build a model to identify outliers (wrong/suspicious labels) and suggest more plausible labels for vegetation mapping, rather than a clean predictive classification with a fixed target. That framing (data cleaning / label correction) does not map to a standard IID predictive task and risks a leaky/ill-defined target, and vegetation mapping can carry geospatial/remote-sensing characteristics. It could be repurposed into a straightforward land-cover classification if a trustworthy label column exists. A human must verify whether a clean categorical target and tabular features (not raster/remote-sensing) are available before considering it. --- diff --git a/curation/records/instadeep_enzyme_classification_challenge.md b/curation/records/instadeep_enzyme_classification_challenge.md index e1790135..0b5a0b7b 100644 --- a/curation/records/instadeep_enzyme_classification_challenge.md +++ b/curation/records/instadeep_enzyme_classification_challenge.md @@ -5,7 +5,6 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- NLP (Text) - Wrong Domain / Source Modality tags: - AI-Filled (Verify) @@ -29,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Wrong Domain / Source Modality`, SEL-12): "InstaDeep Enzyme Classification Challenge" (Zindi, 17 November 2020 to 21 February 2021): "Can you predict the class of an enzyme using only its amino acid sequence?" +- Source: https://zindi.africa/competitions/instadeep-enzyme-classification-challenge; as published: Train.csv (374 MB) and Test.csv with sequence ID, amino-acid sequence, organism and one of 20 labels; 8 organisms in train, 2 in test; plus unlabelled_data.zip (633 MB) "for language modelling or other unsupervised learning tasks". +- Why out: the only input is a raw biological sequence, where protein language models are the tool (SEL-12); the page builds the task around pre-training on unlabelled sequences. The old `NLP (Text)` marker is dropped: the modality is a biological sequence, not language. +- Siblings: `instadeep_kinase_classification_challenge` (its predecessor, UmojaHack Tunisia 2020), `indabax_tunisia_2023` (sequence ddG, same sponsor). +- Metric: accuracy, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. 'Can you predict the class of an enzyme using only its amino acid sequence?'; unlabelled_data.zip (633 MB) is supplied 'for language modelling'. The only input is a raw protein sequence, where protein LMs dominate (SEL-12). https://zindi.africa/competitions/instadeep-enzyme-classification-challenge + This InstaDeep challenge classifies enzymes from raw amino-acid sequences, a biological-sequence (protein language) modality where sequence/embedding models dominate rather than generic tabular learners. The input is a single sequence string per example, not engineered tabular features, so it fails the representative-tabular criterion. It would also require grouped (by-family) splits to avoid leakage. Recommend No as a tabular candidate; a human could reconsider only if a meaningful pre-extracted tabular feature representation is provided. diff --git a/curation/records/instadeep_kinase_classification_challenge.md b/curation/records/instadeep_kinase_classification_challenge.md index 8e294ac0..8d89db02 100644 --- a/curation/records/instadeep_kinase_classification_challenge.md +++ b/curation/records/instadeep_kinase_classification_challenge.md @@ -5,7 +5,6 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- NLP (Text) - Wrong Domain / Source Modality tags: - AI-Filled (Verify) @@ -29,6 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Wrong Domain / Source Modality`, SEL-12): "UmojaHack Tunisia: InstaDeep Kinase Classification Challenge" (Zindi, 17 October 2020; the record's link https://zindi.africa/competitions/umojahack-tunisia is the right page): "Predict the class of a protein kinase enzyme using only its amino acid sequence". +- Source: as published: train.csv (197 MB), test.csv, an ID, the sequence ("up to 560 positions, and each position can take one of 20 values") and an EC class (8 classes, 2.7.13.3 with 634,731 rows the largest), plus 1024-dimensional per-amino-acid embeddings; the API tags the data type "Unstructured". +- Why out: the input is a raw amino-acid sequence, where sequence and protein language models are the tool (SEL-12); the page's own feature suggestions are per-position encodings of the sequence. The old `NLP (Text)` marker is dropped. +- Siblings: `instadeep_enzyme_classification_challenge` (its follow-up), `indabax_tunisia_2023`. +- Metric: multiclass log loss, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. 'Predict the class of a protein kinase enzyme using only its amino acid sequence'. Zindi API: data_type ['Unstructured'], type_of_problem ['Natural Language Processing', 'Classification'], with amino_acid_embeddings.csv and a DL starter notebook. https://zindi.africa/competitions/umojahack-tunisia + The task is to predict the class of a protein kinase using only its amino-acid sequence, a raw biological-sequence modality dominated by protein-sequence models rather than tabular learners. The input is a sequence string, not engineered tabular features, so it does not fit the representative-tabular criterion, and proper evaluation needs grouped/by-family splits. Recommend No; reconsider only if a genuine tabular feature table (not raw sequence) exists. Note the source_link points to umojahack-tunisia, so the link/name mapping should also be checked. --- diff --git a/curation/records/international_womens_day_challenge.md b/curation/records/international_womens_day_challenge.md index fc42c75b..07f0b29e 100644 --- a/curation/records/international_womens_day_challenge.md +++ b/curation/records/international_womens_day_challenge.md @@ -3,9 +3,9 @@ unique_name: international_womens_day_challenge name: international-womens-day-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- No Good Target / Scientific Discovery +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,6 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the International Women's Day Challenge (Zindi, 7 to 31 March 2025, a "Tutorial") re-runs "Womxn in Big Data South Africa: Female-Headed Households in South Africa" (Zindi, 25 November 2019 to 24 February 2020). +- Source: https://zindi.africa/competitions/international-womens-day-challenge; as published: 2011 South African census aggregated to wards, target "the percentage of households per ward that are both female-headed and earn an annual income that is below R19,600", "2,822 wards in the train set" and "1,013 wards in the test set"; the Zindi API lists Train.csv 2,049,620 bytes, Test.csv 1,113,085, SampleSubmission.csv 20,866 and variable_descriptions.csv 4,393, byte for byte the sizes of the 2019 competition, with the same data description and metric. +- Siblings and duplicates: the first publication is `womxn_in_big_data_south_africa_female_headed_households_in_south_africa` (AI `TBD -> 2nd Tier`); the dataset's merits (social science, a thin domain; ward aggregates of the same census as the target) belong on that record. +- Metric: RMSE, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. Zindi API: parent 'womxn-in-big-data-south-africa-female-headed-households-in-south-africa' (2019-11-25 to 2020-02-24), with identical files: Train.csv 2,049,620, Test.csv 1,113,085, SampleSubmission.csv 20,866, variable_descriptions.csv 4,393 B. The 2019 record (AI TBD -> 2nd Tier) carries the dataset. https://zindi.africa/competitions/international-womens-day-challenge + The target is the percentage of households per ward that are female-headed and earn below R19,600, i.e. an aggregate areal/census regression at ward level rather than individual-record prediction. Aggregated census-ward targets are small in row count and lean toward a descriptive/scientific-discovery framing, which weakens the representative-tabular case. It is a real regression task and not obviously out of scope, so 2nd Tier is appropriate pending checks. A human must verify the number of wards (rows), feature richness, and whether the target/features cause aggregation leakage. --- diff --git a/curation/records/ix_mobile_banking_prediction_challenge.md b/curation/records/ix_mobile_banking_prediction_challenge.md index 1ee7be01..5a51d325 100644 --- a/curation/records/ix_mobile_banking_prediction_challenge.md +++ b/curation/records/ix_mobile_banking_prediction_challenge.md @@ -3,12 +3,14 @@ unique_name: ix_mobile_banking_prediction_challenge name: ix-mobile-banking-prediction-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: finance required_split: - Random (IID) @@ -16,6 +18,7 @@ problem_type: Binary Classification original_data_state: One Table source_links: - https://zindi.africa/competitions/ix-mobile-banking-prediction-challenge/data +- https://zindi.africa/competitions/ix-mobile-banking-prediction-challenge source_row: 907 needs_review: - ai_unverified @@ -26,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a large survey table with an unnamed source and a survey answer as the target; the curator should decide whether "uses mobile or internet banking" is a target anyone predicts. +- Source: Zindi "iX Mobile Banking Prediction Challenge", sponsor iXperience, 17 May to 13 June 2021 (https://zindi.africa/competitions/ix-mobile-banking-prediction-challenge); as published: "~100 000" train and "~45 000" test "survey responses from around Africa and the world", Train.csv / Test.csv / VariableDefinitions.csv, survey columns FQ1-FQ37 coded "1 - Yes, 2 - No, 3 - Don't Know 4 - refused to answer", binary target (uses mobile or internet banking). The survey behind it is not named on the page (a Global Findex-style multi-country survey is a guess, not confirmed). +- Use case: the page says "Banks and financial service providers value knowing what habits their clients follow ... tailor products and services"; a provider would not hold these survey answers for its clients, so the fit is approximate at best; regime IID (one row per respondent, no time), though a country column, if present, may call for a grouped check. +- Siblings and duplicates: `indabax_tanzania_mobile_banking_prediction_challenge` (June 2021) ships the same files (Train.csv 11,296,855 bytes, Test.csv 4,748,025 bytes in both Zindi APIs) and the same text, so it is the later duplicate and this record is the first publication; the FinScope "Financial Inclusion in Africa" records (`financial_inclusion_in_africa_for_botswana`, `mobile_money_and_financial_inclusion_in_tanzania_challenge`) are other surveys with other targets (has a bank account), not duplicates (PROV-7). External collections: none found. +- Papers and pages: no paper; competition page only; no public top solution found. +- Metric: ROC AUC, Zindi Evaluation page ("The error metric for this competition is the Area Under the Curve (AUC). Do not set thresholds"); a deliberate leaderboard choice (MET-4); inputs none (MET-1, MET-5, MET-6). +- Risks for /add-dataset: the target is a respondent's own answer in the same interview as the features (SEL-7 / SEL-3 doubt: who predicts it?); FQ columns may hold other account- or payment-usage answers that nearly give the target away (LEAK); the survey's origin, year and weights need tracing (PROV-1, PROV-2). +- Priority: gap 0 (finance, binary, IID not thin), use case 0 (approximate, written for a sponsor), source 0 (survey unnamed), shape + (about 145k rows, many categorical answers), risk - (survey-answer target, likely leaking FQ columns): 1. + Based on the name this is a mobile-banking adoption/usage prediction challenge on Zindi, most likely a financial-inclusion binary classification on demographic/survey tabular features, but the record has no comments. Such tasks are genuine real-world tabular prediction problems, so missing info alone is not a rejection reason. A human must confirm the actual target, check size after cleaning, and verify it does not duplicate the IndabaX Tanzania mobile-banking or broader financial-inclusion datasets. diff --git a/curation/records/job_profitability.md b/curation/records/job_profitability.md index 8cdf9941..da7e2ea5 100644 --- a/curation/records/job_profitability.md +++ b/curation/records/job_profitability.md @@ -4,6 +4,7 @@ name: Job_Profitability checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - No Good Target / Scientific Discovery - Missing source information @@ -30,6 +31,17 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the proposed fast reject (SEL-3 (with SEL-5; PROV-1 supporting)) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`No Good Target / Scientific Discovery`, SEL-3; `Missing source information`, PROV-1): the source's own aim is feature importance, and its target is computed from the shipped columns. +- Source: OpenML 44311 `Job_Profitability`, uploaded 2022-11-03 by "Leon Gross", no origin, paper or citation; the Kaggle copy (ulrikthygepedersen, 2023-02-02) names "https://www.openml.org/" as its source. As published: 14,480 jobs x 31 columns (job number, revenue, cost and hours columns, lead source), target `Jobs_Gross_Margin_Percentage`. +- Use case: none: the description is "Finding which features are most important to job profitability" (OpenML and Kaggle); every column is a closed job's revenue, cost or hours, known only after the job, so no decision before the outcome. +- Deterministic target (SEL-5): on the preview rows `Jobs_Gross_Margin_Percentage` = (`Jobs_Total` - `Jobs_Total_Costs`) / `Jobs_Total` (360 - 105.87 = 254.13 = `Jobs_Gross_Margin`, / 360 = 0.7059; 289 - 137.17 = 151.83, / 289 = 0.5254; 49 - 81.15 = -32.15, / 49 = -0.6561), and the cost components add up to `Jobs_Total_Costs`; dropping the rule's inputs leaves hours and lead flags of a finished job. +- Siblings and duplicates: none in the records. External collections: none found. +- Metric: none found (OpenML, Kaggle pages). +- The curator's workbook note: "Not much information given, but task seems interesting; used for feature selection / scientific discovery". +- Reject check (AI, 2026-10-22): does not stand. The identity is real: Jobs_Gross_Margin_Percentage = (Jobs_Total - Jobs_Total_Costs) / Jobs_Total. SEL-5 then says drop the rule's inputs and check whether a real task remains, and OpenML 44311's columns include quote-time fields: Sold_Hours, Pricebook_Price, Estimates, Jobs_Estimate_Sales_Subtotal, Jobs_Estimate_Sales_Installed, Is_Lead, Opportunity, Warranty, Recall, Converted, 'Lead Generated From Source' (https://www.openml.org/d/44311; the names look like a field-service software export). Margin from estimate-time fields is a plausible decision (price or accept a job), and checking that needs the data. On the source, PROV-1 is not a clear fail under PROV-2 as amended 2026-10-21: there is a named first uploader (OpenML, Leon Gross, 2022) to point to, and PROV-3 has not been run. The curator's workbook note was "task seems interesting". +- For /add-dataset: TBD -> 2nd Tier, priority 1. The add stage must check: (1) drop every input of the margin identity (Jobs_Total, Jobs_Subtotal, all cost, labor, burden and margin columns, the *_as_percent_of_Sales columns, Income), and probe whether the quote-time rest (sold hours, pricebook price, estimates, lead source, warranty/recall flags) beats a baseline (TP-2/TP-4). (2) Which of those fields are known before the job (LEAK). (3) PROV-3 tests on the shipped columns. (4) A search for Leon Gross and the company or software behind the export; if none and PROV-3 marks appear, then `No` with `Missing source information`. + A Kaggle dataset already tagged 2nd Tier / Scientific Discovery and placed in TabArena Reject, with the curator noting little information is given and that it seems geared to feature selection / scientific discovery rather than a clean predictive task. It may be a real profitability regression/classification, but the unclear target and provenance keep it in 2nd Tier. A human must verify the actual target column, dataset size, feature meaning, and original source before any promotion. --- diff --git a/curation/records/kdd_internet_usage.md b/curation/records/kdd_internet_usage.md index 7bc9583b..8c96ce90 100644 --- a/curation/records/kdd_internet_usage.md +++ b/curation/records/kdd_internet_usage.md @@ -3,10 +3,10 @@ unique_name: kdd_internet_usage name: kdd_internet_usage checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: +- Duplicate - No Good Target / Scientific Discovery -- Outdated tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -33,6 +33,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-6; `No Good Target / Scientific Discovery`, SEL-3): a binarized cut of the GVU survey table that `internet_usage` already records, with a target the mirror invented. +- Source: GVU Center, Georgia Tech, WWW User Survey of 10 October to 16 November 1997, "general demographics" subset, donated to UCI by Di Cook in 1999 (https://doi.org/10.24432/C5Q60G). OpenML 981 (2014): "Binarized version of the original data set (see version 1). The multi-class target feature is converted to a two-class nominal target feature by re-labeling the majority class as positive ('P') and all others as negative ('N'). Originally converted by Quan Sun." 10,108 rows x 69 columns, target `Who_Pays_for_Access_Work` (7,393 / 2,715); OpenML 372 `internet_usage` holds the same 10,108 respondents with 72 columns. +- Use case: none: the UCI abstract is "This data contains general demographic information on internet users in 1997"; "This data comes from a survey"; its past usage is the ASA 1999 Data Exposition (graphics), no prediction. +- Siblings and duplicates: `internet_usage` (Lennart: `TBD -> 2nd Tier`, `No Good Target (yet)`, "Maybe a duplicate. Unclear task") links UCI C5Q60G and OpenML 372, the same survey file; this record adds only a relabelled column. External collections: TabSTAR (per the record), OpenML binarized suite. +- Metric: none. +- The curator's workbook note: "Might be survey data. Might actually be an interpretability task. Also might be clustered data". +- Reject check (AI, 2026-10-22): stands. OpenML 981 says of itself: "Binarized version of the original data set (see version 1) ... re-labeling the majority class as positive" (https://www.openml.org/d/981). Version 1 is OpenML 372 internet_usage (target Actual_Time), the same GVU 1997 survey already recorded as internet_usage. A swapped target on the same cohort is the PROV-6 duplicate case by name, and the invented majority-vs-rest label is no observed outcome (SEL-3). + KDD Internet Usage (GVU 1997 web survey, OpenML d/981 and d/372) is questionnaire/survey data already tagged 2nd Tier / Scientific Discovery and in TabArena Reject, with curator concerns about it being survey/interpretability data, possibly clustered by user, and containing sensitive features like race. It is a known tabular multiclass dataset (used by TabSTAR), but the survey nature, 1997 vintage, and ethical/leakage caveats keep it at 2nd Tier rather than a clean Yes. A human must verify the chosen target, whether per-user 'who' features cause grouping/leakage, and the ethics of sensitive attributes. --- diff --git a/curation/records/kyoto_2006_plus.md b/curation/records/kyoto_2006_plus.md index 2404d92f..f6f3ba3f 100644 --- a/curation/records/kyoto_2006_plus.md +++ b/curation/records/kyoto_2006_plus.md @@ -4,8 +4,9 @@ name: Kyoto 2006+ (Kyoto University Honeypot Traffic) checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: -- No Good Target (yet) +- TBD tags: - AI-Filled (Verify) - Non-IID (Temporal) @@ -28,6 +29,15 @@ type_adapter_id: curation-record-v1 ⚠️ **AI-FILLED — UNVERIFIED**: triaged by an AI (2026-08-12), no human has verified it. Check the verdict and the fields before relying on them. +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: real honeypot sessions with a labelled attack flag, but the label comes from the deployed detectors and may track the receiving host. +- Source: Song, Takakura, Okabe et al., Kyoto University honeypots, BADGERS 2011; daily session files 2006-11-01 to 2015-12-31 (https://www.takakura.com/Kyoto_data/). As published: one row per session, 24 features (14 KDD-Cup-style connection statistics, 10 extra), `Label` with "'1' means the session was normal, '-1' means known attack was observed in the session, and '-2' means unknown attack was observed" (format document BenchmarkData-Description-New.pdf). Millions of rows per year. +- Use case: a security operator flags a session as an attack as it ends; fit approximate (the source publishes it as NIDS evaluation data, no actor named); regime temporal because the traffic and attacks drift over the years and the files are dated (`Start Time` ships). +- Siblings and duplicates: no duplicate; the network-IDS family: kddcup99 (`No`, Andrej and Lennart: simulated, outdated), nsl_kdd, unsw_nb15, cic_ids2017 (AI `No`, testbed traffic), ctu_13 and twente_2009_labeled_flows (AI `TBD -> 2nd Tier`, observed traffic). External collections: none found. +- Papers and pages: Song et al. 2011 (BADGERS; not read); format document (read); widely used in NIDS papers. +- Metric: none found (project page and format document; the BADGERS paper not read), so the fallback `roc_auc` on attack vs normal (MET-1, MET-8). +- Risks for /add-dataset: `IDS_detection`, `Malware_detection` and `Ashula_detection` are the detectors' outputs ("We used Symantec IDS to extract this feature") and the label is likely derived from them: drop them and check whether the label is reconstructible from them (LEAK, SEL-5); normal traffic comes from separate hosts (mail server, web crawler), so the label may track the destination address or port (a host shortcut, LEAK); IPs are sanitized per month only. The old marker `No Good Target (yet)` replaced by `TBD`: the source defines a target, its validity is the open point. +- Priority: gap - (technology & internet, temporal not thin), use case - (no actor in the source), source + (paper, format document, university page), shape + (large, mixed types, temporal), risk - (detector-derived label, host shortcut). + Session-level records extracted with Bro 2.4 from Kyoto University's honeypots and darknet sensors, as daily files covering 2006-11-01 to 2015-12-31. The traffic is genuinely observed, so criterion 4B is not the problem here — the target is. **The label is another detector's output.** The canonical page describes the current release as "Bro 2.4 based session extraction. Based on new IDS detection result", i.e. rows are labelled by what the deployed IDS/AV flagged, not by verified ground truth, so a model trained on it distils an IDS rather than learning intrusion. Second concern: the page's "Deployed systems" list separates the honeypots from a mail server and a web crawler, so benign vs attack may track which host received the traffic. Both need checking against the format document linked from the page before this moves either way (Song et al., BADGERS 2011, not read). diff --git a/curation/records/labor.md b/curation/records/labor.md index 2d21685d..cfca1109 100644 --- a/curation/records/labor.md +++ b/curation/records/labor.md @@ -4,6 +4,7 @@ name: labor checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Too Small tags: @@ -26,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict of Lennart and Andrej ("too small") stands, and the source adds a second reason; recommend `No` with `Too Small` and `AHDS (Artifical/Handmade/Deterministic/Simulated)`; the curator has the call. Priority 2 by the factors while the field stays `TBD -> 2nd Tier`. +- Source: UCI Labor Relations, donated by Stan Matwin, November 1988 ("Final settlements in labor negotitions in Canadian industry", Collective Bargaining Review, Labour Canada); OpenML 4, PMLB `labor` are copies. As published: 57 contracts x 16 attributes, 37 good / 20 bad. +- Use case: none named: "testing concept learning software ... to learn the description of an acceptable and unacceptable contract"; regime IID. +- Siblings and duplicates: none in the records (labour_inspection_compliance is another task). External collections: PMLB, OpenML. +- Papers and pages: Bergadano, Matwin, Michalski, Zhang 1988 (EWSL, ISMIS). +- Metric: none deliberate (concept-learning papers), so the fallback `roc_auc` (MET-1, MET-8). +- Against keeping it: 20 bad contracts in all, 4 per test fold under 5-fold CV (TP-5; parkinsons retired at 2-3 per fold, musk kept at 13); and the UCI notes say "The unacceptable contracts were either obtained by interviewing experts, or by inventing near misses", so part of the negative class is handmade (SEL-16, PROV-3). +- Priority: gap + (social science thin), use case -, source + (UCI notes), shape - (57 rows), risk - (invented negatives). + The PMLB 'labor' dataset (labor-negotiation contract acceptability) has only 57 rows, which two TabArena curators (Lennart, Andrej) already flagged as too small, especially under 3-fold CV. While it is a genuine real-world tabular classification, it is far below any usable size threshold for the benchmark. Recommend No on size grounds; no further verification needed beyond confirming the 57-row count. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/laduma_analytics_football_league_winners_prediction_challenge.md b/curation/records/laduma_analytics_football_league_winners_prediction_challenge.md index 9b4060ad..7978e0e1 100644 --- a/curation/records/laduma_analytics_football_league_winners_prediction_challenge.md +++ b/curation/records/laduma_analytics_football_league_winners_prediction_challenge.md @@ -4,6 +4,7 @@ name: laduma-analytics-football-league-winners-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD tags: @@ -11,10 +12,11 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: Other required_split: -- '?' -problem_type: TBD +- Temporal (NON-IID) +problem_type: Multiclass Classification original_data_state: Database (or multiple to-be-joined tables) source_links: - https://zindi.africa/competitions/laduma-analytics-football-league-winners-prediction-challenge/data @@ -28,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a match-outcome classification of one masked league, a few hundred games, with in-match event tables that may hold the result. +- Source: Zindi competition with Laduma Analytics and Layer, 2022-06-06 to 2022-09-04, a rerun of the #ZindiWeekendz hackathon of 2022-06-03 (accuracy); as published: "collected over 3 years, following 23 teams in an African country. The player and team's names have been masked", event data (shots, passes, tackles, dribbles with X/Y), `Train.csv`, `Test.csv`, `train_game_statistics.csv`, `test_game_statistics.csv`; "The train set is two seasons and the test one season". Target: home win / lose / draw. Rows not checked (Zindi data needs a login). +- Use case: "predict the outcome of a football match, based on historical match and player data", for Laduma Analytics "to widen the types of services they offer"; no decision named, fit approximate; regime temporal (the source splits by season). +- Siblings and duplicates: `zindi_weekendz_laduma_analytics_football_league_winners_prediction_challenge` is the same data in the weekend hackathon (22 teams, accuracy), first published three days earlier: keep one record (PROV-4, PROV-8); other sport records: fifa, football_manager_important_matches, profb. External collections: none found. +- Papers and pages: competition page (https://zindi.africa/competitions/laduma-analytics-football-league-winners-prediction-challenge); no paper. +- Metric: log loss over the three outcomes, the leaderboard metric ("The evaluation metric for this challenge is the Log Loss", Evaluation page): deliberate (MET-4); inputs none. +- Risks for /add-dataset: the per-game statistics of the test games are in-match events (goals among the shots), so the outcome may be read off the match itself rather than predicted before kick-off (LEAK, USE); event tables to aggregate to one row per game (joins); a few hundred games over three seasons (TP-5, TP-9). +- Priority: gap - (sport, under Other), use case - (no decision named), source - (a competition page, masked data, no codebook found), shape - (few hundred games, event joins), risk - (in-match statistics). + A Zindi football challenge to predict league/match winners; the record has no detail, so it is unclear whether this is a per-match outcome classification (in scope) or a ranking/forecasting-of-future-results framing (out of scope). Sports outcome data is typically split across multiple match/team tables needing joins and often demands a temporal split to avoid leakage. Given the ambiguity it belongs in 2nd Tier pending inspection. A human must verify the exact target (match result vs league winner vs ranking), the temporal/grouped split needs, table structure, and size. diff --git a/curation/records/laptop_prices_dataset.md b/curation/records/laptop_prices_dataset.md index 3cb561ba..fec88ba0 100644 --- a/curation/records/laptop_prices_dataset.md +++ b/curation/records/laptop_prices_dataset.md @@ -3,15 +3,16 @@ unique_name: laptop_prices_dataset name: Laptop_Prices_Dataset checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - No Good Target / Scientific Discovery -- Data Quality Issue tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) collections: - TabArena Reject +original_source: Kaggle year: '2023' domain: technology & internet required_split: @@ -30,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a one-off scrape of Amazon laptop listings with a price target; real data, but no decision and messy text fields; follows the curators' call on `laptops`. +- Source: Kaggle upload by talhabarkaatahmad, 2023-10-09, "scraped from Amazon.com"; as published: about 4.5K listings x 14 columns (brand, model, screen size, colour, disk, CPU, RAM, OS, special features, graphics, CPU rating, rating, price in USD "as of October 2023"), one file `amazon_laptop_prices_v01.csv`, CC0. +- Use case: a seller or buyer judges a listing's price from its specification; nobody named in the source, fit approximate; regime IID (one snapshot). +- Siblings and duplicates: `laptops` (Lennart, Mustafa: `TBD -> 2nd Tier`, `No Good Target / Scientific Discovery`, another Kaggle scrape, "requires a lot of preprocessing to figure out a task") is the precedent; not the same rows. Other product-price records such as 1000_cameras (Lennart, 2nd Tier). External collections: none found. +- Papers and pages: the Kaggle page only (https://www.kaggle.com/datasets/talhabarkaatahmad/laptop-prices-dataset-october-2023). +- Metric: none found (Kaggle dataset page, no competition), so the fallback `rmse` (MET-1, MET-8). +- Risks for /add-dataset: repeated listings of the same model (the CC note), free-text fields (title-like model, special features) to parse (CLEAN, SEL-12), `rating` measured after listing. The old `Data Quality Issue` marker is dropped: no irreversible damage is shown; duplicates can be removed. +- Priority: gap - (technology & internet not thin), use case -, source - (an anonymous scrape with a data dictionary), shape + (4.5K rows, mixed types), risk - (duplicates and text parsing). + Roughly 4.5k crawled Amazon/Kaggle laptop listings from October 2023 where the nominal task is predicting price. As the curator notes, this is crawled e-commerce data likely with duplicate listings and is more of an analytics/outlier-detection exercise than a genuine predictive task, since price is largely defined by the observed spec features. Already tagged 2nd Tier / TabArena Reject. The data-quality concerns (dedup, scraping artifacts) and weak target make it a poor benchmark candidate. A human should confirm row count after dedup and whether any meaningful price-prediction signal survives, but on current evidence this is a No. --- diff --git a/curation/records/law_school_admission_bianry.md b/curation/records/law_school_admission_bianry.md index 8157fde7..1b37431f 100644 --- a/curation/records/law_school_admission_bianry.md +++ b/curation/records/law_school_admission_bianry.md @@ -30,6 +30,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook note "Not a predictive task" holds for this OpenML cut; the `No` stands. Counter-argument on the markers only: `Ethical Issue` has no basis under SEL-1 (nobody asked that the data not be used; a race column is no ethical bar), the reason is the target. +- Source: OpenML 43889 (2022-05-30), "obtained from the R-package fairml", from the LSAC National Longitudinal Bar Passage Study (Wightman, 1998; students entering law school in 1991). As published here: 20,800 rows x 12 columns, target `ugpagt3`: "The response variable has been changed to a binary version: Whether ugpa is greater than 3." +- Use case: none: undergraduate GPA is known before admission, so nobody predicts it from LSAT and demographics (SEL-3, SEL-8: the mirror relabelled the data into a task of its own); regime IID. +- Siblings and duplicates: no other record of the LSAC data. External collections: fairml, the fairness literature (Kusner et al. 2017 counterfactual fairness used first-year average and bar passage). +- Papers and pages: Wightman 1998, LSAC research report; OpenML page. +- Metric: none; the fallback if ever revived. +- For the curator: the original study's bar-passage outcome is a different, possibly real task (a law school deciding on academic support); it would be a new record from the LSAC source, not this cut. + OpenML d/43889, a 1991 survey of US law-school students (the well-known law-school/LSAC fairness dataset) with race as a feature. The curator flags it as a survey rather than a genuine predictive task, and it is already in TabArena Reject. It is widely used as an algorithmic-fairness benchmark with race/sensitive attributes, raising ethical-scope concerns, and it is a survey table rather than a real-world deployed prediction problem. A human could confirm the exact target (bar passage / admission), but the survey nature plus ethical sensitivity supports a No. --- diff --git a/curation/records/layerai_air_quality_prediction_challenge.md b/curation/records/layerai_air_quality_prediction_challenge.md index e7329a02..f7061aae 100644 --- a/curation/records/layerai_air_quality_prediction_challenge.md +++ b/curation/records/layerai_air_quality_prediction_challenge.md @@ -4,6 +4,7 @@ name: layerai-air-quality-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: - TBD tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: environmental science & climate required_split: - '?' @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: AirQo's PM2.5 readings estimated from Sentinel-5P columns for places without a sensor, a clear use case; watch the forecasting-like setup of the top solution. +- Source: Zindi competition with AirQo (Makerere University) and Layer, 2022-07-04 to 2022-10-02, a rerun of the #ZindiWeekendz hackathon of 2022-07-01; as published: `train.csv` and `test.csv`, PM2.5 from AirQo sensors in Kampala as target, Sentinel-5P L3 products extracted from Google Earth Engine (SO2, CO, NO2, HCHO, aerosol index, O3, cloud) as features, with `date` and `device` columns (named in the 1st-place write-up). Rows not checked (Zindi data needs a login). +- Use case: AirQo estimates air quality "in areas without air quality sensor devices" from satellite data (description page); fit direct; regime grouped by device for new places, or temporal: the 1st-place team found train and test "come from different distributions" (adversarial AUC 0.9988), so the release split is likely by time; one regime to pick from the use case (USE-10). +- Siblings and duplicates: `zindi_weekendz_layerai_air_quality_prediction_challenge` is the same task in the weekend hackathon (AI `No`, forecasting, a claim this screening does not support: the target is the same-day reading, not a horizon), keep one record (PROV-4, PROV-8); related but distinct: airqo_african_air_quality_prediction_challenge (2024, several cities), urban_air_pollution_challenge, airqo_ugandan_air_quality_forecast_challenge (a forecast). External collections: none found. +- Papers and pages: competition page (https://zindi.africa/competitions/layerai-air-quality-prediction-challenge); 1st-place write-up (discussion 13525: imputation with BRITS and a Temporal Fusion Transformer). +- Metric: mean absolute error, the leaderboard metric ("The evaluation metric for this challenge is Mean Absolute Error"): deliberate (MET-4); inputs none. +- Risks for /add-dataset: the top solution treated it as a time series (TFT, possibly with the device's past PM2.5): check TP-10 and SEL-11, and drop any lagged target; Sentinel-5P columns are retrieved atmospheric quantities, read under SEL-15 rather than as image descriptors (SEL-13), to confirm; heavy missingness in the satellite columns. +- Priority: gap + (grouped regime thin, if grouped by device), use case + (the source names the actor and the gap it fills), source + (AirQo and the GEE product list), shape - (size not checked), risk - (forecasting-like solution, shift between train and test). + A Zindi competition (LayerAI) on air-quality prediction; the record has no comments or details beyond the source link. Air-quality challenges are usually tabular regression on sensor/meteorological features, which can be a valid IID or temporal predictive task, but some such challenges are forecasting future pollutant values over a horizon, which would be out of scope. There is too little info in the record to reject outright, so this is a TBD. A human must download the data to confirm it is a fixed predictive (not horizon-forecasting) task, identify the target pollutant, dataset size, and whether a temporal vs random split applies. diff --git a/curation/records/lhc_olympics_2020.md b/curation/records/lhc_olympics_2020.md index 6f0cd04b..3c72241c 100644 --- a/curation/records/lhc_olympics_2020.md +++ b/curation/records/lhc_olympics_2020.md @@ -5,9 +5,8 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- No Good Target / Scientific Discovery -- Wrong Domain / Source Modality - AHDS (Artifical/Handmade/Deterministic/Simulated) +- Wrong Domain / Source Modality tags: - AI-Filled (Verify) original_source: Website @@ -29,6 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16; `Wrong Domain / Source Modality`, SEL-12): simulated collider physics with its own benchmark community, like `higgs`. +- Source: LHC Olympics 2020 (Kasieczka, Nachman, Shih et al., arXiv 2101.08320; https://lhco2020.github.io/homepage/), R&D set on Zenodo (https://doi.org/10.5281/zenodo.2629072): "The events are produced using Pythia8 and Delphes 3.4.1, with no pileup or MPI included"; "1M QCD dijet events and 100k W'->XY events"; each event a list of hadrons "(pT, η, φ, ...) zero-padded up to 700 hadrons", an array of 1.1M x 2101 with a "signal/background truth bit" that "won't be included in the actual challenge". Black boxes: "These samples may contain some new signal(s)". +- Use case: a model-agnostic search for new physics (anomaly detection on unlabelled black boxes); the labelled R&D set is a simulation-vs-simulation classification, the setup `higgs` was rejected for. +- Siblings and duplicates: `higgs` (Lennart, 2026-09-22: `No`, simulated physics with dedicated benchmarks; its note names the LHC Olympics as one of the field's own benchmarks), `susy`, `hepmass`, cern_* records. External collections: HEP-ML living review, Datasets. +- Metric: none for a tabular task (the challenge scored whether teams found the signal). +- Reject check (AI, 2026-10-22): stands. Zenodo R&D set: "produced using Pythia8 and Delphes 3.4.1" (https://doi.org/10.5281/zenodo.2629072). Each event is a zero-padded list of up to 700 hadrons, and the native task is anomaly detection on unlabelled black boxes (https://lhco2020.github.io/homepage/). That is SEL-16 (simulated; the curator's higgs note names the LHC Olympics as one of the field's own benchmarks), with SEL-12 (particle lists) and SEL-3 on top. No real use case survives any of them. + Community challenge for model-agnostic anomaly detection at the LHC (Kasieczka et al., arXiv 2101.08320). Each simulated event is "a list of all hadrons (pT, eta, phi, ...) zero-padded up to 700 hadrons"; a labelled R&D dataset plus unlabelled "black boxes" that "may contain some new signal(s)". Hosted on Zenodo. Out of scope on three counts: the task is unsupervised / weakly supervised anomaly detection, not a fixed classification or regression target; the input is a variable-length particle list, not a table; and the data are simulated (criterion 4B). Listed in the HEP-ML living review Datasets section. Data not inspected. diff --git a/curation/records/loan_default_prediction_challenge.md b/curation/records/loan_default_prediction_challenge.md index f909aa0f..194ad8b3 100644 --- a/curation/records/loan_default_prediction_challenge.md +++ b/curation/records/loan_default_prediction_challenge.md @@ -4,6 +4,7 @@ name: loan-default-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: - TBD tags: @@ -11,12 +12,15 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2018' domain: finance required_split: - '?' problem_type: Binary Classification +original_data_state: Database (or multiple to-be-joined tables) source_links: - https://zindi.africa/competitions/loan-default-prediction-challenge/data +- https://zindi.africa/competitions/data-science-nigeria-challenge-1-loan-default-prediction source_row: 898 needs_review: - ai_unverified @@ -27,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: SuperLender's repeat-loan default at application, a direct credit decision; the record's link is a 2022 rerun of Data Science Nigeria's Challenge 1. +- Source: SuperLender data first released in Zindi's "Data Science Nigeria Challenge #1: Loan Default Prediction" (start 2018-10-11, https://zindi.africa/competitions/data-science-nigeria-challenge-1-loan-default-prediction); the record's link is the AfroPavo Analytics internship rerun (2022-03-15 to 2022-04-16), whose API names the DSN challenge as its parent. As published: three tables per split, `traindemographics.csv`, `trainperf.csv` (the repeat loan to score, target `good_bad_flag`, "good = settled loan on time; bad = did not settled loan on time"), `trainprevloans.csv` (the customer's earlier loans), joined on `customerid`. Rows not checked (Zindi data needs a login). +- Use case: "These two fundamental drivers need to be determined at the point of each application to allow the credit grantor to make a calculated decision ... if an applicant should get a loan, and if so - what the size, price and tenure of the offer will be" (description page); fit direct; regime temporal (`approveddate`, `creationdate` ship; one repeat loan per customer, so grouping adds nothing). +- Siblings and duplicates: no other record of the SuperLender data (grep: superlender, data science nigeria); other credit-default records (zimnat_loan_default_predictio, african_credit_scoring_challenge, lt_vehicle_loan_default_prediction) are other lenders. External collections: none found. +- Papers and pages: the two competition pages; no paper. +- Metric: error rate, "the percentage of loans you predict incorrectly" (Evaluation page of both competitions), the leaderboard metric (MET-4); hard labels at 0.5 with no costs stated, so a candidate for overruling under MET-4 and MET-6 once probed; inputs none. +- Risks for /add-dataset: `trainprevloans` holds `closeddate` and `firstrepaiddate`, known only for loans closed before the application date: filter by date (LEAK); `totaldue` and `termdays` are offer terms set with the decision (USE); aggregation of previous loans (joins); exact birthdate and GPS coordinates to coarsen (personal data). +- Priority: gap - (finance, binary, temporal not thin), use case + (actor and decision named), source + (the page documents every column), shape + (mixed types and dates; the row count not checked), risk - (three tables to join, post-application dates). + A Zindi loan-default competition where the task is predicting whether a loan was good or bad, a classic real-world tabular binary classification problem in credit risk. This maps well to the benchmark criteria: genuine predictive task, clear target, finance domain. The record lacks size and split details, so it cannot be confirmed Yes outright. A human must verify dataset size after cleaning, the exact target definition, absence of leakage, and whether a temporal/grouped split (by applicant or time) is warranted, but it is a plausible Yes. --- diff --git a/curation/records/localised_precipitation_forecasting_in_brazzaville_in_republic_of_congo_using_ai.md b/curation/records/localised_precipitation_forecasting_in_brazzaville_in_republic_of_congo_using_ai.md index a21d781c..0764214d 100644 --- a/curation/records/localised_precipitation_forecasting_in_brazzaville_in_republic_of_congo_using_ai.md +++ b/curation/records/localised_precipitation_forecasting_in_brazzaville_in_republic_of_congo_using_ai.md @@ -3,7 +3,8 @@ unique_name: localised_precipitation_forecasting_in_brazzaville_in_republic_of_c name: localised-precipitation-forecasting-in-brazzaville-in-republic-of-congo-using-ai checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Time-series (Forecasting) tags: @@ -11,12 +12,14 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2025' domain: environmental science & climate required_split: - Temporal (NON-IID) problem_type: Regression source_links: - https://zindi.africa/competitions/localised-precipitation-forecasting-in-brazzaville-in-republic-of-congo-using-ai +- https://power.larc.nasa.gov/ source_row: 948 needs_review: - ai_unverified @@ -27,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a same-day precipitation estimate from NASA POWER gridded daily data for one location, not clearly a forecast, with no decision behind it. +- Source: Zindi hackathon "Localised Precipitation Forecasting in Brazzaville Using AI", 10-31 July 2025, open only to Congolese nationals and students at Congolese universities (Zindi API, rules page). The Data page says: "The data is sampled from the Source Native Resolution Daily Data collected by NASA/POWER from January 01, 2010 to January 01, 2024. It includes weather observations from Brazzaville." Files: Train_data.csv (304,839 bytes), Test_data.csv (123,272 bytes), VariableDefinitions.csv (361 bytes), so roughly 5,100 days with fewer than 10 variables. The column list was not read (it needs a login). +- Use case: the description names urban planning, agriculture, water management and flood prevention, with the goal "predict daily or sub-daily precipitation levels in Brazzaville using meteorological and spatio-temporal data". No actor or decision is named, and nobody acts on a same-day rainfall estimate made from other same-day POWER variables, because POWER publishes the precipitation itself. Fit: approximate at best. Regime: temporal (one daily series), but the sample IDs on the Evaluation page (`ID_ZdrVnE_20110615`, `ID_MdWBtU_20230910`) suggest the test days are interleaved with the training span rather than a future horizon. +- Siblings and duplicates: none in `curation/records/` (searched for brazzaville and NASA POWER). External collections: none known. +- Papers and pages: Zindi page only. NASA POWER (https://power.larc.nasa.gov/) is the upstream source. POWER's daily meteorology comes from the MERRA-2 reanalysis and its precipitation from a gridded product, so features and target are model or satellite estimates for one grid cell, not station readings. +- Metric: RMSE, on the Zindi Evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"; API `error_metric: root_mean_squared_error`). A leaderboard metric (MET-4). Inputs: none. +- Risks for /add-dataset: whether lagged values make this a forecast of a series (SEL-10; the old `Time-series (Forecasting)` marker stays as the open concern); gridded reanalysis inputs and target (SEL-16 / PROV-3 to weigh); a single series of about 5k days with a few weather variables, so `Too Small` is unlikely but the task may be close to `Trivial` (humidity and pressure against rain). +- Priority: gap - (environmental science, regression, temporal: none thin), use case - (no actor or decision), source + (NASA POWER is documented), shape - (one grid cell, about 5k days, few features), risk - (forecast framing, model-generated values); floor 1. + A Zindi competition whose name explicitly states localised precipitation forecasting for Brazzaville. Forecasting future precipitation values over a horizon is time-series forecasting, which is explicitly excluded from the benchmark scope. The name alone strongly signals a forecasting task rather than a fixed predictive task that merely needs a temporal split. A human could double-check whether it is framed as nowcasting from concurrent features (which might change the verdict), but on the name and typical framing this is a No. diff --git a/curation/records/lowbwt.md b/curation/records/lowbwt.md index 9a660258..6d5c4aff 100644 --- a/curation/records/lowbwt.md +++ b/curation/records/lowbwt.md @@ -26,6 +26,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict `No` holds. The linked data is generated, so `AHDS (Artifical/Handmade/Deterministic/Simulated)` is the deciding marker (SEL-16, PROV-3). +- Source: the link is OpenML 1193, named `BNG(lowbwt)`, 31,104 rows x 10 features, uploaded 2014-11-12 by Holmes, Pfahringer, van Rijn and Vanschoren (OpenML API). BNG datasets are rows sampled from a Bayesian network fitted to a small UCI/StatLib dataset, made for the data-stream experiments of van Rijn et al. (2014). The real data behind it is Hosmer & Lemeshow's low-birth-weight study (Baystate Medical Center, 1986), 189 mothers. +- Use case: none for the BNG copy. The original (risk factors of low birth weight) is a 1980s teaching dataset. +- Siblings and duplicates: no other record for the 189-row original (searched for lowbwt, low birth, birthwt). External collections: OpenML BNG suite. +- Metric: none (OpenML has no task metric of its own). +- Counter-check of the old note: "Too Small" was removed on policy, correctly. `Missing source information` is weaker than the old note says: the BNG generator and its parent dataset are known. The deciding fact is that the rows are generated. The curators' notes ("simulated data streams without source information", Lennart; "too small", Andrej) agree with `No`. No change recommended. + OpenML id 1193, a low-birth-weight dataset. The TabArena curation verdict and reviewer comments converge on tiny data (the original has only 189 samples) plus concerns that this version is a simulated BNG/data-stream variant lacking source information. Both the tiny original size and the simulated/missing-provenance status are disqualifying under the criteria. The 'BNG' name suggests a Bayesian-Network-Generator synthetic expansion. A human could confirm whether any usable real version exists, but the existing TabArena reject verdict supports a No. **Update (policy):** removed the "Too Small" marker — size is not a rejection reason (decided post-Data Foundry); this record stays No on its other ground(s). diff --git a/curation/records/lucas_topsoil.md b/curation/records/lucas_topsoil.md index e4ef66d7..204e209f 100644 --- a/curation/records/lucas_topsoil.md +++ b/curation/records/lucas_topsoil.md @@ -4,6 +4,7 @@ name: LUCAS Topsoil Survey (EU, 2009 / 2015 / 2018) checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) - Multi-target @@ -31,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: the EU's measured topsoil survey, documented by the JRC, a spatial (grouped) regression. The task and the covariate join still have to be chosen. +- Source: JRC/ESDAC LUCAS Soil module. Waves: 2009/2012 (about 20k points), 2015 (about 22k), 2018. The 2018 page: "Topsoil data for 18,984 points", most at 0-20 cm, with pH (CaCl2, H2O), organic carbon, CaCO3, N, P, K, EC and oxalate Fe/Al; texture and bulk density come as separate packages. Point coordinates and observed land cover are in the Eurostat LUCAS survey and join on `POINT_ID`; covariates for 2015 ship with that wave, and 2018 covariates are "in preparation". Access needs a request form; the terms (no sub-licensing, no maps that reveal locations) are noted, not weighed (EV-4). +- Use case: digital soil mapping. The JRC predicts topsoil properties at unsampled locations for EU soil and climate policy (organic-carbon and nutrient maps, e.g. Ballabio et al. 2016 and 2019, built from LUCAS). The actor (JRC, national agencies) and the decision (policy maps) come from that literature, not from a statement in the release, so the fit is approximate. Regime: grouped by location (spatial blocks), or temporal across waves for the revisited points. +- Siblings and duplicates: `open_soil_data` (iSDA; mostly spectroscopy predictions), `open_soil_spectral_library` (carries LUCAS spectra), `africa_soil_property_prediction_challenge`, `africa_soil_profiles_database`, `wosis_soil_profiles` (WoSIS includes LUCAS profiles: check overlap). None is a duplicate of the EU survey itself. External collections: OSSL (spectra, with geographic restrictions). +- Papers and pages: Orgiazzi et al. 2018 (EJSS 69:140-153, the review of LUCAS Soil); Fernandez-Ugalde et al. 2022 (EUR 31144 EN, doi:10.2760/215013, the 2018 module); Ballabio et al. 2016 (Geoderma, texture maps from LUCAS). +- Metric: none found as one deliberate choice. The mapping papers report RMSE, R2 and others side by side, so the fallback applies (MET-5). Inputs: none. +- Risks for /add-dataset: the features. Lab properties predicting each other is a discovery table (SEL-3), spectra lean out (SEL-15, raw spectral bins), and covariates need a join from Eurostat or the 2015 covariate package (heavy joins). Also the choice of one target among several (`Multi-target` until one is picked, CLEAN-18), and which wave to use. +- Priority: gap + (Grouped regime thin), use case - (approximate; actor from the literature), source + (JRC documentation and papers), shape + (about 19-22k points per wave, spatial structure), risk - (covariate join and target choice). + EU-wide harmonised topsoil survey run by JRC/ESDAC: 2009/2012 (19,967 points, 25 countries), 2015 (21,859, all 28 member states), 2018 ("18,984 samples", EU + UK), 2022 in release. Measured pH, organic carbon, N, P, K, carbonates, texture, CEC, EC, oxalate Fe/Al; VisNIR spectra for 2015 (also in OSSL as LUCAS). Repeated visits to the same points make a temporal split (2009+2015 -> 2018) possible; otherwise spatially grouped. Access terms, noted only and not a factor in the verdict: data are "free of charge" after a request form, but the 2018 terms say results must go through "proper aggregation process that prevents any individual body from being identified", "geographic location of samples cannot be detectable on maps", and data are "restricted to licensee organization staff", i.e. no redistribution as-is; OSSL carries LUCAS spectra only "with geographic restrictions". On the data itself this is a strong candidate: measured laboratory values, ~19-22k points per wave, repeated visits, spatial or temporal split. To check: usable size per wave after cleaning, target choice, covariate join. Data not inspected. diff --git a/curation/records/medical_charges.md b/curation/records/medical_charges.md index 1c6b8962..c34e8586 100644 --- a/curation/records/medical_charges.md +++ b/curation/records/medical_charges.md @@ -31,6 +31,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the TabArena reject holds. The served copy has no predictive task worth benchmarking, and its main remaining feature is a part of the target. +- Source: OpenML 42130 (163,065 rows x 12) is the CMS Inpatient Utilization and Payment Public Use File for FY2011 (`Inpatient_Data_2011_CSV.zip`), "organized by hospital and Medicare Severity Diagnosis Related Group (MS-DRG)", target `average_total_payments`. OpenML 44146 is Grinsztajn's numeric cut with 3 features (OpenML API). +- Use case: none from the source. The PUF is a published statistical summary. Payments under IPPS follow the DRG weight and the hospital's rates, and "Average Medicare Payments" is a component of the total payment ("The Total Payment Amount includes the DRG amount, ...", CMS catalog), so the 44146 cut is close to an identity. +- Siblings and duplicates: `medicare_inpatient_cost` is the same CMS release series under its current name ("Medicare Inpatient Hospitals - by Provider and Service", data years 2013-2023 in the CMS catalog). The two records should be decided together; under PROV-6 only one cut could enter. External collections: Grinsztajn et al. 2022 benchmark, TabSTAR. +- Metric: none (the benchmark copies score R2 or RMSE by habit). +- Counter-check of the markers: the curator's notes (inference-time availability, string address features, grouping by provider) support `No Good Target / Scientific Discovery`. `Data Quality Issue` (damage that cannot be repaired) is not shown: the payment leak is removable by dropping columns, so that marker could go. No change to the verdict recommended. + OpenML medical-charges dataset (d/42130, 44146), used in Grinsztajn's benchmark in a trivially preprocessed numeric form. The curator notes most available features would not be available at inference time, many are string/address features needing special preprocessing, and the data is likely grouped by hospital/provider, with leakage risk. It is already in TabArena Reject. The combination of inference-time-unavailable features, leakage, and a listing/analytics rather than genuine predictive framing makes it a poor candidate. A human could revisit a leakage-cleaned version, but on current evidence this is a No. --- diff --git a/curation/records/medical_facility_operational_data.md b/curation/records/medical_facility_operational_data.md index d0e2afcc..1420b44e 100644 --- a/curation/records/medical_facility_operational_data.md +++ b/curation/records/medical_facility_operational_data.md @@ -3,23 +3,24 @@ unique_name: medical_facility_operational_data name: Medical Facility Operational Data checked_by: - AI (UNVERIFIED) -suggestion: 'No' -decision_markers: -- No Good Target (yet) -- Missing source information +suggestion: TBD -> Yes +priority: 3 tags: - 2nd Tier / Scientfic Discovery - Non-IID (Temporal) - AI-Filled (Verify) collections: - New (BeyondArena) +original_source: Other +year: '2020' domain: medical & healthcare required_split: -- '?' -problem_type: TBD +- Temporal (NON-IID) +problem_type: Regression original_data_state: One Table source_links: - https://www.kaggle.com/datasets/weirdolucifer/medical-facility-operational-data +- https://doi.org/10.7910/DVN/X4SJWS source_row: 850 needs_review: - ai_unverified @@ -30,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: the Kaggle file is a re-upload of Pianykh's MGH "Operational Predictions Dataset" (Harvard Dataverse, 2020, PLoS One paper), a patient wait-time and delay regression at four imaging facilities. +- Source: Harvard Dataverse doi:10.7910/DVN/X4SJWS, "Operational Predictions Dataset", Oleg Pianykh, deposited 2020-05-24, CC0, one file `WaitData.Published.tab`. The Kaggle copy (weirdolucifer, 2021-02-10) ships the same file name, `WaitData.Published.xlsx`, and the description: "operational features, captured in four different hospital facilities, processing walk-in (F4) or scheduled (F1, F2, F3) patients ... one line per each patient visit event", with timestamps "set to the fictitious future dates ... Only their relative timing was preserved". Data paper: Crowley, Guitron, Son & Pianykh 2020, PLoS One 15(6):e0233810: F1 MRI 15,494 rows, F2 ultrasound 36,154, F3 CT 30,173, F4 X-ray 69,469 (Nov 2017-Jul 2019); 84 candidate features (congestion, customer, resource, task, time); targets delay in minutes (F1-F3) and wait in minutes (F4). Its data availability statement points to the Dataverse. +- Use case: at a patient's arrival, the facility predicts the wait or delay. The earlier MGH work put the prediction on waiting-room displays and flagged patients who waited much longer than predicted (RSNA 2014; Curtis et al. 2018, JACR). Actor: the facility's operations staff. Moment: arrival (scheduled or walk-in). Fit: direct. Regime: temporal (relative order preserved), probably per facility. +- Siblings and duplicates: none for this source (`hospital_patient_data` is a different, unsourced Kaggle wait-time upload). The old reasons "no target" and "missing source" are refuted by the Dataverse and the paper, so those markers are dropped. +- Papers and pages: Crowley et al. 2020 (the data paper); Pianykh & Rosenthal 2015 (JACR, "Can We Predict Patient Wait Time?"); Curtis et al. 2018 (JACR, ML for wait times and delays, MGH). +- Metric: MAE in minutes. Crowley et al. Table 2 is "Predictive performance, measured by mean absolute error", while MSE drives their feature selection and they also give a ratio to an intercept-only model. MAE looks like the deliberate criterion; confirm under MET-5. Inputs: none beyond the rows; the facility is a column. +- Risks for /add-dataset: "captured at the time of patient arrivals and departure", so some features may be known only after the wait (LEAK; check each of the 84 against the arrival moment). Also one task or four: wait (F4) and delay (F1-F3) are different targets, so pick one facility or one target. The timestamps are fictitious, but their order holds for a time split. +- Priority: gap - (medical, regression, temporal: none thin), use case + (direct, named by the source), source + (Dataverse and paper), shape + (about 151k visits, 84 features, temporal), risk - (features recorded at departure). + A Kaggle 'Medical Facility Operational Data' dataset that is a single Excel sheet of mixed operational data with no defined task or target, per the curator's note. Without a clear predictive target it reads as a scientific-discovery / listing table rather than a benchmark task, and provenance is thin. It is tagged 2nd Tier and temporal. There is essentially no clear target to model, so it cannot be a Yes. A human would need to determine whether any meaningful operational target (e.g., occupancy, wait time) can be carved out, but as presented this is a No. --- diff --git a/curation/records/medicare_inpatient_cost.md b/curation/records/medicare_inpatient_cost.md index ab127eac..644bfabc 100644 --- a/curation/records/medicare_inpatient_cost.md +++ b/curation/records/medicare_inpatient_cost.md @@ -4,14 +4,15 @@ name: Medicare Inpatient Cost checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - No Good Target (yet) -- Data Quality Issue tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) collections: - New (BeyondArena) +original_source: GOV Website domain: medical & healthcare required_split: - Grouped (NON-IID) @@ -29,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a CMS statistical summary (hospital x DRG averages) with no task of its own, from the same release series as the rejected `medical_charges`. +- Source: CMS, "Medicare Inpatient Hospitals - by Provider and Service", annual data years 2013-2023 in the CMS catalog (data.cms.gov/data.json). The release "provides information on inpatient discharges for Original Medicare Part A beneficiaries by IPPS hospitals ... organized by hospital and Medicare Severity Diagnosis Related Group (DRG)", and "The Total Payment Amount includes the DRG amount, claim per diem amount, ... and diagnosis related group outlier amount". One row per hospital x DRG with discharges, average covered charges, average total payment and average Medicare payment, plus hospital name, address and location. +- Use case: none stated. CMS publishes the table for transparency. A target such as the average payment follows the IPPS formula (DRG weight x hospital rates), and the other payment columns are components of it. Fit: none from the source. Regime: grouped by hospital, or temporal across data years. +- Siblings and duplicates: `medical_charges` (OpenML 42130/44146) is the FY2011 file of the same series ("Inpatient Utilization and Payment Public Use File"). It is `No` with TabArena Reject and the curator's note that most features are not known at inference time. Under PROV-6 these are cuts of one dataset, so decide them together. External collections: Grinsztajn et al. 2022 and TabSTAR via `medical_charges`. +- Papers and pages: the CMS catalog and data dictionary; no paper defines a predictive task on it. +- Metric: none found (CMS catalog, data dictionary, the OpenML copies), so the fallback. +- Risks for /add-dataset: leakage among the payment and charge columns (CC: "several things being an indicator of each other"); a near-deterministic target under IPPS (SEL-5 to weigh); addresses as strings; aggregated rows, not patients. `Data Quality Issue` (irreparable damage) is dropped: dropping the columns removes the leak. `No Good Target (yet)` stays. +- Priority: gap + (Grouped regime thin), use case - (none), source + (CMS documentation), shape - (after dropping the leaking columns only hospital, DRG, discharges and charges remain), risk - (deterministic payment rule, leak). + CMS Medicare inpatient hospitals summary by provider and service, a real government data release. The curator notes it is essentially a listing of aggregated data without a real predictive task, and that many columns (e.g., payment amounts) are mutually indicative and would leak the target. A regression target (e.g., average cost/payment) could in principle be defined, but it would need careful leakage removal and is a weak benchmark signal. This is borderline: not a clean Yes, not an outright No, so TBD -> 2nd Tier. A human must define a leakage-free target, decide on grouping by provider, and assess size and signal. --- diff --git a/curation/records/mental_health_dataset.md b/curation/records/mental_health_dataset.md index f75dfe86..13160c02 100644 --- a/curation/records/mental_health_dataset.md +++ b/curation/records/mental_health_dataset.md @@ -6,8 +6,7 @@ checked_by: suggestion: 'No' decision_markers: - Missing source information -- No Good Target (yet) -- No Good Target / Scientific Discovery +- AHDS (Artifical/Handmade/Deterministic/Simulated) tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -32,6 +31,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Missing source information`, PROV-1): an anonymous Kaggle upload whose own description does not describe its file, with no traceable upstream. +- Source: Kaggle bhavikjikadara/mental-health-dataset, one file `Mental Health Dataset.csv` (31 MB, 2024-03-18), CC BY 4.0. Re-served as OpenML 46721 (292,364 rows x 17, default target `Occupation`). The uploader's description hedges and describes other data: "This dataset appears to contain a variety of features related to text analysis, sentiment analysis, and psychological indicators, likely derived from posts or text data. Some features include readability indices such as Automated Readability Index (ARI) ... sentiment compound". The file has none of these. Its 17 columns are survey items (`Timestamp, Gender, Country, Occupation, self_employed, family_history, treatment, Days_Indoors, Growing_Stress, Changes_Habits, Mental_Health_History, Mood_Swings, Coping_Struggles, Work_Interest, Social_Weakness, mental_health_interview, care_options`, OpenML feature list). Several of them match the OSMI 2014 tech survey (`mental_health_tech`, 1,259 respondents), which this table exceeds by more than 200 times. The only provenance is "I collected this dataset from the internet", with a link to the OurWorldInData mental-health page (quoted in the curator note below), which publishes country aggregates, not respondents. +- Use case: none; the curator note calls it a discovery table without a clear target, and the OpenML target `Occupation` is no decision. +- Siblings and duplicates: `mental_health_tech` (OSMI survey, `TBD -> 2nd Tier`, Lennart: "no ground truth or objective target"), `busara_mental_health_prediction_challenge`. External collections: TabSTAR (via OpenML 46721). +- Metric: none. +- Also weighed: the row count far above any survey these items come from suggests generated or expanded rows (PROV-3, not tested here). +- Reject check (AI, 2026-10-22): stands. Fails PROV-3 on the data itself, beyond PROV-1: OpenML 46721 (same file as the Kaggle upload, https://www.openml.org/d/46721) has 292,364 rows on only 580 distinct Timestamps, all 8/27/2014-era OSMI survey times, at least 390 rows per timestamp, and 194,641 rows duplicate another once Timestamp is dropped: an expansion of the OSMI 2014 survey (1,259 respondents), not respondents. The description describes other data (readability and sentiment features) and the curator CC finds no target. Add AHDS (SEL-16) to Missing source information; PROV-2's keep needs data that passes PROV-3, which this does not. + A Kaggle/OpenML mental-health dataset (d/46721) the curator describes as parsed from text for scientific discovery, with no clear predictive target and ambiguous provenance ('collected from the internet', linked to OurWorldInData). The record already carries Missing source information and No Good Target markers. It is not a genuine deployed predictive task and provenance is unclear, both disqualifying. A human could revisit if a concrete, well-sourced target emerges, but as presented this is a No. --- diff --git a/curation/records/microsoft_x_data_science_nigeria_2024_ai_bootcamp_qualification_hackathon.md b/curation/records/microsoft_x_data_science_nigeria_2024_ai_bootcamp_qualification_hackathon.md index 4ed39077..ea82e2c1 100644 --- a/curation/records/microsoft_x_data_science_nigeria_2024_ai_bootcamp_qualification_hackathon.md +++ b/curation/records/microsoft_x_data_science_nigeria_2024_ai_bootcamp_qualification_hackathon.md @@ -3,7 +3,8 @@ unique_name: microsoft_x_data_science_nigeria_2024_ai_bootcamp_qualification_hac name: microsoft-x-data-science-nigeria-2024-ai-bootcamp-qualification-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD tags: @@ -12,9 +13,10 @@ collections: - New (BeyondArena) original_source: Zindi year: '2024' +domain: medical & healthcare required_split: - '?' -problem_type: TBD +problem_type: Binary Classification source_links: - https://zindi.africa/competitions/microsoft-x-data-science-nigeria-2024-ai-bootcamp-qualification-hackathon source_row: 968 @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a heart-disease binary task for a training hackathon, about 10k rows of undocumented origin, re-run from a 2023 DSN hackathon. +- Source: Zindi, "Data Science Nigeria X Microsoft 2024 AI Bootcamp Qualification Hackathon", 16-30 September 2024, subtitle "Heart Disease prediction for the DSN AI Bootcamp 2024" (Zindi API). Data page: "There are ~7000 entries in Train and ~3000 entries in Test". Files: `Train Dataset .csv` (317,864 bytes), `Test Dataset.csv` (111,997 bytes), `Variable_Definitions.csv` (470 bytes), so about a dozen short columns. The API names as parent "DSN AI Bootcamp 2023 Mr and Miss Algorithm Hackathon" (August 2023), which has the same description, a 470-byte Variable Definitions file and a Train.csv of 308,417 bytes: the same data, re-split. The origin of the patient records is not stated (Zindi pages, DSN community posts searched); it is probably a public heart-disease table re-uploaded. The columns were not read (login needed). +- Use case: "determining the likelihood of an individual having heart disease ... utilizing easily accessible patient information" for early detection. No actor or moment is named. Fit: approximate. Regime: likely IID (one row per person); unknown until the columns are read. +- Siblings and duplicates: no record for the 2023 parent hackathon. The other DSN records (`dsn_ai_bootcamp_qualification_hackathon_by_data_science_nigeria`, a loan-default task; `dsn_free_ai_classes_in_every_city_2025`, a retail-sales task) are different data. Check against the heart records (`heart_disease*`, `heart_failure*`, `framingham_heart_study_fhs`) and the public 70k cardiovascular or BRFSS heart tables once the columns are known (PROV-4). +- Papers and pages: the Zindi page only; no top solutions found. +- Metric: accuracy ("The error metric for this competition is Accuracy"; API `straight_accuracy`), submitted as hard 0/1 labels. A leaderboard metric (MET-4), but a threshold metric with no stated cost or threshold (MET-6), so the fallback may be preferable. Inputs: none. +- Risks for /add-dataset: unknown origin of the rows (PROV-2: the competition is the source to point to) and a likely duplicate of a known heart dataset (PROV-4). Possibly SEL-7 if the label is a self-reported diagnosis. +- Priority: gap - (medical, binary: not thin), use case - (no actor named), source - (origin undocumented), shape + (about 10k rows), risk - (likely re-upload); floor 1. + A 2024 Zindi qualification hackathon (Data Science Nigeria x Microsoft) with no description in the record beyond the link. Zindi qualification hackathons are typically standard tabular classification/regression tasks, which would be in scope, but the subject matter and target are unknown. There is too little information to reject, so per the heuristics this is a TBD rather than a No. A human must open the competition to identify the actual task, target, problem type, domain, size, and required split before a final verdict. diff --git a/curation/records/mimic_iv.md b/curation/records/mimic_iv.md index e1811e9f..220e5609 100644 --- a/curation/records/mimic_iv.md +++ b/curation/records/mimic_iv.md @@ -4,15 +4,17 @@ name: MIMIC-IV checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Needs extensive data wrangling -- Ethical Issue tags: - '?' - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) collections: - New (BeyondArena) +original_source: Other +year: '2020' domain: medical & healthcare required_split: - '?' @@ -20,6 +22,7 @@ problem_type: TBD original_data_state: Database (or multiple to-be-joined tables) source_links: - https://physionet.org/content/mimiciv/ +- https://doi.org/10.1038/s41597-022-01899-x source_row: 798 needs_review: - ai_unverified @@ -30,6 +33,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a strong, documented clinical database with many real tasks, but it is a relational database with no task of its own. A cohort, target and prediction point have to be defined first, and access is credentialed. +- Source: MIT-LCP on PhysioNet, v1.0 in 2020 (current v3.x). Data paper: Johnson et al. 2023, "MIMIC-IV, a freely accessible electronic health record dataset", Scientific Data 10:1. Beth Israel Deaconess hospital and ICU admissions, 2008-2019 (2008-2022 in v3), in the `hosp` and `icu` modules (admissions, labs, medications, chartevents) plus linked ED and note modules. Many tables join on `subject_id`, `hadm_id` and `stay_id`. +- Use case: the database states none. Established tasks are in-hospital or ICU mortality, length of stay, readmission and decompensation at a fixed hour after admission (benchmark pipelines such as YAIB and the MIMIC-IV extraction benchmarks). Actor: ICU clinicians or bed managers. Fit: direct once one task is fixed. Regime: grouped by patient (repeat admissions), or temporal by anchor year. +- Siblings and duplicates: `icu_length_of_stay_icu_mortality` (TableShift, MIMIC-III via MIMIC-Extract; curator Lennart notes the overlap with a sepsis task), `row_549` (a Kaggle upload claiming MIMIC-III). MIMIC-IV overlaps MIMIC-III in time (2008-2012) for some patients, so one MIMIC task per target. External collections: TableShift (MIMIC-III tasks), YAIB. +- Papers and pages: https://physionet.org/content/mimiciv/ ; Johnson et al. 2023 (doi:10.1038/s41597-022-01899-x). +- Metric: none for the database. Each derived task follows the benchmark that defines it (often AUROC or AUPRC for mortality); otherwise the fallback. +- Risks for /add-dataset: heavy joins and a target to build (`Needs extensive data wrangling`); leakage from charted events after the prediction hour; credentialed access with a data use agreement that bars redistribution (EV-4: noted, not weighed). `Ethical Issue` is dropped: the data is de-identified and released for research, and access terms are no ethical concern under SEL-1. +- Priority: gap - (medical; the regime depends on the task), use case - (none until a task is chosen), source + (PhysioNet and Scientific Data paper), shape + (hundreds of thousands of admissions, mixed types), risk - (joins, target construction, leakage after the prediction point). + MIMIC-IV is the large PhysioNet critical-care relational database (many tables to be joined), supporting real clinical prediction tasks like mortality or readmission. It is a genuine real-world source but requires credentialed access (DUA), substantial wrangling to derive a fixed tabular task, and has patient-privacy/ethical access constraints. The curator is unsure which version/task to use or whether it is even feasible. This is not a drop-in dataset, so TBD -> 2nd Tier. A human must decide the specific cohort/target, handle the credentialing and licensing, define the join/feature pipeline, and choose a patient-grouped split. --- diff --git a/curation/records/mmu_2022.md b/curation/records/mmu_2022.md index 2c4470fb..d787b2ed 100644 --- a/curation/records/mmu_2022.md +++ b/curation/records/mmu_2022.md @@ -3,9 +3,9 @@ unique_name: mmu_2022 name: mmu-2022 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,6 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a re-run of the Tanzania tourism expenditure task of `ai4d_lab_tanzania_tourism_classification_challenge` on the same files. +- Source: Zindi `mmu-2022`, titled "Tanzania Tourism Classification Challenge", 25-28 November 2022, open only to "users from MMU CIT CLUB" (Multimedia University of Kenya), 24 participants (Zindi API). Its Data page is word for word that of the AI4D Africa Anglophone Research Lab challenge (June 2022, 522 participants): "The dataset describes 24,675 rows of up-to-date information on tourist expenditure collected by the National Bureau of Statistics (NBS) in Tanzania". Its files have the same byte sizes: Train.csv 3,147,987, Test.csv 979,230, SampleSubmission.csv 135,818, VariableDefinitions.csv 1,251, zip 483,694. The zip keeps the name `tanzania-tourism-classification-challenge.zip`. Both score log loss over the same six cost classes. +- Canonical record: `ai4d_lab_tanzania_tourism_classification_challenge` (`TBD -> Yes`, the curator's note: "We would use the second dataset", the 2022 multiclass version with 24,675 rows). Same precedent as `indabax_eswatini_2022` (Lennart, 2026-10-12: duplicate by Zindi file size). +- Use case and risks: those of the canonical record (NBS tourism exit survey; tour operators and the Tanzania Tourism Board estimating spend before a visit). +- Reject check (AI, 2026-10-22): stands. Zindi API (api.zindi.world/v1/competitions/mmu-2022): 'Tanzania Tourism Classification Challenge', 25-28 Nov 2022, 24 participants; Data Description word for word the AI4D one ('The dataset describes 24,675 rows of up-to-date information on tourist expenditure collected by the National Bureau of Statistics (NBS) in Tanzania'), same files, the zip still named tanzania-tourism-classification-challenge.zip, Log Loss in both. Same source, same task (PROV-4); keep ai4d_lab_tanzania_tourism_classification_challenge. + A 2022 Zindi competition using Tanzania tourism survey data to classify the range of expenditure a tourist spends, a real-world tabular classification task with a clear ordinal/multiclass target. This maps reasonably to the criteria as a genuine predictive task on survey-derived features. The record lacks size and split detail, so it is not a confirmed Yes. A human must verify dataset size, the exact expenditure-bucket target, feature availability, and whether a random split is appropriate, but it is a plausible Yes. --- diff --git a/curation/records/mobile_money_and_financial_inclusion_in_tanzania_challenge.md b/curation/records/mobile_money_and_financial_inclusion_in_tanzania_challenge.md index 249af69f..930936c2 100644 --- a/curation/records/mobile_money_and_financial_inclusion_in_tanzania_challenge.md +++ b/curation/records/mobile_money_and_financial_inclusion_in_tanzania_challenge.md @@ -4,6 +4,7 @@ name: mobile-money-and-financial-inclusion-in-tanzania-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 decision_markers: - TBD tags: @@ -11,10 +12,11 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: finance required_split: - '?' -problem_type: TBD +problem_type: Multiclass Classification source_links: - https://zindi.africa/competitions/mobile-money-and-financial-inclusion-in-tanzania-challenge/data source_row: 925 @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: FinScope Tanzania 2017 respondents classified into four mutually exclusive groups of financial-service use, with a targeting use the page states; survey data, a risk the curator weighed on a FinScope sibling. +- Source: Zindi competition "Mobile Money and Financial Inclusion in Tanzania Challenge", 26 Mar to 1 Jul 2019, sponsored by HDIF, Data Lab Tanzania and Esri (https://zindi.africa/competitions/mobile-money-and-financial-inclusion-in-tanzania-challenge, read through Zindi's page API); the data "was extracted from the FSDT Finscope 2017 survey and prepared specifically for this challenge": "demographic information and what financial services are used by approximately 10,000 individuals across Tanzania" (training.csv, test.csv, Variable_Codebook.docx), plus FSDT's Financial Access Map (GPS of cash outlets) as an optional layer. Target: `no_financial_services`, `other_only`, `mm_only`, `mm_plus`. Rows and columns not counted (the download needs a Zindi login). +- Use case: "This model can help mobile money providers target new clients and markets across Tanzania more effectively, and also help financial services providers cross-sell other financial services (savings, credit, and insurance)" (Description); fit approximate: the features come from the interview that also records the usage, so this is a look-alike targeting model on demographics, not one a provider feeds with its own data (SEL-7 to weigh: the respondent knows the label, the provider does not). Regime IID (one cross-sectional survey); a grouping by region or enumeration area is to check. +- Siblings and duplicates: `dataorg_financial_health_prediction_challenge` (FinScope MSME surveys, `No`; CC 2026-10-22: "Too risky a dataset: we have no information on the target ... and it is survey data"); here the page defines the target. `indabax_tanzania_mobile_banking_prediction_challenge`, `ix_mobile_banking_prediction_challenge`, `financial_inclusion_in_africa_for_botswana` (other Zindi financial-inclusion surveys): check that none reuses FinScope Tanzania 2017. External collections: none found. +- Papers and pages: FSDT's FinScope Tanzania 2017 survey (not read); the Zindi winners' post linked from the page; no paper on this task. +- Metric: multiclass log loss on the four class probabilities ("The evaluation metric for this challenge is the Log Loss", Evaluation tab; the Rules page's "accuracy score" is Zindi boilerplate); MET-4; inputs none. +- Risks for /add-dataset: the codebook decides which columns describe the usage itself (questions on savings, credit, insurance or mobile money would leak the class, LEAK rules); the four classes are a rule over usage flags (SEL-5: drop the flags if they ship); survey weights; the outlet map is a spatial join. +- Priority: gap 0 (finance has 17 accepted), use case 0 (actor and decision named, fit approximate), source + (FSDT's FinScope survey, a codebook ships), shape + (about 10,000 respondents, mixed demographic features), risk - (survey data, usage questions as leaks, the curator's FinScope precedent). + A Zindi competition on mobile money and financial inclusion in Tanzania, with no description in the record. Financial-inclusion challenges are typically tabular classification (e.g., predicting account ownership or mobile-money usage from survey/demographic features), which would be in scope, but the exact target is unknown. There is too little info to reject, so this is a TBD. A human must open the competition to confirm the target, problem type, size, and split regime. diff --git a/curation/records/mobile_phone_market_in_ghana.md b/curation/records/mobile_phone_market_in_ghana.md index 85064646..0d29f83c 100644 --- a/curation/records/mobile_phone_market_in_ghana.md +++ b/curation/records/mobile_phone_market_in_ghana.md @@ -3,10 +3,11 @@ unique_name: mobile_phone_market_in_ghana name: Mobile_Phone_Market_in_Ghana checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - No Good Target / Scientific Discovery -- Data Quality Issue +- TBD tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -30,6 +31,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: phone listings scraped from an unnamed Ghanaian website with specifications, region and price in cedis; a price regression is possible, but the source is an untraced scrape and the uploader frames it as market analysis. +- Source: Kaggle, "Mobile Phone Market in Ghana" by redpen12, one file Mobile-Phones.csv (493 KB, created 2023-10-18), CC0 (https://www.kaggle.com/datasets/redpen12/mobile-phone-market-in-ghana, metadata read with the Kaggle CLI): "The data was collected through web scraping"; described columns: brand and model, SD card, main camera, resolution, display, SIM, operating system, colour, region and location, screen size, battery, storage, RAM, selfie camera, price. The scraped site, the scrape date and whether a row is a listing or a catalogue model are not stated. Rows not counted. The record's year 2024 is unconfirmed (the file dates from October 2023). +- Use case: none stated ("It can be used for market analysis, consumer insights, and decision-making related to mobile phone purchases"); one we could write is a seller pricing a listing on a classifieds site (fit approximate, our wording). Regime IID if the rows are listings from one scrape. +- Siblings and duplicates: none among the records (searched ghana, mobile phone); TabArena v0.1 put it on its reject list (collection `TabArena Reject`; CC: "Simply scraped from a website. Predict the price - more for analysis and insights than for predictive performance"). External collections: none found. +- Papers and pages: none; the Kaggle keywords are "exploratory data analysis", "data cleaning", "gradient boosting". +- Metric: none found (the Kaggle page; no competition, task or paper), so the fallback (rmse; the price likely on a log scale, MET-3). +- Risks for /add-dataset: the first platform of the scrape (PROV-2, approximate fit at best); repeated listings of one model; the CC's concern that the price is set by model and storage (TP-4 to probe); the date and currency of the prices. +- Markers: `Data Quality Issue` dropped (no irreparable damage is documented: that marker needs an undone supervised transform or a manipulated table); `No Good Target / Scientific Discovery` kept as the curator's concern. +- Priority: gap 0 (technology & internet has 5 accepted), use case 0 (none stated), source 0 (an anonymous scrape), shape 0 (size not counted, mixed types), risk - (untraced source, a possibly near-trivial price). + Scraped Kaggle data on phone models in Ghana with the nominal task of predicting price. As the curator notes, this is web-scraped data better suited to analysis/insights than predictive performance, where price is largely determined by the observed spec features, making it a weak/circular target. It is already in TabArena Reject. The scraping data-quality and weak-target concerns make it a poor benchmark candidate. On current evidence this is a No. --- diff --git a/curation/records/mobility_prediction.md b/curation/records/mobility_prediction.md index 8bc3f806..c1c5adfa 100644 --- a/curation/records/mobility_prediction.md +++ b/curation/records/mobility_prediction.md @@ -4,6 +4,7 @@ name: mobility_prediction checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD - Time-series (Forecasting) @@ -12,10 +13,11 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: technology & internet required_split: - '?' -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/lusaka-mobility-prediction-hackathon/data - https://zindi.africa/competitions/luanda-mobility-prediction-hackathon @@ -31,4 +33,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: four one-day Yango hackathons (Lusaka, Luanda, Maputo, Kinshasa, Nov to Dec 2023) with one design: average speed per road section and weekday time slot from September 2023 track logs; a speed profile to fill in, tagged "Forecast" by Zindi, and four city copies of one task. +- Source: Zindi, "Lusaka / Luanda / Maputo / Kinshasa Mobility Prediction Hackathon Powered" by Yango, 4 Nov, 25 Nov, 25 Nov and 16 Dec 2023 (the four links in `source_links`, read through Zindi's page API); as published: "aggregated statistics on the numbers of cars and speed of travel along [the city's] main highways" from "the track logs of transportation service providers (Yango partners) in September 2023"; two tables, statistics per time interval and `persistent_id` (a road section), and a road graph (geometry, name); Train.csv, Test.csv, Graph.csv and variable definitions per city. A test ID reads `5405080210124813414_X_second_weekday_X_evening_rush_hour`: road section x weekday x time slot. Rows not counted (login). +- Use case: "Knowing the average traffic speeds helps in finding the quickest and most reliable routes. It also allows drivers and Yango to provide accurate estimated time of arrival (ETA) times" (Description); fit approximate: a router has the measured speeds of the logs, so the task fills in sections and slots of one month rather than predicting ahead. Regime unclear: grouped by road section, or slots of one month; how the test rows were chosen is not stated. +- Siblings and duplicates: `accra_mobility_prediction_hackathon` (the same design for Accra, September 2024; re-screened today as `TBD -> 2nd Tier`, priority 1); the Yango ride-time hackathons `2024_lusaka_mobility_prediction_hackathon`, `2025_accra_mobility_prediction_hackathon_powered`, `2025_maputo_mobility_prediction_hackathon_powered_by_yango` are a different task (per-trip ride time). The four cities here are one task: at most one would ship (PROV-6). External collections: none found. +- Papers and pages: none; Yango starter notebooks per city. +- Metric: RMSE on the average speed ("The evaluation metric for this competition is Root Mean Squared Error", Evaluation tab, same on all four); MET-4; inputs none. +- Risks for /add-dataset: whether it is a forecast of later intervals (SEL-10) or a fill-in of a profile (SEL-11, TP-10 against each section's naive profile); speed statistics of neighbouring slots of the same section in the features; country-restricted hackathons (download is a question for the curator); four copies. +- Markers: `Time-series (Forecasting)` kept as a concern (Zindi's own "Forecast" tag), not a fail: the page asks for a speed per section and slot, not values over a horizon. +- Priority: gap 0 (technology & internet has 5 accepted), use case 0 (named by Yango, fit approximate), source 0 (a hackathon page, no documentation beyond variable definitions), shape 0 (size not counted), risk - (forecasting-like profile, unclear test design, four city copies). + A group of Zindi 'mobility prediction' hackathons across several African cities (Lusaka, Luanda, Maputo, Kinshasa). 'Mobility prediction' over time often implies forecasting movement/flows, which would be out of scope, but it could also be a fixed origin/destination or trip-attribute classification task. The record has no detail, and the multiple linked competitions complicate a single verdict. Because it may be forecasting but might be a valid fixed task, this is TBD -> 2nd Tier. A human must inspect each competition's actual task and target before deciding and avoid double-counting near-duplicate city variants. diff --git a/curation/records/moneyball.md b/curation/records/moneyball.md index 522e618c..e9e2634c 100644 --- a/curation/records/moneyball.md +++ b/curation/records/moneyball.md @@ -4,6 +4,7 @@ name: Moneyball checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - No Good Target / Scientific Discovery tags: @@ -31,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: MLB team seasons 1962-2012 with runs scored as the target and same-season batting rates as features; a teaching dataset for the Moneyball story, an explanatory regression more than a prediction. +- Source: The Analytics Edge (MITx 15.071x on edX), data "gathered from baseball-reference.com"; re-uploaded to Kaggle by wduckett (baseball.csv, 67 KB) and to OpenML as 41021 "Moneyball" (2018, target `RS`). As published: 1,232 rows x 15 columns (Team, League, Year, RS, RA, W, OBP, SLG, BA, Playoffs, RankSeason, RankPlayoffs, G, OOBP, OSLG; 3,600 missing values, the playoff ranks and the opponent rates mostly). Year in the record (2000) is the Moneyball era, not the publication (the course, about 2013; Kaggle about 2017). +- Use case: the source's own framing is understanding: the data "can be used to better understand their methods", with questions such as "How does a team make the playoffs?" (OpenML description, copied from Kaggle). The course used the regression of RS on OBP and SLG to estimate how many runs a roster would score from its players' rates; here the rates are measured in the same season as the runs, so the fit to a decision is approximate at best. Regime temporal by Year if kept (the CC's point). +- Siblings and duplicates: none among the records; OpenML 41021 and the Kaggle copy are the same table (one record). External collections: TabSTAR. TabArena v0.1 put it on its reject list (CC: "clearly not a predictive performance task but an analytics task. Linear model best in TabRepo. Could include to also cover this bias"). +- Papers and pages: The Analytics Edge lecture on Moneyball; no paper defines a task. +- Metric: none found (OpenML 41021, the Kaggle page, the course description; the course reports R-squared of a linear model, not a deliberate criterion), so the fallback (rmse). +- Risks for /add-dataset: same-season outcomes as features (W, Playoffs, RankSeason, RankPlayoffs follow from RS; RA is the opponents' runs), to drop under LEAK rules; RS is close to linear in OBP and SLG (TP-4 to probe, the CC's "linear model best"); temporal split by Year. +- Priority: gap 0 (domain Other is never thin), use case 0 (explanatory framing), source 0 (a course dataset re-uploaded, no codebook beyond column names), shape + (1,232 team seasons over 50 years, a temporal structure), risk - (same-season leaks, a near-linear target). + The Moneyball MLB stats dataset (1962-2012, OpenML 41021), estimating runs scored from team-season statistics. The curator notes it is clearly an analytics task rather than a predictive-performance task, with a linear model best in TabRepo, and may require a temporal split. It is in TabArena Reject but the curator suggests it could be included to cover the linear-model bias. This is borderline: a real dataset with a clear regression target but weak as a predictive benchmark and likely small. TBD -> 2nd Tier; a human must confirm size, the temporal split, and whether the analytics framing is acceptable. --- diff --git a/curation/records/monthly_insurance_claim_prediction_challenge.md b/curation/records/monthly_insurance_claim_prediction_challenge.md index 7d6ac71d..613fa721 100644 --- a/curation/records/monthly_insurance_claim_prediction_challenge.md +++ b/curation/records/monthly_insurance_claim_prediction_challenge.md @@ -4,6 +4,7 @@ name: Monthly Insurance Claim Prediction Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD - Time-series (Forecasting) @@ -29,6 +30,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: Zimnat's claim values per client, "per month for a whole year", from a two-day UmojaHack; a real insurer's cost task in a thin domain, but what one row is (a claim, or a client-month) and whether the test months lie after the training months is not stated. +- Source: Zindi hackathon "UmojaHack Africa 2022: Monthly Insurance Claim Prediction Challenge (INTERMEDIATE)", 19-20 March 2022, data from Zimnat (Zimbabwe) (https://zindi.africa/competitions/umojahack-africa-2022-intermediate-challenge, read through Zindi's page API); as published: train.csv, test.csv, policies.csv ("extra information regarding the policies"), DataDictionary.csv, a starter notebook; "The target column is the claim column which indicates the value of a claim. Policy terms are annual and as a result a policy is renewed every 12 months from the date of inception"; submissions keyed by `claim_id`. Rows and columns not counted (login). +- Use case: "Zimnat Insurance wants to predict the value of future claims per client, in order to better forecast annual costs ... help Zimnat in being better prepared to address claims" (Description); the actor and the aim are named, the decision (reserving, budgeting) only implied; fit approximate until the row unit is known. Regime likely temporal (claims over months); grouped by client if a client has several rows. +- Siblings and duplicates: other Zimnat and insurance Zindi records (`autoinland_vehicle_insurance_claim_challenge`, `axa_vehicle_insurance_claim_challenge`; the 2020 "Zimnat Insurance Assurance Challenge" is a different task, policy lapse); the UmojaHack 2022 siblings `african_snake_antivenom_binding_challenge`, `faulty_air_quality_sensor_challenge` are other tasks. No duplicate found. External collections: none found. +- Papers and pages: the Zindi winners' post: the winners "concatenated the two different data sets" (train and policies), built "date-time features", log-transformed the labels, used LightGBM and CatBoost. +- Metric: MAE on the claim value ("The error metric for this competition is the Mean Absolute Error", Evaluation tab); MET-4; inputs none. +- Risks for /add-dataset: the row unit and the test design from the data dictionary (a panel of client-months is forecasting-like: SEL-10 against SEL-11 and TP-10); the policies join (one to many?); many zero claims (a hurdle shape); dates after the prediction point. +- Markers: `Time-series (Forecasting)` kept as a concern ("per month for a whole year"); not a fail from the page, which Zindi tags "Prediction". +- Priority: gap + (insurance has 4 accepted), use case 0 (actor named, fit unclear), source 0 (a hackathon page with a data dictionary, no paper), shape 0 (size not counted; mixed types and a policy table), risk - (unclear row unit and target timing, a join). + A Zindi UmojaHack 2022 challenge to predict how much will be claimed per month. Predicting monthly claim amounts could be a fixed regression task on policy/customer features, but 'per month' phrasing raises the possibility of a temporal forecasting framing, which would be out of scope. The record lacks data details. Because it is plausibly a valid insurance regression task but may be forecasting, this is TBD -> 2nd Tier. A human must open the competition to confirm whether it is a fixed predictive task vs horizon forecasting, the target, size, and split. --- diff --git a/curation/records/mtoto_news_childline_kenya_call_volume_prediction_challenge.md b/curation/records/mtoto_news_childline_kenya_call_volume_prediction_challenge.md index a4d4f567..4aa2779a 100644 --- a/curation/records/mtoto_news_childline_kenya_call_volume_prediction_challenge.md +++ b/curation/records/mtoto_news_childline_kenya_call_volume_prediction_challenge.md @@ -27,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): hourly call counts of Childline Kenya's 116 hotline, trained on 1 January to 12 July 2016 and predicted for 13 July to 6 September 2016: future values of one series over a horizon. +- Source: Zindi competition "Mtoto News Childline Kenya Call Volume Prediction Challenge", 11 Apr to 5 Aug 2019, sponsored by Mtoto News and Childline Kenya (https://zindi.africa/competitions/mtoto-news-childline-kenya-call-volume-prediction-challenge, read through Zindi's page API; Zindi's problem type "Forecast"); as published: train.csv of "all the calls (over 135,000) that were received from 1 January 2016 to 12 July 2016" (call date and time, case status, category, priority, caller and child age, gender and county, abuser relationships), plus Nairobi school dates, Nairobi weather and Kenyan public holidays 2016; the submission is `time_index` (an hour, e.g. 2016071300) and `calls`. +- Use case: "Your solution will help Childline manage its resources and staff their call center more efficiently" (Description): staffing from a forecast of hourly demand, which is the forecasting protocol SEL-10 excludes; the call records describe past calls, not the hours to predict. +- Siblings and duplicates: `childline_kenya_call_volume_prediction_challenge` (Zindi Mentorship 3 re-run of the same task, AI `No` with the same marker): a duplicate of this one by provenance (PROV-4) if the data is the same. +- Metric: RMSE on hourly call counts ("The evaluation metric for this challenge is the Root Mean Squared Error", Evaluation tab). +- Reject check (AI, 2026-10-22): stands. 'create a forecast model to predict the number of incoming calls ... per hour per day', trained on 1 Jan to 12 Jul 2016 and predicted for 13 Jul to 6 Sep 2016; type_of_problem ['Forecast']. It is one series (hotline-wide hourly counts), not a panel, so SEL-11/TP-10 cannot apply, and the per-call fields do not exist for the future hours. https://zindi.africa/competitions/mtoto-news-childline-kenya-call-volume-prediction-challenge + A Zindi competition whose name explicitly states predicting call volume for Childline Kenya. Call-volume prediction over time is a demand/forecasting task, which is explicitly excluded from the benchmark scope. The name strongly signals forecasting future call counts over a horizon rather than a fixed predictive task. A human could verify whether any concurrent-feature (non-horizon) framing exists, but on the name and typical setup this is a No. diff --git a/curation/records/national_longitudinal_survey_binary.md b/curation/records/national_longitudinal_survey_binary.md index 3f28f230..05146f48 100644 --- a/curation/records/national_longitudinal_survey_binary.md +++ b/curation/records/national_longitudinal_survey_binary.md @@ -31,6 +31,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** agree with `No` (the CC: "Survey data. Not a real predictive task."); recommend replacing `Duplicate` by nothing or by a note, since no other copy is among the records; the curator has the call. +- Source: U.S. Bureau of Labor Statistics, National Longitudinal Survey of Youth 1979 (https://www.nlsinfo.org/), as cut by the R package fairml (`national.longitudinal.survey`: 4,908 rows, 18 variables; "The data was preprocessed differently from Komiyama et al. (2018) ... In that paper, `income90` is the response, and `gender` and `age` are the sensitive attributes", rdrr.io fairml manual); OpenML 43892 (2022-05-30) re-uploads it binarized: 4,908 x 17, target `income96gt17`, description "Whether income90 is greater than 1.7" (10,000 USD units), with `income96` kept as a feature and `income90`, `income06` dropped. +- Original task (SEL-8): Komiyama et al. 2018 (ICML), fair regression with sensitive attributes: a benchmark for fairness-constrained methods on survey answers, not a decision anyone makes with the prediction (SEL-3, USE-1). +- Use case: none from the source. A steelman: the 1989-1990 answers (`afqt89`, `grade90`, `jobsnum90`, `typejob90`) precede 1996 income, so an income-in-six-years prediction exists in principle, but no actor acts on it, and `age`, `height`, `weight`, `illegalact` are survey answers of the same panel. +- Leak: if the target is income96 > 1.7, the feature `income96` decides it (fixable by dropping, LEAK rules); the OpenML text names income90, the column name income96: the OpenML upload contradicts itself. +- Markers: `No Good Target / Scientific Discovery` fits (SEL-3); `Duplicate` has no other record behind it (OpenML 43892 is the only copy we hold; it is a cut of fairml's cut of NLSY79, PROV-5 would apply only against another record). +- Metric: none for a classification (the paper's response is continuous income); the fallback. + Re-upload of the NLSY data via the R fairml package with a binarized target and the original continuous target left in the data, framed as an Adult-like survey task. The curator (CC) judged it 'not a real predictive task' and it is already in the TabArena Reject collection. It is essentially survey/scientific-discovery data with a constructed target plus a leakage concern from the retained original target. Suggest No. A human should confirm the leakage and that this duplicates the well-known fairml/NLSY upload. --- diff --git a/curation/records/new_zealand_datasets.md b/curation/records/new_zealand_datasets.md index bf8c7775..af7fd74c 100644 --- a/curation/records/new_zealand_datasets.md +++ b/curation/records/new_zealand_datasets.md @@ -4,6 +4,7 @@ name: new zealand datasets checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Missing source information - TBD @@ -24,4 +25,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a bare OpenReview link with the name "new zealand datasets"; the page could not be read, so what the data is, its target and its size are unknown. +- Source: https://openreview.net/forum?id=70XgWMiBeH7 only. OpenReview answered every request (the forum page, the PDF, both APIs) with a bot challenge; the Wayback Machine has no copy; web searches for the ID and for New Zealand tabular datasets on OpenReview found nothing. The record came with the workbook import (source row 1046, between `bemtpl97` and `sports_bet_datasets`), with no comment. +- Use case: unknown. Regime unknown. +- Siblings and duplicates: none found among the records (searched new zealand, the forum ID). External collections: unknown. +- Papers and pages: the OpenReview submission itself, unread. +- Metric: none found (the page could not be read), so the fallback until the paper is read. +- Risks for /add-dataset: everything: the curator should open the forum in a browser, name the paper and the datasets it uses, and split the record into one record per dataset. Not rejected: the origin is a paper on OpenReview, so this is too little information, not an untraceable source (PROV-2; `Missing source information` stays as a state of the record until the paper is read). +- Priority: gap 0, use case 0, source 0, shape 0 (all unknown), no known risk beyond that. + The record contains only an OpenReview forum link and a vague name ('new zealand datasets'), with no comments, target, size, or modality information. There is far too little here to judge scope, task, or representativeness. Per the heuristics, insufficient info alone is not grounds for rejection. Suggest TBD -> 2nd Tier; a human must open the OpenReview paper to identify the actual dataset(s), target(s), size, and whether the data is genuinely tabular. diff --git a/curation/records/nhanes_age_prediction.md b/curation/records/nhanes_age_prediction.md index 6b77da48..6e8d69a9 100644 --- a/curation/records/nhanes_age_prediction.md +++ b/curation/records/nhanes_age_prediction.md @@ -30,6 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** agree with the curator's `No` (CC: "No (after preprocessing)"); the AI-set `TBD -> 2nd Tier` above contradicts that CC line, so the curator may want to restore `No`; the curator has the call. +- Source: UCI 887, "National Health and Nutrition Health Survey 2013-2014 (NHANES) Age Prediction Subset" (https://doi.org/10.24432/C5BS66, read through the UCI API): 6,287 instances listed, 7 features (gender, weekly physical activity, BMI, fasting glucose, diabetes answer, oral glucose tolerance, insulin), target `age_group` ("respondents 65 years old and older were labeled as 'senior' and all individuals under 65 years old as 'non-senior'"), `RIDAGEYR` (exact age) shipped with role Other; data from CDC/NCHS NHANES 2013-2014; intro paper Dinh et al. 2019 (BMC Medical Informatics and Decision Making), which predicts diabetes and cardiovascular disease, not age. +- Use case: none: age is known at every point where these labs are taken, so nobody predicts "senior" from glucose and BMI (SEL-3, USE-1); `RIDAGEYR` would decide the label (fixable, but the task stays). `No Good Target / Scientific Discovery` would state that better than the current `No Good Target (yet)`. +- Siblings: `nhanes_undiagnosed_diabetes` (CC 2026-10-15) is the NHANES task with a decision (screening for undiagnosed diabetes, 1999-2023 cycles, temporal), and its notes already name this record as a 2013-2014 subset with an age-group target. The curator's alternative "predict the actual age" has no decision either. +- `Data Quality Issue` does not fit the vocabulary meaning (a leak that can be dropped is not irreparable damage). +- Metric: none (no source scores this target). + NHANES health/nutrition survey data; the OpenML-style version frames an age-group prediction target but contains an 'age_group' leakage feature, and the curator marked it 'No (after preprocessing)'. A fuller raw NHANES source exists from CDC and a 2019 paper used it for diabetes/cardiovascular prediction, so a meaningful tabular predictive task may be recoverable after careful preprocessing. The current preprocessed form has a leakage problem and an unclear real target. Suggest TBD -> 2nd Tier; a human must remove the leak, define the true target (e.g. diabetes/CVD vs age), and check post-cleaning size and feature set. --- diff --git a/curation/records/nhtsa_crss_crash_severity.md b/curation/records/nhtsa_crss_crash_severity.md index 6ecaceb4..6195cd3a 100644 --- a/curation/records/nhtsa_crss_crash_severity.md +++ b/curation/records/nhtsa_crss_crash_severity.md @@ -4,6 +4,7 @@ name: NHTSA Crash Report Sampling System (CRSS), crash injury severity checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: - Needs extensive data wrangling tags: @@ -31,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: NHTSA's documented national sample of police crash reports with the injury outcome on every crash, a thin domain; the use case is ours, and the release is three tables to join. +- Source: NHTSA National Center for Statistics and Analysis, Crash Report Sampling System, data years 2016 onward (replaced NASS-GES in 2016); about 50,000 police crash reports a year drawn by a three-stage probability sample from 6-7 million, about 120 coded elements per report, imputation for missing items, survey weights (DOT HS 812 795; DOT HS 812 688). As published: yearly ACCIDENT / VEHICLE / PERSON (and further) files, CSV and SAS; crash-level maximum injury severity (KABCO) and per-person injury severity. US government work. The NHTSA pages answer 403 to automated fetches, so the file layout and the MAX_SEV coding were not read. +- Use case: no actor named by the source (CRSS exists for national estimates, not prediction). Closest real decision: triage of a crash from what is known at notification (automatic crash notification, the EMS level to dispatch), or a road-safety agency scoring crash configurations; fit approximate, since police-coded fields are filled after the scene. Regime temporal: yearly releases, coding changes across years (SPLIT-6). +- Siblings and duplicates: `fars` (NHTSA fatal-crash census, Lennart `TBD -> 2nd Tier`, a different system and no severity spread), `nyc_motor_vehicle_collisions` and `chicago_traffic_crashes` (AI `TBD -> Yes`, city police crash tables, the same use-case family: the curator may want one or two of the three), `us_accidents` (Lennart `No`, its Severity is a delay code; this record was opened as its injury-severity counterpart), `road_safety` (`No`). External collections: none found. +- Papers and pages: DOT HS 812 688 (design overview, analytic guidance), DOT HS 812 795 (CRSS 2016 sample design), CRSS Analytical User's Manual 2016-2022 (DOT HS 813 557); injury-severity models on CRSS/GES are common in road-safety research. No competition. +- Metric: none found (NHTSA's manuals and the CRSS pages define estimation, not a prediction task; no competition), so the fallback: `log_loss` for the KABCO classes, or `roc_auc` if framed as injury vs none (MET-1, MET-8, MET-11). Inputs: none; the survey weight `WEIGHT` ships and must not be a feature (MET-2: decide whether an unweighted score is the task). +- Risks for /add-dataset: outcome fields recorded after the crash (harmful events, towing, EMS arrival and hospital times, person-level injury fields that define the target) must go (LEAK-10); the crash-level target needs a join from PERSON or the MAX_SEV field; imputed values are flagged by NHTSA (`*_IM` columns) and need a policy; coding changes across years (PROV-13). +- Priority: gap + (social science thin), use case 0 (the source names no actor), source + (NHTSA manuals), shape + (about 50k crashes a year, mixed types, temporal), risk - (three-table join, post-crash fields). + NHTSA's nationally representative probability sample of police crash reports, in place since data year 2016 when it replaced NASS-GES; target sample "about 50,000" police accident reports a year drawn from the ~7 million issued annually (NHTSA, "Crash Report Sampling System: Design Overview, Analytic Guidance", DOT HS 812 688; 2016 file: 47,515 eligible PCRs). Annual releases as ACCIDENT / VEHICLE / PERSON tables (SAS + CSV) with a crash-level maximum injury severity (KABCO) and per-person injury severity; US government work, public domain. Sibling `fars` (fatal-crash census, OpenML/PMLB copy of unknown vintage) is a different NHTSA system; CRSS covers all severities, which is what makes it a severity task. To decide: crash-level target (MAX_SEV, 5 imbalanced classes, or injury vs none) from crash / vehicle / driver descriptors known from the scene report; drop outcome fields (HARM_EV after the fact, tow, EMS). The file carries survey weights (WEIGHT) and a stratified design; an unweighted classification is a different task from NHTSA's estimates, same caveat as `higgsml_2014`. Yearly files -> temporal split, mind coding changes across years. NHTSA pages returned 403 here, so the download layout was not checked. Data not inspected. diff --git a/curation/records/nsl_kdd.md b/curation/records/nsl_kdd.md index 9219c22f..09c69ed1 100644 --- a/curation/records/nsl_kdd.md +++ b/curation/records/nsl_kdd.md @@ -32,6 +32,16 @@ type_adapter_id: curation-record-v1 ⚠️ **AI-FILLED — UNVERIFIED**: this record was triaged by an AI (2026-08-12) and no human has verified it. Check the verdict and the fields before relying on them. +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, `Duplicate`, `Outdated`; SEL-16, PROV-6): a deduplicated, difficulty-resampled cut of KDD Cup 99, whose rows are the synthesized DARPA'98 traffic; the parent `kddcup99` is a curator `No` (Andrej, Lennart) with `AHDS`. +- Source: Tavallaee, Bagheri, Lu, Ghorbani, "A Detailed Analysis of the KDD CUP 99 Data Set", IEEE CISDA 2009 (UNB). As published: KDDTrain+ 125,973 x 43 and KDDTest+ 22,544 x 43 (41 features, label, a difficulty count), plus KDDTrain+_20Percent and KDDTest-21. The UNB page now says the dataset "is no longer available"; mirrors on Kaggle and GitHub. +- Deciding words (paper, Sec. III): "KDD'99 is built based on the data captured in DARPA'98 which has been criticized by McHugh, mainly because of the characteristics of the synthetic data"; "For the sake of privacy, the experiments chose to synthesize both the background and the attack data"; and of NSL-KDD itself (Sec. VI): "the data set still suffers from some of the problems discussed by McHugh and may not be a perfect representative of existing real networks". The dedup and resampling change the row selection, not where the rows come from. +- Duplicate: same rows as `kddcup99` after removing duplicates and resampling by difficulty; no distinct task (PROV-6). +- Use case: intrusion detection is real, but this traffic is simulated (SEL-16), and the 1998 attack mix is outdated. +- Siblings: `kddcup99` (`No`, Andrej and Lennart, `AHDS`, `Outdated`, `Data Quality Issue`), `kdd` (`No`), `intrusion_detection` (Lennart `No`). +- Metric: not needed for a `No` (the paper reports accuracy of 21 learners). +- Leak note if ever revisited: the 43rd column counts how many of 21 classifiers got the row right (label-derived, LEAK-27). +- Reject check (AI, 2026-10-22): stands. Tavallaee et al. 2009 (https://www.ecb.torontomu.ca/~bagheri/papers/cisda.pdf), Sec. III: 'the experiments chose to synthesize both the background and the attack data'. NSL-KDD is a deduplicated and resampled cut of the same rows as `kddcup99`, a curator No (Andrej, Lennart) with AHDS and Outdated, and it adds no distinct task (PROV-6). + Deduplicated, difficulty-resampled cut of [[kddcup99]], published by Tavallaee et al. (CISDA 2009) to fix that dataset's two measured defects (78%/75% duplicate rows; saturation). Shipped as flat files: KDDTrain+ 125,973 x 43 and KDDTest+ 22,544 x 43, plus KDDTrain+_20Percent (25,192) and the harder diff --git a/curation/records/nyc_motor_vehicle_collisions.md b/curation/records/nyc_motor_vehicle_collisions.md index 3d0f9a16..6f9748d7 100644 --- a/curation/records/nyc_motor_vehicle_collisions.md +++ b/curation/records/nyc_motor_vehicle_collisions.md @@ -4,6 +4,7 @@ name: NYC Motor Vehicle Collisions - Crashes (NYPD MV-104AN) checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) - Non-IID (Temporal) @@ -29,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: NYPD's police crash table on NYC Open Data, 2.27M crashes 2012-2026 with injury counts, a thin domain; few features and post-hoc contributing factors keep it from the top. +- Source: NYPD via NYC Open Data, dataset h9gi-nx95, "Motor Vehicle Collisions - Crashes" (data from 2012-07-01; the page describes TAMS from 1999 and FORMS from March 2016). Page: "The police report (MV104-AN) is required to be filled out for collisions where someone is injured or killed, or where there is at least $1000 worth of damage"; the data "is preliminary and subject to change". As published: one table, one row per crash, 29 columns (date, time, borough, zip, lat/lon, street names, persons/pedestrians/cyclists/motorists injured and killed, contributing factor and vehicle type for up to 5 vehicles, COLLISION_ID); API count on 2026-10-22: 2,269,187 rows, 2012-07-01 to 2026-06-11. Companion tables (Vehicles, Person) exist on the portal and join by COLLISION_ID. +- Use case: no actor named by the source; the page points to Vision Zero safety analysis. Closest decision: dispatch or triage from the reported location, time and vehicles, or a city traffic-safety team prioritising locations; fit approximate. Regime temporal: 14 years, the reporting system changed in 2016 (SPLIT-6). +- Siblings and duplicates: `chicago_traffic_crashes` (AI `TBD -> Yes`, same family), `nhtsa_crss_crash_severity` (AI `TBD -> Yes`, national sample), `us_accidents` (Lennart `No`; this record was opened as its injury counterpart). External collections: none found. +- Papers and pages: the NYC Open Data page and the NYPD MV-104AN form; many Kaggle notebooks, no competition or paper that defines a task. +- Metric: none found (the Open Data page, no competition, no defining paper), so the fallback `roc_auc` for any-injury (MET-1, MET-8, MET-11). +- Risks for /add-dataset: every injured/killed sub-count is the target and must go (LEAK-10); contributing factors are the officer's judgement after the fact, check whether they encode the injury (for example how often "Unspecified" appears for injury crashes); the reporting threshold (injury or $1000 damage) makes the negatives property-damage crashes only; preliminary data that is amended; street names are free text. +- Priority: gap + (social science thin), use case 0 (no actor named by the source), source + (NYPD documents the table), shape + (2.27M rows, categorical mix, temporal), risk - (post-hoc factors, few real features). + NYC Open Data, NYPD: one row per police-reported collision; "The police report (MV104-AN) is required to be filled out for collisions where someone is injured or killed, or where there is at least $1000 worth of damage." Queried via the Socrata API on 2026-09-23: 2,269,187 crashes, 2012-07-01 to 2026-06-11; NUMBER OF PERSONS INJURED = 0 for 1,710,566 rows, >= 1 for ~558k (1: 433,878; 2: 81,463; 3: 26,827); NUMBER OF PERSONS KILLED > 0 in 3,469 crashes. Columns: date, time, borough, zip, lat/lon, on/cross/off street, contributing factor per vehicle (1-5), vehicle type per vehicle (1-5), injured/killed counts split by pedestrians, cyclists, motorists. Task: any-injury (binary, ~25% positive) or injured-count regression from time, place, vehicle types and contributing factors; the killed count is too rare for a target. Contributing factors are the officer's post-hoc judgement but are scene descriptors, not outcomes; the injured/killed sub-counts are the target and must all be dropped. Reporting practice changes over 14 years -> temporal split. No PII in the crash table. Data inspected only through API aggregates. diff --git a/curation/records/nz_springs.md b/curation/records/nz_springs.md index d92ffaf8..bacedc3d 100644 --- a/curation/records/nz_springs.md +++ b/curation/records/nz_springs.md @@ -3,10 +3,9 @@ unique_name: nz_springs name: nz_springs checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- TBD tags: - Many features - AI-Filled (Verify) @@ -28,6 +27,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): a microbial biogeography survey of 925 New Zealand geothermal springs, published to find what drives community composition; the candidate targets (temperature, pH) are measured on site with a probe, so no decision needs them predicted. +- Source: Power et al., "Microbial biogeography of 925 geothermal springs in New Zealand", Nature Communications 9:2876, 2018 (doi 10.1038/s41467-018-05020-y), the 1000 Springs Project. As published: 1,019 water samples from 974 features in 18 fields, 925 springs after QC; 46 physicochemical parameters (ICP-MS metals, nitrogen species, H2S, sulfate, conductivity, DO, ORP, pH, turbidity, temperature in situ); 16S rRNA reads in ENA (PRJEB24353); metadata on 1000springs.org.nz; the record's OTU table and metadata came "via email from Matthew Stott". +- Deciding words (paper, abstract and aim): "We measured bacterial and archaeal community composition, 46 physicochemical parameters"; "We determined that diversity is primarily influenced by pH at temperatures <70 °C"; the study's aim is "to determine factors that influence biogeographical patterns". The methods are correlational (Mantel, PERMANOVA, ANOSIM, constrained correspondence analysis, edgeR); no predictive model is trained or validated. The paper only remarks that geochemistry "could be predictive of community composition". +- Use case: none; temperature and pH are read with a thermocouple and a handheld meter at sampling, and predicting a community from chemistry is the discovery question itself (SEL-3, USE-2). +- Siblings: none in `curation/records/`. +- Metric: none (no prediction task in the paper). +- Doubt for the reject-checker: a framing "predict community diversity from the 46 parameters" could be argued; it still has no actor or decision. +- Reject check (AI, 2026-10-22): stands. Power et al. 2018 (https://pmc.ncbi.nlm.nih.gov/articles/PMC6056493/) set out 'to determine factors that influence biogeographical patterns' with correlational methods only. No predictive task is published. The candidate targets (temperature, pH) are measured on site, so no actor's decision needs them predicted (SEL-3, USE-2). I found no later predictive use of the 1000 Springs data. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + Sampling of New Zealand geothermal springs with metadata (temperature 13.9-100.6C, pH <1-9.7) plus a microbial OTU/16S count table (Nature Communications 2018, doi 10.1038/s41467-018-05020-y). A regression target such as temperature or pH from the OTU/metadata features is conceivable, but this is microbiome scientific-discovery data with very many (sparse OTU) features and an unclear, possibly small sample count. It leans toward scientific discovery rather than a representative tabular task. Suggest TBD -> 2nd Tier; a human must confirm the sample size after joining tables, the intended target, and whether OTU features make it representative. --- diff --git a/curation/records/okcupid_stem.md b/curation/records/okcupid_stem.md index dde04d0a..4e36fbd8 100644 --- a/curation/records/okcupid_stem.md +++ b/curation/records/okcupid_stem.md @@ -31,6 +31,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** agrees with the curator's `No` (the CC line: "not a meaningful task ... ethical concerns ... meme dataset"; also in `TabArena Reject`); no priority on a `No`. +- Source: Kim and Escobedo-Land, "OkCupid Profile Data for Introductory Statistics and Data Science Courses", Journal of Statistics Education 23(2), 2015 (revised 2021); github.com/rudeboybert/JSE_OkCupid: 59,946 San Francisco profiles; "Permission to use this data set was explicitly granted by OkCupid"; the 2021 revision "Removed the exact date the data was collected". Published for teaching statistics, not for a prediction task. The `okcupid-stem` task (job in STEM, open-text essays dropped) is a later re-framing on OpenML, carried by TabSTAR. +- Use case: none; nobody decides anything from whether a dating-profile user works in STEM (SEL-3, USE-1). This alone decides the `No`; the marker `No Good Target / Scientific Discovery` would name it better than `Not Representative`. +- Siblings: `okcupid` (Alex, Mustafa, Lennart: `No`, `No Good Target / Scientific Discovery`, `Ethical Issue`): the same data, decided the same way. +- Counter-point on the ethics marker only: the JSE data was released with OkCupid's explicit permission for teaching, so `Ethical Issue` rests on the use (inferring a person's occupation from a dating profile), not on objecting subjects (SEL-1); the verdict stands on SEL-3 either way. +- Metric: none defined by the source (JSE paper, repo); not needed for a `No`. + OkCupid dating-profile data where the task is predicting whether a user works in STEM, with open-text features already stripped out and the data otherwise preprocessed. The curator flags it as a meaningless/'meme' task, notes ethical concerns about this use of human data, and it already sits in the TabArena Reject collection. The lack of a meaningful target plus the ethical and representativeness issues argue against inclusion. Suggest No. --- diff --git a/curation/records/one_purchase_prediction_challenge_by_industrial_engineers_club.md b/curation/records/one_purchase_prediction_challenge_by_industrial_engineers_club.md index 97385c43..5a72b28b 100644 --- a/curation/records/one_purchase_prediction_challenge_by_industrial_engineers_club.md +++ b/curation/records/one_purchase_prediction_challenge_by_industrial_engineers_club.md @@ -4,6 +4,7 @@ name: One Purchase Prediction Challenge by Industrial Engineers Club checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 decision_markers: - TBD tags: @@ -11,9 +12,10 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: business & marketing required_split: -- Temporal (NON-IID) +- Random (IID) problem_type: Binary Classification source_links: - https://zindi.africa/competitions/iec-algeria-data-cup-by-temtem-purchase-prediction-challenge @@ -27,6 +29,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: a real repeat-purchase task from an Algerian shopping app with a stated business use and AUC; small, one snapshot, and the order items must be aggregated per customer. +- Source: Zindi hackathon "IEC ALGERIA Data Cup by Temtem One Purchase Prediction Challenge", Industrial Engineers Club with Temtem One, 17-21 December 2021, private to Algerian Data Cup participants (93 participants, 49 on the leaderboard). As published: Train.csv (2.3 MB) and Test.csv (1.6 MB), one row per order item; "The data contains 29774 order" from 1 January to 15 February 2021; columns Order ID, Order Status, Category Name, Product Name (French), Quantity, Unit Price, Total Price, Subtotal, Payment Method, Customer ID, Merchant, Order Time, Customer Country, and the target "Purchase Again": "1 corresponds to that the customer will make a new purchase after february 15th, 0 else". Customers, not orders, are the unit (the submission is per customer). +- Use case: Temtem, on 15 February, decides which customers to target for retention ("help Temtem improve it's UX, it's marketing strategy"); fit direct. Regime IID across customers: "Both train and test set contains order during the same period", one prediction point for every customer, so a temporal split has nothing to order; the old `Temporal (NON-IID)` value was AI-filled and is changed to `Random (IID)`. +- Siblings and duplicates: none in `curation/records/` (grep on temtem, purchase). External collections: none found. +- Papers and pages: the Zindi page (description, data, evaluation); discussions are private. No paper. +- Metric: "The evaluation metric for this challenge is Area Under the Curve (AUC)" (Zindi Evaluation page; API `error_metric: area_under_curve`), a competition's metric (MET-4): `roc_auc` per customer. Inputs: none. +- Risks for /add-dataset: build one row per customer from the order items (the aggregation is ours; keep Order Status, since rejected orders are not purchases); the number of customers is not stated (a 9.9 KB submission file suggests about a thousand test customers), so check `Too Small` (TP-5); Zindi data needs a login and the hackathon was private, so access is a question for the curator; the target window (15 February to 20 March) lies after the prediction point, which is fine. +- Priority: gap 0 (business), use case + (the host names its decision), source + (the competition page documents the columns and the target), shape 0 (customer count unknown, likely a few thousand), risk - (aggregation, size). + A Zindi competition whose stated goal is to predict which customers will purchase again, i.e. a repeat-purchase / churn-style binary classification over customer records. This is a genuine real-world predictive task rather than CTR/ranking, and such e-commerce transaction data is typically tabular and adequately sized. It may need a temporal split and some transaction aggregation. Suggest TBD -> Yes; a human must verify the data layout (single table vs transaction log to aggregate), the exact target definition, and post-processing size. --- diff --git a/curation/records/open_food_facts.md b/curation/records/open_food_facts.md index e717e901..1d91d7f5 100644 --- a/curation/records/open_food_facts.md +++ b/curation/records/open_food_facts.md @@ -4,14 +4,16 @@ name: Open Food Facts checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- No Good Target / Scientific Discovery +- No Good Target (yet) - Needs extensive data wrangling tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) collections: - New (BeyondArena) +original_source: Website domain: Other required_split: - Random (IID) @@ -19,6 +21,7 @@ problem_type: TBD original_data_state: Database (or multiple to-be-joined tables) source_links: - https://www.kaggle.com/datasets/openfoodfacts/world-food-facts +- https://world.openfoodfacts.org/data source_row: 836 needs_review: - ai_unverified @@ -29,4 +32,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a crowdsourced product database with no predictive target of its own; the obvious target (Nutri-Score) is computed by a published rule from the nutrient columns. +- Source: Open Food Facts, a volunteer non-profit (world.openfoodfacts.org), crowdsourced from barcode scans; the Kaggle copy `openfoodfacts/world-food-facts` is uploaded by the organisation itself (file `en.openfoodfacts.org.products.tsv`, 1.0 GB, 2019-09-19 snapshot; the Kaggle description now says "over 4 million products from 150 countries" and "over 150 columns"). As published: one wide table of products (ingredients, allergens, categories, countries, nutrients per 100 g, Nutri-Score and NOVA where computed), heavy missingness. +- Use case: none from the source as a prediction; the organisation's list of AI tasks (github.com/openfoodfacts/openfoodfacts-ai) is mostly extraction from product photos and text. A tabular framing (the NOVA group or the category from nutrients) would be ours. Fit: none yet. Regime IID by product, but near-duplicate products across countries would need grouping. +- Siblings and duplicates: none in `curation/records/`. External collections: none found. +- Papers and pages: the Kaggle page and the Open Food Facts data page; no paper defines a task. +- Metric: none found (Kaggle dataset page, Open Food Facts data page), so the fallback for whichever target is chosen (MET-1, MET-11). +- Risks for /add-dataset: Nutri-Score grade and points are a deterministic function of the nutrient columns (SEL-5, `Trivial`); NOVA is assigned by a rule on the ingredient list (text); the table is a moving snapshot of a living database; text-heavy columns. +- Priority: gap 0, use case 0, source + (the organisation documents the data), shape + (millions of rows, mixed types), risk - (no target; the natural one is rule-computed). Markers: `No Good Target (yet)` replaces `No Good Target / Scientific Discovery` (a database built for another purpose, waiting for a target, not a discovery table); `Needs extensive data wrangling` kept. + Open Food Facts is a large crowdsourced product/nutrition database (Kaggle world-food-facts) rather than a dataset with a defined predictive target. It is multi-table/database-shaped with heavy missingness and would need extensive wrangling and a chosen target (e.g. nutri-score or food category). As-is it reads as a reference/scientific-discovery table. Suggest TBD -> 2nd Tier; a human must define a meaningful, non-leaking target (e.g. Nutri-Score prediction) and assess usable row count after cleaning. diff --git a/curation/records/open_soil_spectral_library.md b/curation/records/open_soil_spectral_library.md index ca265fa3..82744dbe 100644 --- a/curation/records/open_soil_spectral_library.md +++ b/curation/records/open_soil_spectral_library.md @@ -3,7 +3,8 @@ unique_name: open_soil_spectral_library name: Open Soil Spectral Library (OSSL) / KSSL MIR soil spectra checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: TBD -> Yes +priority: 4 decision_markers: - Wrong Domain / Source Modality tags: @@ -34,6 +35,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: a well documented open library of lab spectra with wet-chemistry reference values, built to calibrate the models that predict soil properties from a scan, in a spatially grouped regime; the open point is whether spectra are in scope (SEL-15). +- Source: Soil Spectroscopy for Global Good, Open Soil Spectral Library; data on Zenodo (doi 10.5281/zenodo.7599269, v1.2), paper Safanelli et al., PLOS ONE 20(1): e0296545, 2025. As published: 135,651 entries harmonised from KSSL, LUCAS, AfSIS1/2, ICRAF-ISRIC and smaller libraries; MIR and VisNIR scans plus about 40 soil properties (SOC, clay/silt/sand, pH, CEC, extractable cations, ...); 87,707 entries with coordinates; separate tables for site, lab values and each spectral range, joined by sample id. Licences differ per library (LUCAS under a JRC agreement; noted only, EV-4). +- Use case: the paper: users "need to predict soil properties from spectra collected on any spectrometer", and the OSSL Engine returns predictions for uploaded spectra; the actor is a soil lab or survey that scans a sample instead of running wet chemistry; fit direct. Regime grouped by location or profile (several layers per site); the paper's own 10-fold CV does not say it was grouped (SPLIT-3). +- Siblings and duplicates: `open_soil_data` (Lennart `No`: 96% of its values are spectroscopy-model predictions; this is a measured parent), `lucas_topsoil` (AI `TBD -> Yes`; LUCAS VisNIR spectra are inside OSSL), `africa_soil_property_prediction_challenge` (AfSIS MIR is inside OSSL), `bloodsai_blood_spectroscopy_classification_challenge` (the same spectroscopy question). Precedent for spectra: `micro_mass` (MALDI-TOF spectra, `Yes`, Lennart and Andrej). External collections: none found. +- Papers and pages: the PLOS ONE paper (Cubist ranked best; SNV and the first 120 principal components), docs.soilspectroscopy.org, github.com/soilspectroscopy/ossl-models. +- Metric: the paper reports RMSE, bias, R2, Lin's CCC and RPIQ and ranks models by CCC together with RPIQ, several side by side, so the fallback `rmse` on one property (MET-5, MET-11). Inputs: none. +- Risks for /add-dataset: pick one spectral range and one library cut (KSSL MIR is the largest single-instrument set) to avoid overlap with `lucas_topsoil` and the AfSIS records (PROV-6); one target, or a declared multi-target (CLEAN-18); some KSSL coordinates are county centroids, so group by profile or site, not by coordinate; about 1,700 MIR bins (raw spectral bins weigh -, SEL-15). +- Priority: gap + (Grouped thin), use case + (the paper names the users and the decision), source + (paper, Zenodo, docs), shape + (tens of thousands of samples, grouped), risk - (joins across tables, overlap with sibling libraries, the spectra question). + Safanelli et al., PLOS One 20(1):e0296545, 2025. Compilation of soil spectra with laboratory reference values, v1.2 on Zenodo (CC BY 4.0, 2021-12-26): "MIR (91,631) and VisNIR (65,063) spectral scans + soil calibration data (>60,000 unique locations)"; "from more than 135,000 entries in the OSSL, 87,707 samples have spatial coordinates". Table 1: KSSL (USA) 21,857 MIR + 10,521 VisNIR, CC; LUCAS (EU) 21,711 VisNIR under a "JRC license agreement" with geographic restrictions; AFSIS1 1,457 MIR (ODbL); AFSIS2 3,573 MIR (CC); ICRAF-ISRIC 1,841 MIR; CAF 715; Schiedung 422; Garrett 263; Serbia 410 (shared internally, not redistributable). Reference properties: >40 harmonised (SOC, clay/silt/sand, pH, CEC, extractable K/Ca/Mg/Na/P/B/Zn/Mn, EC, bulk density, ...). Own models: "Cubist was adopted as many studies have found it to be one of the top performing models", SNV preprocessing, "first 120 principal components" of each range; 10-fold CV Lin's CCC 0.95 for SOC (MIR), 0.84 pH, 0.74 clay. Why it is the dataset to settle the spectroscopy question on: the incumbent models in this field are Cubist, PLSR and memory-based learning, i.e. tabular-style learners on ~1,700 wavenumber bins, so "modality-specific models clearly superior" (criterion 4A) is not obvious here; the `Wrong Domain / Source Modality` marker is the provisional flag, not a verdict. Same call would settle `bloodsai_blood_spectroscopy_classification_challenge`. Licence mix (LUCAS under a JRC agreement, Serbia internal) is noted, not a factor. Points to decide: one spectral range (MIR), spatially grouped split by site (many layers per location), and the library "spatially over-represents USA and European Union". Data not inspected. diff --git a/curation/records/opengeoai_2023_hack.md b/curation/records/opengeoai_2023_hack.md index 8189684f..714b7603 100644 --- a/curation/records/opengeoai_2023_hack.md +++ b/curation/records/opengeoai_2023_hack.md @@ -4,9 +4,9 @@ name: opengeoai-2023-hack checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Missing source information -- TBD +- Image tags: - AI-Filled (Verify) collections: @@ -18,6 +18,7 @@ required_split: problem_type: TBD source_links: - https://zindi.africa/competitions/opengeoai-2023-hack/data +- https://zindi.africa/competitions/opengeoai-2023-hack source_row: 954 needs_review: - ai_unverified @@ -28,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: olive-parcel classification from Sentinel-2 image time series in a 24-hour student hackathon; the inputs are satellite imagery, which SEL-13 counts as image-derived, but a curator kept a sibling Sentinel-2 cropland task at `TBD -> Yes`. +- Source: Zindi hackathon "OpenGeoAI 2023 Hack", 11-12 November 2023, open to Tunisian participants, by AGEOS (a Tunisian geospatial association) and the IEEE Sup'Com GRSS chapter, within the EU-funded Innovi project. Subtitle: "Using Sentinel-2 image time series, can you map Olive crops at parcel level in Central Tunisia?". Data description: "classify and map olive parcels using open Sentinel-2 multispectral time series data. Please find the images for the challenge here." Files: train.rar (143 KB), test.rar (74 KB), a starter notebook (1.7 MB); the imagery is linked elsewhere. The old `Missing source information` marker is refuted: the host and the page are known. +- Use case: Tunisian agricultural planners map olive parcels; fit approximate (a 24-hour hackathon task). Regime: spatially grouped by parcel or area if the rows are pixels; not settled. +- Siblings and duplicates: `geoai_challenge_for_cropland_mapping_in_dry_environments` (Andrej `TBD -> Yes`, Sentinel time series), `crop_type_classification_challenge` (AI `TBD -> 2nd Tier`, `Wrong Domain / Source Modality`). External collections: none. +- Papers and pages: the Zindi page and the starter notebook only; discussions private. +- Metric: "The error metric for this competition is Accuracy" (Zindi Evaluation page; API `straight_accuracy`), a competition's metric (MET-4). +- Risks for /add-dataset: the features are remote-sensing spectra of an image (SEL-13, `Image`) unless the starter notebook ships per-parcel band statistics that tabular models are used on; the label files are tiny (likely a few hundred parcels, a `Too Small` risk, TP-5); access was restricted to Tunisian participants (a question for the curator). +- Priority: gap 0, use case 0, source 0 (a hackathon page), shape 0 (tiny label files, size unknown), risk - (image modality). Markers: `Image` added as the provisional flag; `Missing source information` and `TBD` dropped. Consistency question for the curator: if `Image` holds here (SEL-13), does it hold for the Andrej-kept cropland sibling too? + A 2023 Zindi 'OpenGeoAI' hackathon with no description in the record. The 'GeoAI' framing strongly suggests geospatial and likely satellite-imagery or raster inputs, which would put it outside representative tabular ML, but this is not certain from the record. Too little info to reject outright. Suggest TBD -> 2nd Tier; a human must open the competition data page to confirm whether inputs are tabular features or raw imagery and identify the target. diff --git a/curation/records/operating_system_and_browser_identification_in_tls_traffic.md b/curation/records/operating_system_and_browser_identification_in_tls_traffic.md index 7b40d275..de72f02f 100644 --- a/curation/records/operating_system_and_browser_identification_in_tls_traffic.md +++ b/curation/records/operating_system_and_browser_identification_in_tls_traffic.md @@ -3,20 +3,24 @@ unique_name: operating_system_and_browser_identification_in_tls_traffic name: operating-system-and-browser-identification-in-tls-traffic checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- TBD +- Not Representative +- Trivial tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: technology & internet required_split: - Random (IID) problem_type: Multiclass Classification source_links: - https://zindi.africa/competitions/operating-system-and-browser-identification-in-tls-traffic/data +- https://zindi.africa/competitions/operating-system-and-browser-identification-in-tls-traffic source_row: 970 needs_review: - ai_unverified @@ -27,4 +31,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: TLS sessions captured in a vendor's lab while scripted browsers visit a fixed website list, two labels (OS, browser); real protocol traffic but a generated setting, and the handshake fields likely fingerprint the label almost by rule. +- Source: Zindi hackathon "Operating system and browser identification in TLS traffic", 18-19 February 2023, private to the IEEE ENSI Student Branch (254 participants), data from Venari Security. Data description: "The dataset was generated in Venari virtual environment and was obtained by capturing the TLS traffic generated by visiting a predefined list of websites using different web browsers (Chrome, Firefox, Opera, Edge) installed on machines with different OS families"; "Each row in the data represents a session"; features from the Client Hello / Server Hello and concatenated per-packet fields (ip.ttl, tcp.window_size_value, tls.handshake.type). Files: Train_one_hot.zip (219 MB), Test.zip (107 MB), VariableDefinitions.csv. Targets: OS name (11 one-hot columns) and browser (4 one-hot columns). +- Use case: the page: "traffic engineering or detection of the organization's security policy violation"; actor a network security team flagging unmanaged devices; fit approximate (lab traffic, not an enterprise's). Regime: IID by session is the page's setup, but sessions of one capture (machine, website) are near copies, so grouped by capture may be the honest split (SPLIT-3). +- Siblings and duplicates: none in `curation/records/` for TLS fingerprinting; the network-traffic siblings `cic_ids2017`, `unsw_nb15`, `kyoto_2006_plus`, `ctu_13` are intrusion tasks, a different target. External collections: none found. +- Papers and pages: the Zindi page; discussions private; no paper. +- Metric: "The error metric for this competition is the Hamming Loss" over the one-hot label columns of both targets (Zindi Evaluation page; API `hamming_loss`), a competition's hard-label metric (MET-4). Inputs: none. It fixes a two-target task; shipping one target would need a stand-in (MET-11). +- Risks for /add-dataset: generated in a virtual environment with scripted visits (SEL-16, PROV-3: real software stacks, but not a real random draw of users); TLS Client Hello fields (cipher list, extensions) and ip.ttl identify browser and OS family nearly by rule (`Trivial`, TP-4); two targets (multi-target, CLEAN-18); access limited to a private hackathon. +- Priority: gap 0 (technology), use case 0 (approximate), source 0 (vendor data on a hackathon page, no documentation beyond it), shape + (hundreds of MB of sessions), risk - (generated setting, likely trivial). Markers: `Not Representative` (lab-generated traffic) and `Trivial` (handshake fingerprint) as provisional flags to test; `TBD` dropped. + A Zindi competition to identify the operating system and browser from TLS network-traffic features. Network-traffic fingerprinting from extracted handshake/flow features is a standard tabular multiclass classification task with real-world relevance. The split is likely random (IID) unless connections are grouped by host/session. Suggest TBD -> Yes; a human must confirm the features are pre-extracted tabular fields (not raw packet captures), the target cardinality, dataset size, and whether grouping is needed. diff --git a/curation/records/outsmarting_outbreaks_challenge.md b/curation/records/outsmarting_outbreaks_challenge.md index 26cd484c..5aef0a71 100644 --- a/curation/records/outsmarting_outbreaks_challenge.md +++ b/curation/records/outsmarting_outbreaks_challenge.md @@ -4,6 +4,7 @@ name: outsmarting-outbreaks-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Needs extensive data wrangling - TBD @@ -13,13 +14,15 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2024' domain: medical & healthcare required_split: - Temporal (NON-IID) -problem_type: TBD +problem_type: Regression original_data_state: Database (or multiple to-be-joined tables) source_links: - https://zindi.africa/competitions/outsmarting-outbreaks-challenge/data +- https://zindi.africa/competitions/outsmarting-outbreaks-challenge source_row: 870 needs_review: - ai_unverified @@ -30,4 +33,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: monthly counts of waterborne-disease cases per hospital in a Tanzanian region, a panel that may be forecasting (SEL-10 / SEL-11, TP-10 decides). +- Source: Zindi "SUA Outsmarting Outbreaks Challenge" (Sokoine University of Agriculture with IRCAI and AWS, 2024-12-06 to 2025-01-31, 816 participants, CC-BY-SA 4.0); as published: Train.csv (2019-2022) and Test.csv (2023) with ID per hospital, month, year and disease (typhoid, amoebiasis, diarrhoea, schistosomiasis, intestinal worms), target `Target`, plus waste_management.csv, toilets.csv, water_sources.csv (locations, masked) and a climate table 2019-2024 (ASDI); rows not checked (Train.csv 4.1 MB). +- Use case: the source says the model "will enable governments and health organisations to implement timely, targeted interventions and optimize resource allocation"; actor a district health office, moment before a month, decision where to send supplies; fit approximate (the climate table covers the test months, which a planner would not have ahead of time); regime temporal: "The train set contains information from 2019 to 2022, the test set is 2023". +- Siblings and duplicates: none found in `curation/records/` (grep outbreak, waterborne); external collections: none known. +- Papers and pages: the competition pages (Description, Data, Evaluation) read through the Zindi API; no paper; winners @Yisakberhanu, @AhmedElFazouani, @JuliusFx (solutions not read). +- Metric: Mean Absolute Error, Zindi Evaluation page ("The evaluation metric for this competition is Mean Absolute Error"); deliberate (a leaderboard metric, MET-4); inputs none beyond the target. +- Risks for /add-dataset: the task is "to predict the total number of disease outbreaks per hospital for each month of 2023" from 2019-2022, the future values of a count series per hospital: run TP-10 (a model on the features against each hospital's naive profile) before anything else, else `Time-series (Forecasting)` (SEL-10); top solutions may use lagged targets (not checked); the location tables join by masked coordinates (heavy joins); climate for the target month is a future value at a monthly prediction point (LEAK); the counts are skewed with many zeros. +- Priority: gap 0 (medical, temporal not thin), use case + (the source names the actors and the decision), source 0 (a competition page, no codebook beyond VariableDefinitions.txt), shape + (a hospital-month panel with side tables), risk - (forecasting-like panel, joins). + A Zindi disease-outbreak challenge provided as multiple to-be-joined tables with temporal structure. Depending on framing it could be a legitimate temporal-split predictive task (e.g. predicting case counts/outbreak risk for a location-period) or a forecasting task, which would be out of scope. It needs substantial wrangling to join tables and define a fixed predictive target. Suggest TBD -> 2nd Tier; a human must verify whether the task is a fixed predictive classification/regression with a temporal split versus a forecasting-over-horizon problem, and assess size after joins. diff --git a/curation/records/phoenix_ke_analytics_hackathon.md b/curation/records/phoenix_ke_analytics_hackathon.md index 7ada77f4..e5b577cf 100644 --- a/curation/records/phoenix_ke_analytics_hackathon.md +++ b/curation/records/phoenix_ke_analytics_hackathon.md @@ -3,10 +3,9 @@ unique_name: phoenix_ke_analytics_hackathon name: phoenix-ke-analytics-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +26,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a 2024 learning hackathon by the Phoenix KE Analytics community that re-uses the data of the Zindi Sendy Logistics Challenge (2019), recorded as `sendy_logistics_challenge`. +- Source: Zindi "Phoenix Ke Analytics Data Science Project" (2024-04-26 to 2024-05-19, 41 participants, no prizes); its Description and Data pages repeat the Sendy challenge word for word ("Sendy, in partnership with insight2impact facility, is hosting a Zindi challenge to predict the estimated time of delivery of orders"; "The dataset provided by Sendy includes order details and rider metrics"); its files have the same sizes as the 2019 Sendy Logistics Challenge's: Train.csv 4,094,263 bytes, Test.csv 1,209,061, Riders.csv 29,751, VariableDefinitions.csv 2,661, SampleSubmission.csv 117,450 (Zindi API, both competitions). +- Use case: Sendy's delivery ETA from pickup to arrival (the canonical record's task); no distinct task. +- Siblings and duplicates: `sendy_logistics_challenge` (TBD -> Yes, the first publication, 2019-08-23 to 2019-11-26, 2,201 participants), `sendy_logistics_challenge_2` (marked `Duplicate`, the mentorship and EDSA re-runs). +- Metric: Root Mean Squared Error (the Evaluation page, identical to Sendy's). +- Reject check (AI, 2026-10-22): stands. I checked the Zindi API myself. phoenix-ke-analytics-hackathon and sendy-logistics-challenge (https://zindi.africa/competitions/sendy-logistics-challenge) have byte-identical file sizes: Train.csv 4094263, Test.csv 1209061, Riders.csv 29751, VariableDefinitions.csv 2661, SampleSubmission.csv 117450. The Description is the same Sendy text, so this is the same source and task (PROV-4), and the canonical sendy_logistics_challenge is TBD -> Yes. + A Zindi 'Phoenix KE analytics hackathon' with no description, target, size, or modality given in the record. There is not enough information to determine the task or whether the data is tabular. Insufficient info is not by itself grounds for rejection. Suggest TBD -> 2nd Tier; a human must open the competition data page to determine the predictive target, data layout, and size. diff --git a/curation/records/pnwflights14.md b/curation/records/pnwflights14.md index 1b3c60af..1a472584 100644 --- a/curation/records/pnwflights14.md +++ b/curation/records/pnwflights14.md @@ -3,7 +3,8 @@ unique_name: pnwflights14 name: pnwflights14 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD tags: @@ -26,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a 2014 teaching cut of the BTS on-time table for SEA and PDX, likely a duplicate of the flight-delay task aeolus already covers (PROV-6). +- Source: the R data package pnwflights14 by Chester Ismay and Hadley Wickham (GitHub, created 2015-08-13, CC0), "a modified version of Hadley Wickham's `nycflights13` package"; upstream the US DOT BTS on-time data and hourly airport weather; as published: `flights` (162,049 flights departing SEA and PDX in 2014) plus `weather`, `planes`, `airports`, `airlines` to join; no target named (the README: "To help understand what causes delays"). +- Use case: an airline or passenger before departure predicts the arrival delay (the use case of aeolus); fit approximate (two airports, one year, a teaching package); regime temporal (one year of dated flights). +- Siblings and duplicates: `aeolus` (`Yes`, shipped as `aeolus_1m`, BTS 2016-2024 with weather), `airlines` (TBD -> Yes, CC asks whether it duplicates aeolus), `airlines_depdelay_10m` (assessment of 2026-10-22: a cut of the same BTS source); all end at the BTS on-time table; external collections: none known. +- Papers and pages: the package README and DESCRIPTION; used in ModernDive teaching material (not read); no paper. +- Metric: none found (README, DESCRIPTION; BTS publishes no prediction task), so the fallback (`rmse` for an arrival-delay regression or `roc_auc` for a 15-minute delay flag). +- Risks for /add-dataset: PROV-6 against aeolus first (a different year and region is not a distinct task by itself); `dep_delay`, `dep_time`, `air_time` and `arr_time` are known only after departure (LEAK); weather joined by the scheduled hour; the aeolus notes found delay minutes a weak regression target. +- Priority: gap 0, use case 0 (no actor named by the source), source 0 (a teaching package over BTS), shape + (162k flights, joins and a temporal structure), risk - (likely duplicate of aeolus). + Pacific Northwest 2014 flights data (analogous to the well-known nycflights13 teaching dataset), distributed as flights plus auxiliary tables (airlines/airports/weather). It supports standard tabular tasks such as flight delay classification or arrival-delay regression with a natural temporal split, and is large and real-world. Some joining and target definition is needed, and the standard departure-time leakage must be avoided. Suggest TBD -> Yes; a human must pick the non-leaking target (e.g. arrival delay) and confirm usable size after cleaning. **Assessment (AI, 2026-10-21):** aeolus (BTS on-time flights 2016-2024 with hourly weather at both airports, a temporal task in the TabArena v0.2 working copy as `aeolus_1m`) covers US flight delays, and pnwflights14 ends at the same BTS on-time table (aeolus review panel, precedent check, 2026-10-20). When this record is processed, judge it against aeolus under PROV-6: a second flight-delay task, or a duplicate of aeolus (a cut of one source without a distinct task), as for airlines and airlines_depdelay_10m. diff --git a/curation/records/pollution.md b/curation/records/pollution.md index 9cfe6561..fc8f850e 100644 --- a/curation/records/pollution.md +++ b/curation/records/pollution.md @@ -4,11 +4,12 @@ name: pollution checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small tags: - AI-Filled (Verify) -year: '?' +year: '1973' domain: environmental science & climate required_split: - '?' @@ -25,6 +26,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: 60 US metropolitan areas from a 1973 study of regression instability, an explanatory table with the workbook's "Tiny data" by Lennart and Andrej; a curator may well reject it (`Too Small`, TP-5, or `No Good Target / Scientific Discovery`, SEL-3). +- Source: McDonald, G.C. and Schwing, R.C. (1973), "Instabilities of regression estimates relating air pollution to mortality", Technometrics 15, 463-; served by StatLib and OpenML 542 ("the pollution data so loved by writers of papers on ridge regression"); as published: 60 rows (SMSAs, 1959-1961) x 15 numeric predictors (climate, demography, HC, NOX, SO2 potentials), target `MORT` (age-adjusted mortality per 100,000). +- Use case: none the source states: the paper estimates the effect of pollution on mortality across cities, a look at the coefficients, not a decision on a new city; fit none or approximate at best; regime IID (one row per metropolitan area, no time). +- Siblings and duplicates: none found (grep McDonald, Schwing, 542); other pollution records are different data; external collections: OpenML, StatLib. +- Papers and pages: the Technometrics paper (not read); many ridge-regression papers used it. +- Metric: none found as a prediction metric (the OpenML page states none; the paper studies coefficient stability), so the fallback `rmse`. +- Risks for /add-dataset: 60 rows give about 20 test rows per fold (TP-5); the target is an observed rate, but the task is explanatory (SEL-3, USE-2). +- Priority: gap 0, use case 0, source + (a peer-reviewed paper), shape 0 (60 rows), risk - (too small, unclear decision). + OpenML dataset id 542, a classic pollution/mortality regression table with only 60 samples. Prior TabArena verdict and both Lennart and Andrej flagged it as too small for the 3-fold CV protocol. With 60 rows it cannot support reliable benchmarking. Suggest No on size grounds. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/ports.md b/curation/records/ports.md index ad98d71c..95243683 100644 --- a/curation/records/ports.md +++ b/curation/records/ports.md @@ -3,10 +3,9 @@ unique_name: ports name: ports checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- TBD tags: - Many features - AI-Filled (Verify) @@ -27,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3, USE-2): microbial communities sampled in shipping ports, published for a biogeography study; the classifiers of port or region serve to find which taxa differ between places, not a decision. +- Source: Ghannam, Schaerer, Butler, Techtmann (2020), "Biogeographic Patterns in Members of Globally Distributed and Dominant Taxa Found in Port Microbial Communities", mSphere (10.1128/mSphere.00481-19), with the companion Schaerer et al. (2019), "Global Comparison of the Bacterial Communities of Bilge Water, Boat Surfaces, and External Port Water", AEM (10.1128/AEM.01804-19); data on GitHub rghannam/portmicrobes (GPL-3.0): ASV count and taxonomy tables as R .rds files and pm_metadata.csv (port, region, season, water chemistry); a few hundred samples from 20 ports in five regions (not counted). +- Use case: none the source states: "Here, we display how supervised machine learning approaches can be useful to understand microbial biodiversity and biogeography using microbes from globally distributed shipping ports. Our findings indicate that the members of globally dominant phyla are important for differentiating locations". The repository's random-forest port classifiers reach 0.98 to 0.995 accuracy from genus or ASV counts (data/machine_learning/model_stats/localstats.csv), near `Trivial` as well. +- Siblings and duplicates: none in `curation/records/`; external collections: none known. +- Metric: the repository reports log loss, accuracy, mean precision and recall side by side (MET-5: no unique one). +- Reject check (AI, 2026-10-22): stands. mSphere abstract (https://doi.org/10.1128/mSphere.00481-19): ML is used 'to understand microbial biodiversity and biogeography'. The full text (PMC6992368) only speculates that this 'may allow for the future prediction or classification of a geospatial location', and no actor or decision is named. The companion AEM paper (invasive bacteria on boats) concerns bilge and hull samples, not a prediction task. The port classifier reaches accuracy 0.994 on 1,218 samples x 3,214 ASVs, close to trivial. SEL-3 / USE-2. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + 16S rRNA microbiome data from ports with separate OTU count and metadata tables (refs mSphere/AEM 2019). A predictive target (e.g. port/region or an environmental variable from OTU features) might be constructed, but this is microbiome scientific-discovery data with very many sparse OTU features and likely a modest sample count. It is borderline for representative tabular ML. Suggest TBD -> 2nd Tier; a human must define a meaningful target, join the count and metadata tables, and check sample size and feature representativeness. --- diff --git a/curation/records/postoperative_patient_data.md b/curation/records/postoperative_patient_data.md index bcdba7cf..802d67a2 100644 --- a/curation/records/postoperative_patient_data.md +++ b/curation/records/postoperative_patient_data.md @@ -3,8 +3,9 @@ unique_name: postoperative_patient_data name: postoperative_patient_data checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: +- Duplicate - Too Small tags: - AI-Filled (Verify) @@ -26,6 +27,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the PMLB copy of the UCI Post-Operative Patient dataset (DOI 10.24432/C5DG6Q), recorded and rejected by Lennart as `post_operative_patient` ("Too small data (less than 96)"). +- Source: UCI Machine Learning Repository, Post-Operative Patient (Summers et al., 1993); PMLB `postoperative_patient_data` re-serves it without its own description ("None yet") with the same eight attributes L-CORE, L-SURF, L-O2, L-BP, SURF-STBL, CORE-STBL, BP-STBL, COMFORT and the discharge decision as the target; 88 to 90 rows. +- Siblings and duplicates: `post_operative_patient` (`No`, `Too Small`, checked by Lennart); the workbook note in this record agrees ("88 samples, too small with 3-fold CV. Lennart: too small. Andrej: too small"). +- Metric: not needed for a duplicate. +- Reject check (AI, 2026-10-22): stands. PMLB metadata.yaml (https://github.com/EpistasisLab/pmlb/blob/master/datasets/postoperative_patient_data/metadata.yaml) has the same eight attributes as UCI Post-Operative Patient (https://doi.org/10.24432/C5DG6Q). That source is post_operative_patient, which Lennart checked and rejected ('Too small data (less than 96)'). The 88 vs 90 rows come from PMLB dropping rows with missing values, so it is a processed copy (PROV-4). + The UCI/PMLB post-operative patient dataset (predicting discharge decision) has only 88 samples. Prior TabArena verdict and both Lennart and Andrej flagged it as too small for the 3-fold CV protocol. With under 90 rows it is unsuitable for reliable benchmarking. Suggest No on size grounds. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/predict_fire_extent.md b/curation/records/predict_fire_extent.md index a537b504..c5f41e10 100644 --- a/curation/records/predict_fire_extent.md +++ b/curation/records/predict_fire_extent.md @@ -3,9 +3,9 @@ unique_name: predict_fire_extent name: predict-fire-extent checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,6 +27,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): Zindi's competition `predict-fire-extent` (shown as "Wildfire Prediction Challenge", 2024-07-16 to 2024-08-14) is a re-run of the Zimbabwe burned-area data first used in the Zindi "Fighting Fire with Data Hackathon" (2020-06-27). +- Source: the Zindi API lists the same files for `fighting-fire-with-data-hackathon` (2020), `cmu-africa-fighting-fire-with-data` (2020), `data-science-zw-2022-fire-prediction` (2022), `predict-fire-extent` (2024) and `wildfire-prediction-challenge` (2024): Train.csv 19,531,802 bytes, Test.csv 5,802,277, SampleSubmission.csv 429,661, variable_definitions.csv 2,344, and a starter notebook named "Fighting_Fire_with_Data_Starter_Notebook.ipynb"; as published: monthly burned-area fraction per area square across Zimbabwe since 2001 with TerraClimate, MODIS land cover, population and elevation covariates; train to 2013, test 2014-2016. +- Siblings and duplicates: `data_science_zw_2022_fire_prediction` (TBD -> 2nd Tier, its `source_links` carry the 2020 fighting-fire hackathons, so it holds the first publication) and `wildfire_prediction_challenge` (`No`, `Time-series (Forecasting)`), both the same files. Whichever record the curator keeps as canonical, this one adds no distinct task. +- Metric: Root Mean Squared Error on the burned fraction (the Evaluation page). +- Note for the canonical record: the task predicts "the burned area in different locations over 2014 to 2016" from the series to 2013 with covariates "that extends into the test period", a forecasting-like panel (SEL-10 / SEL-11, TP-10). +- Reject check (AI, 2026-10-22): stands. I checked the Zindi API myself. predict-fire-extent and fighting-fire-with-data-hackathon (https://zindi.africa/competitions/fighting-fire-with-data-hackathon, 2020-06-27) have identical sizes: Train.csv 19531802, Test.csv 5802277, SampleSubmission.csv 429661, variable_definitions.csv 2344. The 2024 Description repeats the 2020 task (predict burned area 2014-2016 from data to 2013), so PROV-4 applies. data_science_zw_2022_fire_prediction already carries the 2020 links and stays the canonical record. + A Zindi competition to predict the burned area of wildfires in Zimbabwe, i.e. a regression target. Such challenges often supply geospatial/satellite-derived features which may be raster imagery (out of scope) or pre-extracted tabular covariates per location-time. There is also a risk this is a forecasting task over time. Suggest TBD -> 2nd Tier; a human must verify whether inputs are tabular features versus imagery, whether the task is a fixed predictive (temporal-split) regression rather than forecasting, and the dataset size. --- diff --git a/curation/records/predictive_insights_youth_income_prediction_challenge.md b/curation/records/predictive_insights_youth_income_prediction_challenge.md index 5f44fb53..57f70328 100644 --- a/curation/records/predictive_insights_youth_income_prediction_challenge.md +++ b/curation/records/predictive_insights_youth_income_prediction_challenge.md @@ -4,6 +4,7 @@ name: predictive-insights-youth-income-prediction-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 4 decision_markers: - TBD tags: @@ -11,12 +12,14 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2023' domain: social science required_split: - Random (IID) problem_type: Binary Classification source_links: - https://zindi.africa/competitions/predictive-insights-youth-income-prediction-challenge/data +- https://zindi.africa/competitions/predictive-insights-youth-income-prediction-challenge source_row: 889 needs_review: - ai_unverified @@ -27,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 4: whether a young South African is employed one year after a baseline labour-market survey, a clear follow-up target in a thin domain. +- Source: Zindi "Predictive Insights Youth Income Prediction Challenge" (Predictive Insights, 2023-06-08 to 2023-10-01, 648 participants); the data "comes from four rounds of a survey of youth in the South African labour market, conducted at 6-month intervals" (the survey is not named on the page); as published: Train.csv (one row per person, baseline answers plus the target "whether they were employed or not one year later") and Test.csv; numerical, categorical and free-text answers, demographics and school results; rows not checked (Train.csv 405 KB, likely a few thousand). +- Use case: the source: a baseline prediction of young people's employment outcomes, "allowing them to design and test interventions to help youth make a transition into the labour market"; actor a youth-employment programme, moment the baseline interview, decision who gets additional help; fit direct (baseline features, outcome a year later); regime IID by person, with the survey round as a possible temporal axis (to check). +- Siblings and duplicates: none found (grep youth, Predictive Insights); `adult`, `absa_customer_income_prediction_challenge` and `income_prediction_challenge_for_azubian` are different data; external collections: none known. +- Papers and pages: the competition pages read through the Zindi API; no paper; solutions not read. +- Metric: Area Under the Curve, Zindi Evaluation page ("The evaluation metric for this challenge is the Area Under the Curve"), `roc_auc`; deliberate (leaderboard metric, MET-4); inputs none. +- Risks for /add-dataset: the record's name says income but the target is employment; check that every column is from the baseline and none from the follow-up (LEAK); free-text answers (drop or keep as text); the round column decides IID against temporal (SPLIT); the survey's publisher and licence to be traced (PROV-2). +- Priority: gap + (social science is thin), use case + (the source names the decision), source 0 (a competition page only), shape + (mixed types, likely above 1,000 people), risk 0. + A Zindi competition to predict whether a young person in South Africa is employed, a clear real-world binary classification over socio-economic survey/individual records. This is a genuine predictive tabular task with a meaningful target, similar in spirit to the Adult income benchmark. It may have a temporal component if it tracks individuals over survey waves. Suggest TBD -> Yes; a human must confirm the data is a single tabular feature set (not a panel requiring grouping/temporal split), the exact target, and post-processing size. --- diff --git a/curation/records/profb.md b/curation/records/profb.md index 81a0ac23..7114f656 100644 --- a/curation/records/profb.md +++ b/curation/records/profb.md @@ -5,8 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- No Good Target (yet) -- Not Representative +- No Good Target / Scientific Discovery tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) @@ -32,6 +31,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3, SEL-8): NFL scores and point spreads for 1989-1991 compiled by Hal Stern for a 1992 Joint Statistical Meetings talk, published on StatLib for investigating questions, not for a prediction; the OpenML target is a relabelling. +- Source: StatLib `profb` (Hal Stern, Robin Lock; 1992), OpenML 470 (2014); as published: 672 games (224 regular-season games per season) x 9 variables (Home/Away, Favorite Points, Underdog Points, Pointspread, team codes, Year, Week, Special). +- Use case: none the source states. It asks "How well do the oddsmakers of Las Vegas predict the outcome of professional football games? Is there really a home field advantage - if so how large is it?" and says "These questions and a host of others can be investigated using this data set": a look-back study (SEL-3, USE-2). OpenML's default target `Home/Away` ("Favored team is at home (1) or away (0)") is a fact of the schedule known before the game, not an outcome anyone predicts (SEL-8). +- Siblings and duplicates: none in `curation/records/`; external collections: TabSTAR, the TabArena workbook's rejects ("TabArena Reject"). +- Human notes: the CC line ("wrong target used on OpenML, need more thought on what the actual target could be. In general, the task was rather presented as an interpretability task"). +- A constructed target (the favourite covers the spread) would be a betting task on 672 games with the spread as the main input; the source does not define it. +- Reject check (AI, 2026-10-22): stands. StatLib/OpenML 470 (https://www.openml.org/d/470): 'These questions and a host of others can be investigated using this data set' (home advantage, oddsmakers' accuracy), which is a look-back study (SEL-3). OpenML's target 'Home/Away = Favored team is at home' is a fixed fact of the schedule, a relabelling (SEL-8). The record's CC says the same: 'wrong target used on OpenML ... the task was rather presented as an interpretability task'. + NFL game-score data for the 1989-1991 seasons (StatLib/OpenML d/470). The curator notes the OpenML target is wrong, the intended use was an interpretability illustration rather than prediction, and it is unclear what a valid target would be; it already sits in TabArena Reject. The lack of a clear meaningful target and its origin as an interpretability example make it unrepresentative. Suggest No; a human could revisit whether a sensible predictive target (e.g. game outcome) exists, but it currently lacks one. --- diff --git a/curation/records/protein_location_prediction_challenge.md b/curation/records/protein_location_prediction_challenge.md index d2906d6a..8432dd67 100644 --- a/curation/records/protein_location_prediction_challenge.md +++ b/curation/records/protein_location_prediction_challenge.md @@ -3,7 +3,10 @@ unique_name: protein_location_prediction_challenge name: protein-location-prediction-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' +decision_markers: +- Wrong Domain / Source Modality +- NLP (Text) tags: - AI-Filled (Verify) collections: @@ -26,6 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Wrong Domain / Source Modality`, `NLP (Text)`, SEL-12): predict a protein's subcellular location "using only its amino acid sequence"; the input is a raw sequence string, a modality where protein language models are the tool. +- Source: Zindi "Protein Location Prediction Challenge" (an AI Hack Tunisia hackathon, 2022-08-29 to 2022-08-30, 133 participants, secret code required); data "extracted from UniProtKB/Swiss-Prot", 18,206 sequences; as published: train.csv (~18,000) and test.csv (~6,500) with ID, Sequence, Kingdom, Seq_length and the target among Nucleus, Cytoplasm, Membrane, Cell membrane, Extracellular; plus sequence_esm1b_cls_embeddings_df.zip and sequence_esm1b_mean_embeddings_df.zip (ESM-1b embeddings, 160 MB each). +- Use case: annotating proteins in research; the only inputs besides the sequence are the kingdom and the length. +- Siblings and duplicates: `instadeep_kinase_classification_challenge` (`No`, the same pattern: a class from the amino-acid sequence); `antigen_avida_hil6` (`No`, `Wrong Domain / Source Modality`, `NLP (Text)`, Lennart: "NLP task / data"). The data is a cut of UniProt, where DeepLoc and similar sequence models are the field's tools. +- Metric: Log Loss over five classes (the Evaluation page). +- Reject check (AI, 2026-10-22): stands. Zindi API (https://zindi.africa/competitions/protein-location-prediction-challenge): 'predict the location in the cell where this protein is likely to be located, using just the amino acid sequence'. The inputs are ID, Sequence, Kingdom and Seq_length, and the organisers ship ESM-1b embeddings, which are the output of a protein language model. Sequence models are the field's tool here (SEL-12). This matches Lennart's 'NLP task / data' No on antigen_avida_hil6. + Zindi competition to predict the subcellular location of a protein, a classic bioinformatics multiclass classification task that is typically solved on engineered tabular sequence-derived features. If the released data is genuinely tabular (per-protein feature rows) it is a real predictive task with a clear target and likely a random split. A human must verify the feature representation (engineered tabular vs raw sequences), dataset size, number of location classes, and the data license/terms on Zindi. --- diff --git a/curation/records/protein_model_quality_rf_pcp.md b/curation/records/protein_model_quality_rf_pcp.md index ad2a3364..cbedcd9b 100644 --- a/curation/records/protein_model_quality_rf_pcp.md +++ b/curation/records/protein_model_quality_rf_pcp.md @@ -4,6 +4,7 @@ name: RF-PCP (quality assessment of modeled protein structures, CASP and decoy s checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) - Non-IID (Grouped) @@ -26,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata and notes below were drafted by an AI assistant from the release files and the paper's abstract; a human must verify them before relying on them.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: predict the quality (RMSD, TM-score or GDT_TS) of a modelled protein structure without its native, from six physicochemical properties; the decoy file names give the target protein, so the split can be grouped by target. +- Source: Rana, Sharma, Bhattacharya, Shukla, "Quality assessment of modeled protein structure using physicochemical properties", J Bioinform Comput Biol 13(2):1550005, April 2015 (doi:10.1142/S0219720015500055), with the authors' release on SourceForge (project rf-pcp, RF-PCP-DataSet.zip, 5.3 MB, 2014-08-18, user psrana; no licence stated). The abstract (Crossref): "a total of 95,091 modeled structures of 4896 native targets", features "total surface area, euclidean distance (ED), total empirical energy, secondary structure penalty (SS), sequence length (SL), and pair number (PN)", targets RMSD, TM-score and GDT_TS, k-fold cross-validation over structures. As published (per the old note, not re-opened): a full file of ~100k rows and CASP5-9 / CASP10 files with PDB-ID, RMSD, TM, GDT and the six features. +- Use case: a structure-prediction group or a CASP assessor ranks the candidate models of a new target before its native structure is known; fit approximate (the paper says the method makes "the prediction faster and inexpensive" but names no actor or decision); regime grouped by target protein because a new target has none of its decoys in training (the leak that retired `physiochemical_protein`). +- Siblings and duplicates: `physiochemical_protein` (UCI 2013, the same group's earlier file, `No (Retired)` by Lennart on 2026-10-06 because no copy has target ids; the CC line names RF-PCP as a candidate to triage) and `physiochemical_protein_duplicate`; this release has other features and cannot be joined to the UCI rows, so it is a distinct dataset (PROV-7). External collections: none found. +- Papers and pages: the 2015 paper above; Pathak, Rana et al., Int J Data Mining and Bioinformatics (online 2015-11, RMSD <= 5 A with the same six properties) used the same features. +- Metric: the paper reports RMSE, correlation, R^2 and accuracy (within +-0.1) for each of the three targets, none ranked, so the fallback `rmse` on the chosen target (MET-5); inputs none. +- Risks for /add-dataset: which target to ship (GDT_TS or TM-score, the bounded scores QA methods predict, against RMSD of the UCI file); the rows mix CASP decoys with classic decoy sets and native structures (RMSD 0, not known at prediction time for a new target); Energy outliers up to 7e11; SL and PN near-constant within a target; six coarse features that structure-based deep QA models have since replaced (`Outdated` question, SEL-12 boundary). +- Priority: gap + (biology & life sciences and the grouped regime are thin), use case 0 (no actor named by the source), source +, shape +, risk - (target choice, natives, outdated features). + **What it is (AI, 2026-10-06).** The supplement of Rana, Sharma, Bhattacharya and Shukla (2015), "Quality assessment of modeled protein structure using physicochemical properties" (J Bioinform Comput Biol 13(2):1550005): predict the RMSD, TM-score and GDT_TS of a modeled protein structure, without its native structure, from six physicochemical properties (total surface area, Euclidean distance, total empirical energy, secondary structure penalty, sequence length, pair number). The abstract reports "95,091 modeled structures of 4896 native targets" and k-fold cross-validation over the structures (random forest best, R^2 0.92 for RMSD). The files are on SourceForge (project rf-pcp, registered 2014-08-18, RF-PCP-DataSet.zip, 5.3 MB): RF-PCP-Full-dataset.csv (99,987 rows), RF-PCP-Training-Testing-CASP-5-9.csv (92,597) and RF-PCP-Validation-CASP-10.csv (5,119), with the columns PDB-ID, RMSD, TM, GDT, Area, ED, Energy, SS, SL and PN. **Why a candidate.** It can replace physiochemical_protein (retired 2026-10-06): the same group's UCI file has no target ids, so its decoys could not be split by protein. Here the PDB-ID names each decoy file, so the target is known: CASP5-10 decoys of 342 targets (34,288 rows, a median of 101 per target; e.g. `CASP10_T0719_PconsD_TS5.pdb`), classic decoy sets named by the native's PDB code (ROSETTA 17,077, lattice 14,329, fisa 8,334, rosetta62 7,189, John2002 5,923, 4state 4,160, lmds 3,863 rows) and 4,478 native structures. The use case, judging the models of a new target, needs grouped splits by target (the native protein; one native can occur in several decoy sets); the source's CASP10 file is a natural test on a new CASP round. diff --git a/curation/records/qos_prediction_challenge.md b/curation/records/qos_prediction_challenge.md index 251bc0e1..200d462c 100644 --- a/curation/records/qos_prediction_challenge.md +++ b/curation/records/qos_prediction_challenge.md @@ -3,14 +3,14 @@ unique_name: qos_prediction_challenge name: qos-prediction-challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier -decision_markers: -- Data Quality Issue +suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: technology & internet required_split: - '?' @@ -18,6 +18,8 @@ problem_type: Regression original_data_state: One Table source_links: - https://zindi.africa/competitions/qos-prediction-challenge/data +- https://ieee-dataport.org/open-access/berlin-v2x +- https://arxiv.org/abs/2212.10343 source_row: 897 needs_review: - ai_unverified @@ -28,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: predict the achieved LTE downlink throughput of a driving car from physical-layer, cell, GPS and traffic/weather information; a real measurement campaign, but a stream of readings with scheduler fields that may nearly determine the target. +- Source: first published as Berlin V2X (Hernangómez et al., "Berlin V2X: A Machine Learning Dataset from Multiple Vehicles and Radio Access Technologies", arXiv:2212.10343, December 2022, IEEE VTC2023-Spring; data on IEEE DataPort, doi:10.21227/8cj7-q373, open access, Parquet files for cellular and sidelink; up to four cars over three days in Berlin, two operators). The Zindi QoS Prediction Challenge (ITU AI/ML in 5G Challenge, 2023-06-02 to 2023-07-15, 261 participants) uses it: "leverage the Berlin V2X dataset to predict QoS ... features from the primary and secondary cells of two commercial LTE mobile network operators ... alongside GPS and context information to predict the achieved throughput in the downlink". As published on Zindi: Train.csv (10.4 MB), Test.csv (5.2 MB), itu-ffs.zip, VariableDefinitions.csv. +- Use case: a network or V2X application anticipates the QoS of a link to manage radio resources "proactively" (the page); fit approximate, because the features are measured at the same moment as the throughput, not before it; regime not settled: grouped by drive, car or operator ("multi-domain setup (i.e., across operators, devices and/or link directions)") or temporal along the drives; `required_split` stays `?`. +- Siblings and duplicates: `row_1040` (the ITU challenge archive that lists this challenge, "QoS Prediction Challenge by ITU AIML in 5G Challenge" and "Multi-environment automotive QoS prediction"); `5g_energy_consumption` (another ITU challenge, `Yes`) is unrelated data. External collections: none found. +- Papers and pages: https://arxiv.org/abs/2212.10343, https://ieee-dataport.org/open-access/berlin-v2x, https://zindi.africa/competitions/qos-prediction-challenge. +- Metric: `rmse` (Zindi Evaluation: "The evaluation metric for this competition is Root Mean Squared Error"), a leaderboard metric (MET-4); inputs none. The target is in bit/s (sample values 16,500,000), a scale question (MET-3). +- Risks for /add-dataset: build from Berlin V2X (the first publication, PROV-8) or from the Zindi files; scheduler fields (MCS, assigned resource blocks, carrier aggregation) may make throughput near-deterministic (TP-4 check); per-second readings along drives (SPLIT-7, `Time-series (Regression)` question). The old `Data Quality Issue` marker stood for a "no-distribution license"; a licence is no reason for a marker (EV-4), so it is dropped; the DataPort page states open access. +- Priority: gap 0, use case 0, source +, shape +, risk - (stream, scheduler fields). + Zindi competition to predict quality-of-service for a wireless vehicle network, a plausible real-world tabular regression task. The record explicitly notes a 'no-distribution license', which is a hard blocker for redistribution in a public benchmark and the main reason this is not a clear Yes. A human must confirm the licensing terms (whether redistribution/use is actually permitted), the exact target and units, dataset size, and whether the split should be random or temporal (vehicular networks may have temporal structure). --- diff --git a/curation/records/rmb_monthly_mini_challenge_4_nowcast_july.md b/curation/records/rmb_monthly_mini_challenge_4_nowcast_july.md index 8b9c07e5..6fa7539a 100644 --- a/curation/records/rmb_monthly_mini_challenge_4_nowcast_july.md +++ b/curation/records/rmb_monthly_mini_challenge_4_nowcast_july.md @@ -28,6 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): the fourth of five RMB Monthly Mini Challenges on Zindi (2023-07-01 to 2023-08-01): nowcast South Africa's July 2023 CPI, the headline value and 12 sub-categories, scored against the Stats SA release of 23 August 2023. +- Source: https://zindi.africa/competitions/rmb-monthly-mini-challenge-4-nowcast-july; as published: monthly CPI history from 2022 (`CPI_Historic_Values_Zindi*.csv`), CPI weights, Sentinel-5P air quality, JSE indices, vehicle sales and other macro series from March 2022 to May 2023; the submission has 13 rows (`July_headline CPI`, `July_food and non-alcoholic beverages`, ...). +- Why out: the target is the next value of 13 monthly price series, which is time-series forecasting (SEL-10); there is no row-level task, only 13 values per month. The record's note ("Could use multiple historical months to create new splits") describes building a forecasting panel, still SEL-10. +- Metric: `rmse`, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. SEL-10 (Must): Zindi Evaluation: "You are required to Nowcast the July CPI headline value as well as 12 sub-categories", scored on the Stats SA release of 23 Aug 2023; SampleSubmission.csv is 399 bytes (13 rows), type_of_problem 'Forecast' (api.zindi.world/v1/competitions/rmb-monthly-mini-challenge-4-nowcast-july). Next values of 13 national series, no row-level task; SEL-11 does not apply. + Zindi 'mini-challenge' to nowcast South Africa's July CPI inflation figure. Nowcasting an economic aggregate for a future/just-passed period is fundamentally a time-series forecasting task over a horizon, which is explicitly out of scope for this benchmark. The record's own note about using multiple historical months to create splits confirms its temporal/forecasting nature. Excluded as forecasting. --- diff --git a/curation/records/row_1040.md b/curation/records/row_1040.md index 0f123cc8..ce7e8af2 100644 --- a/curation/records/row_1040.md +++ b/curation/records/row_1040.md @@ -4,8 +4,9 @@ name: row_1040 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Missing source information +- Needs extensive data wrangling tags: - AI-Filled (Verify) required_split: @@ -23,4 +24,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: not one dataset but the ITU AI Challenges archive; the tabular challenges in it are best screened as records of their own. +- Source: https://github.com/ITU-AI-ML-in-5G-Challenge/Challenge_Archive ("a curated collection of AI and ML challenges organized by the International Telecommunication Union (ITU) from 2020 to 2026"), folders per year, 118 index entries, among them keynotes, talks and webinars. +- As published: one Readme per challenge with links to its data. Titles that may be tabular prediction tasks: Radio Link Failure Prediction (2021), Network failure classification model, Network failure detection and root cause analysis in 5GC, Classification of Home Network Users, Next-Gen WiFi Throughput Prediction, Machine Learning for Throughput Prediction in Coordinated IEEE 802.11be Wi-Fi networks, QoS Prediction Challenge (record `qos_prediction_challenge`), AIML for 5G-Energy Consumption Modelling (record `5g_energy_consumption`, `Yes`), GeoAI Ground-level NO2 Estimation, INEGI UN-GGIM Vegetation Mapping (record `inegi_gcim_vegetation_mapping_challenge`), Location Estimation Using RSSI of Wireless LAN (record `3d_estimation_using_rssi_of_wlan_dataset_complete_1_target`). Most other entries are forecasting (traffic, beam-level traffic), images or beams (SEL-12), graph or RL tasks, or text and LLM tasks. +- Use case: per challenge; none for the bundle. Regime: per challenge. +- Siblings and duplicates: as listed; the same pattern as `indabax_hackathons` (a bundle of competitions, `TBD -> 2nd Tier`, priority 1). +- Metric: per challenge. +- Risks for /add-dataset: none as a dataset; the next step is a record per promising challenge (Radio Link Failure Prediction, the Wi-Fi throughput challenges, network failure classification), screened on its own. +- Priority: gap 0, use case 0, source +, shape 0 (a collection), risk - (no target: the record itself can never be added). The old `Missing source information` marker is dropped: the source is ITU's own archive. + The record contains only a link to the ITU AI/ML in 5G Challenge archive (a repository of many telecom/networking challenges) with no name, comments, or target description. There is far too little information to identify a single dataset or task, and the archive aggregates heterogeneous problems that may include forecasting, ranking, or non-tabular data. A human must open the archive, pick a specific sub-challenge, and verify it is a genuine tabular predictive task with adequate size and a clear target before any Yes is possible. diff --git a/curation/records/row_876.md b/curation/records/row_876.md index 46c5a974..c6c1d4cb 100644 --- a/curation/records/row_876.md +++ b/curation/records/row_876.md @@ -4,6 +4,7 @@ name: row_876 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Missing source information tags: @@ -23,6 +24,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the proposed fast reject (PROV-4) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Duplicate`, PROV-4): a second record of the FAIR Forward catalog, the same link as `fair_forward`. +- Source: https://fair-forward.github.io/datasets/, "FAIR Forward - Catalog of Open Source AI Datasets, Models & AI Systems" (GIZ, the German development agency's FAIR Forward initiative): 89 projects in `data/catalog.json`, data types geospatial/remote sensing 37, text 32, voice 21, images 19, tabular 14, drone imagery 8, meteorological 5. +- Why out: not a dataset but a catalog, and recorded already as `fair_forward` (source_row 1042) with the same single link. This record's only note, "Several interesting datasets", belongs in `fair_forward`. +- Next step for the catalog (in `fair_forward`): the tabular entries with a dataset link that may be prediction tasks, e.g. Croppie coffee yield (https://huggingface.co/datasets/rgautroncgiar/croppie_coffee_ug), the Agrivoltaic Dataset Ghana (Kaggle), Residential Energy and Weather Data Pakistan (LUMS), Solar irradiance (Marconi Lab, GitHub), each as its own record. +- Metric: none (a catalog). +- Reject check (AI, 2026-10-22): does not stand. row_876 (source_row 876) is the older entry of the same single link https://fair-forward.github.io/datasets/ and carries the curator's own note "Several interesting datasets"; fair_forward (source_row 1042) is the later copy. Under PROV-4 the later record is the duplicate, so row_876 keeps the link and fair_forward is `No` (`Duplicate` of row_876). The catalog is not a dataset either way. +- For /add-dataset: Keep row_876 as `TBD -> 2nd Tier`, priority 1, marker `Missing source information` (a catalog pointer, not a dataset); move the screener's catalog summary and the tabular candidates (Croppie coffee yield, Agrivoltaic Ghana, Residential Energy and Weather Pakistan, Marconi Lab solar irradiance) into it; each candidate gets its own record before any add stage. Mark fair_forward `No` (`Duplicate`, PROV-4) pointing to row_876. (Priority set to 1 by the main session, like row_1040: a catalog pointer, not a dataset.) + The record points to a fair-forward datasets index page with only the note 'Several interesting datasets', so it is a pointer to a collection rather than a single curated dataset. There is not enough information to assess a specific predictive task, target, modality, or size. A human must browse the linked index, select individual candidate datasets, and create separate records, verifying each is a tabular predictive task with a clear target. --- diff --git a/curation/records/sbtic_xente_credit_scoring_challenge.md b/curation/records/sbtic_xente_credit_scoring_challenge.md index 34f5e7dd..3fb8fba3 100644 --- a/curation/records/sbtic_xente_credit_scoring_challenge.md +++ b/curation/records/sbtic_xente_credit_scoring_challenge.md @@ -4,11 +4,13 @@ name: sbtic-xente-credit-scoring-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: finance required_split: - Temporal (NON-IID) @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: predict whether a Xente pay-later loan defaults from the customer's e-commerce transactions and the loan's fields; a real credit decision with a chronological split, but small and with columns known only after repayment. +- Source: the Standard Bank Tech Impact Challenge: Xente Credit Scoring Challenge on Zindi (2019-08-30 at the SBTIC finals, reopened as a points challenge until 2019-12-01, 695 participants), data from Xente (Uganda). As published: "approximately 2,665 unique e-commerce transactions that occurred between 21 September 2018 and 17 July 2019. During this period, 1,631 loans were issued"; "The data have been split into a test and training set. This was done chronologically" (Train 21 Sep 2018 to 31 Mar 2019, Test 31 Mar to 17 Jul 2019); columns CustomerId, TransactionStartTime, Value, Amount, ProviderId, ProductId, ProductCategory, ChannelId, TransactionStatus, IssuedDateLoan, AmountLoan, LoanId, PaidOnDate, IsFinalPayBack, InvestorId, DueDate, PayBackId, ThirdPartyId, IsThirdPartyConfirmed, target IsDefaulted ("Exceeded agreed payback time"); a second table `unlinked_masked_final.csv` (2.4 MB) of transactions not linked to loans. +- Use case: Xente decides on a loan at application, "to refine their credit decision processes, and ... assess the creditworthiness of new and existing clients" (the page); fit direct once the post-outcome columns are dropped; regime temporal because the source split by time and a lender scores future applicants. +- Siblings and duplicates: `xente-credit-scoring-challenge` (DSN AI+ Unilorin private hackathon, 2020-11; its Zindi parent is this competition and its files have the same sizes: Train.csv 642,190, Test.csv 207,046, unlinked 2,444,377 bytes), already in this record's `source_links`; `xente_fraud_detection_challenge` and `xente_purchase_prediction_challenge` are other Xente data and tasks. +- Papers and pages: https://zindi.africa/competitions/sbtic-xente-credit-scoring-challenge. +- Metric: `roc_auc` (Zindi Evaluation: "The evaluation metric for this challenge is the Area Under the Curve (AUC)"), a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: PaidOnDate, IsFinalPayBack, PayBackId and ThirdPartyId are known only after repayment (LEAK, drop; Test.csv lacks some loan columns, per the page); rows are transactions and payback installments, while the unit is the loan (dedupe to one row per loan); the number of defaults may be small for three temporal windows (TP-5, TP-9); joining the unlinked transactions as customer history. +- Priority: gap 0, use case +, source +, shape + (borderline), risk -. + Zindi Xente credit scoring challenge, predicting loan default/repayment from financial-transaction and loan features, which is a canonical real-world tabular binary classification task in finance. Credit scoring is well-suited to the benchmark with a clear target and adequate data, though it may require a temporal split since lending data is time-ordered. A human must verify whether the original data is a single table or transactions to be aggregated/joined, the exact target definition, size, and the appropriate split regime, plus the Zindi data license. diff --git a/curation/records/scku_datathon.md b/curation/records/scku_datathon.md index e1ba90c8..bd8eaa94 100644 --- a/curation/records/scku_datathon.md +++ b/curation/records/scku_datathon.md @@ -3,9 +3,9 @@ unique_name: scku_datathon name: scku-datathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information +- Duplicate tags: - AI-Filled (Verify) collections: @@ -26,4 +26,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the SCKU Datathon (Zindi, 2024-02-25 to 2024-03-08, "a private hackathon only open to SKCU members") re-runs the Ernst & Young Carbon Prediction Competition (2023-02-23 to 2023-05-01), its parent in the Zindi API. +- Source: https://zindi.africa/competitions/scku-datathon; files Train.csv 95,570,638 bytes, Test.csv 36,024,298, SampleSubmission.csv 725,308, EY_Carbon_Prediction_StarterNotebook.ipynb and GetStarted.pdf, the same names and byte sizes as https://zindi.africa/competitions/ey-carbon-prediction-hackathon; the same task text (Sentinel-5P weekly pollutant columns at ~500 South African locations, 2019-01 to 2022-11, target `emission`, IDs `ID_LAT_LON_YEAR_WEEK`). +- Siblings and duplicates: the first publication, the EY hackathon, is in `source_links` of `carbon_dioxide_prediction_challenge` (AI-screened `TBD -> 2nd Tier`, priority 2, 2026-10-22), which keeps the five Zindi pages of this dataset in one record; this record ends at the same source (PROV-4, PROV-8). +- Metric: `rmse`, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. PROV-4/PROV-5 confirmed from the Zindi API: scku-datathon (2024-02-25) has parent 'ey-carbon-prediction-hackathon' (2023-02-23) and identical files and sizes (Train.csv 95,570,638; Test.csv 36,024,298; SampleSubmission.csv 725,308; EY_Carbon_Prediction_StarterNotebook.ipynb 2,679,805; GetStarted.pdf 106,935). The EY page is already in source_links of carbon_dioxide_prediction_challenge. + This Zindi 'SCKU datathon' record has no comments, target description, or metadata beyond the competition link, so the task type, modality, target, and size are all unknown. There is too little information to decide, and datathons can span forecasting, NLP, or image tasks. A human must open the competition page to determine whether it is a genuine tabular predictive classification/regression task with a clear target and adequate size. diff --git a/curation/records/sea_turtle_rescue_error_detection_challenge.md b/curation/records/sea_turtle_rescue_error_detection_challenge.md index 6a0a9fd1..6796bcb2 100644 --- a/curation/records/sea_turtle_rescue_error_detection_challenge.md +++ b/curation/records/sea_turtle_rescue_error_detection_challenge.md @@ -4,20 +4,21 @@ name: sea-turtle-rescue-error-detection-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: -- Data Quality Issue +- Needs extensive data wrangling tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2018' required_split: - '?' -problem_type: TBD +problem_type: Binary Classification original_data_state: One Table source_links: - https://zindi.africa/competitions/sea-turtle-rescue-error-detection-challenge/data -- https://zindi.africa/competitions/sea-turtle-rescue-forecast-challenge/data source_row: 924 needs_review: - ai_unverified @@ -28,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: predict, for each field of each rescue record, whether it was mistyped from the handwritten logbook, so Local Ocean Conservation's staff check fewer fields; a real decision, but a per-cell task whose label comes from diffing a dirty and a cleaned table. +- Source: Zindi, "Sea Turtle Rescue: Error Detection Challenge" (2018-11-30 to 2019-04-29, 329 participants), data from Local Ocean Conservation (Watamu, Kenya), By-Catch Release Programme, "unique sea turtle rescues that took place from 1998 until 2011". As published: `dirty_data.csv` (1.1 MB, "all of the data from 1998 to 2010 (except for 2008 and 2009) as they were originally entered into the database from the handwritten logbooks"), `cleaned_data.csv` (1.2 MB, the reviewed version), `variable_definitions.csv`; the submission scores one probability per rescue and field (`2011_RE_0001 x Date_Caught`, `x CaptureSite`, `x Researcher`). +- Use case: "At the end of the year, LOC staff review each rescue/field to compare the database entry with the hand-written log book"; a model that ranks fields by error probability saves that review; fit direct for the actor and moment; regime temporal or grouped by year (train on reviewed years, flag a new year; the test IDs are 2011), to settle when added. +- Siblings and duplicates: the Sea Turtle Rescue: Forecast Challenge (Zindi, 2019-11-29, weekly rescues per capture site, RMSE) is a different task and a forecast (SEL-10); it was in this record's `source_links` and is moved here as related work (PROV-12): https://zindi.africa/competitions/sea-turtle-rescue-forecast-challenge. No other record found. +- Papers and pages: https://zindi.africa/competitions/sea-turtle-rescue-error-detection-challenge. +- Metric: mean absolute error between the predicted probability and the 0/1 error flag (Zindi Evaluation: "The Mean Absolute Error will be used ... the average distance between the predicted likelihood that a entry is an error and the actual value"), a leaderboard metric (MET-4); MAE on probabilities rewards 0/1 answers rather than calibrated ones, so the curator may prefer the fallback (`log_loss` or `roc_auc`) when adding (MET-4 departure to argue). +- Risks for /add-dataset: build the target by aligning the dirty and cleaned rows field by field (keys, deleted or added rows); the unit is a cell, so the table is rows x fields with the field name as a feature; whether the cleaned values leak into features; the error rate per field may be tiny. +- Priority: gap 0, use case +, source +, shape 0 (size unknown), risk - (built target, per-cell unit). + Zindi 'sea turtle rescue error detection' challenge: the framing is detecting data-entry errors in a rescue records table, which is an unusual target and points to a data-cleaning rather than a standard predictive task; the companion link is a 'forecast' variant which would be out of scope. It is unclear whether the error-detection target yields a clean, well-defined tabular classification problem or is too idiosyncratic/trivial. A human must inspect the data and target definition to decide whether the error-detection task is a representative tabular predictive problem (and confirm the forecast variant is excluded). diff --git a/curation/records/sendy_logistics_challenge.md b/curation/records/sendy_logistics_challenge.md index 7f5525d2..3012023e 100644 --- a/curation/records/sendy_logistics_challenge.md +++ b/curation/records/sendy_logistics_challenge.md @@ -4,14 +4,16 @@ name: sendy-logistics-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: business & marketing required_split: -- Temporal (NON-IID) +- Random (IID) problem_type: Regression original_data_state: Database (or multiple to-be-joined tables) source_links: @@ -26,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: predict the time from pickup to arrival of a Sendy motorbike delivery in Nairobi from the order, its timestamps up to pickup, distance, weather and rider metrics; a real ETA task with a named user, but no month or year in the data. +- Source: Zindi, "Sendy Logistics Challenge" (2019-08-23 to 2019-11-25, 2,201 participants, $7,000), hosted by Sendy and insight2impact. As published: Train.csv (4.1 MB), Test.csv (1.2 MB), Riders.csv, VariableDefinitions.csv; "a subset of over 20,000 orders and only includes direct orders (i.e. Sendy 'express' orders) with bikes in Nairobi. All data in this subset have been fully anonymized while preserving the distribution", with weather added; columns Order No, User Id, Vehicle Type, Platform Type, Personal or Business, day of month / weekday / time for placement, confirmation, arrival at pickup and pickup, Distance (KM), Temperature, Precipitation, pickup and destination lat/long, Rider Id; Arrival at Delivery times only in Train; target "Time from Pickup to Arrival" in seconds. +- Use case: Sendy tells the customer an arrival time when the rider picks up the package, and plans scheduling ("enhance customer communication ... improved resource management and planning for order scheduling"); fit direct at pickup; regime: a temporal task by nature, but the release has only day of month and weekday, no month or year ("Placement - Day of Month i.e 1-31"), so the windows SPLIT-14 asks for cannot be built and it ships IID, forced from temporal (SPLIT-8). The AI-filled `required_split` `Temporal (NON-IID)` is changed to `Random (IID)` for that reason. +- Siblings and duplicates: `sendy_logistics_challenge_2` (the Zindi Mentorship #2 and EDSA 2020 re-runs, both with this competition as parent and the same file sizes; proposed `Duplicate` in this batch); `delivery_rider_response_challenge_by_galsenai` (IndabaX Senegal 2021, Sendy rider matching, AI `No`, another task). External collections: none found. +- Papers and pages: https://zindi.africa/competitions/sendy-logistics-challenge (winners linked from the page, not read). +- Metric: `rmse` on seconds (Zindi Evaluation: "The error metric for this competition is the Root Mean Squared Error"), a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: Riders.csv holds rider aggregates (number of orders, age on platform, average rating, number of ratings), probably taken at extraction time over all orders (LEAK check); drop the Arrival at Delivery columns; rider-marked times give many near-zero or implausible targets (known from the competition, to verify; CLEAN, not a reason to drop rows silently); join the rider table on Rider Id. +- Priority: gap 0, use case +, source +, shape +, risk - (rider aggregates, noisy target, forced IID). + Zindi Sendy logistics challenge to predict delivery ETA for parcel/motorbike deliveries in Nairobi, a genuine real-world tabular regression task with order, rider, and location features. This is a strong fit for the benchmark, though it likely involves joining multiple provided tables (orders + riders) and may need a temporal split since deliveries are time-ordered. A human must confirm the target (ETA/duration), table structure to be joined, size, split regime, and license; note potential overlap with the 'sendy_logistics_challenge_2' record. diff --git a/curation/records/sendy_logistics_challenge_2.md b/curation/records/sendy_logistics_challenge_2.md index 7dc586d3..0f8f3130 100644 --- a/curation/records/sendy_logistics_challenge_2.md +++ b/curation/records/sendy_logistics_challenge_2.md @@ -3,7 +3,7 @@ unique_name: sendy_logistics_challenge_2 name: Sendy Logistics Challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: - Duplicate tags: @@ -29,6 +29,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the Zindi Mentorship #2 (2020-07-20 to 2020-08-26) and EDSA 2020 (2020-05-11 to 2020-09-07) private re-runs of the Sendy Logistics Challenge (2019-08-23 to 2019-11-25). +- Source: both pages name `sendy-logistics-challenge` as their parent in the Zindi API, repeat its description ("predict the estimated time of delivery of orders, from the point of driver pickup to the point of arrival at final destination") and ship the same files with the same byte sizes (Train.csv 4,094,263, Test.csv 1,209,061, Riders.csv 29,751, SampleSubmission.csv 117,450). +- Siblings and duplicates: the first publication is `sendy_logistics_challenge` (screened `TBD -> Yes`, priority 3, 2026-10-22); this record ends at the same source (PROV-4, PROV-8). Its two links can move to that record's comments as re-runs. +- Metric: `rmse` on seconds, Zindi Evaluation page. +- Reject check (AI, 2026-10-22): stands. PROV-4/PROV-5 confirmed from the Zindi API: zindi-mentorship-2-sendy-logistics-challenge (2020-07-20) and edsa-sendy-logistics-challenge (2020-05-11) both have parent 'sendy-logistics-challenge' (2019-08-23) and the same Train.csv 4,094,263 / Test.csv 1,209,061 / Riders.csv 29,751 / VariableDefinitions.csv 2,661 / SampleSubmission.csv 117,450 bytes; the original is the record sendy_logistics_challenge. + This is another Sendy logistics ETA-prediction competition (mentorship/EDSA re-runs) predicting estimated time of arrival for motorbike deliveries in Nairobi, a real tabular regression task. It is almost certainly the same underlying data as 'sendy_logistics_challenge' and should be de-duplicated so only one version is kept. A human must compare the two records' data files, confirm which is the canonical source, the join/table structure, target, size, and split, and drop the duplicate. --- diff --git a/curation/records/sensory.md b/curation/records/sensory.md index 5c0782c7..5f951cd9 100644 --- a/curation/records/sensory.md +++ b/curation/records/sensory.md @@ -31,6 +31,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line, TabArena Reject) stands on SEL-3; one counter-argument on a marker: the data is a real experiment, not a simulation. +- Source: OpenML 546 (uploaded 2014), from Brien and Payne (1996), "Tiers, structure formulae and the analysis of complicated experiments"; 576 rows x 12 columns: 11 design factors (Occasion, Judges, Interval, Sittings, Position, Squares, Rows, Columns, Halfplot, Trellis, Method) and the response `Score`. +- Use case: none. OpenML: a split-plot vineyard trial "to investigate the differences between 4 types of trellising and 2 methods of pruning", the 24 wines scored by 6 judges in a latinized design. The question is the treatment effects (an ANOVA of a designed experiment), a discovery question; predicting a judge's score from the layout of the design simulates no decision (SEL-3, USE-2: `No Good Target / Scientific Discovery`). Regime would be grouped (judges, plots). +- Siblings and duplicates: none (`wine_quality` is other data). External collections: TabSTAR. +- Metric: none (an ANOVA paper), so the fallback had it shipped. +- **Counter-argument (AI):** `AHDS (Artifical/Handmade/Deterministic/Simulated)` (SEL-16, PROV-3) does not fit the source's words: the scores are real tastings of real wines from a field trial; only the allocation is designed. The verdict rests on SEL-3 alone; suggest dropping the AHDS marker. The curator has the call. + + OpenML 'sensory' dataset (Brien & Payne 1996) from a designed wine-tasting experiment recording control variables and tier/grouping structure, already tagged as a TabArena Reject and 2nd-Tier/Scientific-Discovery. The curator notes it appears to be a constructed/simulated experimental design rather than a genuine predictive task with a meaningful target. It is excluded as artificial/designed-experiment scientific-discovery data without a real predictive target. --- diff --git a/curation/records/sf_police_incidents.md b/curation/records/sf_police_incidents.md index b11f7081..afe02dfb 100644 --- a/curation/records/sf_police_incidents.md +++ b/curation/records/sf_police_incidents.md @@ -33,6 +33,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line, TabArena Reject: "Ethical problems") stands; one counter-argument on a marker: the task is no forecast. +- Source: City and County of San Francisco, Police Department Incident Reports, Historical 2003 to May 2018 (data.sfgov.org, dataset tmnf-yvry, PDDL); OpenML 42732 (2020) took a download of 5 Nov 2018 and built the binary target `ViolentCrime` from `Category` and `Descript` (assault, robbery, forcible sex offences, kidnapping, purse snatching), dropped Category, Descript, Date, Time, Resolution, Location and PdId, and added Hour, DayOfWeek, Month, Year; 2,215,023 rows x 9 columns, incidents with several categories span several rows (`IncidntNum`). +- Use case: the target was made by the OpenML uploader ("Using this dataset for machine learning was inspired by Nina Zumel's blogpost" on impact coding), not by the source (SEL-8). The only decision it would fit is where and when to expect violent crime from district, address and time, which is predictive policing, the ethical concern the curator names (SEL-1). Once a report exists its category is known, so no actor predicts it from the report. +- Siblings and duplicates: none among the records (grep for sfgov, SF crime). The Kaggle "San Francisco Crime Classification" (2015) uses the same source; no record. External collections: TabSTAR. +- Metric: none from the source; the fallback. +- **Counter-argument (AI):** `Time-series (Forecasting)` (SEL-10) does not fit: each row is one incident classified from its own time and place, not future values of a series; the temporal split the CC asks for is a split, not a scope rule. The verdict rests on SEL-1 and SEL-8/USE-2; suggest dropping the forecasting marker. The curator has the call. + + OpenML 42732 San Francisco police incident reports (2003-2018), where the typical task is predicting crime category/resolution; it is already a TabArena Reject. The curator flags ethical problems (predictive policing bias, collection bias) and notes it would need a temporal split and that a newer version exists. Given the documented ethical concerns and biased collection, it is excluded; a human could instead consider a fresh, ethically reviewed extract if desired. --- diff --git a/curation/records/sfc_paygo_solar_credit_repayment_competition.md b/curation/records/sfc_paygo_solar_credit_repayment_competition.md index ea41c19e..18d78350 100644 --- a/curation/records/sfc_paygo_solar_credit_repayment_competition.md +++ b/curation/records/sfc_paygo_solar_credit_repayment_competition.md @@ -3,19 +3,22 @@ unique_name: sfc_paygo_solar_credit_repayment_competition name: sfc-paygo-solar-credit-repayment-competition checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 2 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: finance required_split: - Temporal (NON-IID) -problem_type: Binary Classification +problem_type: Regression original_data_state: Database (or multiple to-be-joined tables) source_links: - https://zindi.africa/competitions/sfc-paygo-solar-credit-repayment-competition/data +- https://zindi.africa/competitions/sfc-paygo-solar-credit-repayment-competition source_row: 910 needs_review: - ai_unverified @@ -26,4 +29,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real PAYGo solar loan book (~37,000 contracts), but the task as published is the next six monthly payments of each contract from its payment history, a forecasting-like panel that has to pass TP-10. +- Source: Zindi competition "SFC PAYGo Solar Credit Repayment Competition", host Solar Frontier Capital (an impact investor financing PAYGo solar distributors), 6 June to 29 August 2021, $5,000, 1,072 participants (https://zindi.africa/competitions/sfc-paygo-solar-credit-repayment-competition, read through the Zindi API). As published: "a dataset consisting of over ~37000 PAYGo SHS contracts"; Train.csv "contains the target (next 6 months to predict)", Test.csv the same without target columns, metadata.csv "some extra features", VariablesDefinition.txt (a codebook, not read: download needs a login). The target is six monthly amounts per contract (submission rows `ID x m1` ... `ID x m6`). +- Use case: the page says "predict the next six months of payments for different customers. This will allow PAYGo distributors to provide appropriate services and customer support"; the data description frames it as credit scoring ("predicts PAYGo SHS contract repayments or defaults"). Actor: the distributor or its financier, at a cut-off date, projecting each running contract's collections (or default). Fit approximate: the shipped target is a monthly payment series, not a default label; a contract-level target (sum over six months, or fully paid vs defaulted) is a re-framing to decide at /add-dataset. Regime temporal (one cut-off date; the future months are later than all features). +- Siblings and duplicates: `sfc_paygo_solar_credit_repayment_hackathon` is the 3-day hackathon of 4-6 June 2021 on "over ~3000 PAYGo SHS contracts"; this page calls it "the hackathon which was a subset of this dataset", so the hackathon is the duplicate (proposed `No`, PROV-6) and this record is the one to keep. No other PAYGo record. External collections: none found. +- Papers and pages: no paper; no top-solution write-up found (web search). Discussions report contracts with payments above the contract value (~168) and transaction months past the cut-off in the hackathon data (threads 6401, 6924, 6994; organiser replies not readable). +- Metric: RMSE over the contract-month rows (Evaluation page: "The error metric for this competition is the Root Mean Squared Error"), a leaderboard metric, deliberate (MET-4); inputs: none beyond the target rows (MET-1). +- Risks for /add-dataset: SEL-10 / TP-10: the inputs likely include the contract's own past payments (discussions cite transaction dates and payment histories; the codebook was not read), a lag of the target series, which TP-10 leg 2 forbids; the task is in scope only if it is re-targeted at contract level or passes the naive-profile test. Join of metadata.csv to Train by ID; multi-output target (six months); payments above contract value. `problem_type` was `Binary Classification` (AI-filled); the source scores a regression (RMSE on amounts), so it is now `Regression`; a default label would be our re-framing. License: Zindi rules forbid redistribution (no criterion, EV-4). +- Priority: gap - (finance and temporal not thin), use case - (the source's actor and decision are vague, fit approximate), source + (host named, codebook shipped), shape + (~37,000 contracts, mixed contract metadata, temporal), risk - (forecasting-like target, SEL-10/TP-10). + + Zindi SFC PAYGo solar credit repayment competition, predicting whether pay-as-you-go solar customers will repay/make payments, a genuine real-world tabular credit-scoring task in finance/fintech. This is a good fit with a clear repayment target and likely customer + payment-history tables, though it probably needs a temporal split and may require joining tables. A human must confirm the exact target (default vs payment amount, classification vs regression), table structure, size, split, and license; note likely overlap with the 'hackathon' variant. diff --git a/curation/records/sfc_paygo_solar_credit_repayment_hackathon.md b/curation/records/sfc_paygo_solar_credit_repayment_hackathon.md index 024440a9..8b690b6b 100644 --- a/curation/records/sfc_paygo_solar_credit_repayment_hackathon.md +++ b/curation/records/sfc_paygo_solar_credit_repayment_hackathon.md @@ -3,7 +3,7 @@ unique_name: sfc_paygo_solar_credit_repayment_hackathon name: SFC PAYGo Solar Credit Repayment Hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: - Duplicate tags: @@ -28,6 +28,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-6): the 3-day hackathon (4-6 June 2021, $300) on "over ~3000 PAYGo SHS contracts" is a subset of the follow-up SFC PAYGo Solar Credit Repayment Competition (~37,000 contracts, same host, same page text, same task and metric), which calls it "the hackathon which was a subset of this dataset". +- Source: Zindi hackathon "SFC PAYGo Solar Credit Repayment Hackathon", host Solar Frontier Capital, 142 participants (https://zindi.africa/competitions/sfc-paygo-solar-credit-repayment-hackathon, read through the Zindi API); files Train.csv, Test.csv, metadata.csv, VariablesDefinition.txt, StarterNotebook.ipynb; target the next six monthly payments per contract; metric RMSE ("The error metric for this competition is the Root Mean Squared Error"). +- Duplicate of: `sfc_paygo_solar_credit_repayment_competition` (https://zindi.africa/competitions/sfc-paygo-solar-credit-repayment-competition): its Data Description says "StarterNotebook.ipynb - this is the starter notebook for the hackathon which was a subset of this dataset", and this hackathon's page says "Once this hackathon is finished join the follow-up competition here - SFC PAYGo Solar Credit Repayment Competition". A cut with fewer rows and no distinct task is a duplicate (PROV-6); the fuller competition is the record to keep (screened `TBD -> 2nd Tier`, priority 2, for its forecasting-like target). +- Also: a discussion thread (6401) reports target months past the cut-off in the hackathon data, one more reason to prefer the competition's release. +- Reject check (AI, 2026-10-22): stands. Zindi API: the competition's Data Description says 'StarterNotebook.ipynb - this is the starter notebook for the hackathon which was a subset of this dataset' (~37000 contracts) and the hackathon (4-6 June 2021) has '~3000 PAYGo SHS contracts' and 'Once this hackathon is finished join the follow-up competition'; same host, target and RMSE (https://zindi.africa/competitions/sfc-paygo-solar-credit-repayment-competition). A smaller cut with no distinct task (PROV-6); keep sfc_paygo_solar_credit_repayment_competition. + Zindi SFC PAYGo solar credit repayment 'hackathon', predicting PAYGo solar customer payments, a real fintech credit-repayment tabular task. It is very likely the same underlying data as the 'competition' variant and should be de-duplicated so only one is kept. A human must compare the two records, confirm the canonical source, the target/problem type, table join structure, size, temporal split, and license, and drop the duplicate. --- diff --git a/curation/records/shop_customer_data.md b/curation/records/shop_customer_data.md index f6e4bcad..2cf927f3 100644 --- a/curation/records/shop_customer_data.md +++ b/curation/records/shop_customer_data.md @@ -27,6 +27,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line, TabArena Reject) stands; the source's own words decide it. +- Source: Kaggle dataset datascientistanna/customers-dataset, one file `Customers.csv` (created 2023-02-07), DbCL-1.0, usability 1. As published: "Shop Customer Data is a detailed analysis of a imaginative shop's ideal customers ... Dataset consists of 2000 records and 8 columns": Customer ID, Gender, Age, Annual Income, Spending Score ("Score assigned by the shop"), Profession, Work Experience, Family Size (read with the Kaggle CLI metadata). +- Use case: none; "imaginative shop" says the data is made up (SEL-16, PROV-3: `AHDS`), and no target is named (keywords classification, clustering); predicting the shop's own `Spending Score` simulates no decision (SEL-3). The column set extends the well-known Mall Customers toy table. +- Siblings and duplicates: none among the records (grep for customers-dataset, mall customers). External collections: none found. +- Metric: none; would be the fallback. +- No counter-argument: both markers follow from the source's words. + + Kaggle 'Shop Customer Data' describing an imaginative/fictional shop's ideal customers, already tagged as a TabArena Reject. The curator notes it is likely artificial data and was not published as a predictive task (some users ad hoc predict spending score). It is excluded as artificial/handmade data without a genuine real-world predictive target. --- diff --git a/curation/records/silicon_valley_21st_century_education_hackathon.md b/curation/records/silicon_valley_21st_century_education_hackathon.md index 633bb993..18a23394 100644 --- a/curation/records/silicon_valley_21st_century_education_hackathon.md +++ b/curation/records/silicon_valley_21st_century_education_hackathon.md @@ -4,6 +4,7 @@ name: silicon-valley-21st-century-education-hackathon checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Missing source information tags: @@ -11,12 +12,14 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: education required_split: - '?' -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/silicon-valley-21st-century-education-hackathon/data +- https://zindi.africa/competitions/silicon-valley-21st-century-education-hackathon source_row: 1001 needs_review: - ai_unverified @@ -27,4 +30,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a 2-day Zindi hackathon to predict 2010 literacy rates from "historical survey data"; the upstream survey, the unit of a row and the columns are not named, so the target and the regime are unclear. +- Source: Zindi hackathon "Silicon Valley 21st Century Education and Silicon Valley Propellers Hackathon" (subtitle "Semaine Africaine de l'Intelligence Artificielle"), 30 June to 2 July 2021, $700, 87 participants, page in French (https://zindi.africa/competitions/silicon-valley-21st-century-education-hackathon, read through the Zindi API). As published: "Le jeu de données d'entraînement contient ~14 000 réponses et celui de test contient ~1000 réponses à un sondage"; target column `target` (example values 68.92, 41.67, 98.12, a percentage); Train.csv 1.96 MB, Test.csv 145 kB, VariablesDefinition.txt 879 bytes (not read: download needs a login). +- Use case: the page states only the goal: "prédire le taux d'alphabétisation des populations en Afrique et dans le monde pour l'année 2010 à partir de données historiques de sondages". No actor or decision is named; a literacy rate per population is an aggregate, so a row is probably a population-year or area, not a respondent (unverified). Fit unknown. Regime unknown: "2010 from historical data" suggests temporal, a repeated area suggests grouped. +- Siblings and duplicates: no other record on literacy (grep over `curation/records/`). The upstream data (UNESCO UIS, DHS or a census series) is not named on the page; no external collection found. +- Papers and pages: none found (one web search on the title and the French goal). +- Metric: MAE (Évaluation page: "La métrique d'évaluation pour ce challenge est l'Erreur Absolue Moyenne"), a leaderboard metric (MET-4); inputs none (MET-1). +- Risks for /add-dataset: PROV-1/PROV-2: name the upstream survey before anything else (the existing `Missing source information` marker stays as that concern); the unit of a row and whether 2010 rows of the same area appear in train (a temporal or grouped leak); whether it is forecasting of a country series (SEL-10). Zindi rules forbid redistribution (no criterion, EV-4). +- Priority: gap + (education is thin), use case - (none named), source - (upstream unnamed, no codebook read), shape + (~14,000 rows), risk - (unclear unit and target, possible forecasting). + + This Zindi education hackathon record has no comments, target description, or metadata beyond the competition link, so the task type, target, modality, and size are unknown. There is too little information to decide and education hackathons can be survey/non-predictive or NLP-heavy. A human must open the competition page to verify whether it is a genuine tabular predictive classification/regression task with a clear target and adequate size. diff --git a/curation/records/skillcraft1_master_table_dataset.md b/curation/records/skillcraft1_master_table_dataset.md index a09c5063..615ae0d2 100644 --- a/curation/records/skillcraft1_master_table_dataset.md +++ b/curation/records/skillcraft1_master_table_dataset.md @@ -4,6 +4,7 @@ name: SkillCraft1 Master Table Dataset checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - No Good Target (yet) - Not Representative @@ -30,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's doubt (CC: "Unclear target / predictive task"; TabArena Reject) is kept and the verdict `TBD -> 2nd Tier` left as it is; priority 1. A counter-argument on the use case and the grouping for the curator. +- Source: UCI 272 (doi 10.24432/C5161N), Thompson, Blair, Chen and Henrey (2013), "Video game telemetry as a critical tool in the study of complex skill learning", PLoS ONE 8(9): e75129. As published: 3,395 rows, `GameID` ("Unique ID number for each game"), the target `LeagueIndex` (Bronze to Professional, coded 1-8, ordinal), self-reported Age, HoursPerWeek, TotalHours, and 15 per-timestamp telemetry rates (APM, hotkey use, perception-action cycles, action latency, units made); all numeric. +- Use case: the paper studies how skill differs across leagues (a discovery aim). A decision that fits is placement: a matchmaking service assigns a new player a league from their play; that use case is written by the curation, not the source (USE-1, fit approximate). Regime IID: the CC fears several games per player, but the paper says "From each respondent we gathered a replay file" and "we have only a single game from each participant" (3,360 players, 55 professionals from public sources), so no player group can leak a league. +- Siblings and duplicates: none among the records. External collections: not checked. +- Metric: the paper fits pairwise two-league conditional inference forests and reports a "classification rate" per pair against a baseline (e.g. Bronze-Gold 82.32% vs 77.81%), accuracy with no costs, a habit (MET-5, MET-6), so the fallback. +- **Counter-argument (AI):** the target is clear (the league the ladder assigned) and the grouping concern does not hold (one game per player), so this is an IID ordinal task with a plausible placement use case; what stays open is the use case, which the source does not state. Note that Professional players (league 8) lack the self-reported fields. The curator has the call. +- Priority: gap - (domain Other, regression), use case - (curation-written), source + (paper and UCI page), shape - (3,395 rows but all numeric, no structure), risk - (the curator's unclear-task doubt; the use case is ours). + + UCI SkillCraft1 (Thompson et al. 2013): StarCraft II telemetry aggregated into per-game engineered tabular features, with LeagueIndex (player skill league) as the usual target, already a TabArena Reject / 2nd-Tier. The curator notes the data is grouped by player (a player's league could leak across their games), the predictive task/target is somewhat unclear, and temporal nature needs checking. It is engineered tabular data so it is not raw non-tabular modality, but the leakage/grouping concerns make it borderline; a human must verify the grouping/leakage handling, the precise target, and whether a grouped split makes it a sound predictive task. --- diff --git a/curation/records/sleuth_case1202.md b/curation/records/sleuth_case1202.md index aa3d52ca..346d323a 100644 --- a/curation/records/sleuth_case1202.md +++ b/curation/records/sleuth_case1202.md @@ -4,6 +4,7 @@ name: sleuth_case1202 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small tags: @@ -25,6 +26,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict ("Tiny data"; Lennart and Andrej: too small) and the current `TBD -> 2nd Tier` are left as they are; priority 1. A counter-argument on the reason. +- Source: The Statistical Sleuth (Ramsey and Schafer, Duxbury 1997), Case Study 12.2, via StatLib (lib.stat.cmu.edu/datasets, 1997) and OpenML 706 (2014). 93 rows x 7 columns (bsal, sal77, fsex, senior, age, educ, exper): the Harris Trust and Savings Bank employees in a sex-discrimination case (the book's study of beginning salaries by sex). OpenML's default target `exper` (prior experience) is an arbitrary column. +- Use case: none: the source asks whether women were paid less at hire, a discovery question; no actor predicts an employee's prior experience, and predicting salary from sex would encode the bias under study (SEL-3, USE-2). +- Siblings and duplicates: `sleuth_case2002`, `sleuth_ex1605`, `sleuth_ex1714` (the same book); Lennart kept `sleuth_case2002` at `TBD -> 2nd Tier` as "a real study but for scientific discovery". External collections: OpenML. +- Metric: none (a textbook regression analysis); the fallback. +- **Counter-argument (AI):** the stronger reason is SEL-3 (`No Good Target / Scientific Discovery`), not size: under TP-5 a 93-row regression is a tiny-data task, not `Too Small` by itself. By SEL-3 the verdict would be `No`; treated like `sleuth_case2002` it stays `TBD -> 2nd Tier`. The curator has the call. +- Priority: gap -, use case -, source + (textbook, StatLib), shape - (93 rows), risk - (arbitrary target). + + OpenML 706, a Sleuth statistics-textbook dataset with only 93 samples, far too small for the benchmark's 3-fold CV protocol. The TabArena curation verdict and two reviewers (Lennart, Andrej) already agree it is too small. It is excluded on size; it is one of a family of tiny Sleuth textbook tables. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/sleuth_ex1605.md b/curation/records/sleuth_ex1605.md index c865e97a..c0385566 100644 --- a/curation/records/sleuth_ex1605.md +++ b/curation/records/sleuth_ex1605.md @@ -4,6 +4,7 @@ name: sleuth_ex1605 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small tags: @@ -25,6 +26,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict ("Tiny data"; Lennart and Andrej: too small) and the current `TBD -> 2nd Tier` are left as they are; priority 1. A counter-argument on the reason. +- Source: The Statistical Sleuth (Ramsey and Schafer), Exercise 16.5, via StatLib (1997) and OpenML 687 (2014); the data is "a subset from an observational, longitudinal, study on adopted children", Skodak and Skeels (1949) (Sleuth3 R documentation, https://search.r-project.org/CRAN/refmans/Sleuth3/html/ex1605.html). 62 rows x 6 columns: FMED (adoptive mother's years of education), TMIQ (biological mother's IQ), the child's IQ at ages 2, 4, 8 and 13; OpenML's target `Age13IQ`. +- Use case: none: the exercise asks "whether child's intelligence related to intelligence of the biological mother and the intelligence of the adoptive mother", a discovery question on a 1949 cohort; no actor decides on a predicted IQ at 13 (SEL-3, USE-2; `Outdated` would also apply). +- Siblings and duplicates: `sleuth_case1202`, `sleuth_case2002` (Lennart: "a real study but for scientific discovery", `TBD -> 2nd Tier`), `sleuth_ex1714`. External collections: OpenML. +- Metric: none; the fallback. +- **Counter-argument (AI):** the stronger reason is SEL-3 (`No Good Target / Scientific Discovery`), not size alone; with 62 rows the size is borderline as well (TP-5 counts the spread per test fold). Treated like `sleuth_case2002` it stays `TBD -> 2nd Tier`. The curator has the call. +- Priority: gap -, use case -, source + (textbook, StatLib, the 1949 study), shape - (62 rows), risk - (target picked by the uploader). + + OpenML 687, a Sleuth statistics-textbook dataset with only 62 samples, far too small for the benchmark's 3-fold CV protocol. The TabArena curation verdict and two reviewers (Lennart, Andrej) already agree it is too small. It is excluded on size; it is one of a family of tiny Sleuth textbook tables. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/sleuth_ex1714.md b/curation/records/sleuth_ex1714.md index 2635d960..ca5f05ac 100644 --- a/curation/records/sleuth_ex1714.md +++ b/curation/records/sleuth_ex1714.md @@ -4,7 +4,9 @@ name: sleuth_ex1714 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: +- No Good Target / Scientific Discovery - Too Small tags: - AI-Filled (Verify) @@ -25,6 +27,16 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the proposed fast reject (SEL-3 (with SEL-8; Lennart and Andrej's 'too small' supports, TP-5)) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`No Good Target / Scientific Discovery`, SEL-3): a textbook table from a civil-rights inquiry into insurance redlining, 47 Chicago zip codes; no predictive task, and the OpenML target `income` was picked by default. +- Source: Ramsey and Schafer, The Statistical Sleuth (2nd ed., Duxbury 2002), exercise 17.14 (R package Sleuth2, `ex1714`); the data from Andrews and Herzberg, Data (1985): Illinois Department of Insurance counts of homeowner policies by zip code, December 1977 to February 1978, fire and theft from 1975. As published: 47 x 8 (Zip, Fire, Theft, Age, Income, Race, Vol, Invol). OpenML 659 (StatLib upload) sets `income` as target with "CLASSINDEX: none specific". +- Use case: none. Sleuth2: "In the 1970's the U.S. Commission on Civil Rights investigated charges that insurance companies were attempting to redefine Chicago 'neighborhoods' in order to cancel existing homeowner insurance policies or refuse to issue new ones": the question is whether involuntary (FAIR plan) policies relate to the minority share after adjusting for fire, theft and housing age, an inference on 47 areas (SEL-3, SEL-8). Median family income per zip code as a target has no actor. +- Siblings and duplicates: the same data is faraway's `chredlin` (no record); `sleuth_case2002`, `sleuth_ex1605`, `sleuth_case1202` are other Sleuth tables (Lennart kept sleuth_case2002 as `TBD -> 2nd Tier`, `Outdated`). External collections: OpenML 659. +- Metric: none (a regression exercise in a textbook). +- The workbook notes (TabArena "Tiny data"; Lennart and Andrej: "too small") agree on dropping it; size alone would be TP-5, the target decides here. +- Reject check (AI, 2026-10-22): does not stand. The SEL-3 reading is sound (Sleuth2: "the U.S. Commission on Civil Rights investigated charges that insurance companies were attempting to redefine Chicago 'neighborhoods'", https://search.r-project.org/CRAN/refmans/Sleuth2/html/ex1714.html; OpenML 659 "CLASSINDEX: none specific", https://www.openml.org/d/659). But the curator's precedent goes the other way: Lennart kept sleuth_case2002 at TBD -> 2nd Tier although he called it "a real study but for scientific discovery", and today's run left the sibling Sleuth tables (sleuth_ex1605, sleuth_case1202) at TBD -> 2nd Tier, priority 1, with the same SEL-3 counter-argument. A No here alone would be inconsistent; size alone is not a reason (TP-5), and the record had already been moved off No by the size policy. +- For /add-dataset: TBD -> 2nd Tier, priority 1. Treat it like sleuth_ex1605/sleuth_case1202: put the SEL-3 counter-argument (no actor predicts zip-level income; Invol is the inference response) in the record for the curator to settle the Sleuth family in one decision; if it is added, choose a target with a use case (not OpenML's default `income`), and check TP-5 on 47 rows. + OpenML 659, a Sleuth statistics-textbook dataset with only 47 samples, far too small for the benchmark's 3-fold CV protocol. The TabArena curation verdict and two reviewers (Lennart, Andrej) already agree it is too small. It is excluded on size; it is one of a family of tiny Sleuth textbook tables. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/snakes_and_sequences_senegalese_serpent_venom_sequencing_hackathon.md b/curation/records/snakes_and_sequences_senegalese_serpent_venom_sequencing_hackathon.md index 4172c2df..c1618aa8 100644 --- a/curation/records/snakes_and_sequences_senegalese_serpent_venom_sequencing_hackathon.md +++ b/curation/records/snakes_and_sequences_senegalese_serpent_venom_sequencing_hackathon.md @@ -6,7 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - Wrong Domain / Source Modality -- No Good Target / Scientific Discovery tags: - AI-Filled (Verify) collections: @@ -28,4 +27,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Wrong Domain / Source Modality`, SEL-12): de novo peptide sequencing from raw mass spectra; the target is a peptide sequence. +- Source: Zindi hackathon by InstaDeep at Deep Learning Indaba Senegal, 2-6 September 2024, 101 participants, 5,000 USD (https://zindi.africa/competitions/snakes-and-sequences-senegalese-serpent-venom-sequencing-hackathon, read through the Zindi API); as published: Train.csv (507 MB, 100,000 MS2 spectra from InstaNovo's validation set), Test.csv (223 MB, spectra of Senegalese snake venoms), columns precursor m/z, mass, charge, retention time, `mz_array`, `intensity_array`, ten InstaNovo beam outputs and log-probabilities, and `target`, "the ground truth peptide". +- Use case: identifying venom peptides for antivenom research; the task is sequence decoding from spectra, where spectrum models (InstaNovo, a transformer) are the tool. +- Siblings: `african_snake_antivenom_binding_challenge` (UmojaHack 2022, binding, `TBD -> 2nd Tier`, AI) is another task. +- Metric: area under the precision-recall curve of the predicted peptides ranked by confidence (Zindi Evaluation page). +- Reject check (AI, 2026-10-22): stands. SEL-12 (Must), from the source's words: "Each row contains an MS2 spectrum representing a peptide to be identified ... mz_array: Mass-to-charge values of the MS2 spectrum ... intensity_array: Intensity values ... target: This is what you are predicting: the ground truth peptide"; submissions are a peptide string plus a confidence ("ID_3qQWny VSEELED 0.4") (https://api.zindi.world/v1/competitions/snakes-and-sequences-senegalese-serpent-venom-sequencing-hackathon). De novo sequencing from raw spectra is the domain of spectrum models (InstaNovo); the target is a sequence, neither a class nor a number. A tabular 'is beam 0 correct' rescoring task would be our own reframing (SEL-8). + A Zindi hackathon about Senegalese serpent venom sequencing. By its name this is genomic/protein-sequencing biology data (a scientific-discovery / sequence-modeling task) rather than a standard tabular predictive problem with a clear target. The raw modality (sequence data) and discovery framing put it out of scope for representative tabular ML. The record has no field details, so a human should confirm whether any flat tabular target table actually exists, but the strong prior is reject. diff --git a/curation/records/spanish_wine_quality_dataset.md b/curation/records/spanish_wine_quality_dataset.md index 22f0364c..89b29cdd 100644 --- a/curation/records/spanish_wine_quality_dataset.md +++ b/curation/records/spanish_wine_quality_dataset.md @@ -4,14 +4,17 @@ name: Spanish Wine Quality Dataset checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Duplicate - No Good Target / Scientific Discovery +- Not Representative tags: - 2nd Tier / Scientfic Discovery - AI-Filled (Verify) collections: - New (BeyondArena) +original_source: Kaggle +year: '2022' domain: Other required_split: - Random (IID) @@ -29,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a scraped list of Spanish red wines with the average user rating as target; 5,452 of 7,500 rows are exact duplicates and the ratings start at 4.2, so a selected sample with no named decision. +- Source: Kaggle upload by fedesoriano, April 2022 (citation request on the page), "collected by me using web scraping from different sources (from wine specialized pages to supermarkets)", the sources not named; as published: one CSV `wines_SPA.csv`, 7,500 x 11 (winery, wine, year, rating, num_reviews, country, region, price, type, body, acidity). Preview of 2026-10-22: 5,452 exact duplicate rows (about 2,048 unique), `type` missing in 545 rows, `body` and `acidity` in 1,169; rating takes only 4.2 to 4.9 (4.2 in 5,679 rows). +- Use case: the page says "predict either the quality of wine or the prices", with no actor; a retailer pricing a wine, or a buyer screening one, is our reading (approximate fit). The rating is an aggregate of app users' scores, and the 4.2 floor says the scrape kept only top-rated wines (`Not Representative`). Regime IID (one row per wine and vintage; a grouped split by winery is worth a look). +- Siblings and duplicates: `winevivino` (Lennart, `TBD -> 2nd Tier`, "Wine in Spain", a different Kaggle scrape of Vivino) may overlap: body and acidity scores of 1-5 are Vivino's fields, so check by rows before adding; `wine_quality` (Cortez et al., physicochemical tests) is a different task. The old `Duplicate` marker named no original, so it is dropped (PROV-4 needs one). External collections: none found. +- Papers and pages: https://www.kaggle.com/datasets/fedesoriano/spanish-wine-quality-dataset ; no paper. +- Metric: none found (the Kaggle page names "classification or regression tasks" only), so the fallback `rmse` for the rating. +- Risks for /add-dataset: deduplicate first (PROV-13); `num_reviews` and `rating` are scraped at one date (a later state than a new wine's listing); with price as target, the rating is a consequence of the same listing; winery and wine names are high-cardinality strings. +- Priority: gap - (no thin domain), use case - (none named), source - (unnamed scraped sources), shape + (about 2,000 unique rows, mixed types), risk - (73% duplicate rows, a truncated target). + A Kaggle 'Spanish Wine Quality' dataset (fedesoriano) with attributes such as winery, region, type, price and a quality/rating column. The curator already flagged it as likely a duplicate of the many wine datasets and questionable as a real predictive task; quality ratings here are aggregated review scores rather than a natural deployment target. It is a plausible small tabular regression but overlaps heavily with the well-known wine-quality family and is tagged 2nd-tier/scientific-discovery. A human should verify uniqueness vs other wine datasets, the true target (quality vs price), and size after cleaning. --- diff --git a/curation/records/spatio_temporal_beam_level_traffic_forecasting_challenge.md b/curation/records/spatio_temporal_beam_level_traffic_forecasting_challenge.md index c9046c17..6ba80fde 100644 --- a/curation/records/spatio_temporal_beam_level_traffic_forecasting_challenge.md +++ b/curation/records/spatio_temporal_beam_level_traffic_forecasting_challenge.md @@ -27,6 +27,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): forecast future hourly traffic volume per beam from the past five weeks of the same and three companion series. +- Source: Zindi competition by ITU AI for Good, 24 July to 11 October 2024, 742 participants, 12,000 CHF (https://zindi.africa/competitions/spatio-temporal-beam-level-traffic-forecasting-challenge, read through the Zindi API); as published: four wide hourly tables (traffic_DLThpVol, DLThpTime, DLPRB, MR_number; about 35-39 MB each) for 2,880 beams (30 base stations x 3 cells x 32 beams) over five weeks; licence CC-BY-SA 4.0. +- Use case: network resource planning by an operator; the task is a multi-horizon forecast of the series itself (weeks 6 and 11), with no other features than the series. +- TP-10 cannot rescue it: there are no non-lag features; the curator's note that the first place used tree models does not change the task (trees on lags are still forecasting). +- Metric: MAE (Zindi Evaluation page). +- Reject check (AI, 2026-10-22): stands. SEL-10 (Must), from the source's words: "The objective of this challenge is to develop a multivariate time series forecasting model for traffic volume (DLThpVol) at the beam level"; "We remind the participants that the objective is to forecast future values of traffic volume"; target weeks 6 and 11 after five weeks of history, type_of_problem Forecast (https://api.zindi.world/v1/competitions/spatio-temporal-beam-level-traffic-forecasting-challenge). The only files are the four hourly series, with nothing given for the target weeks, so every feature is a lag of the target or of a companion series. TP-10 leg (2) cannot hold, whatever model type the winner used. + A Zindi challenge to forecast spatio-temporal beam-level cellular traffic. The name and the comment ('time-series') make clear this is forecasting future traffic over a horizon, which is explicitly out of scope for the benchmark even though the winning solution used tree models. The forecasting setup (predicting future values per beam/time) is not a fixed predictive task that merely needs a temporal split. Recommend reject; a human could confirm there is no reframable fixed-target variant. --- diff --git a/curation/records/speeddating.md b/curation/records/speeddating.md index bec46dc5..6ad45dc7 100644 --- a/curation/records/speeddating.md +++ b/curation/records/speeddating.md @@ -33,6 +33,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line, TabArena Reject) stands; the research agrees on the target and adds the leak in the rating columns. +- Source: Fisman, Iyengar, Kamenica, Simonson, "Gender Differences in Mate Selection: Evidence From a Speed Dating Experiment", QJE 121(2), 2006 (see Reference); experiments run with Columbia University students in several waves (2002-2004 by the public codebook, not re-checked); the public copy is Gelman's page at Columbia (https://sites.stat.columbia.edu/gelman/arm/examples/speed.dating/) and OpenML 40536 (8,378 date pairs x about 120 columns, target `match`). +- Use case: none fits. The paper is an economics study of stated and revealed preferences; `match` means both partners said yes at the same event, and the ratings of the partner (`attractive_partner`, `like`, `guess_prob_liked` and the like) are filled in on the same scorecard as the decision, so they are not known before it (LEAK, USE-4). A dating service predicting a match before the date would have only the pre-event survey, which is a different, much weaker task that the source did not define. +- Regime: grouped by participant and wave, as the CC says (each person rates 10 to 20 partners); an IID split repeats the same people. +- Siblings and duplicates: none among the records. External collections: TabSTAR; OpenML 40536. +- Metric: none in the paper (it reports regressions, no prediction score), so the fallback `roc_auc` had it shipped. +- No counter-argument: the CC's "interpretability task" matches the paper's aim, which is to estimate gender differences in mate selection, not to predict matches. + The well-known OpenML SpeedDating dataset (Fisman et al. mate-selection experiment) predicting whether a match occurs. It is already in the TabArena Reject collection: data came from a designed multi-wave experiment, the target is influenced by the study design, several features are questionable/leaky ('samerace', 'met'), and it reads as an interpretability/social-science study rather than a deployable predictive task. The grouped/wave structure further complicates honest evaluation. Recommend reject, consistent with prior curation. --- diff --git a/curation/records/speoau_data_hackathon.md b/curation/records/speoau_data_hackathon.md index 7f92adf2..25a6be8b 100644 --- a/curation/records/speoau_data_hackathon.md +++ b/curation/records/speoau_data_hackathon.md @@ -4,6 +4,7 @@ name: speoau-data-hackathon checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Missing source information - TBD @@ -12,9 +13,10 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2023' required_split: -- '?' -problem_type: TBD +- Temporal (NON-IID) +problem_type: Regression source_links: - https://zindi.africa/competitions/speoau-data-hackathon/data source_row: 972 @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a 4-day private student hackathon (SPE OAU chapter, Nigeria, March 2023) to predict "the Actual Price of Electricity at a given time"; the data's origin is not named and the host frames it as price forecasting. +- Source: Zindi, https://zindi.africa/competitions/speoau-data-hackathon (read through the Zindi API on 2026-10-22), 9-12 March 2023, hosted for the Society of Petroleum Engineers OAU student chapter with Chemotronix, 46 participants; as published: Train.csv (5.1 MB), Test.csv (1.4 MB), Variable_description.csv; the test IDs start at ID_027500, so about 27,500 training rows. Columns not readable without a login; the page names no origin of the data. +- Use case: the page: "forecasting in energy markets is identified as one of the highest leverage contribution areas" and "By predicting demand and price in advance, energy companies can optimize their power generation". An actor (a utility or a trader) is named only in general; fit approximate. Regime temporal: a price series over time; whether it is a fixed task with ex-ante inputs (SEL-11) or forecasting of a series (SEL-10) depends on the columns, which could not be read. +- Siblings and duplicates: `speoau_energy_data_hackathon` (same host, 2024, solar PV output: a different task). The size and the column name "actual price" fit a re-upload of a public market-price table (for example the Irish SEM prices that circulate on Kaggle); unconfirmed, check by rows (PROV-4). External collections: none found. +- Papers and pages: none beyond the Zindi page. +- Metric: RMSE, the Zindi Evaluation page ("The error metric for this competition is the Root Mean Squared Error"); a leaderboard metric, deliberate (MET-4); inputs none. +- Risks for /add-dataset: SEL-10 against SEL-11 and TP-10 once the columns are seen (lagged prices as features would make it forecasting); the origin of the data (PROV-1, PROV-2; `Missing source information` kept as the open question); a likely re-upload. +- Priority: gap - (no thin value), use case - (generic), source - (origin not named), shape + (about 38,000 rows in all), risk - (forecasting framing, unknown origin). + A generic Zindi 'speoau-data-hackathon' with no description, comments, or fields in the record. Without any information on the target, features, modality, or size it cannot be assessed, and Zindi student hackathons vary widely (some are valid tabular tasks, others are forecasting/text/image). Per the heuristic, lack of information alone is not grounds for rejection. A human must open the Zindi page to determine the actual task, target, modality and dataset size before deciding. diff --git a/curation/records/speoau_energy_data_hackathon.md b/curation/records/speoau_energy_data_hackathon.md index 039c6bc6..0523d975 100644 --- a/curation/records/speoau_energy_data_hackathon.md +++ b/curation/records/speoau_energy_data_hackathon.md @@ -4,6 +4,7 @@ name: speoau-energy-data-hackathon checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Missing source information - TBD @@ -12,9 +13,10 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2024' required_split: - '?' -problem_type: TBD +problem_type: Regression source_links: - https://zindi.africa/competitions/speoau-energy-data-hackathon/data source_row: 956 @@ -27,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a one-week student hackathon (SPE OAU, Nigeria, 2024): solar PV power output from environmental conditions, 2,920 rows x 10 features; the data's origin is not named. +- Source: Zindi, https://zindi.africa/competitions/speoau-energy-data-hackathon (read through the Zindi API on 2026-10-22), 27 January to 2 February 2024, 23 participants; as published: "The dataset comprises 2920 data points containing 10 features", "30% of the data reserved for testing"; Train.csv (121 KB), Test.csv (49 KB), VariableDescriptions.csv. Features named: "temperature, pressure, wind speed, humidity and other factors"; no site, period or provider named. +- Use case: the page: "predicting the output of solar power systems based on environmental conditions ... for the efficient daily and hourly management of power grid production"; actor a grid or solar-farm operator, in general terms; fit approximate. Regime: temporal if the rows are a time series of one or a few plants (2,920 = 365 x 8, three-hourly for a year, is a guess); the columns could not be read without a login. +- Siblings and duplicates: `speoau_data_hackathon` (same host, electricity price, a different task). Possibly a cut of a public PV table such as the Northern Hemisphere horizontal PV data of Pasion et al. (2020, Mendeley hfhwmn8w24, 12 sites, the same weather variables); unconfirmed, check by rows (PROV-4). External collections: none found. +- Papers and pages: none beyond the Zindi page. +- Metric: RMSE, the Zindi Evaluation page ("The evaluation metric for this competition is Root Mean Squared Error"); deliberate (MET-4); inputs none. +- Risks for /add-dataset: origin (PROV-1, PROV-2; `Missing source information` kept as the open question), a likely re-upload, measured weather at the same time as the output (fine for a nowcast, a leak for a day-ahead forecast), small after the split. +- Priority: gap - (no thin value), use case - (generic), source - (origin not named), shape + (2,920 rows, about 2,000 to train), risk - (unknown origin, likely a cut of another table). + A Zindi 'speoau-energy-data-hackathon' with no details in the record. Energy hackathons frequently turn out to be consumption/generation forecasting, which would be out of scope, but it could also be a fixed predictive task. There is too little information to judge the target, modality, or split regime. A human must inspect the Zindi competition to confirm whether it is forecasting (reject) or a genuine fixed-target tabular regression/classification. diff --git a/curation/records/sports_bet_datasets.md b/curation/records/sports_bet_datasets.md index 315d2625..93b042f5 100644 --- a/curation/records/sports_bet_datasets.md +++ b/curation/records/sports_bet_datasets.md @@ -4,8 +4,9 @@ name: sports bet datasets checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Missing source information +- Needs extensive data wrangling - TBD tags: - AI-Filled (Verify) @@ -25,4 +26,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the link is a review's list of data sources, not one dataset; split it into concrete records before anything else. +- Source: Galekwa, Tshimula, Tajeuna, Kyandoghere, "A Systematic Review of Machine Learning in Sports Betting: Techniques, Challenges, and Future Directions", arXiv 2410.21484 (28 October 2024), section 6.1 "Datasets": for soccer football-data.co.uk, transfermarkt and "more than 200,000 match results from 52 leagues, sourced from the Open International Soccer Database"; for basketball NBA seasons; for tennis Jeff Sackmann's dataset, Tennis-Data.co.uk and the Match Charting Project; cricket, NFL, baseball, horse racing, rugby, golf and hockey sources. As published: no table; each source is its own dataset. +- Use case: a bettor or bookmaker pricing a match outcome before kick-off is a real decision; it holds per source, not for the list. Regime temporal (seasons, matches in time) and grouped by team. +- Siblings and duplicates: no soccer, tennis or betting record exists; NBA records (`nba_draft`, `nba_shots`, `nba_players_2016_2019`, `basketball_c`) are other tasks. External collections: none found. +- Papers and pages: the strongest lead is the Open International Soccer Database (Dubitzky, Lopes, Davis, Berrar, Machine Learning 108, 2019) with the 2017 Soccer Prediction Challenge, scored by the ranked probability score; worth its own record. +- Metric: none for the list; section 6.3 names accuracy, RPS and others per sport. Decide per new record. +- Risks for /add-dataset: not addable as is (`Needs extensive data wrangling`: a collection to split into datasets); match tables need pre-match features only (LEAK) and a temporal split; some sources are scraped sites with unclear terms (a note, never a reason, EV-4). +- Priority: gap - (no thin value as a list), use case - (per source), source - (a review, not the publisher), shape - (no table), risk 0. The old `Missing source information` is dropped: the review names its sources. + An entry pointing only to an arXiv paper section on 'sports bet datasets', with no fields, target, or description in the record. Sports-betting data is frequently framed as odds/ranking or outcome prediction with temporal structure, and the vague 'datasets' framing suggests it may be a collection rather than one well-defined predictive task. There is not enough information to decide scope, uniqueness, or modality. A human must read the arXiv source to identify whether a concrete, well-targeted tabular classification/regression task exists and whether it risks ranking/CTR-style framing. diff --git a/curation/records/stackoverflow.md b/curation/records/stackoverflow.md index 563d7827..2d6993de 100644 --- a/curation/records/stackoverflow.md +++ b/curation/records/stackoverflow.md @@ -30,6 +30,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` (CC line) stands on the text and wrangling load; one counter-argument: "no purpose" does not hold, the host named a use and the decision. +- Source: Kaggle "Predict Closed Questions on Stack Overflow", hosted by Stack Exchange, August to November 2012, 161 teams, 20,000 USD; as published: train.csv with the question at posting time (user and account fields such as reputation at post creation, title, body markdown, up to 5 tags) and the label OpenStatus (open, not a real question, not constructive, off topic, too localized), plus the full 2012-07 Stack Overflow dump (6.7 GB 7z) and leaderboard files. +- Use case: Stack Exchange's blog (https://stackoverflow.blog/2012/08/21/stack-exchange-machine-learning-contest/): "an algorithm that predicts whether (and for what reason) a question will be closed", for "giving users who are composing questions advice on how to improve them while they're composing them", and for better flagging; "everything we knew about them right before they were posted". Actor, moment and decision are named; fit direct. Regime temporal (questions over 2008-2012; the final ranking used fresh questions). +- Siblings and duplicates: `stackoverflow_salary_data` and `stackoverflow_polarity` are other tasks. External collections: none found. +- Papers and pages: Lezina et al. (Uchenye Zapiski Kazanskogo Universiteta 155(4), 2013, pp. 118-133) on the contest, who report user characteristics and topical features as the most important; Stanford CS224d report (lefrankl) scored by the contest's multiclass log loss. +- Metric: multiclass log loss on the Kaggle leaderboard (MET-4); inputs none. +- **Counter-argument (AI):** the CC's facts stand (much of the signal is in the title and body, and the user history needs joins), so `NLP (Text)` and `Needs extensive data wrangling` fit; "no purpose" does not: the host stated the decision above. A tabular cut (user and post metadata plus tags, text left out or reduced to lengths) would be a different, weaker task than the source's, which is why the `No` still holds. The curator has the call. + The Kaggle 'Predict Closed Questions on Stack Overflow' competition data. The curator notes it is dominated by free text, requires heavy feature engineering from user/post IDs and special tag preprocessing, and is described as 'just a collection of data, no purpose.' This is essentially an NLP/text task masquerading as tabular, where text models would dominate and substantial wrangling is needed. Recommend reject as not representative of standard tabular ML. --- diff --git a/curation/records/store_sales_forecasting_challenge_for_beginners.md b/curation/records/store_sales_forecasting_challenge_for_beginners.md index 2a0d693c..68a5f610 100644 --- a/curation/records/store_sales_forecasting_challenge_for_beginners.md +++ b/curation/records/store_sales_forecasting_challenge_for_beginners.md @@ -5,7 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Time-series (Forecasting) +- Duplicate tags: - AI-Filled (Verify) collections: @@ -27,4 +27,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): forecast weekly sales per store and product category for the next eight weeks. +- Source: Zindi beginner hackathon (Young AI Leaders Kampala Hub, Tufuna Technologies), 23 May to 3 June 2025, 113 participants (https://zindi.africa/competitions/store-sales-forecasting-challenge-for-beginners, read through the Zindi API); a child of `grocery_store_product_forecasting_challenge` (AI Hack Tunisia 2022, `No`, `Time-series (Forecasting)`, AI); as published: train.csv (88 MB, "transaction information for 3 years and 6 months"), test.csv, stores, dates, holidays; "anonymised real data", 54 stores and 33 product categories (the counts of the Favorita store-sales data on Kaggle; unconfirmed). +- Use case: stock ordering for small chain stores, from a forecast of the series. +- Metric: RMSE on log1p of the target (Zindi Evaluation page). +- Reject check (AI, 2026-10-22): stands. It stands on PROV-4 (Duplicate), not on SEL-10. The Zindi API lists grocery-store-product-forecasting-challenge as its parent, and all six files have byte-identical sizes (train.csv 87,712,551; test.csv 2,766,947; dates.csv 81,765; holidays.csv 1,822; stores.csv 864; SampleSubmission.csv 520,786) (https://api.zindi.world/v1/competitions/store-sales-forecasting-challenge-for-beginners vs .../grocery-store-product-forecasting-challenge). The SEL-10 half alone does not stand: today's reject check refuted it on the parent (rossmann_store_sales ships as Yes; TP-10 needs the data), and the parent is TBD -> 2nd Tier, priority 2. The marker should be `Duplicate` (PROV-4), like grocery_store_forecasting_challenge_for_azubian, and not `Time-series (Forecasting)`. + A Zindi 'store-sales-forecasting-challenge-for-beginners'. The name makes clear it is a sales-forecasting task (predicting future store sales over time), which is explicitly excluded as time-series forecasting. There is no indication of a reframable fixed predictive target. Recommend reject. diff --git a/curation/records/strikes.md b/curation/records/strikes.md index 1d1a0a10..35b8ede7 100644 --- a/curation/records/strikes.md +++ b/curation/records/strikes.md @@ -32,6 +32,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` stands: a country-year macro panel analysed to compare explanatory models, with no decision that a prediction would serve. +- Source: Bruce Western, StatLib 1999 (OpenML 549, uploaded 2014): "annual observations on the level of strike volume (days lost due to industrial disputes per 1000 wage salary earners), and their covariates in 18 OECD countries from 1951-1985"; 625 rows x 7 columns (country, year, strike volume, unemployment, inflation, social-democratic parliamentary representation, a time-invariant union-centralization measure); "analyzed in ... 'Vague Theory and Model Uncertainty in Macrosociology'" (Sociological Methodology). +- Use case: none named; the paper is about model uncertainty in explaining strike volume, not about predicting it. A real prediction would be next year's strike volume per country, which is forecasting (SEL-10); the same-year covariates make it an explanation exercise (SEL-3). Regime if kept: temporal or grouped by country. +- Siblings and duplicates: OpenML 770 is "Binarized version of the original data set ... computing the mean" (a derived copy, same record). External collections: TabSTAR; TabArena Reject. +- Papers and pages: https://www.openml.org/d/549, https://lib.stat.cmu.edu/datasets/; Western 1996/1999 paper not read. +- Metric: none found (OpenML and StatLib descriptions); the fallback would apply. +- Counter-argument: none found; the marker matches the curator's reason ("Rather interpretability than predictive task"). 625 rows over 18 countries would also be small for a grouped split. + The OpenML 'strikes' regression dataset (volume of strikes by country/year, 1951-1985) from a macrosociology study. It is already in the TabArena Reject collection; the curator notes it is an interpretability rather than predictive task, is small, and needs group/time-based splits. The dataset is tiny and tied to a sociological modeling exercise, not representative real-world tabular prediction. Recommend reject, consistent with prior curation. **Update (policy):** removed the "Too Small" marker — size is not a rejection reason (decided post-Data Foundry); this record stays No on its other ground(s). diff --git a/curation/records/subseafloor_arc.md b/curation/records/subseafloor_arc.md index d4336eba..ffbad8ff 100644 --- a/curation/records/subseafloor_arc.md +++ b/curation/records/subseafloor_arc.md @@ -3,10 +3,9 @@ unique_name: subseafloor_arc name: subseafloor_arc checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Needs extensive data wrangling tags: - Many features - AI-Filled (Verify) @@ -27,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the Archaea amplicon table of a census of marine-sediment microbial diversity; the source publishes no predictive task and the record names no target. +- Source: Hoshino et al., "Global diversity of microbial communities in marine sediment", PNAS 2020 (https://doi.org/10.1073/pnas.1919139117); as published: 16S amplicon sequence variants (Archaea and Bacteria) from "40 globally distributed sampling locations, spanning sediment depths of 0.1 to 678 m", sample metadata in SI Tables 1 and 2 (PDF); the record's OTU-table link is empty. +- Use case: none; the paper's aim is descriptive ("the global taxonomic diversity ... remain unclear. We investigated microbial composition", Significance), and its finding is which environmental factors explain composition. A target such as oxygen presence or organic carbon is measured on the core, not predicted from sequencing; fit none. +- Siblings and duplicates: `subseafloor_arc_2` is the Bacteria table of the same study (its `name` field also says `subseafloor_arc`); the other microbiome records of this set (`bog_lakes`, `bedford_basin_*`, `boso_fish`, `nz_springs` proposed `No` in this re-screen; `australia`, `atl_ocean_transect`, `wastewater_treatment` `No` (AI)). External collections: none found. +- Metric: none (a discovery study). +- Reject check (AI, 2026-10-22): stands. SEL-3: Hoshino et al., PNAS 2020 (https://doi.org/10.1073/pnas.1919139117, PMID 33077589) abstract: the taxonomic diversity "ha[s] been poorly constrained on a global scale. We investigated 299 globally distributed sediment core samples from 40 different sites ... Statistical analysis reveals significant correlations between taxonomic composition, sedimentary organic carbon concentration, and presence or absence of dissolved oxygen": a census, no prediction a decision uses; oxygen and organic carbon are measured on the core. The record has no target and an empty OTU-table link. Same family call as bog_lakes (reject check stood). +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + A subseafloor microbiome dataset (Archaea) built by joining a PDF-supplement metadata table with an OTU abundance table (PNAS 10.1073/pnas.1919139117). This is a microbiome/metagenomics scientific-discovery dataset with many features and substantial wrangling needed to assemble; whether a clean, deployable tabular target (e.g. an environment/site label) exists is unclear. The microbiome/OTU framing leans toward scientific discovery rather than representative tabular ML. A human must define the target, assess sample size after joining (microbiome studies are often very small), and confirm it is not duplicative of subseafloor_arc_2. --- diff --git a/curation/records/subseafloor_arc_2.md b/curation/records/subseafloor_arc_2.md index 88cffe58..d72d6444 100644 --- a/curation/records/subseafloor_arc_2.md +++ b/curation/records/subseafloor_arc_2.md @@ -3,10 +3,9 @@ unique_name: subseafloor_arc_2 name: subseafloor_arc checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Needs extensive data wrangling tags: - Many features - AI-Filled (Verify) @@ -27,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): the Bacteria amplicon table of the same marine-sediment diversity census as `subseafloor_arc`; no predictive task in the source. +- Source: Hoshino et al., PNAS 2020 (https://doi.org/10.1073/pnas.1919139117); 40 sampling locations, sediment depths 0.1-678 m; metadata in SI Tables 1 and 2 (PDF), the record's OTU-table link is empty. The record's `name` reads `subseafloor_arc` though the note says Bacteria. +- Use case: none; the paper describes composition and the environmental factors that explain it (Significance). Fit none. +- Siblings and duplicates: `subseafloor_arc` (Archaea, same samples and metadata): a second cut of one study that adds no distinct task (PROV-6) even if a target were found. +- Metric: none (a discovery study). +- Reject check (AI, 2026-10-22): stands. SEL-3 as for subseafloor_arc (same paper, Bacteria table of the same 299 samples, https://doi.org/10.1073/pnas.1919139117); also PROV-6, a second cut of one study adding no distinct task. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + The Bacteria counterpart of the subseafloor microbiome project (same PNAS source 10.1073/pnas.1919139117), again assembled by joining PDF-supplement metadata with an OTU table. Same concerns apply: many features, heavy wrangling, microbiome scientific-discovery framing, and an unclear/likely-small sample size with no defined deployable target. It also overlaps strongly with subseafloor_arc (Archaea), raising a near-duplicate question. A human must define the target, check sample size, and decide whether the Archaea/Bacteria variants are distinct enough to keep. --- diff --git a/curation/records/sustainable_development_goals_sdgs_text_classification_challenge.md b/curation/records/sustainable_development_goals_sdgs_text_classification_challenge.md index 9394fcf2..8663714d 100644 --- a/curation/records/sustainable_development_goals_sdgs_text_classification_challenge.md +++ b/curation/records/sustainable_development_goals_sdgs_text_classification_challenge.md @@ -28,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`NLP (Text)`, SEL-12): a multi-label classification of raw English text (contract names, news titles, organisation and opportunity descriptions) into SDG 3 indicators; language models are the tool. +- Source: Zindi competition hosted by Devex, 5 Sept to 13 Nov 2018 (https://zindi.africa/competitions/sustainable-development-goals-sdgs-text-classification-challenge, read through Zindi's page API); as published: devex_train.csv with ID, Type, Text and Label_1 to Label_12 ("manually classified by Devex analysts"), about 3,000 texts; 27 indicator labels. +- Use case: Devex tags development projects and documents by SDG indicator to "align resources, find partners" (Description); fit direct, but the input is the text itself. +- Siblings and duplicates: none found in `curation/records/`. +- Metric: Hamming loss over 27 labels ("The Hamming Loss metric be used", Evaluation tab), multi-label (also no single-target task). +- Reject check (AI, 2026-10-22): stands. SEL-12: Zindi Data Description: "Text: text to be classified ... The text contains some HTML mark-up. Dealing with this 'dirtiness' of the text is part of the challenge. All the text is in English"; Description: "approximately 3,000 web-scraped text from tenders, programs, and documents, as well as news articles"; type_of_problem 'Natural Language Processing' (api.zindi.world/v1/competitions/sustainable-development-goals-sdgs-text-classification-challenge). The only other input is the Type column; language models are the tool. Multi-label over 27 indicators (Hamming loss) as well. + A Zindi 'SDGs text classification challenge'. By name and design this is a text-classification task (classifying documents into Sustainable Development Goals), where NLP/text models dominate, so it is not representative of standard tabular ML. Recommend reject as a wrong-modality (text) task. diff --git a/curation/records/susy.md b/curation/records/susy.md index 30d47b82..80ff1e32 100644 --- a/curation/records/susy.md +++ b/curation/records/susy.md @@ -3,7 +3,7 @@ unique_name: susy name: SUSY checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - AHDS (Artifical/Handmade/Deterministic/Simulated) tags: @@ -29,6 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16): simulated collider events, the second benchmark of the paper behind `higgs`, which the curator rejected on the same ground. +- Source: Baldi, Sadowski & Whiteson 2014 (arXiv 1402.4735), UCI donated 2014-02-11 (https://archive.ics.uci.edu/dataset/279/susy): "The data has been produced using Monte Carlo simulations"; 5,000,000 events, 8 low-level and 10 derived features, label = simulated signal (chargino pair) vs background. +- Use case: physics analyses train on simulation, but the benchmark scores simulation-vs-simulation separability on balanced classes, as for `higgs` (CC 2026-09-22, Lennart). +- Siblings and duplicates: `higgs` (`No`, Lennart; SEL-16 Cases), `hepmass`, `higgs_tautau` (same group and pipeline). Not a duplicate of `higgs` (different process), but the same verdict applies. +- Metric: AUC reported in the paper (one of several measures). +- Reject check (AI, 2026-10-22): stands. SEL-16 (Must, Cases: Higgs): UCI https://archive.ics.uci.edu/dataset/279/susy: "The data has been produced using Monte Carlo simulations"; the second benchmark of the same paper (arXiv 1402.4735) and pipeline as higgs, which Lennart rejected (CC 2026-09-22) as simulated physics with dedicated benchmarks scored as balanced sim-vs-sim AUC. No fact sets SUSY apart (like cases alike). + Second benchmark of the same paper as `higgs` (Baldi, Sadowski & Whiteson 2014, arXiv 1402.4735, "Benchmark Case for Supersymmetry Particles (SUSY)", p. 4-5): 5M fully simulated 8 TeV events (MadGraph5 + Pythia + Delphes), signal = chargino pair production "chi+- -> W chi0" with "m chi+- = 200 GeV and m chi0 = 100 GeV", background = W-pair production; final state two leptons + missing momentum. 8 low-level + 10 high-level features, 46% signal, last 500k rows = test set (UCI, donated 2014-02-11, CC BY 4.0). Features are globally standardized the same way as `higgs` (p. 10), same judgement applies. Paper reports little gain from deep nets or high-level features here (AUC ~0.87-0.88, Table II, p. 8), i.e. a smoother, less feature-hungry task than `higgs`. Not a duplicate of `higgs` (different process and feature set), but the same pipeline and the same curation question, so the verdict should follow whatever is decided for `higgs` (see its 2026-09-22 comment: simulated physics with dedicated benchmarks, criterion 4B). Data not inspected here beyond the UCI page. diff --git a/curation/records/tara_oceans.md b/curation/records/tara_oceans.md index c4c44701..0625ae10 100644 --- a/curation/records/tara_oceans.md +++ b/curation/records/tara_oceans.md @@ -3,10 +3,9 @@ unique_name: tara_oceans name: tara_oceans checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Needs extensive data wrangling tags: - Many features - AI-Filled (Verify) @@ -27,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): taxonomic profiles of Tara Oceans samples joined to their metadata, published to describe ocean-microbiome structure and its ecological determinants; no predictive task, no target in the record. +- Source: Sunagawa et al., "Structure and function of the global ocean microbiome", Science 2015 (https://doi.org/10.1126/science.1261359); companion site http://ocean-microbiome.embl.de/companion.html (Tables W1 and W8, miTAG.taxonomic.profiles.release.tsv); 243 samples from 68 locations, 139 prokaryote-enriched. +- Use case: none; the abstract's aim is "a global picture of ... microbial community structure, and their ecological determinants". Depth, temperature or region are measured at sampling, not predicted from sequencing; fit none. About 140-240 samples would also be small. +- Siblings and duplicates: the microbiome set (`subseafloor_arc`, `subseafloor_arc_2`, `bog_lakes`, `bedford_basin_*`, `boso_fish`, `nz_springs` proposed `No` in this re-screen; `australia`, `atl_ocean_transect` `No` (AI)). External collections: none found. +- Metric: none (a discovery study). +- Reject check (AI, 2026-10-22): stands. SEL-3: Sunagawa et al., Science 2015 (https://doi.org/10.1126/science.1261359, PMID 25999513), "Structure and function of the global ocean microbiome": a description of community structure and its drivers ("epipelagic community composition mostly driven by temperature"); no prediction a decision uses, depth/temperature/region are measured at sampling, and the record names no target. Same family call as bog_lakes. +- Collection (AI, 2026-10-22): this record is one of the environmental-metabarcoding datasets of Zschaubitz et al. 2025, "A benchmark analysis of feature selection and machine learning methods for environmental metabarcoding datasets" (CSBJ, doi 10.1016/j.csbj.2025.04.017), whose Table 1 fixes one target per dataset (a measured environmental variable such as depth, pH, temperature or a nutrient). The collection picked targets "based on an expectation that there is a correlation" to benchmark feature selection, not for a decision, and the targets are measured at sampling anyway; so the reject holds under SEL-3 and SEL-8 (reject check 6). + The Tara Oceans marine microbiome dataset (Science 10.1126/science.1261359), assembled by joining companion-website metadata tables (W1/W8) with a 16S OTU abundance table. As with the subseafloor records, this is a metagenomics/microbiome scientific-discovery resource with many features and heavy wrangling; the existence of a clean, deployable tabular target and adequate sample size is uncertain. The OTU/microbiome framing tends toward scientific discovery rather than representative tabular ML. A human must define a target (e.g. ocean region/depth/biome), check post-join sample size, and confirm it is not better treated as discovery. --- diff --git a/curation/records/telangana_crop_health_challenge.md b/curation/records/telangana_crop_health_challenge.md index 445ac8d4..2fc704f1 100644 --- a/curation/records/telangana_crop_health_challenge.md +++ b/curation/records/telangana_crop_health_challenge.md @@ -7,6 +7,7 @@ checked_by: data_foundry_status: - WIP (Triage) suggestion: TBD -> Yes +priority: 3 decision_markers: - TBD tags: @@ -32,6 +33,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** recommend keeping `TBD -> Yes` with priority 3; the inputs are a real cultivation-practice table, with Sentinel-2 imagery as an optional extra that the benchmark version should leave out. The curator has the call. +- Source: Zindi competition by GIZ (FAIR Forward), the Government of Telangana and ADeX, 8 Nov 2024 to 10 Feb 2025 (https://zindi.africa/competitions/telangana-crop-health-challenge, read through Zindi's page API); data_type "Structured, Satellite"; as published: Train.csv, Test.csv, VariableDefinitions.docx; "The tabular data includes information on cultivation practices such as crop type, sowing and irrigation methods, past yield, and crop coverage (boundaries)", with a notebook to pull Sentinel-2 time series "from sowing ... to harvest"; data also on the ADeX platform, CC-BY-SA 4.0. +- Use case: "This model will help both farmers and government bodies monitor crop conditions and take proactive measures" (Description); fit approximate (the label is the field's condition over the season, and the season's imagery runs to harvest). Regime: grouped by district (the fairness metric is computed "based on crop district"), as the record has. +- Siblings and duplicates: none found in `curation/records/` (other GIZ/ADeX Telangana challenges, if any, have no record). +- Metric: multiclass F1 ("The error metric for this competition is F1 Multi Class", Evaluation tab; the averaging is not stated), plus a fairness score by district for the final rank; inputs: the district column for the fairness part (MET-1, MET-5). +- Risks for /add-dataset: keep only columns known before the season's outcome; Sentinel-2 features would be image-derived (SEL-13); the crop-coverage boundaries are geometry; the four classes are probably imbalanced (Healthy majority); check the F1 averaging. +- Priority: gap + (Grouped regime thin), use case + (named by the source), source 0 (a competition page; ADeX not read), shape + (field-level table with mixed types and districts), risk - (imagery may carry the signal). + A Zindi 'Telangana Crop Health Challenge' to predict crop health, already flagged Non-IID (Grouped) and marked WIP by curator Andrej (data_foundry_status WIP (Triage)). Crop-health challenges can be either tabular (field/sensor/weather features) or remote-sensing image based, so the modality is the key open question. Given it is actively being worked on and has a clear classification target with a grouped split, it is a plausible Yes. A human must verify the input modality is genuinely tabular (not satellite imagery) and confirm the target and group/split structure. --- diff --git a/curation/records/the_ultimate_hacking_challenge_by_uct_cryptocurrency_and_ai_society.md b/curation/records/the_ultimate_hacking_challenge_by_uct_cryptocurrency_and_ai_society.md index c03880be..a693bbe0 100644 --- a/curation/records/the_ultimate_hacking_challenge_by_uct_cryptocurrency_and_ai_society.md +++ b/curation/records/the_ultimate_hacking_challenge_by_uct_cryptocurrency_and_ai_society.md @@ -4,6 +4,7 @@ name: The Ultimate Hacking Challenge by UCT Cryptocurrency and AI Society checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 decision_markers: - TBD tags: @@ -11,12 +12,15 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: insurance required_split: - Random (IID) problem_type: Binary Classification source_links: - https://zindi.africa/competitions/uct-datathon-the-ultimate-hacking-challenge/data +- '# first publication: Zindi Insurance Prediction Challenge (DSN Bootcamp 2019); this record''s link is a 2020 private re-run' +- https://zindi.africa/competitions/insurance-prediction-challenge source_row: 1016 needs_review: - ai_unverified @@ -27,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: a building-insurance claim classification (at least one claim over the insured period), first published as Zindi's 2019 DSN "Insurance Prediction Challenge"; this record is a later private re-run of it, and the data's origin is undocumented. +- Source: first publication Zindi "Insurance Prediction Challenge", "A learning competition designed for DSN Bootcamp 2019", from 19 Nov 2019 (https://zindi.africa/competitions/insurance-prediction-challenge); the same brief re-hosted as the DSN AI+ Unilorin hackathon (June 2020, https://zindi.africa/competitions/insurance-prediction) and this UCT Cryptocurrency and AI Society hackathon (19 Sept 2020, https://zindi.africa/competitions/uct-datathon-the-ultimate-hacking-challenge, private, 25 participants); all read through Zindi's page API. As published: train_data.csv, test_data.csv, VariableDescription.csv; target `Claim` "1 if the building has at least a claim over the insured period"; "The model will be based on the building characteristics". Rows and columns not counted (login). +- Use case: "Olusola Insurance Company offers a building insurance policy ... You have been appointed as the Lead Data Analyst to build a predictive model" (Description): an insurer prices or underwrites a building policy; a role-play scenario, no named real insurer; fit approximate. Regime: likely temporal if a year-of-observation column ships (check), else IID; the record's `Random (IID)` is kept for now. +- Siblings and duplicates: no record for the 2019 original or the Unilorin re-host; the record should point to the 2019 competition (rename or note). Other building/vehicle-claim records: `autoinland_vehicle_insurance_claim_challenge`, `axa_vehicle_insurance_claim_challenge`, `monthly_insurance_claim_prediction_challenge` (different data). External collections: none found. +- Papers and pages: no paper; Zindi pages only. +- Metric: ROC AUC ("The evaluation metric for this competition is the Area Under the ROC curve (AUC)", Evaluation tab of all three hosts); a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: provenance: who supplied the data to DSN is not stated (PROV-2); an insured-period column may be partly known only after the period (LEAK); the year of observation decides the split. +- Priority: gap + (insurance thin), use case 0 (a scenario, not a named actor), source 0 (no codebook or publisher), shape + (building characteristics of mixed types), risk - (undocumented origin, several re-hosts). + A UCT student datathon on Zindi whose stated task is to predict whether a building will have an insurance claim. This is a clear binary classification problem on what is most likely tabular building/insurance features, which fits the benchmark scope. The record lacks field-level details, so a human should confirm the modality is tabular, the dataset size after cleaning is adequate, and the appropriate split regime. Provisionally a plausible Yes. --- diff --git a/curation/records/the_zimnat_insurance_assurance_challenge.md b/curation/records/the_zimnat_insurance_assurance_challenge.md index 3889968f..e74f9250 100644 --- a/curation/records/the_zimnat_insurance_assurance_challenge.md +++ b/curation/records/the_zimnat_insurance_assurance_challenge.md @@ -4,17 +4,20 @@ name: The Zimnat Insurance Assurance Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: -- TBD +- Needs extensive data wrangling tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2020' domain: insurance required_split: - Temporal (NON-IID) problem_type: Binary Classification +original_data_state: Database (or multiple to-be-joined tables) source_links: - https://zindi.africa/competitions/the-zimnat-insurance-assurance-challenge source_row: 1025 @@ -27,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: a real insurer's policy-lapse task with a clear temporal structure; the release is several tables and the public labels are lapse years 2017-2019, so the benchmark target has to be rebuilt (for example lapse in 2019 among policies active at the end of 2018). +- Source: Zindi #ZindiWeekendz hackathon with Zimnat (Zimbabwe), 22-24 May 2020 (https://zindi.africa/competitions/the-zimnat-insurance-assurance-challenge, read through Zindi's page API); as published: "51,685 life assurance policies", client_data.csv (location, branch, agent code, age), policy_data.csv ("multiple rows for each Policy ID since policies can cover more than one person"), payment_history.csv ("up to the end of 2018"), train.csv (Lapse = 1 and the year for lapses in 2017, 2018 or 2019; '?' for policies still active at the end of 2019, which form the test set for 2020); "Numerical quantities have been transformed, and many categories have been assigned unique IDs". +- Use case: the insurer scores active policies for lapse so "churn prediction can be used to offer targeted support and tailored services to certain customers vulnerable to churning" (Description); fit direct for a yearly scoring run. Regime temporal: lapses by year, payment history cut at a date. +- Siblings and duplicates: `zimnat_insurance_recommendation_challenge` (another Zimnat task, a product recommendation, `No` (AI)); `zimnat_loan_default_predictio` (another task). Not a duplicate. External collections: none found. +- Papers and pages: no paper; the winner interview is linked from the Description. +- Metric: log loss on the lapse probability ("The evaluation metric for this challenge is Log Loss", Evaluation tab); a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: joins across three tables with many-to-one policy rows; the 2020 labels are not public, so the target and cut-off must be built from the lapse years with features only up to the cut-off (payment history ends 2018, which fits a 2019 target); transformed values; a lapse year is known only after it. +- Priority: gap + (insurance thin), use case + (named by the source), source 0 (a competition page), shape + (51k policies, mixed tables, temporal), risk - (joins and a rebuilt target). + A Zindi challenge (Zimnat, Zimbabwe) to predict when an insurance policy will lapse. Predicting policy lapse from customer/policy attributes is a standard tabular insurance classification (or possibly time-to-event) task that fits the benchmark scope. The 'when' framing may imply a temporal/duration target, so a human must confirm whether it is a clean binary lapse classification or a survival/temporal task, plus the modality, size, and split. Provisionally a plausible Yes. --- diff --git a/curation/records/titanic.md b/curation/records/titanic.md index 7ff81468..4192bcc2 100644 --- a/curation/records/titanic.md +++ b/curation/records/titanic.md @@ -33,6 +33,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the curator's `No` stands, on the ground the CC line gives ("Not a real predictive task"): no one makes a decision with a survival prediction for the passengers of a ship that sank in 1912. +- Source: passenger list of the RMS Titanic (1912) with survival, compiled from Eaton & Haas (1994) and Encyclopedia Titanica; the common 1,309-row version (Harrell's titanic3, OpenML 40945) and Kaggle's tutorial split (891 + 418). Countless re-uploads. +- Use case: none; the table is used to teach and to look back at who survived (SEL-3; `passenger_list_for_the_estonia_ferry_disaster` was judged the same way: "Like Titanic, a scientific discovery task", Lennart). Regime if kept: IID. +- Siblings and duplicates: `passenger_list_for_the_estonia_ferry_disaster` (`TBD -> 2nd Tier`, Lennart). External collections: TabSTAR; TabArena Reject. +- Metric: Kaggle's tutorial competition scores accuracy (a tutorial's habit, not a decision's metric). +- Counter-argument (markers only): `Trivial` has no evidence from a task probe (TP-4); a sex-and-class rule is strong but not a perfect solve. `No Good Target / Scientific Discovery` matches the stated reason better than `Trivial`; the curator has the call. + The ubiquitous Titanic survival dataset, already in the TabArena Reject collection. The curator notes it is a toy/interpretability example with no real deployment value (a survival predictor for a sinking ship is not a meaningful predictive task), and it is one of thousands of re-uploaded copies. It is small, toy, and not representative of real-world tabular ML. Recommend reject, consistent with prior curation. --- diff --git a/curation/records/total_energies_revolution_hackathon.md b/curation/records/total_energies_revolution_hackathon.md index a9ce9d8f..0037c7ff 100644 --- a/curation/records/total_energies_revolution_hackathon.md +++ b/curation/records/total_energies_revolution_hackathon.md @@ -3,10 +3,9 @@ unique_name: total_energies_revolution_hackathon name: total-energies-revolution-hackathon checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- Missing source information -- TBD +- Duplicate tags: - Non-IID (Temporal) - AI-Filled (Verify) @@ -30,4 +29,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the TotalEnergies qualifier (Uganda, 2023) re-runs the Mobiticket "Traffic Jam" task of 2018 on the same ticket data with the same metric; the finals have no data. +- Source: Zindi "TotalEnergies rEVolution Hackathon" (2023-07-25 to 2023-08-25, a qualifier for Ugandan teams, 405 participants) and its finals (2023-09-18 to 20, 61 participants), read through the Zindi API. The finals' brief is EV charging locations in Kampala, judged by a panel ("there are no leaderboard scores"), with a 433-byte zip as the only file. The qualifier's own text is the Nairobi one: "predicts the number of seats that Mobiticket can expect to sell for each ride", Train.csv "tickets purchased from Mobiticket for the 14 routes ... between 17 October 2017 and 20 April 2018" with `ride_id`, `seat_number`, `payment_method`, `payment_receipt`, `travel_date`, `travel_time`, `travel_from`, `travel_to`, `car_type`, `max_capacity`; Test.csv the rides of 21 April to 9 May 2018; MAE on `number_of_ticket`. +- Use case and regime: those of `traffic_jam_predicting_peoples_movement_into_nairobi` (Mobiticket demand per ride, temporal). +- Siblings and duplicates: the original is `traffic_jam_predicting_peoples_movement_into_nairobi` (Zindi 2018, the same company data, task, columns, date range and metric; screened `TBD -> 2nd Tier` in this batch). No distinct task (PROV-6, PROV-7 does not apply). +- Metric: MAE (the qualifier's Evaluation page), the same as the original. +- The old `Missing source information` marker is refuted: the source is the 2018 Mobiticket release. `domain` `Other` and the energy reading came from the title only. +- Reject check (AI, 2026-10-22): stands. PROV-4 (Must): the qualifier's Data Description and Evaluation pages are word for word those of traffic-jam-predicting-peoples-movement-into-nairobi (Zindi 2018): "This challenge asks you to build a model that predicts the number of seats that Mobiticket can expect to sell for each ride ... There are 14 routes"; "The Mean Absolute Error will be used", submission ride_id,number_of_ticket (api.zindi.world/v1/competitions/total-energies-revolution-hackathon vs .../traffic-jam-predicting-peoples-movement-into-nairobi). Same source, same task, so no distinct task (PROV-7 does not apply); the EV-charging finals are judged by a panel with no data. + A Zindi 'Total Energies Revolution Hackathon' (with a finals stage) tagged Non-IID (Temporal) but otherwise with no description in the record. The energy domain plus the temporal tag raise a real risk that this is a forecasting task, which would be out of scope, but it could also be a fixed-target predictive problem needing a temporal split. There is too little information to decide the target, modality, or whether it is forecasting. A human must inspect the Zindi pages to determine the actual task before deciding. diff --git a/curation/records/traffic_jam_predicting_peoples_movement_into_nairobi.md b/curation/records/traffic_jam_predicting_peoples_movement_into_nairobi.md index 863c656d..27438da6 100644 --- a/curation/records/traffic_jam_predicting_peoples_movement_into_nairobi.md +++ b/curation/records/traffic_jam_predicting_peoples_movement_into_nairobi.md @@ -3,7 +3,8 @@ unique_name: traffic_jam_predicting_peoples_movement_into_nairobi name: traffic-jam-predicting-peoples-movement-into-nairobi checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - TBD tags: @@ -11,12 +12,14 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2018' domain: business & marketing required_split: - Temporal (NON-IID) problem_type: Regression source_links: - https://zindi.africa/competitions/traffic-jam-predicting-peoples-movement-into-nairobi/data +- https://zindi.africa/competitions/traffic-jam-predicting-peoples-movement-into-nairobi source_row: 922 needs_review: - ai_unverified @@ -27,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: tickets sold per scheduled bus ride into Nairobi, a real operator task, but the inputs only index the route, date and time, so it is a forecasting-like demand panel that TP-10 has to clear. +- Source: Zindi competition "Traffic Jam: Predicting People's Movement into Nairobi" (2018-09-06 to 2019-01-14, 1,517 participants), sponsored by Uber, Mobiticket and insight2impact; the data is Mobiticket's ticket sales (read through the Zindi API, `api.zindi.africa/v1/competitions/traffic-jam-predicting-peoples-movement-into-nairobi`). As published: one row per ticket for 14 routes "between 17 October 2017 and 20 April 2018" with `ride_id`, `seat_number`, `payment_method`, `payment_receipt`, `travel_date`, `travel_time`, `travel_from`, `travel_to`, `car_type`, `max_capacity` (train zip 0.68 MB); the target `number_of_ticket` per `ride_id` has to be counted from the ticket rows; the test file holds the rides of the following weeks without the ticket columns. Uber Movement hourly travel times were an allowed external source (the platform has since closed). Rows and rides not counted here. +- Use case: Mobiticket and the bus operators, before a scheduled departure, decide on vehicles and promotions ("anticipate customer demand for certain rides, to manage resources and vehicles more efficiently"); fit direct for the count per ride. Regime temporal: the test is the period after the training window, and demand per route and slot drifts. +- Siblings and duplicates: `total_energies_revolution_hackathon` (2023 qualifier) re-runs this same task on the same Mobiticket files (proposed as its duplicate in this batch); the Yango mobility hackathons (`2024_lusaka_mobility_prediction_hackathon` and others) are other data. External collections: none found. +- Papers and pages: the Zindi pages; a public tutorial blog by Johnowhitaker linked from the data page; no paper. +- Metric: MAE on `number_of_ticket` per ride (Zindi Evaluation page: "The Mean Absolute Error will be used"), a competition's deliberate choice (MET-4); inputs none. +- Risks for /add-dataset: the target is built by counting ticket rows (rides with zero sales are absent from train, USE-7); the features are route, departure date and time, vehicle type and capacity only, so it is demand per series slot (SEL-10 against SEL-11): run TP-10 (a per-route-and-slot naive profile against a model) before shipping; `payment_method`, `seat_number` and `payment_receipt` are per-ticket and not known before sales (LEAK checks). +- Priority: gap 0 (transport demand, temporal regime not thin), use case + (operator and decision named by the source), source + (company data, variables documented on the competition page), shape 0 (some thousands of rides but five features that only index the series), risk - (forecasting-like, SEL-10 / TP-10). + A well-known Zindi 'Traffic Jam: Predicting People's Movement into Nairobi' challenge, where the task is to predict the number of seats/tickets sold for bus rides into Nairobi from route and time features. This is a concrete tabular regression with a clear count target, plausibly fitting the benchmark scope, with a temporal structure (ride dates) suggesting a temporal split. A human should confirm the exact target (seats sold per ride), the modality is tabular, dataset size, and the correct split regime. Provisionally a plausible Yes. diff --git a/curation/records/traffic_violations.md b/curation/records/traffic_violations.md index 16bcfa28..94711aa0 100644 --- a/curation/records/traffic_violations.md +++ b/curation/records/traffic_violations.md @@ -4,6 +4,7 @@ name: Traffic_violations checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - No Good Target / Scientific Discovery tags: @@ -33,6 +34,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: a real county traffic-stop log, but the OpenML target (citation, warning or repair order) is the officer's own disposition, which no actor predicts, and the charge columns largely set it. +- Source: Montgomery County, Maryland, "all electronic traffic violations issued in the County" (open data portal, mirrored on data.gov; the CC line counts about 1.5M rows first, 2M now). OpenML 42345 (`Traffic_violations` v5, uploaded 2020-04-03, CC0): "downsampled to 5% of the original size", "some features were removed", 70,340 rows x 21 columns, target `Violation.Type` with 3 classes (34,382 in the largest, 3,506 in the smallest; "SERO and ESERO were collapsed"); columns `Description`, `Charge`, `Belts`, `Personal.Injury`, `Property.Damage`, `Commercial.License`, `Commercial.Vehicle`, `State`, `VehicleType`, `Year`, `Make`, `Model`, `Color`, `Contributed.To.Accident`, `Race`, `Gender`, `Driver.City`, `Driver.State`, `DL.State`, `Arrest.Type`. The Kaggle link is a later re-upload. +- Use case: none stated; the OpenML version was cut as a high-cardinality categorical benchmark. Who would predict, at a stop, whether the officer writes a citation or a warning? The officer decides it; a model of it describes enforcement practice (a look back, SEL-3, USE-2). A curated alternative (predict `Contributed.To.Accident` or `Personal.Injury`) would be ours. Fit approximate at best. Regime: the stops are dated in the county data (the OpenML cut dropped the date), so temporal if the date is restored. +- Siblings and duplicates: none among `curation/records/` (grep traffic violation, Montgomery; `employee_salaries` is another Montgomery County table). External collections: TabSTAR; tagged `TabArena Reject`. +- Papers and pages: OpenML 42345; the county open-data page; no paper that defines a task found. +- Metric: none found (OpenML description, data.gov entry), so the fallback. +- Risks for /add-dataset: `Charge` and `Description` (the statute and its text) likely determine the repair-order class and much of the rest (SEL-5, TP-4); `Race` and `Gender` as inputs to a model of police enforcement need a look under SEL-1 (no objection by the source found); the 5% OpenML cut versus the full county data (PROV-6, PROV-13, use the origin); free-text `Description` (SEL-12). +- Priority: gap 0 (`domain` says social science, which is thin, but a police record fits it only loosely; not counted), use case 0, source + (a county's open data with a column guide), shape + (70k-2M rows, mixed categorical), risk - (no decision for the target; charge columns may fix it). + Maryland traffic-violations log (~1.5-2M rows) with spatial and temporal fields, available on OpenML/data.gov/Kaggle. The data is large and genuinely tabular, but the CC comment flags that it may simply have been crawled because it is available and may not correspond to a meaningful predictive task; columns like violation type, race, gender, or fine could each be a target. It carries TabArena Reject and 2nd Tier/Scientific Discovery tags. A human must define a non-arbitrary target, confirm a sensible temporal split, and check for leakage and ethical concerns (race/demographic fields), so a provisional 2nd-tier hold is appropriate. --- diff --git a/curation/records/trec_2022_clinical_trials_dataset.md b/curation/records/trec_2022_clinical_trials_dataset.md index 20dc72a7..67411b03 100644 --- a/curation/records/trec_2022_clinical_trials_dataset.md +++ b/curation/records/trec_2022_clinical_trials_dataset.md @@ -28,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Out-of-scope Task (CTR/RecSys/Ranking)`, SEL-9; `NLP (Text)`, SEL-12): the TREC 2022 Clinical Trials track, a retrieval task that ranks trials for free-text patient cases. +- Source: NIST, TREC 2022 Clinical Trials track (data.gov entry; trec-cds.org/2022.html): "given a free text summary of a patient health record, find suitable clinical trials for that patient". As published: 50 topics, each "a lengthy (5-10 sentence) patient case description", "synthetic patient cases created by individuals with medical training"; the corpus is an April 2021 snapshot of ClinicalTrials.gov in XML; qrels from physicians who judge each pooled trial eligible, excluded or not relevant. +- Use case: trial matching for a patient, solved as retrieval: systems return a ranked list of up to 1,000 trial IDs per topic. No tabular table and no per-row target (the qrels label topic-trial pairs from pooled runs). +- Siblings and duplicates: `clinical_trials` and `covid_trials` are other trial-registry records (different data); no duplicate. +- Metric: NDCG over the ranked list ("the performance of the retrieval submissions will be measured using normalized discounted cumulative gain"). +- Reject check (AI, 2026-10-22): stands. SEL-9 (Must): trec-cds.org/2022.html: "Participants of the track will be challenged with retrieving clinical trials from ClinicalTrials.gov", runs are "a ranked list of up to one thousand IDs", "measured using normalized discounted cumulative gain (NDCG)"; topics are free-text synthetic admission notes (SEL-12). An IR ranking task on text with no row-level target. + The TREC 2022 Clinical Trials track is an information-retrieval / ranking task: given a patient case description (free text), rank/match eligible clinical trials. This is a text-based IR/ranking problem, not an IID/temporal/grouped tabular predictive classification or regression task. It falls under out-of-scope (ranking/IR) and NLP. No clear tabular target exists, so this should be rejected. diff --git a/curation/records/tunisian_fraud_detection_challenge.md b/curation/records/tunisian_fraud_detection_challenge.md index c639e7d8..44e09c84 100644 --- a/curation/records/tunisian_fraud_detection_challenge.md +++ b/curation/records/tunisian_fraud_detection_challenge.md @@ -4,16 +4,19 @@ name: Tunisian Fraud Detection Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: finance required_split: - '?' -problem_type: Binary Classification +problem_type: Regression source_links: +- https://zindi.africa/competitions/tunisian-fraud-detection - https://zindi.africa/competitions/supcom-tunisian-fraud-detection-challenge source_row: 1015 needs_review: @@ -25,6 +28,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: taxpayer-year data of the Tunisian Ministry of Finance, the target the tax liability an inspection adjusted; a real audit-selection task, scored by RMSE. +- Source: Zindi "Tunisian Fraud Detection Challenge" (`tunisian-fraud-detection`, opened 2019-09-20, a knowledge competition, 1,529 participants), data "provided by the Tunisian Ministry of Finance includes variables about tax analysis, taxpayer inspection, and VAT returns"; "a subset of over 25,000 samples aggregated by year", "an anonymized dataset containing a large number of numeric variables"; "The "TARGET" column ... equals the amount of the tax liability that the taxpayer has to adjust". Files: Train.csv (9,679,520 B), Test.csv, VariableDescription.csv ("Full list of variables and their explanations"). The linked `supcom-tunisian-fraud-detection-challenge` (2020-09, a private Sup'Com student hackathon, 71 participants, "over 21,295 samples") ships files of the same sizes: the same data, a later re-run. Both read through the Zindi API. +- Use case: the tax authority, after returns are filed, picks taxpayers to inspect ("a supervised machine learning technique that detects potential fraudulent taxpayers will increase the operational efficiency of the tax supervision process"); fit direct when the features are return-time data, approximate if inspection-stage variables are in. Regime: grouped by taxpayer or temporal by year, to be read from the columns (rows are "aggregated by year"; whether an id or year ships is not checked); `required_split` left at `?`. +- Siblings and duplicates: none among `curation/records/` (grep tax, Tunisia: `fraud_detection_in_electricity_and_gas_consumption` is STEG electricity fraud, another task; `indabax_tunisia_2023` another hackathon). External collections: none found. +- Papers and pages: the two Zindi pages; no paper found. +- Metric: RMSE on the adjustment amount (Zindi Evaluation page: "The evaluation metric for this challenge is Root Mean Square Error"), the competition's choice (MET-4); inputs none. The old `problem_type` `Binary Classification` (AI-filled) is contradicted by the source: the target is an amount, so `Regression`. +- Risks for /add-dataset: "taxpayer inspection" variables may be recorded during or after the inspection that set the target (USE-4, LEAK checks); anonymized numeric columns, so the meaning rests on VariableDescription.csv; a likely zero-inflated, heavy-tailed target (check MET-3 for the scale); taxpayer-years of one taxpayer across folds if no id ships (SPLIT checks); download needs a Zindi login and the rules forbid redistribution (no criterion, EV-4). +- Priority: gap 0 (finance not thin; grouped would be thin but the regime is not confirmed), use case + (actor and decision named by the source), source + (a ministry's data with a variable description file), shape + (about 25,000 rows, many columns), risk - (inspection-stage variables, unknown grouping). + A Zindi competition to detect tax fraud using the Tunisian Ministry of Finance's data — a real-world, high-value tabular classification task with a clear binary target (fraud / not fraud). This maps well to the benchmark criteria: unique original source, predictive task, representative tabular ML. A human must verify dataset size after cleaning, the exact feature schema, whether a temporal or grouped (per-taxpayer) split is required, and class balance / potential leakage, and confirm data accessibility from Zindi. --- diff --git a/curation/records/turkiye_student_evaluation.md b/curation/records/turkiye_student_evaluation.md index b5844414..f70c2bdf 100644 --- a/curation/records/turkiye_student_evaluation.md +++ b/curation/records/turkiye_student_evaluation.md @@ -29,6 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** agrees with the curator's `No` (CC line; `TabArena Reject`): no target that a decision needs. +- Source: UCI 262, donated 2013-08-31 by Ernest Fokoue and Necla Gunduz: "a total 5820 evaluation scores provided by students from Gazi University in Ankara (Turkey)", "28 course specific questions and additional 5 attributes": `instr` (3 instructors), `class` (13 courses), `repeat`, `attendance` (0-4), `difficulty` (1-5), Q1-Q28 Likert 1-5. UCI lists "Classification, Clustering" and names no target; no paper. +- Use case: none. Every column is answered by the student in the one post-course form, so predicting `difficulty`, `attendance` or the instructor from the other answers is a look at the survey, not a decision (SEL-3, SEL-7: answers of the same interview); the record's CC says the same. Regime would be grouped by instructor and course (only 3 and 13). +- Siblings and duplicates: none among `curation/records/` (grep Turkiye, student evaluation). +- Metric: none found (UCI page), so the fallback would apply. +- Counterpoint checked: education is thin (3 accepted), so a target here would fill a gap, but no candidate column is observed after or apart from the survey, so the gap does not change the verdict. The marker `No Good Target / Scientific Discovery` fits; `problem_type` `Other` matches. + UCI Turkiye Student Evaluation: ~5820 course evaluations with 28 Likert survey items plus instructor/course/attendance fields. As the CC comment notes, most features were collected in the same post-course survey as any plausible target, making it a scientific-discovery / no-clear-supervised-target dataset (it is commonly used for clustering, not prediction). It is also clustered/grouped by instructor and course. Given the lack of a meaningful, non-leaky predictive target, this should be rejected; a human could confirm there is no defensible target. --- diff --git a/curation/records/twente_2009_labeled_flows.md b/curation/records/twente_2009_labeled_flows.md index c328760a..2ec18402 100644 --- a/curation/records/twente_2009_labeled_flows.md +++ b/curation/records/twente_2009_labeled_flows.md @@ -4,6 +4,7 @@ name: Twente 2009 Labeled Flow-based IDS Data Set checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - No Good Target (yet) tags: @@ -15,10 +16,11 @@ domain: technology & internet required_split: - Temporal (NON-IID) problem_type: Binary Classification -original_data_state: One Table +original_data_state: Database (or multiple to-be-joined tables) source_links: - https://link.springer.com/chapter/10.1007/978-3-642-04968-2_4 - https://www.simpleweb.org/wiki/index.php/Traces +- https://www.simpleweb.org/wiki/index.php/Labeled_Dataset_for_Intrusion_Detection needs_review: - ai_unverified type_adapter_id: curation-record-v1 @@ -28,6 +30,15 @@ type_adapter_id: curation-record-v1 ⚠️ **AI-FILLED — UNVERIFIED**: triaged by an AI (2026-08-12), no human has verified it. Check the verdict and the fields before relying on them. +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: real flow records of one honeypot labelled by its logs, but almost all traffic is suspicious by design, so the IDS task (attack against benign) is not in the data, and the host does not answer. +- Source: Sperotto, Sadre, van Vliet, Pras, "A Labeled Data Set For Flow-based Intrusion Detection", IPOM 2009, LNCS 5843, pp. 39-50 (University of Twente). The data page (simpleweb.org, unreachable from here; read from the Wayback copy of 2022-03-08): "traffic collected at a honeypot hosted in the network of the University of Twente in September 2008", "By design, the honeypot only captured "unexpected" traffic, hence it can be assumed that most of the recorded traffic is at least suspicious"; a gzipped MySQL dump with tables `flows` (anonymized IPs, packets, octets, start and end times, ports, TCP flags, protocol), `alert_type` (service and type, e.g. `icmp_sideeffect`), `alerts` (automated, succeeded, description, timestamp), `flow_alert` and cluster and causality tables. About 14.2M flows, over 98% labelled (from the paper's abstract, via search). IPs are anonymized and do not preserve order. +- Use case: a network operator flags malicious flows; the data holds only traffic that reached a honeypot, so a malicious-against-benign model cannot be trained on it, and an alert-type classifier (ssh, http, ftp, irc, side effect) mirrors the honeypot's log correlation rather than a decision. Fit approximate at best. Regime temporal (one month of flows; scans repeat). +- Siblings and duplicates: `kyoto_2006_plus` (AI, `TBD -> 2nd Tier`, `No Good Target (yet)`, the same honeypot-label concern); no duplicate found. External collections: none found. +- Papers and pages: the IPOM 2009 paper; cited in NIDS dataset surveys as small, one honeypot, little benign traffic (the NTLFlowLyzer survey, Shafi et al. 2025, per the earlier note). +- Metric: none found (the data page, the paper's abstract), so the fallback. +- Risks for /add-dataset: the target is to be built from the alert tables (several tables to join, `Needs extensive data wrangling`); the label is the honeypot's log correlation, so features of the alert side leak it; availability (simpleweb.org down, no mirror checked). The old `problem_type` `Binary Classification` (AI-filled) does not hold for a data set without a benign class; left as set until the labels are read. `original_data_state` changed from `One Table` to `Database (or multiple to-be-joined tables)`: the page lists several tables. +- Priority: gap 0 (technology not thin), use case 0, source + (paper and table guide), shape + (14M flows, temporal), risk - (no benign class, joins, the host down). + Labelled NetFlow data from a honeypot run at the University of Twente (Sperotto et al., IPOM 2009) — real observed traffic, so criterion 4B is fine, but it carries the same honeypot-labelling concern as [[kyoto_2006_plus]]: traffic reaching a honeypot is unsolicited by construction, so check whether the label is anything more than "arrived at the honeypot". **Source needs recovering first.** simpleweb.org, the distribution point, did not respond at all from here (repeated connection failures, not a 404), so availability must be re-established — try Wayback or the authors — before anyone plans work on it. The paper is paywalled at Springer and was not read; everything above about the labelling is a hypothesis to test, not a finding. Attack variety and volume are also reported as limited by the NTLFlowLyzer survey (Shafi et al., Computers & Security 148:104160, 2025, Sec. 5.1, p. 10). diff --git a/curation/records/uber_movement_sanral_cape_town_challenge.md b/curation/records/uber_movement_sanral_cape_town_challenge.md index e722bce9..18523135 100644 --- a/curation/records/uber_movement_sanral_cape_town_challenge.md +++ b/curation/records/uber_movement_sanral_cape_town_challenge.md @@ -3,7 +3,8 @@ unique_name: uber_movement_sanral_cape_town_challenge name: uber-movement-sanral-cape-town-challenge checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Time-series (Forecasting) tags: @@ -27,4 +28,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-10) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Time-series (Forecasting)`, SEL-10): forecast, for every hour of a coming quarter and every 500 m freeway segment of Cape Town, whether an incident occurs, from the incident log of the three years before. +- Source: Zindi "Uber Movement SANRAL Cape Town Challenge" (2019-10-11 to 2020-02-10, 865 participants), incident data of SANRAL's Cape Town Freeway Management System and Uber Movement travel times, read through the Zindi API: "The aim of this challenge is to forecast if an incident will occur for each hour of each day per 500m road segment along the major roadways in Cape Town for 1 January 2019 to 31 March 2019". As published: `train.csv`, "all reported incidents from 1 January 2016 to 31 December 2018" (6 MB), road-segment shapefiles, a SANRAL zip (injuries, vehicles, vehicle-detector counts), Uber Movement aggregates; the submission grid is every hour x segment of the quarter (a 36 MB sample file). +- Use case: the authorities pre-position response "to anticipate where they will be needed next"; the rows to score are future hour-by-segment cells with no features of their own besides the segment, the time and aggregates; the target grid is built from the event log. That is a series of event counts per segment forecast over a three-month horizon (SEL-10), not a fixed task on rows with their own attributes (SEL-11). +- Siblings and duplicates: `uber_nairobi_ambulance_perambulation_challenge` (the same Uber Movement series, crash data of Nairobi; proposed `No` in this batch); `accra_mobility_prediction_hackathon` (AI, 2nd Tier, priority 1, a speed panel left to TP-10 because its page decided nothing). +- Metric: F1 on the 0/1 predictions per hour and segment (Zindi Evaluation page). +- Counterpoint for the checker: SEL-11 lets a forecasting-like panel in when it passes TP-10, which was not run here; what sets this apart from the accra precedent is the source's own words ("forecast ... for 1 January 2019 to 31 March 2019") and a training file that is only the event log, so the rows to predict exist only as a calendar grid. +- Reject check (AI, 2026-10-22): does not stand. The source publishes a fixed classification on rows ("forecast if an incident will occur for each hour of each day per 500m road segment", F1 on 0/1 per datetime x segment_id; api.zindi.world/v1/competitions/uber-movement-sanral-cape-town-challenge), with per-segment attributes (lanes, surface, length, condition), SANRAL vehicle counts, Uber travel times and weather. That is a forecasting-like panel, which SEL-11 lets in when TP-10 holds, and TP-10 was not run. The organiser-linked tutorial (datasciencecastnet.home.blog/2019/10/21/zindi-uberct-part-2-stepping-up/) builds a tabular model from segment and weather features without the incident records (no lag of the target), so TP-10 leg 2 is plausible. Unlike stock and appliances_energy_prediction, this is not a single series' next values. Deciding it needs the data, as for the accra_mobility_prediction_hackathon precedent. +- For /add-dataset: TBD -> 2nd Tier, priority 2 (use case + authorities pre-positioning response; shape + large segment x hour grid with segment attributes; risk - forecasting-like panel, very sparse positives, joins across shapefiles, SANRAL and Uber files). The add stage must run TP-10: does a model on the segment, time, weather and SANRAL features clearly beat each segment's naive incident rate (same hour/weekday profile from the year before) by F1 or a probabilistic metric on the shipped temporal windows? Check the winners' features for lagged incident counts per segment (leg 2). Also check the target is buildable from train.csv (incidents to hour x segment cells) and that the positive rate is not too thin for a stable metric; if TP-10 fails, `No` (SEL-10). + A Zindi challenge based on Uber Movement travel-time / traffic data for Cape Town (with SANRAL road sensor data), aimed at predicting traffic conditions / travel times over time. This is a spatio-temporal forecasting task over a horizon rather than a fixed predictive classification/regression problem, placing it out of scope as time-series forecasting. A human should confirm the precise target, but the forecasting nature makes a No appropriate. diff --git a/curation/records/uber_nairobi_ambulance_perambulation_challenge.md b/curation/records/uber_nairobi_ambulance_perambulation_challenge.md index f2065164..9e365753 100644 --- a/curation/records/uber_nairobi_ambulance_perambulation_challenge.md +++ b/curation/records/uber_nairobi_ambulance_perambulation_challenge.md @@ -5,6 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: +- No Good Target / Scientific Discovery - Time-series (Forecasting) tags: - AI-Filled (Verify) @@ -27,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3; `Time-series (Forecasting)`, SEL-10): the task is a placement schedule for six ambulances, scored by the distance to future crashes, and the data is a list of crash times and coordinates with no target. +- Source: Zindi "Uber Nairobi Ambulance Perambulation Challenge" (2020-09-17 to 2021-01-24, 1,097 participants), crash data of the World Bank DIME team and Flare, read through the Zindi API. As published: `Train.csv`, "time and location for 6318 crashes in the training period (2018-01-01 to 2019-06-30)" (`Uid`, `Datetime`, `Latitude`, `Longitude`), daily GFS weather, a road-segment survey with obfuscated columns, segment geometries; Uber Movement speeds allowed. +- Use case: Flare dispatch positions ambulances ahead of crashes; the source frames it as an optimisation: "Instead of predicting crash locations or frequency, you must instead submit a location schedule for six ambulances every 3 hours over the test period". A curated crash-count-per-cell target would be ours and would be a forecast of a spatio-temporal event series over the test half-year (SEL-10). +- Siblings and duplicates: `uber_movement_sanral_cape_town_challenge` (Cape Town incidents, proposed `No` with SEL-10 in this batch); no duplicate. +- Metric: the summed distance from each test crash to the nearest scheduled ambulance (Zindi Evaluation page; metric id `uber_ambulance`), a score of a placement, not of a prediction per row. +- Reject check (AI, 2026-10-22): stands. SEL-3 (Must): the source's own Evaluation page rules out a predictive task: "Instead of predicting crash locations or frequency, you must instead submit a location schedule for six ambulances every 3 hours over the test period", scored by the summed distance to the nearest ambulance (api.zindi.world/v1/competitions/uber-nairobi-ambulance-perambulation-challenge). Train.csv is only Uid, Datetime, Latitude, Longitude of 6,318 crashes, so any classification or regression target would be ours. Unlike the Cape Town sibling, the source publishes no per-row target or metric. + A Zindi challenge to optimally place ambulances in Nairobi by predicting where/when road traffic crashes will occur, using Uber Movement and crash data. The objective is spatio-temporal crash forecasting / facility placement over future periods, not a fixed IID/grouped tabular classification or regression task, so it falls under time-series forecasting and out of scope. A human could double-check, but the forecasting framing supports a No. diff --git a/curation/records/umojahack_africa_2023_intermediate_challenge.md b/curation/records/umojahack_africa_2023_intermediate_challenge.md index 01e1b336..0ddb0521 100644 --- a/curation/records/umojahack_africa_2023_intermediate_challenge.md +++ b/curation/records/umojahack_africa_2023_intermediate_challenge.md @@ -4,17 +4,17 @@ name: umojahack-africa-2023-intermediate-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier -decision_markers: -- TBD +priority: 2 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi year: '2023' +domain: technology & internet required_split: -- '?' -problem_type: TBD +- Random (IID) +problem_type: Binary Classification source_links: - https://zindi.africa/competitions/umojahack-africa-2023-intermediate-challenge/data - https://zindi.africa/competitions/indabax-ghana-2023/data @@ -28,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: cryptojacking detection from 18 host and network counters per visited website; the canonical Zindi copy (March 2023) of data first published by Hernandez-Suarez et al. (IPN, Applied Sciences 2022); three later Zindi re-runs share its files. +- Source: first publication Hernandez-Suarez et al., "Detecting Cryptojacking Web Threats: An Approach with Autoencoders and Deep Dense Neural Networks", Applied Sciences 12(7):3234, 2022 (doi:10.3390/app12073234, Instituto Politécnico Nacional): "9292 benign sites and 3434 sites labeled and validated as cryptojacking infected", 18 host and network features (processor idle states and usage, paging, disk I/O, bytes and packets sent and received), as described by the re-use paper Aponte-Novoa et al., Sensors 22(23):9219, 2022 (Universidad del Norte, ), whose authors helped Zindi get the data. As published on Zindi (UmojaHack Africa 2023: Cryptojacking Detection Challenge (INTERMEDIATE), 18-19 March 2023, 435 participants, read through Zindi's API): Train.csv (1,374,711 B, "~9 000 readings", with the target), Test.csv (581,950 B, ~4,000 rows, no target), Variable_Definitions.csv; target 1 = cryptojacking, 0 = benign; one table. Whether the 2022 paper released the data was not read (the MDPI page returned 403). +- Use case: a browser or endpoint security tool flags and blocks a website as cryptojacking from its resource and traffic counters while it is open; the source states the classification goal ("classify network activity from various websites as either cryptojacking or not"), no actor, so fit approximate; regime IID (one row per site in the paper, no time column), grouped by site if rows turn out to be several windows of one site. +- Siblings and duplicates: the same Train.csv and Test.csv (identical byte sizes) in `indabax-ghana-2023` (Hack4Ghana, May 2023, this record's second link), `indabax-rwanda-2024-hacktathon` and `ai-national-summit-2024-competition` (record `ai_national_summit_2024_competition`), all with this competition as Zindi `parent`, and the Benin 2024 link of `indabax_hackathons`: this record is the canonical copy, the others are duplicates (PROV-4). No other cryptojacking record. External collections: none found. +- Papers and pages: the two papers above; (Description, Data, Evaluation); winners named (Team DSTune first), no solution read. +- Metric: F1 of the cryptojacking class ("The error metric for this competition is the F1 score", Evaluation tab); a competition's leaderboard metric on hard labels (MET-4); inputs none. The papers report accuracy, precision, recall and F1 side by side (no single one, MET-5). +- Risks for /add-dataset: the first source is the 2022 paper, so take its release if public and fix `original_source` and `year` (Zindi, 2023 now; PROV-8); only Train.csv is labelled (~9,000 rows); what one row is (a site or a time window of a site: SPLIT, grouping); duplicate rows; the Sensors paper finds simple models as good as deep ones, so check for a near-perfect score (TP-4). +- Priority: gap 0 (technology & internet, binary, IID not thin), use case 0 (no actor named), source + (a paper documents the collection), shape 0 (about 9,000 labelled rows, 18 numeric counters, no mixed types or structure), risk 0 (triviality unmeasured). + A 2023 UmojaHack Africa / IndabaX Ghana intermediate hackathon on Zindi, but the record gives no description of the underlying data or target, so the task type and domain cannot be determined from the record or general knowledge. Per the rules, too little information is not itself a reason to reject. A human must open the Zindi competition to determine the actual target, problem type, data modality (some hackathons use image/text/audio), size, and split regime before deciding. diff --git a/curation/records/unsw_nb15.md b/curation/records/unsw_nb15.md index 1a89f931..0baacaa5 100644 --- a/curation/records/unsw_nb15.md +++ b/curation/records/unsw_nb15.md @@ -27,6 +27,14 @@ type_adapter_id: curation-record-v1 ⚠️ **AI-FILLED — UNVERIFIED**: triaged by an AI (2026-08-12), no human has verified it. Check the verdict and the fields before relying on them. +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16; PROV-3): the traffic was generated in a lab by a commercial traffic generator, with synthetic attacks, like `kddcup99` (Andrej, `No`) and `cic_ids2017`. +- Source: UNSW Canberra Cyber Range Lab, Moustafa & Slay, "UNSW-NB15: a comprehensive data set for network intrusion detection systems", MilCIS 2015 (); as published: "The total number of records is two million and 540,044", 49 features "with the class label", nine attack categories, a prescribed training set (175,341 records) and testing set (82,332 records). +- Decisive words (project page): "The raw network packets of the UNSW-NB 15 dataset was created by the IXIA PerfectStorm tool in the Cyber Range Lab of UNSW Canberra for generating a hybrid of real modern normal activities and synthetic contemporary attack behaviours." Both classes come out of the traffic generator: the attacks are synthetic, the "normal activities" are profiles replayed by the tool, not observed users. +- Siblings: `kddcup99` (`No`, Andrej: simulated, outdated), `intrusion_detection` (Lennart, `No`), `cic_ids2017` (AI `No`, testbed traffic); `ctu_13` (AI `TBD -> Yes`) differs by its real background traffic; `kyoto_2006_plus` (real honeypot traffic). External collections: many IDS benchmarks, none of ours. +- Metric: not needed for a reject (the papers report accuracy, detection and false-alarm rates side by side). +- Not checked: the MilCIS 2015 paper's text (not read); the old AI note quoted the same page. +- Reject check (AI, 2026-10-22): stands. SEL-16/PROV-3 (Must) holds on the source's own words, not a reading of them: "The raw network packets of the UNSW-NB 15 dataset was created by the IXIA PerfectStorm tool in the Cyber Range Lab of UNSW Canberra for generating a hybrid of real modern normal activities and synthetic contemporary attack behaviours" (https://research.unsw.edu.au/projects/unsw-nb15-dataset, re-read 2026-10-22). Both classes come out of a commercial traffic generator; the 'real ... normal activities' are profiles the tool emulates, not observed users. The curator-checked siblings take the same call: kddcup99 (Andrej, Lennart: No, AHDS) and intrusion_detection (Lennart: No, AHDS); ctu_13 differs because its background traffic was captured, not generated. + The most-benchmarked modern tabular IDS dataset and the usual stand-in for KDD'99: 2,540,044 records with 49 features across four CSVs, nine attack categories, plus a prescribed train/test partition (`UNSW_NB15_training-set.csv` / `_testing-set.csv`). Recorded mainly so we have an answer when asked about it. **Suggested No on criterion 4B**, from the canonical page's own description: the raw packets "was created by the **IXIA PerfectStorm tool** in the Cyber Range Lab of UNSW Canberra for generating a hybrid of real modern normal activities and **synthetic** contemporary attack behaviours". Both sides of the traffic come out of a commercial traffic generator in a lab, so it is the same scripted-testbed situation as [[kddcup99]] and [[intrusion_detection]], only newer. The `attack_cat` column also allows a 9-class framing if the verdict is ever revisited. The paper (Moustafa & Slay, MilCIS 2015) is not read — the quote above is from the project page. diff --git a/curation/records/urban_air_pollution_challenge.md b/curation/records/urban_air_pollution_challenge.md index b57b0335..2a36e00c 100644 --- a/curation/records/urban_air_pollution_challenge.md +++ b/curation/records/urban_air_pollution_challenge.md @@ -3,9 +3,8 @@ unique_name: urban_air_pollution_challenge name: Urban Air Pollution Challenge checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier -decision_markers: -- Time-series (Forecasting) +suggestion: TBD -> Yes +priority: 2 tags: - AI-Filled (Verify) collections: @@ -13,7 +12,7 @@ collections: original_source: Zindi domain: environmental science & climate required_split: -- Temporal (NON-IID) +- Grouped (NON-IID) problem_type: Regression original_data_state: One Table source_links: @@ -28,6 +27,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: same-day PM2.5 estimation for cities worldwide from Sentinel-5P gas columns and GFS weather, test on other locations; not forecasting, so the old `Time-series (Forecasting)` marker is dropped. +- Source: #ZindiWeekendz hackathon "Urban Air Pollution Challenge", 10-12 April 2020, with the World Air Quality Index project and Microsoft (236 participants; read through Zindi's API, ); as published: Train.csv (37,182,338 B, 349 locations, about three months from 2020-01-02), Test.csv (19,024,059 B, "covers 179 different locations", no target), SampleSubmission.csv; target the daily mean PM2.5 from ground sensors (WAQI); inputs GFS humidity, temperature and wind and every band of the Sentinel-5P L3 products from Google Earth Engine (NO2, CO, SO2, O3, HCHO, CH4, aerosol index, cloud), one table keyed by `Place_ID X Date`. +- Use case: "finding ways to track air quality and how it is changing, even in places without ground-based sensors" (Description): estimate today's PM2.5 for a place without a monitor; no actor named, fit approximate; regime grouped by place (the test holds other locations; SPLIT-3), not temporal: the target is the same-day level, not a future value (SEL-10 does not apply), so `required_split` changes from the AI-filled `Temporal (NON-IID)` to `Grouped (NON-IID)`. +- Siblings and duplicates: `urban_air_pollution_challenge_daust_students` is a 2024 re-run with this competition as Zindi `parent` and identical file sizes (a duplicate, proposed `No`); the same kind of task on other data: `airqo_african_air_quality_prediction_challenge` (2024, 8 African cities, AI `TBD -> Yes`, priority 4), `layerai_air_quality_prediction_challenge` (AI `TBD -> Yes`, priority 3) and `zindi_weekendz_layerai_air_quality_prediction_challenge`; a curator may want one or two Sentinel-5P PM2.5 tasks, and this one has by far the most locations. External collections: none found. +- Papers and pages: the Zindi Description, Data and Evaluation tabs; winners' code on GitHub per the rules, not read; no paper. +- Metric: RMSE ("The error metric for this competition is the Root Mean Squared Error", Evaluation tab); a competition's leaderboard metric (MET-4); inputs none (the group `Place_ID` is a split column, not a feature). +- Risks for /add-dataset: drop the target-related columns of Train.csv ("columns for minimum and maximum readings on that day, the variance of the readings and the total number (count) of sensor readings", LEAK); only Train.csv is labelled, so the split is by `Place_ID` inside it; the Sentinel-5P columns are satellite retrievals: SEL-13 names remote-sensing spectra as image-derived, these are retrieved gas columns used as measurements (SEL-15), a call for the curator as on airqo; the "metadata like the satellite altitude" columns and the many missing retrievals (CH4); three months during the 2020 lockdowns. +- Priority: gap + (grouped regime thin), use case 0 (no actor named), source 0 (a weekend hackathon page, products documented on GEE), shape + (about 30,000 rows over 349 places, many numeric columns, grouped), risk - (the modality question and three sibling tasks). + A Zindi challenge to predict air quality (PM2.5) in cities worldwide using engineered features derived from Sentinel-5P satellite data plus weather, provided as a tabular feature table. While the underlying source is satellite imagery, the competition exposes aggregated tabular features, so it could be a valid regression task; however it has a strong temporal/forecasting character (predicting daily pollution levels). A human must verify whether the released data is genuinely tabular features (not raw imagery), the exact target and temporal-split protocol, and dataset size, so a provisional 2nd-tier hold is appropriate. --- diff --git a/curation/records/urban_air_pollution_challenge_daust_students.md b/curation/records/urban_air_pollution_challenge_daust_students.md index 5b59418a..36e8af31 100644 --- a/curation/records/urban_air_pollution_challenge_daust_students.md +++ b/curation/records/urban_air_pollution_challenge_daust_students.md @@ -28,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a 2024 student re-run of `urban_air_pollution_challenge` (2020) with the same files. +- Source: Zindi "Urban Air Pollution Challenge: DAUST Students", 18 Nov to 28 Dec 2024, 20 participants, read through Zindi's API (). +- Decisive evidence: Zindi's API lists `parent`: `urban-air-pollution-challenge` (the 2020 #ZindiWeekendz hackathon); the files are byte-identical in size to the parent's (Train.csv 37,182,338 B, Test.csv 19,024,059 B, SampleSubmission.csv 371,151 B), and the Description and Data pages repeat the parent's text ("The data covers the last three months, spanning hundreds of cities across the globe"; the sample IDs start at 2020-01-02). +- Canonical record: `urban_air_pollution_challenge` (screened 2026-10-22, `TBD -> Yes`, priority 2). The DAUST link can be kept as a `#` note on it. +- Metric: RMSE, as the parent. +- Reject check (AI, 2026-10-22): stands. PROV-4 (Must): the Zindi API (https://api.zindi.world/v1/competitions/urban-air-pollution-challenge-daust-students) lists parent = urban-air-pollution-challenge. Its three files match the 2020 parent's byte sizes exactly (Train.csv 37,182,338; Test.csv 19,024,059; SampleSubmission.csv 371,151), the Data page is word for word the same, and the metric is the same (RMSE). It is a 2024 class re-run (20 participants) with no distinct task (PROV-7 does not apply). The canonical record is urban_air_pollution_challenge (TBD -> Yes, priority 2). + This is a student/practice variant ('daust-students') of the Urban Air Pollution Challenge, almost certainly re-using the same data and target as the main challenge. It is therefore a duplicate / re-upload rather than a unique original source. A human should confirm it shares data with urban_air_pollution_challenge, but it should be rejected as a duplicate; the original entry (if any) carries the canonical decision. diff --git a/curation/records/usaids_intelligent_forecasting_challenge_model_future_contraceptive_use.md b/curation/records/usaids_intelligent_forecasting_challenge_model_future_contraceptive_use.md index f1ad0560..417a4119 100644 --- a/curation/records/usaids_intelligent_forecasting_challenge_model_future_contraceptive_use.md +++ b/curation/records/usaids_intelligent_forecasting_challenge_model_future_contraceptive_use.md @@ -3,7 +3,8 @@ unique_name: usaids_intelligent_forecasting_challenge_model_future_contraceptive name: usaids-intelligent-forecasting-challenge-model-future-contraceptive-use checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Time-series (Forecasting) tags: @@ -27,4 +28,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: the proposed fast reject (SEL-10) did not hold in the reject check, so it stays a candidate with the doubt listed. +- Proposed reject (refuted): `No` (`Time-series (Forecasting)`, SEL-10): monthly consumption of 11 contraceptives at 156 sites, predicted for the next three months from 42 months of history. +- Source: USAID's Intelligent Forecasting Challenge (), mirrored on Zindi 27 Jul to 30 Aug 2020 (526 participants, read through Zindi's API, ); as published: an extract of Côte d'Ivoire's eLMIS, Train.csv (2.5 MB) with site, product, year and month, plus site, product and case tables. +- Decisive words (Data page): "The goal of the Intelligent Forecasting Competition is to predict consumption (stock_distributed in the Primary Data Dictionary) for 11 contraceptives across 156 health service delivery sites ... The predictions should be made monthly for three months: October 2019, November 2019, and December 2019 ... Zindi competitors will have access to 42 months, January 2016 through June 2019, of data." The Description: USAID seeks "more accurate methods of predicting future contraceptive use", "to predict the consumption of contraceptives over three months". +- Use case: real (procurement quantities per site), but the task is future values of per-site, per-product series over a horizon, so SEL-10 applies; nothing sets it apart as a fixed task with a temporal split (SEL-11). +- Metric: RMSE on the Zindi leaderboard; USAID judged its own submissions qualitatively. +- Reject check (AI, 2026-10-22): does not stand. This is a site x product panel (156 sites x 11 products = 1,716 series, monthly). TP-10 (amended 2026-10-22 on g_research, which had history-only features) can admit such a panel, so SEL-10 alone does not decide it. Against it: the source frames the task as forecasting ("Accurate forecasting of contraceptive consumption ..."; USAID judges whether models "Outperform traditional methods"). The test months (Jul-Sep 2019) follow the end of training (June 2019), and submission rows carry only year x month x site_code x product_code, so no future-month covariates exist, unlike grocery (promotions) or hydropower (climate) (https://api.zindi.world/v1/competitions/usaids-intelligent-forecasting-challenge-model-future-contraceptive-use). What is available is the static site table (lat/lon, site type), the product table, and other eLMIS columns up to the cutoff (stock_end, stock_received, average_monthly_consumption, stockout days). Under TP-10 leg 2 those may be allowed point-in-time summaries. Leg 1 (beat each series' naive profile) needs the data. One winner (Inventec, arXiv 2304.06344) describes 'feature engineering for time-series data', and its text was not read, so leg 2 is open too. Weaker than grocery: if the measurement fails, the No comes back, as for fossil_stock_forecasting_challenge (stood, checker 10, an SKU panel with no exogenous inputs). +- For /add-dataset: TBD -> 2nd Tier, priority 2. Add stage: (1) run TP-10 at the June 2019 cutoff with horizons 1-3. Compare GBM on site/product attributes plus point-in-time eLMIS summaries against each site-product's naive profile (last value, same month a year earlier, trailing mean) under RMSE. Treat average_monthly_consumption as a trailing summary of the target, not a same-month feature. (2) Read the Inventec paper (arXiv 2304.06344) and the Zindi top-solution threads for target lags (leg 2). (3) LEAK: same-month stock_initial + stock_received - stock_end is close to stock_distributed (stock identity), so these must never be same-row features. (4) The source warns of 'invalid 0 values' (non-reported months), which are a data-quality and target-cleaning question. (5) Licence: the rules say the data is 'the sole property of USAID' and must not be redistributed; check the USAID call (competitions4dev.org/forecastingprize) for terms that allow shipping. If (1) fails, return to No with SEL-10. (Priority set to 2 by the main session, like the other refuted forecasting-like panels: grocery_store_product_forecasting_challenge, uber_movement_sanral_cape_town_challenge.) + A USAID Zindi challenge to model/forecast future contraceptive (family-planning consumption) use from historical service-delivery logistics data. The explicit objective is forecasting future values over a horizon, which is out of scope for the benchmark (time-series forecasting). The name itself ('model future ... use') makes the forecasting nature clear, so this should be rejected. diff --git a/curation/records/uscrime1978.md b/curation/records/uscrime1978.md index ef883c30..a6e395e2 100644 --- a/curation/records/uscrime1978.md +++ b/curation/records/uscrime1978.md @@ -4,6 +4,7 @@ name: USCrime1978 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small tags: @@ -25,6 +26,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** recommend `No` with `Too Small` (TP-5) and `No Good Target / Scientific Discovery` (SEL-3), as the workbook's "Tiny data" and both curators' "too small" say; the `TBD -> 2nd Tier` came from an AI "policy" update, not from a curator, and the current rules keep `Too Small` as a reason for `No`. The curator has the call. +- Source: Vandaele, W. (1978), "Participation in illegitimate activities: Ehrlich revisited", in Blumstein, Cohen and Nagin (eds.), Deterrence and Incapacitation, National Academy of Sciences, 270-335; reprinted in DASL (USCrime) and Hand et al. (1994), A Handbook of Small Data Sets; on OpenML as 1089 `USCrime` (uploaded 2014 from TunedIT); as published: 47 US states in 1960 x 14 numeric columns, the crime rate `R` and 13 demographic, labour and police-spending variables; OpenML's default target is `X` ("The number of families per 1000 earning below 1/2 the median income"), not the crime rate. +- Use case: none; the data were "collected from the FBI's Uniform Crime Report and other government agencies to determine how the variable crime rate depends on the other variables measured in the study" (OpenML description, from DASL): an explanatory study of deterrence, one row per state in one year; nobody predicts a state's crime rate from these at a decision point (SEL-3). +- Siblings and duplicates: no other record of this data; it also ships as `UScrime` in R's MASS package. External collections: none found. +- Metric: none found (OpenML and DASL descriptions), so the fallback `rmse`. +- Risks: 47 rows, about 16 per test fold under 3-fold CV (TP-5); OpenML's target is a covariate of the study. +- Priority: 1 (gap + for social science, use case 0, source 0, shape - (47 rows), risk - (no real target)); set only because the verdict is not `No`. + The classic US crime dataset (OpenML id 1089) with only 47 samples. Both the TabArena curation verdict and reviewers (Lennart, Andrej) mark it as too small, which is too few for reliable 3-fold CV evaluation. It fails the adequate-size criterion and should be rejected as Too Small; no further verification needed. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/used_cars_24.md b/curation/records/used_cars_24.md index b7d1af84..04fdf2dc 100644 --- a/curation/records/used_cars_24.md +++ b/curation/records/used_cars_24.md @@ -4,6 +4,7 @@ name: Used cars 24 checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: - Missing source information tags: @@ -31,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: Indian used-car resale prices with mixed string columns; the rows look real (Indian cities, model names, units in the strings), so the "likely synthetic" note is not supported, but the first publisher is not named on the Kaggle page. +- Source: Kaggle upload by avikasliwal, 26 Oct 2019, "Used Cars Price Prediction: Predict the price of an unknown car. Build your own Algo for cars 24 !!" (, no description); as published: train-data.csv 6,019 rows x 14 columns (`Name`, `Location` (11 cities), `Year`, `Kilometers_Driven`, `Fuel_Type`, `Transmission`, `Owner_Type`, `Mileage` "26.6 km/kg", `Engine` "998 CC", `Power` "58.16 bhp", `Seats`, `New_Price` "8.61 Lakh" (missing in 5,195 rows), target `Price` in lakh) and test-data.csv without the price (preview read 2026-10-22). The columns and the 6,019 training rows match MachineHack's "Predicting The Costs Of Used Cars" hackathon (sponsored by Imarticus Learning, June 2019, over 1,700 participants), which is the likely first publication; its page and the site the data came from (the cars24 link is a guess) were not reached. +- Use case: a dealer or a listing platform suggests a resale price for a car from its listing attributes; no actor named by the source, fit approximate; regime IID (one snapshot of listings, no listing date). +- Siblings and duplicates: other used-car price records: `fiat_500` (`Yes`), `used_cars_dataset_cardekho` (Lennart, `TBD -> 2nd Tier`, `No Good Target (yet)`), `used_cars_pakistan`, `used_cars_saudi_arabia`, `used_cars_benz_italy`, `exploreml_funaab_hackathon_nigerian_cars_price_prediction`; none shares these rows. External collections: CARTE/TARTE (as `used_cars_24`). +- Papers and pages: the Kaggle page; Analytics India Magazine's report of the MachineHack winners (feature work on `Mileage`, `Engine`, `Power`, `New_Price`, brand from `Name`); no paper. +- Metric: none found (the Kaggle page has no task; the MachineHack evaluation page was not reached), so the fallback `rmse`, with the price likely shipped on a log scale (MET-3). +- Risks for /add-dataset: confirm the MachineHack origin and its metric (PROV-2, MET-1); the unit strings to parse (CLEAN); `New_Price` is the new-car list price, fine at listing time but mostly missing; only the train file is labelled. +- Priority: gap 0 (business & marketing, regression, IID), use case 0, source 0 (an undescribed Kaggle upload, likely a hackathon), shape + (6,019 rows, mixed numeric, categorical and unit-string columns), risk 0. The `Missing source information` marker stays until the MachineHack origin is confirmed. + A used-cars price-prediction dataset (Kaggle, possibly from cars24.com) — a standard tabular regression task with several string columns benefiting from preprocessing. The CC comment flags that the source is unclear and the data may be synthetic, which is the main risk; if real it would be a reasonable IID regression dataset. A human must verify the data provenance (genuine scraped listings vs synthetic), size, and whether it duplicates other used-car datasets before promoting it, so a provisional 2nd-tier hold is appropriate. --- diff --git a/curation/records/used_cars_benz_italy.md b/curation/records/used_cars_benz_italy.md index 9d482473..241d52f4 100644 --- a/curation/records/used_cars_benz_italy.md +++ b/curation/records/used_cars_benz_italy.md @@ -3,15 +3,16 @@ unique_name: used_cars_benz_italy name: Used Cars Benz Italy checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes -decision_markers: -- Missing source information +suggestion: TBD -> 2nd Tier +priority: 1 tags: - New IID - Free Text (Short) - AI-Filled (Verify) collections: - CARTE/TARTE +original_source: Kaggle +year: '2023' domain: business & marketing required_split: - Random (IID) @@ -29,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: second-hand Mercedes-Benz listing prices in Italy, scraped by the uploader from unnamed Italian sites; a real but narrow single-brand task with price errors and duplicate rows. +- Source: Kaggle upload by bogdansorin, 15 Apr 2023, "Second-hand Mercedes Benz price Italy: This is a dataset created by web scraping second-hand cars italian websites" (); as published: mercedes-benz.csv (semicolon-separated) 16,392 rows x 10 columns: an index, `brand` (one value), `model` (186), `first_reg` (month-year), `fuel`, `mileage_km`, `seller_type` (dealer or private), `swift` (the gearbox, named `shift` in the codebook), target `price` (EUR), `power_hp` (preview read 2026-10-22). +- Use case: a seller or a dealer prices a used Mercedes from its listing attributes; no actor named, fit approximate; regime IID (one scrape, no listing date). +- Siblings and duplicates: the used-car price records (`fiat_500` `Yes`, `used_cars_dataset_cardekho` Lennart `TBD -> 2nd Tier`, `used_cars_24`, `used_cars_pakistan`, `used_cars_saudi_arabia`); no other Mercedes listing record (`mercedes_benz_greener_manufacturing` is a test-bench time task); no duplicate found. External collections: CARTE/TARTE. +- Papers and pages: the Kaggle page; no paper. +- Metric: none found (a Kaggle dataset without a task), so the fallback `rmse`, with the price likely on a log scale (MET-3). +- Risks for /add-dataset: `price` runs from 1 to 500,000,000 (placeholder and typo prices to clean, CLEAN); 853 duplicate rows apart from the index; which sites were scraped (PROV-2: plausible, unclear in a detail, so the old `Missing source information` marker is dropped); only 8 features, and one brand. +- Priority: gap 0, use case 0, source 0 (an uploader's scrape, sites unnamed), shape + (16,392 rows, mixed types), risk - (price errors and duplicates). + Second-hand Mercedes-Benz listings registered 2000-2023 in Italy (Kaggle), a valid tabular price-regression task with short free-text/categorical fields useful for feature engineering. The main risks noted are that it may duplicate other Mercedes-Benz used-car datasets and that the scraping source is not fully documented. A human must check for overlap/duplication with other used-car records, confirm the price target and size, and verify the data is genuinely scraped real listings before final acceptance. --- diff --git a/curation/records/used_cars_pakistan.md b/curation/records/used_cars_pakistan.md index 28f6f5e0..d3ba5403 100644 --- a/curation/records/used_cars_pakistan.md +++ b/curation/records/used_cars_pakistan.md @@ -4,12 +4,15 @@ name: Used Cars Pakistan checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 tags: - Free Text (Short) - New IID - AI-Filled (Verify) collections: - CARTE/TARTE +original_source: Kaggle +year: '2021' domain: business & marketing required_split: - Random (IID) @@ -28,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: 72,655 used-car listings scraped from PakWheels in 2021 with their asking price; a real mixed-type price task, one of several used-car records. +- Source: Kaggle upload by mustafaimam, 22 Apr 2021, "Used car prices in Pakistan 2021: Dataset scraped from Pakwheels" (, CC0): "we created our own dataset using web scraping technique in python to extract data of used vehicles from www.pakwheels.com"; as published: a raw file (6.1 MB) and Used_car_prices_in_Pakistan_cleaned.csv, 72,655 rows x 10 columns (`Make`, `Model`, `Version` (1,328 values), target `Price` (PKR), `Make_Year`, `CC`, `Assembly` (local or imported), `Mileage`, `Registered City` (182), `Transmission`), no missing values (preview read 2026-10-22). +- Use case: a seller or the platform suggests an asking price for a listed car ("features which play an important role in suggesting and evaluating the price"); no actor named, fit approximate; regime IID (one scrape, no listing date). +- Siblings and duplicates: the used-car price records (`fiat_500` `Yes`, `used_cars_dataset_cardekho` Lennart `TBD -> 2nd Tier`, `used_cars_24`, `used_cars_saudi_arabia`, `used_cars_benz_italy`); other Pakistani used-car uploads on Kaggle exist (older PakWheels scrapes, per the uploader), none recorded. External collections: CARTE/TARTE. +- Papers and pages: the Kaggle page; no paper. +- Metric: none found (a Kaggle dataset without a task), so the fallback `rmse`, with the price likely on a log scale (MET-3). +- Risks for /add-dataset: 1,507 duplicate rows in the cleaned file (re-listed cars?); start from the raw file to see what the uploader's cleaning removed (PROV-13); asking price, not sale price. +- Priority: gap 0 (business & marketing, regression, IID), use case 0, source 0 (the uploader's own scrape, described in a paragraph), shape + (72,655 rows, high-cardinality categoricals and numerics), risk 0. + Used-car listings scraped from pakwheels.com (Pakistan, ~2021), a genuine real-world tabular price-prediction regression task; the comment clarifies it is a feature-engineering / string-preprocessing problem rather than free-text handling. It satisfies the core criteria (unique source, predictive regression, representative tabular ML). A human should confirm dataset size after cleaning, the exact price target, and that it does not duplicate other regional used-car datasets in the collection. --- diff --git a/curation/records/used_cars_saudi_arabia.md b/curation/records/used_cars_saudi_arabia.md index 5ae03009..e486ba7c 100644 --- a/curation/records/used_cars_saudi_arabia.md +++ b/curation/records/used_cars_saudi_arabia.md @@ -4,12 +4,15 @@ name: Used Cars Saudi Arabia checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 2 tags: - Free Text (Short) - New IID - AI-Filled (Verify) collections: - CARTE/TARTE +original_source: Kaggle +year: '2021' domain: business & marketing required_split: - Random (IID) @@ -28,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: used-car listings scraped from syarah.com in 2021; a real mixed-type price task once the 2,526 negotiable listings without a price are dropped. +- Source: Kaggle upload by turkibintalib, 4 Jul 2021, "Saudi Arabia Used Cars Dataset: Used Cars Prices and Spec Scraped From Syarah Website" (): "8248 records of used cars collected from syarah.com"; as published: UsedCarsSA_Unclean_EN.csv, an Arabic xlsx and UsedCarsSA_Clean_EN.csv with 8,035 rows x 13 columns (`Make`, `Type` (381), `Year`, `Origin`, `Color`, `Options`, `Engine_Size`, `Fuel_Type`, `Gear_Type`, `Mileage`, `Region` (27), target `Price` (SAR), `Negotiable`) (preview read 2026-10-22). +- Use case: a seller or the platform prices a used car from its listing; no actor named, fit approximate; regime IID (one scrape, no listing date). +- Siblings and duplicates: the used-car price records (`fiat_500` `Yes`, `used_cars_dataset_cardekho` Lennart `TBD -> 2nd Tier`, `used_cars_24`, `used_cars_pakistan`, `used_cars_benz_italy`); no duplicate found. External collections: CARTE/TARTE. +- Papers and pages: the Kaggle page; no paper. +- Metric: none found (a Kaggle dataset without a task), so the fallback `rmse`, with the price likely on a log scale (MET-3). +- Risks for /add-dataset: `Price` is 0 in 2,527 rows, 2,526 of them `Negotiable` = True (no price asked): drop them and drop `Negotiable`, which is then True in one row only, as the old note says; 8 priced rows below 1,000 SAR; start from the unclean file to see the uploader's cleaning (PROV-13); 5,508 priced rows remain. +- Priority: gap 0, use case 0, source 0 (the uploader's own scrape, one paragraph), shape + (5,508 priced rows, 12 mixed features), risk 0. + Saudi Arabia used-car listings scraped from syarah.com (Kaggle), a valid tabular price-regression task with categorical/short-text features. The comment notes the task likely needs filtering by whether the price is negotiable, which is a cleaning step rather than a blocker. A human must define the exact target (price), apply the negotiable filter, confirm size, and check for duplication with other regional used-car datasets before acceptance. --- diff --git a/curation/records/valuation.md b/curation/records/valuation.md index b5a95aa5..d141c277 100644 --- a/curation/records/valuation.md +++ b/curation/records/valuation.md @@ -4,8 +4,9 @@ name: Valuation checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 2 decision_markers: -- No Good Target / Scientific Discovery +- No Good Target (yet) - Needs extensive data wrangling tags: - 2nd Tier / Scientfic Discovery @@ -19,6 +20,7 @@ problem_type: Regression original_data_state: Database (or multiple to-be-joined tables) source_links: - https://catalog.data.gov/dataset/property-valuation-and-assessment-data-db7c2 +- https://data.cityofnewyork.us/City-Government/Property-Valuation-and-Assessment-Data/yjxr-fw8i source_row: 830 needs_review: - ai_unverified @@ -29,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: New York City's yearly property assessment roll; the only target is the assessor's own market value, a model output, so no real task yet (as the CC says); the sale-price task belongs to `nyc_property_sales`. +- Source: NYC Department of Finance, "Property Valuation and Assessment Data" on NYC Open Data (dataset yjxr-fw8i, attribution "Department of Finance (DOF)", created 2019, rows updated 2020; the data.gov page in the links mirrors it): "The Department of Finance values properties every year as one step in calculating property tax bills". As published (portal metadata, read 2026-10-22): 9,845,857 rows over several rolls (`year`, `period`, `valtype`), about 40 columns: lot keys (`bble`, `boro`, `block`, `lot`), `owner`, building and tax class, lot and building dimensions, `stories`, `fullval` (market value), assessed and exempt values (`avland`, `avtot`, `exland`, `extot` and their `2` versions), exemption codes, address, zip, coordinates and admin areas; the column descriptions are empty. +- Use case: none in the source. Predicting `fullval` copies the DOF's mass appraisal, not an observed outcome (SEL-3), and the assessed values are fixed shares of it by tax class (SEL-5); the outcome an actor acts on is a sale price (`nyc_property_sales`, Lennart: `TBD -> 2nd Tier`, `No Good Target (yet)`), so the marker is aligned to it (`No Good Target / Scientific Discovery` replaced). Regime temporal (yearly rolls; the record's `Custom` kept) or grouped by lot. +- Siblings and duplicates: `nyc_property_sales` (DOF rolling sales; a join by `bble` could give a sale-price task with the roll as features). External collections: none found. +- Papers and pages: none; the NYC Open Data page and the DOF property assessment page. +- Metric: none found (NYC Open Data page, DOF assessment page), so the fallback (rmse). +- Risks for /add-dataset: no target to ship as is (SEL-3); value columns derived from each other (SEL-5, LEAK); `owner` holds names (drop); repeated lots over rolls (grouped or temporal); size and joins (millions of rows per roll). +- Priority: gap 0 (finance is not thin), use case 0 (none), source + (the city agency's own portal), shape + (millions of lots, mixed types, yearly rolls), risk - (the target is an appraisal output, derived columns). + NYC property valuation and assessment data (data.gov) — essentially a government SQL dump with geospatial and text fields and no clean predictive task out of the box. The CC comment notes that assessed value may follow a fixed formula or come from a third party, and that extensive expert feature engineering plus multi-year collection would be needed to build a sound dataset. It is potentially salvageable into a property-value regression task but requires substantial wrangling and leakage checks, so a provisional 2nd-tier hold is appropriate pending human inspection of the target and feature provenance. --- diff --git a/curation/records/vineyard.md b/curation/records/vineyard.md index 4eac372a..3fea9446 100644 --- a/curation/records/vineyard.md +++ b/curation/records/vineyard.md @@ -4,6 +4,7 @@ name: vineyard checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small tags: @@ -25,6 +26,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Assessment (AI, 2026-10-22):** the workbook verdict (Tiny data; Lennart and Andrej: too small) stands, priority 1; recommend `No` with `Too Small` (TP-5); the curator has the call. +- Source: OpenML 192 (uploaded 2014): "Dataset from Smoothing Methods in Statistics (ftp stat.cmu.edu/datasets)", Simonoff, J.S. (1996), Springer; "Source: Unknown". As published: 52 rows x 4 numeric columns, `row_number`, `lugs_1989`, `lugs_1990` and the target `lugs_1991` (grape harvest in lugs per vineyard row). +- Use case: a grower's yield of a row in 1991 from the two previous harvests: the target is the next value of each row's own series (close to SEL-10 forecasting, SEL-11 only with more years); a textbook smoothing example, not a decision. Regime would be temporal, with only one target year. +- Siblings and duplicates: none in `curation/records/`; OpenML and the CMU StatLib archive carry it. +- Papers and pages: Simonoff (1996), as an example for smoothing. +- Metric: none found (OpenML page), so the fallback (rmse). +- Risks for /add-dataset: 52 rows, about 17 per test fold (TP-5); the features are lags of the target. +- Priority: gap 0, use case 0 (none), source - (a textbook example, source unknown on OpenML), shape - (52 x 3), risk - (lagged target). + The OpenML 'vineyard' dataset (id 192) has only 52 samples. The TabArena curation verdict and reviewers (Lennart, Andrej) all mark it as too small for reliable 3-fold CV evaluation. It fails the adequate-size criterion and should be rejected as Too Small; no further verification needed. **Update (policy):** "Too Small" is no longer a rejection reason — dataset size is assessed *after* Data Foundry processing. Re-classified off No to TBD -> 2nd Tier pending that post-DF size check. diff --git a/curation/records/wastewater_treatment.md b/curation/records/wastewater_treatment.md index e24d4d2d..634a9278 100644 --- a/curation/records/wastewater_treatment.md +++ b/curation/records/wastewater_treatment.md @@ -6,7 +6,6 @@ checked_by: suggestion: 'No' decision_markers: - No Good Target / Scientific Discovery -- Not Representative tags: - Many features - AI-Filled (Verify) @@ -26,6 +25,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`No Good Target / Scientific Discovery`, SEL-3): a global microbiome survey of wastewater plants published to explain diversity and biogeography, with no prediction target. +- Source: Wu et al. (Global Water Microbiome Consortium), "Global diversity and biogeography of bacterial communities in wastewater treatment plants", Nature Microbiology, 13 May 2019 (doi 10.1038/s41564-019-0426-5); as published: 16S rRNA OTU tables "from ~1,200 activated sludge samples taken from 269 WWTPs in 23 countries on 6 continents" (http://gwmc.ou.edu/data-disclose.html) and plant metadata in SI Table 1, to be joined. +- The task: none. The abstract frames the study as discovery: "the diversity of microorganisms and the factors that control it are poorly understood"; it reports a core community "strongly linked to activated sludge performance" and that turnover "appears to be largely driven by stochastic processes"; "Our findings enhance our mechanistic understanding of the global diversity and biogeography of activated sludge bacterial communities within a theoretical ecology framework". No target, no prediction. +- Siblings and duplicates: none in `curation/records/`. +- Metric: none. +- Doubt to weigh: SI Table 1 may hold plant performance (removal rates) that a task could predict from the microbiome; that would be a new task of ours on a sparse, very wide OTU table, not the source's. The old `Not Representative` marker is dropped. +- Reject check (AI, 2026-10-22): stands. Original task (SEL-8) is discovery: Wu et al., Nature Microbiology 2019 (https://doi.org/10.1038/s41564-019-0426-5), "Our findings enhance our mechanistic understanding of the global diversity and biogeography of activated sludge bacterial communities". The collection's own target (Zschaubitz et al. 2025, CSBJ, Table 1, https://pmc.ncbi.nlm.nih.gov/articles/PMC12049816/) is 'avg. temp' / 'Continent' on 687 samples x 66,743 OTUs: quantities measured at sampling, predicted by no one, the same pattern as tara_oceans, bog_lakes, ports (stand today). Nothing sets wastewater apart: a plant-performance target from SI Table 1 would be our own new task, not the source's. SEL-3 (Must). + A global microbiome study: 16S rRNA OTU tables from ~1,200 activated-sludge samples across 269 WWTPs in 23 countries (Nature Microbiology 10.1038/s41564-019-0426-5), joined with metadata. This is an OTU abundance table — extremely high-dimensional and sparse, used for ecological/scientific discovery rather than a standard real-world tabular predictive task, and not representative of typical tabular ML. There is no clear, defensible predictive target out of the box. It should be rejected as scientific-discovery / not representative, though a human could confirm no usable target (e.g., continent/process type) is salvageable. --- diff --git a/curation/records/water_potability_classification_hackathon_by_indabax_nigeria_2021.md b/curation/records/water_potability_classification_hackathon_by_indabax_nigeria_2021.md index 5594ffd2..79cc6f03 100644 --- a/curation/records/water_potability_classification_hackathon_by_indabax_nigeria_2021.md +++ b/curation/records/water_potability_classification_hackathon_by_indabax_nigeria_2021.md @@ -3,9 +3,10 @@ unique_name: water_potability_classification_hackathon_by_indabax_nigeria_2021 name: Water Potability Classification Hackathon by IndabaX Nigeria 2021 checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier +suggestion: 'No' decision_markers: -- TBD +- Duplicate +- AHDS (Artifical/Handmade/Deterministic/Simulated) tags: - AI-Filled (Verify) collections: @@ -29,6 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, `AHDS (Artifical/Handmade/Deterministic/Simulated)`, PROV-4, SEL-16): the IndabaX Nigeria 2021 hackathon serves the synthetic Kaggle "water potability" file, split into train and test. +- Source: Zindi "Water Potability Classification Hackathon", IndabaX Nigeria 2021, 16 to 23 Sept 2021 (https://zindi.africa/competitions/indabax-nigeria-2021, read through Zindi's API): "The train contains 2260 rows and the test 1016" (3,276 together, the row count of Kaggle `adityakadiwal/water-potability`); the variable descriptions are the Kaggle card's text word for word, including "The current investigation ranges were 6.52–6.83 which are in the range of WHO standards" (pH, Hardness, Solids, Chloramines, Sulfate, Conductivity, ...). The framing story ("You have been contracted by Water Corporation in Nigeria") is the hackathon's; no Nigerian sampling is described. +- Siblings and duplicates: `water_quality_and_potability` (`No`, `Missing source information` and `AHDS`; CC 2026-10-22, Lennart): the Kaggle first upload's source field reads "Synthetically generated", TDS is 50 times the conductivity, and 1,277 of 1,278 "potable" rows exceed the WHO TDS maximum. +- Metric: F1 (Zindi Evaluation page), moot. +- Not checked: the files (login); the match rests on the row count and the verbatim data card. +- Reject check (AI, 2026-10-22): stands. Zindi Data page (https://zindi.africa/competitions/indabax-nigeria-2021, api.zindi.world, read 2026-10-22): "The train contains 2260 rows and the test 1016" (3,276 = Kaggle adityakadiwal/water-potability) and the variable text is that Kaggle card verbatim ("The current investigation ranges were 6.52-6.83 ..."); no Nigerian sampling is described, the 'Water Corporation' framing is the hackathon's. The first upload's source field reads "Synthetically generated" (record water_quality_and_potability, CC 2026-10-22 Lennart: keep No with AHDS). PROV-4 + SEL-16, both Must. + A Zindi / IndabaX Nigeria 2021 hackathon to classify whether water is potable from physico-chemical measurements — a binary classification task. The key risk is that many 'water potability' datasets (notably the popular Kaggle one) are synthetic/imaginary, so provenance must be checked; if this reuses such synthetic data it should be rejected, but if it is genuine measurements it could qualify. A human must verify the data source/realness, dataset size, class balance, and whether it duplicates the synthetic water_quality dataset before deciding, so a provisional 2nd-tier hold is appropriate. --- diff --git a/curation/records/water_quality.md b/curation/records/water_quality.md index 8ac1e504..3dce5d39 100644 --- a/curation/records/water_quality.md +++ b/curation/records/water_quality.md @@ -29,6 +29,12 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`AHDS (Artifical/Handmade/Deterministic/Simulated)`, SEL-16): the uploader says the data is made up. +- Source: Kaggle `mssmartypants/water-quality`, "Water quality", one file waterQuality1.csv (840,687 bytes, 2021-06-30), licence "copyright-authors"; 20 numeric contaminant columns (aluminium to uranium, each listed with "dangerous if greater than" a limit) and `is_safe`. The card: "This is a set of data created from imaginary data of water quality in an urban environment. I recommend using this dataset for educational purposes", and "Idea was to create dataset that is good for binary classification and that gives adequate results." No place, date or lab. +- Siblings and duplicates: `water_quality_and_potability` (`No`, `AHDS`, another synthetic potability file) and `water_potability_classification_hackathon_by_indabax_nigeria_2021` (a copy of it) are different files. Already tagged TabArena Reject; the CC line quotes the same sentence. +- Metric: none (a Kaggle dataset), moot. +- Reject check (AI, 2026-10-22): stands. The uploader's own words on Kaggle mssmartypants/water-quality (https://www.kaggle.com/datasets/mssmartypants/water-quality): "This is a set of data created from imaginary data of water quality in an urban environment. I recommend using this dataset for educational purposes". SEL-16 (Must), detection PROV-3; source's words, top evidence rank. + A Kaggle water-quality dataset that, per the CC comment and the Kaggle description, was created from imaginary/synthetic data of water quality in an urban environment and is recommended only for educational use. As artificial/simulated data it is not representative of real-world tabular ML and fails the criteria; it is already tagged TabArena Reject. It should be rejected as AHDS; no further verification needed. --- diff --git a/curation/records/wazihub_soil_moisture_prediction_challenge.md b/curation/records/wazihub_soil_moisture_prediction_challenge.md index 1cc56ac7..306d2296 100644 --- a/curation/records/wazihub_soil_moisture_prediction_challenge.md +++ b/curation/records/wazihub_soil_moisture_prediction_challenge.md @@ -28,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Time-series (Forecasting)`, SEL-10): a forecast of each field's soil humidity over the next days, at 5-minute steps, from its own past. +- Source: Zindi "Wazihub Soil Moisture Prediction Challenge", 29 Jul to 21 Oct 2019, sponsored by Wazihub and Microsoft (read through Zindi's API; the API's problem type is "Forecast"); as published: IoT readings every five minutes "over 4 months in 4 fields growing maize and peanuts in Senegal" (soil humidity per field, air temperature, humidity, pressure, wind, an irrigation flag) and hand-collected daily context files; Train.csv 2,033,452 bytes. +- The task: "create a machine learning model capable of predicting the humidity for a particular plot in the next few days, using data from the past"; "You will need to predict, in 5-minute increments, the last four days for soil humidities in fields 1 and 3 and ... the last six days for soil humidities in fields 2 and 4"; "You may not use data from the future". Future values of four series over a horizon, with no fixed per-row target (SEL-11 does not apply). +- Siblings and duplicates: none in `curation/records/`. +- Metric: RMSE (Evaluation page), moot. +- Reject check (AI, 2026-10-22): stands. Zindi (https://zindi.africa/competitions/wazihub-soil-moisture-prediction-challenge, API type_of_problem ['Forecast']): "predicting the humidity for a particular plot in the next few days, using data from the past" and "predict, in 5-minute increments, the last four days for soil humidities in fields 1 and 3 ... the last six days ... in fields 2 and 4". Future values of four sensor series over a horizon: SEL-10 (Must); SEL-11/TP-10 does not rescue it (no fixed per-row target, only 4 entities, the use case is anticipating the series). + WaziHub soil moisture prediction challenge (Zindi) asks competitors to forecast future soil moisture values from IoT sensor time-series streams. This is a time-series forecasting task over a horizon, which is explicitly excluded from the benchmark scope. The data is sensor logs rather than a single predictive cross-sectional table. A human should confirm the framing, but the name and known task strongly indicate forecasting, so the suggestion is No. diff --git a/curation/records/wildfire_prediction_challenge.md b/curation/records/wildfire_prediction_challenge.md index 248e0fcc..e794b29f 100644 --- a/curation/records/wildfire_prediction_challenge.md +++ b/curation/records/wildfire_prediction_challenge.md @@ -5,7 +5,7 @@ checked_by: - AI (UNVERIFIED) suggestion: 'No' decision_markers: -- Time-series (Forecasting) +- Duplicate tags: - AI-Filled (Verify) collections: @@ -28,4 +28,10 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): the IndabaX Eswatini 2024 "Wildfire Prediction Challenge" re-runs Zindi's 2020 "Fighting Fire with Data" hackathon with the same files. +- Source: https://zindi.africa/competitions/wildfire-prediction-challenge (28 Aug to 30 Sept 2024, Deep Learning IndabaX Eswatini, read through Zindi's API). Its parent is "predict-fire-extent" (Jul-Aug 2024, record `predict_fire_extent`), whose parent is "fighting-fire-with-data-hackathon" (27 June 2020, the first publication). All three list Train.csv 19,531,802 bytes, Test.csv 5,802,277, SampleSubmission.csv 429,661 and variable_definitions.csv 2,344, with the same text: "We've aggregated data on burned areas across Zimbabwe for each month since 2001. You'll be given the burn area data up to the end of 2013", target `burn_area` per cell and month, test 2014 to 2016. +- Siblings and duplicates: the family's record is `data_science_zw_2022_fire_prediction` (its links include the 2020 hackathon and cmu-africa-fighting-fire-with-data, same byte sizes; `TBD -> 2nd Tier`, a forecasting-like panel that needs TP-10); `predict_fire_extent` is a further re-run of the same files (one for the main session to align). +- Metric: RMSE (Evaluation page), as in 2020. +- Reject check (AI, 2026-10-22): stands. Zindi API (https://zindi.africa/competitions/wildfire-prediction-challenge): parent 'predict-fire-extent'; files Train.csv 19,531,802, Test.csv 5,802,277, SampleSubmission.csv 429,661, variable_definitions.csv 2,344 bytes and the 'Fighting_Fire_with_Data_Starter_Notebook.ipynb', the same as the 2020 fighting-fire-with-data-hackathon; same task text ("burn area data up to the end of 2013", predict 2014 to 2016). Same task, same source: PROV-4 (Must); the family is carried by data_science_zw_2022_fire_prediction (TBD -> 2nd Tier, TP-10 pending), so no task is lost. Replace the record's old 'Time-series (Forecasting)' marker with 'Duplicate'. + Zindi wildfire prediction challenge typically asks for forecasting burned area per spatial cell across future months using historical climate/satellite time-series. This is spatio-temporal forecasting over a horizon, which is out of scope for the benchmark. Source modality leans toward gridded/satellite-derived features rather than a natural cross-sectional tabular task. A human should verify whether any fixed cross-sectional framing exists, but as known it is forecasting, so No. diff --git a/curation/records/womens_mentorship_hackathon.md b/curation/records/womens_mentorship_hackathon.md index 1d624bd8..f6fad9da 100644 --- a/curation/records/womens_mentorship_hackathon.md +++ b/curation/records/womens_mentorship_hackathon.md @@ -4,24 +4,22 @@ name: womens-mentorship-hackathon checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier -decision_markers: -- Ethical Issue -- Too Small -- TBD +priority: 2 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: social science required_split: - '?' problem_type: Binary Classification +original_data_state: One Table source_links: -- https://zindi.africa/competitions/womens-mentorship-hackathon - https://zindi.africa/competitions/womens-hackathon +- https://zindi.africa/competitions/womens-mentorship-hackathon - https://zindi.africa/competitions/she-solves-hackathon-womens-hack-for-safety -- https://zindi.africa/competitions/womens-hackathon source_row: 1003 needs_review: - ai_unverified @@ -32,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 2: whether a South African woman experienced a crime in the last five years, from survey answers; real and large enough, but the label likely comes from the same interview as the features (SEL-7) and no decision is named. +- Source: first publication Zindi "Women's Hack for Safety", a one-day women-only hackathon on 10 April 2021 by the Agence Française de Développement, with DataFirst (UCT) and Stats SA named as partners (https://zindi.africa/competitions/womens-hackathon, read through Zindi's API). This record's "Women's Mentorship #1: Hack for Safety" (14 May to 19 July 2021) and "She Solves Hackathon - Women's Hack for Safety" (Feb to Mar 2022) are Zindi children of it with byte-identical files (Train.csv 2,495,328 bytes, Test.csv 1,063,766, VariableDefinitions.csv 948). As published: "The train set contains ~7700 individuals with various information including if they have experienced a crime in the last 5 years or not"; the test set "has ~3300 individuals" without labels. The survey wave is not named; Stats SA's Victims of Crime Survey (on DataFirst) is the likely origin, not confirmed. Columns not read (login). +- Use case: "build a machine learning model that predicts a woman's level of risk of being victimized by a crime given basic information about her and her life" (Description); no actor or decision is named (a safety programme that targets women at risk would be ours). Fit approximate at best. Regime IID (one survey cross-section), unless households or areas group the respondents. +- Siblings and duplicates: no other record; the three Zindi pages are one dataset and this record stands for the family (links reordered, first publication first, the repeated link dropped). External collections: none found. +- Papers and pages: none beyond the competition pages. +- Metric: AUC ("The error metric for this competition is Area Under the Curve", Evaluation page): roc_auc, a competition's leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: SEL-7, a label the respondent already knows, asked in the same interview as the features: check that the features are "basic information" (age, place, household) and not answers that follow a crime (feeling unsafe, avoidance, reporting) (LEAK); the survey wave and its weights. The old `Ethical Issue` marker is dropped: the source states no objection to ML use and the task as stated is not harmful (SEL-1), though the curator may weigh the sensitivity; the old `Too Small` is refuted by ~7,700 labelled rows. +- Priority: gap + (social science, thin), use case 0 (no decision named), source 0 (a competition page, survey not named), shape + (~7,700 respondents), risk - (SEL-7 and post-crime answers). + This Zindi women's mentorship hackathon asks to predict which women are at highest risk of being made a victim of a crime in South Africa. It is plausibly a genuine tabular classification task, but predicting individual crime-victimization risk for women raises clear ethical sensitivity, and hackathon datasets are usually small. There is almost no detail in the record (no size, features, target definition, or split regime). A human must inspect dataset size, the exact target, and weigh the ethical concern; provisionally 2nd Tier. --- diff --git a/curation/records/womxn_in_big_data_south_africa_female_headed_households_in_south_africa.md b/curation/records/womxn_in_big_data_south_africa_female_headed_households_in_south_africa.md index b24d2118..e981bcc1 100644 --- a/curation/records/womxn_in_big_data_south_africa_female_headed_households_in_south_africa.md +++ b/curation/records/womxn_in_big_data_south_africa_female_headed_households_in_south_africa.md @@ -3,19 +3,19 @@ unique_name: womxn_in_big_data_south_africa_female_headed_households_in_south_af name: womxn-in-big-data-south-africa-female-headed-households-in-south-africa checked_by: - AI (UNVERIFIED) -suggestion: TBD -> 2nd Tier -decision_markers: -- Too Small -- TBD +suggestion: TBD -> Yes +priority: 3 tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: social science required_split: - '?' problem_type: Regression +original_data_state: One Table source_links: - https://zindi.africa/competitions/womxn-in-big-data-south-africa-female-headed-households-in-south-africa/data - https://zindi.africa/competitions/indabax-sa-2022-beginner/data @@ -30,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: the ward-level share of poor female-headed households from 2011 South African census aggregates; the first publication of data Zindi re-ran at least four times; social science is thin, and 2,822 labelled wards refute the old `Too Small`. +- Source: Zindi competition "Womxn in Big Data South Africa: Female-Headed Households in South Africa", 25 Nov 2019 to 24 Feb 2020, sponsored by Women in Big Data South Africa and Microsoft with HERE Technologies (page read through Zindi's API); as published: "We have aggregated the target indicator and the other predictive variables from the 2011 census across all the households within each ward"; target "the percentage of households per ward that are both female-headed and earn an annual income that is below R19,600"; "2,822 wards in the train set" (labelled) and "1,013 wards in the test set" (no labels); Train.csv 2,049,620 bytes, Test.csv 1,113,085, variable_descriptions.csv 4,393. Named columns: ward centre `lat`, `lon`, ward code `ADM4_PCODE`, nightlights `NL`; the other columns not read (login). +- Use case: "it is important for policymakers and other actors to have accurate estimates of these statistics even in between census years ... by using data points that can be collected through other means without an intensive household survey like the census" (Description). Fit approximate: the features are themselves 2011 census aggregates, not the between-census sources the use case names (only `NL` is). Regime grouped (spatial): neighbouring wards share their municipality (in the ward code), so a random split lets neighbours inform each other; one census, no time axis. +- Siblings and duplicates: re-runs with the same text and byte sizes, proposed `No` (`Duplicate`) in this re-screen: `international_womens_day_challenge` (2025), `atf_2023_hackathon` (2023); `be_a_trailblazer` carries the same target text (likely a re-run, not checked). The record's second and third links (indabax-sa-2022-beginner, Dec 2022; dsa-2021, Oct 2021) are Zindi children of this competition (the API's parent field), same data description and metric. External collections: none found. +- Papers and pages: no paper; the Description, Data and Evaluation pages; the winners' solutions were promised public (not read). +- Metric: RMSE ("The error metric for this competition is the Root Mean Squared Error", Evaluation page); a competition's leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the column list is unread: census aggregates close to the target's parts (share of female heads, income bands) would make it near-deterministic (SEL-5, LEAK); only the 2,822 train wards are labelled; the spatial grouping key to choose (municipality from `ADM4_PCODE`; SPLIT); the rules forbid redistribution (no criterion, EV-4). +- Priority: gap + (social science, 1 accepted dataset), use case 0 (named by the source, but the fit is approximate), source 0 (a competition page on Stats SA's census, no codebook read), shape + (2,822 wards, spatial structure), risk 0. + Womxn in Big Data (Zindi) asks to predict the percentage of female-headed households per area, a regression on census/survey-derived aggregate features (the original-source modality is often satellite/aggregate covariates). It is a real predictive regression task with a clear target, but such aggregate-area datasets tend to be small with limited rows. The record gives no size or feature details and the split regime (likely grouped/spatial) is unknown. A human must check row count after cleaning, feature provenance, and whether it is representative tabular ML; provisionally 2nd Tier. --- diff --git a/curation/records/wosis_soil_profiles.md b/curation/records/wosis_soil_profiles.md index 6477fa1c..105aa016 100644 --- a/curation/records/wosis_soil_profiles.md +++ b/curation/records/wosis_soil_profiles.md @@ -4,6 +4,7 @@ name: WoSIS snapshot 2023 (World Soil Information Service, ISRIC) checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 3 decision_markers: - Needs extensive data wrangling tags: @@ -33,6 +34,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 3: ISRIC's global database of measured, standardised soil profiles; documented and large, but the source defines no task, ships no covariates, and contains `africa_soil_profiles_database`. +- Source: ISRIC - World Soil Information, WoSIS snapshot 2023 (DOI 10.17027/isric-wdcsoils-20231130), described in Batjes, Calisto and de Sousa, Earth System Science Data 16:4735-4765, 2024; as published (ESSD abstract, read 2026-10-22): "quality-assessed and standardised data for 228 000 geo-referenced profiles", "from 174 countries", "more than 900 000 soil layers (or horizons)", "over 6 million records" of chemical (carbon, pH, phosphorus, CEC, ...) and physical (texture, bulk density, water retention, ...) properties grouped by comparable analytical method; distributed as a static snapshot and through web services ("wosis_latest"); profile, layer and attribute tables to join. Coverage skewed to North America, Oceania and Europe (the old note's shares, from the paper). +- Use case: the source's purpose is to "support digital soil mapping" (abstract), as the training points of SoilGrids; no prediction task is defined. Candidates: a property at a location from covariates (the covariates are not shipped, a remote-sensing join), or a pedotransfer function (a costly property such as CEC, bulk density or water retention from routine ones such as texture and organic carbon), which soil surveys use; fit approximate either way. Regime grouped: layers of one profile, and spatial blocks. +- Siblings and duplicates: `africa_soil_profiles_database` (`TBD -> 2nd Tier`, priority 3) is ingested by WoSIS: keep one; `lucas_topsoil` (`TBD -> Yes`, EU survey, also partly in WoSIS), `open_soil_data` (`No`, mostly model predictions), `open_soil_spectral_library`, `africa_soil_property_prediction_challenge` are the same family. External collections: none found. +- Papers and pages: the ESSD paper; ISRIC's WoSIS page; SoilGrids uses it. +- Metric: none found (ESSD abstract, ISRIC WoSIS page), so the fallback (rmse). +- Risks for /add-dataset: the target and inputs are ours to define (SEL-3, USE); joins of profile and layer tables; methods harmonised after the fact across decades; "almost 27% lack documented sampling dates"; overlap with AfSP and LUCAS (PROV-6); per-provider licences (no criterion, EV-4). +- Priority: gap + (a grouped regime is thin), use case 0 (none in the source), source + (an ESSD data paper and an institution's DOI), shape + (228k profiles, many properties, grouped structure), risk - (no defined target, joins, overlap with siblings). + Batjes et al., Earth System Science Data 16:4735, 2024: ISRIC's standardised global soil profile database, snapshot 2023 with 228,000 georeferenced profiles from 174 countries, >900,000 layers, >6 million records; standardised chemical (organic/total/inorganic C, N, P, pH, CEC, EC) and physical (texture, bulk density, coarse fragments, water retention) properties. Snapshot distributed under CC BY 4.0; some sources stay internal or embargoed. Coverage is skewed: North America 35%, Oceania 19%, Europe 17%, Asia 3%; 54% sampled 1980-2020 and "almost 27% lack documented sampling dates". Largest measured-label source for a digital-soil-mapping regression (points + covariates -> properties, spatially blocked split), but a legacy compilation: heterogeneous methods harmonised after the fact, many layers per profile, no covariates shipped, and it contains `africa_soil_profiles_database` (and much of what the iSDA maps trained on), so choose one of the two. Data not inspected. diff --git a/curation/records/xente_fraud_detection_challenge.md b/curation/records/xente_fraud_detection_challenge.md index 3fb487a3..9af4c182 100644 --- a/curation/records/xente_fraud_detection_challenge.md +++ b/curation/records/xente_fraud_detection_challenge.md @@ -4,6 +4,7 @@ name: xente-fraud-detection-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: - TBD tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2019' domain: finance required_split: - Temporal (NON-IID) @@ -31,4 +33,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: Xente's own e-commerce transactions with a fraud label, a direct fraud-screening task with a dated temporal split. +- Source: Zindi competition "Xente Fraud Detection Challenge" with Xente, Innovation Village and insight2impact, 2019-05-20 to 2019-09-23 (https://zindi.africa/competitions/xente-fraud-detection-challenge, read through Zindi's page API); as published: "a sample of approximately 140,000 transactions that occurred between 15 November 2018 and 15 March 2019"; `training.csv` ("Transactions from 15 November 2018 to 13 February 2019, including whether or not each transaction is fraudulent"), `test.csv` (13 February to 14 March 2019, unlabelled), `Xente_Variable_Definitions.csv`; target `FraudResult` (1 fraud, 0 not); "the data is highly imbalanced". Rows and columns not counted (login). +- Use case: "create a machine learning model to detect fraudulent transactions ... help Xente provide improved and safer service" (Description): Xente scores each transaction as it happens and blocks or reviews it; fit direct if the columns are the transaction fields known at authorisation; regime temporal (the source splits by date; customers recur, so a grouped check is worth one look). +- Siblings and duplicates: the other three links are reruns of the same data: `xente-fraud-dection-hackathon` (Alliance4AI SUP'COM, Nov 2020, API parent `xente-fraud-detection-challenge`), `multimedia-university-of-kenya-hackathon` (Apr 2024, same parent, same Data Description), `indabax-benin-2022` ("La détection des fraudes", Oct 2022, the same "approximately 140,000 transactions ... between 15 November 2018 and 15 March 2019" with the company name removed; also listed in `indabax_hackathons`). Other Xente records are different tasks: `sbtic_xente_credit_scoring_challenge` (loans), `xente_purchase_prediction_challenge` (recommendation). External collections: none found. +- Papers and pages: the competition page and its Evaluation tab; no paper by the publisher; public solutions on GitHub (claims, not read). +- Metric: F1 ("The error metric for this competition is the F1 score", Evaluation tab), on submitted 0/1 labels: a leaderboard metric (MET-4), threshold left to the participant (MET-6; precedent african_credit_scoring_challenge kept `f1`); inputs none. +- Risks for /add-dataset: only `training.csv` is labelled, so a temporal split covers three months with very few frauds per window (TP-5, TP-9); `Amount` and `Value` and the id columns (customer, account, subscription, batch) to check for redundancy and id leakage (LEAK); whether the fraud label is known only after a chargeback, so a gap may be needed (SPLIT). +- Priority: gap 0 (finance, binary, temporal not thin), use case + (Xente named, the decision is the transaction screen), source + (the publisher's dates and variable definitions), shape + (about 140k transactions, mixed types, timestamps), risk - (few positives per temporal window). + Xente fraud detection (Zindi) is a transaction-level financial fraud classification task with a clear binary target (fraudulent vs not) and standard tabular transaction features (amount, channel, product, customer/timestamp). This is a representative real-world tabular classification problem, well known, and likely benefits from a temporal split given timestamps. The main caveats are extreme class imbalance and the need to confirm dataset size and feature usability after cleaning. Provisionally a Yes pending human verification of size, target, and leakage. diff --git a/curation/records/xente_purchase_prediction_challenge.md b/curation/records/xente_purchase_prediction_challenge.md index 25aade2a..3348a20b 100644 --- a/curation/records/xente_purchase_prediction_challenge.md +++ b/curation/records/xente_purchase_prediction_challenge.md @@ -28,6 +28,14 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Out-of-scope Task (CTR/RecSys/Ranking)`, SEL-9): a next-purchase recommendation over an account x day x product grid, built from a bare purchase log. +- Source: Zindi one-day student hackathon "UmojaHack #2: Xente Purchase Prediction Challenge (INTERMEDIATE)", 2020-03-21, Xente (Uganda) (https://zindi.africa/competitions/umojahack-2-xente-purchase-prediction-challenge, read through Zindi's page API); as published: `Train.csv` of transactions 2019-11-01 to 2020-02-23, "Each row contains an account ID ('acc'), the time and date of the transaction ('date') and the product ID ('PID')", `PID_Categories.csv`; the test is the week from 2020-02-23, "one row for each account for each day for each product". +- Use case: "predict what and when individuals will purchase next, based on their purchase history ... help Xente with target marketing": a recommender for the app; Zindi tags it "Forecast", "Product Recommendation", "Market Basket Analysis". +- Siblings and duplicates: other Xente records are other tasks (`xente_fraud_detection_challenge`, `sbtic_xente_credit_scoring_challenge`); the same pattern as `farm_to_feed_shopping_basket_recommendation_challenge` (customer x SKU x week grid, the 2026-10-07 assessment recommended `No` for the same reason) and `zimnat_insurance_recommendation_challenge`. +- Metric: F1 over the grid ("The error metric for this competition is the F1 Score", Evaluation tab). +- Why it fails: the task is which product each account buys on each day, a recommendation and next-event forecast over user-item pairs (SEL-9, and SEL-10 for the "when"); the data holds no attribute beyond the interaction log, so no tabular task remains after removing the framing. +- Reject check (AI, 2026-10-22): stands. SEL-9 (Must), SEL-10 as well. Zindi: "predict what and when individuals will purchase next, based on their purchase history ... help Xente with target marketing". Data: "Each row contains an account ID ('acc'), the time and date of the transaction ('date') and the product ID ('PID')". Submission: "one row for each account for each day for each product". Zindi tags include Product Recommendation, Market Basket Analysis and Personalization. The data has no attribute beyond the interaction log, so no tabular task is left once the recommender framing is removed. Andrej's open 'Yes' on farm_to_feed_shopping_basket_recommendation_challenge is not a like case: that grid has customer and SKU attributes, and it is undecided. https://zindi.africa/competitions/umojahack-2-xente-purchase-prediction-challenge + This UmojaHack/Xente challenge asks to predict when and what a customer will purchase on the Xente app, which is a recommendation/next-purchase prediction task. Such 'what will the user buy next' framings are out-of-scope RecSys/ranking tasks rather than a fixed tabular classification/regression. The data is user-item interaction logs rather than a clean predictive table. A human could confirm, but as described this is out of scope, so No. --- diff --git a/curation/records/yassir_eta_prediction_challenge.md b/curation/records/yassir_eta_prediction_challenge.md index e80958ee..7f64c416 100644 --- a/curation/records/yassir_eta_prediction_challenge.md +++ b/curation/records/yassir_eta_prediction_challenge.md @@ -4,6 +4,7 @@ name: Yassir ETA Prediction Challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 3 decision_markers: - TBD tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2020' domain: technology & internet required_split: - Temporal (NON-IID) @@ -18,6 +20,8 @@ problem_type: Regression original_data_state: One Table source_links: - https://zindi.africa/competitions/umojahack-south-africa-yassir-eta-prediction-challenge +- '# the sister hackathon on the same day with the same data (Algeria; Zindi API, no parent)' +- https://zindi.africa/competitions/umojahack-algeria-yassir-eta-prediction-challenge source_row: 1026 needs_review: - ai_unverified @@ -28,6 +32,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 3: Yassir's trip-time (ETA) regression from origin, destination, route distance and time, with a date split; a clear ride-hailing quote task. +- Source: Zindi one-day hackathons "UmojaHack South Africa: Yassir ETA Prediction Challenge" and "UmojaHack Algeria: Yassir ETA Prediction Challenge", both 2020-07-25 (https://zindi.africa/competitions/umojahack-south-africa-yassir-eta-prediction-challenge, https://zindi.africa/competitions/umojahack-algeria-yassir-eta-prediction-challenge, read through Zindi's page API); as published: "details for 119,549 trips (train and test are split by date)"; columns `ID`, `Timestamp` ("Time that the trip was started"), `Origin_lat`, `Origin_lon`, `Destination_lat`, `Destination_lon` ("to within approximately 100m"), `Trip_distance` ("Distance in meters on a driving route"), target `ETA` ("Estimated trip time in seconds"); `Weather.csv` with daily ERA5 temperature, rainfall and wind, joined by date. +- Use case: "predict the estimated time of arrival at the dropoff point for a single Yassir journey" to make "Yassir's services more reliable": the app quotes a trip time when a ride is booked; fit direct (every column is known at booking); regime temporal (the source splits by date; quoting future trips). +- Siblings and duplicates: `yassir_eta_prediction_challenge_for_azubian` is a knowledge rerun of the Algeria hackathon (API parent `umojahack-algeria-yassir-eta-prediction-challenge`, the same Data Description), so a duplicate of this record (proposed `No` in the same batch); the Yango ride-time hackathons (`2024_lusaka_mobility_prediction_hackathon` and its Accra and Maputo siblings) are other cities and another company; ride-time precedents `nyc_taxi_trip_duration`, `pkdd_15_taxi_trip_time_prediction_ii`, `maps_router_eta` (TabReD). External collections: none found. +- Papers and pages: the two competition pages; no paper. +- Metric: RMSE on `ETA` in seconds ("The error metric for this competition is the Root Mean Squared Error", Evaluation tab), a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the target is named "Estimated trip time": check whether it is the realised duration or Yassir's routing estimate (USE, an unclear target); `Trip_distance` is "along the fastest route", known at booking, fine; few features (6 plus daily weather), so a check against a distance-only baseline (TP-4); only Train.csv is labelled. +- Priority: gap 0 (technology & internet, regression, temporal not thin), use case + (Yassir and the ETA quote named), source + (the page documents every column), shape + (about 120k trips, coordinates, time, a weather join), risk - (the target's meaning to confirm). + Yassir ETA prediction (Zindi/UmojaHack) is a ride-hailing estimated-time-of-arrival regression: trip-level features (origin/destination, distance, time of day, etc.) map to trip duration. This is a representative real-world tabular regression with a clear continuous target, not forecasting a future-value series. A temporal split is likely appropriate given trips occur over time. A human must verify dataset size, feature columns, and that geospatial coordinates do not constitute the dominant signal; provisionally Yes. --- diff --git a/curation/records/yassir_eta_prediction_challenge_for_azubian.md b/curation/records/yassir_eta_prediction_challenge_for_azubian.md index 7eb54c4a..78c3dbb5 100644 --- a/curation/records/yassir_eta_prediction_challenge_for_azubian.md +++ b/curation/records/yassir_eta_prediction_challenge_for_azubian.md @@ -3,10 +3,9 @@ unique_name: yassir_eta_prediction_challenge_for_azubian name: yassir-eta-prediction-challenge-for-azubian checked_by: - AI (UNVERIFIED) -suggestion: TBD -> Yes +suggestion: 'No' decision_markers: - Duplicate -- TBD tags: - AI-Filled (Verify) collections: @@ -29,4 +28,11 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Duplicate`, PROV-4): a knowledge-competition rerun of the UmojaHack Yassir ETA hackathon, the same data as `yassir_eta_prediction_challenge`. +- Source: Zindi knowledge competition "Yassir ETA Prediction Challenge For Azubian", 2022-11-07 to 2024-06-06 (https://zindi.africa/competitions/yassir-eta-prediction-challenge-for-azubian, read through Zindi's page API); Zindi's API names its parent: `umojahack-algeria-yassir-eta-prediction-challenge` (2020-07-25); the Data Description is word for word the UmojaHack one: "The data contains details for 119,549 trips (train and test are split by date)", the same variables (`Timestamp`, origin and destination coordinates, `Trip_distance`, target `ETA`) and `Weather.csv`; "This is a learning competition. Aside from knowledge, there are no prizes". +- Siblings and duplicates: `yassir_eta_prediction_challenge` (UmojaHack South Africa, the same day and the same data as UmojaHack Algeria) is the canonical record; this rerun adds no task. +- Metric: RMSE, as in the original. +- Why it fails: two records that end at the same original source (the UmojaHack Yassir ETA data of 2020-07-25) are duplicates (PROV-4); the first publication is the hackathon (PROV-8). +- Reject check (AI, 2026-10-22): stands. PROV-4/PROV-5 (Must), source confirmed. Zindi API: parent = umojahack-algeria-yassir-eta-prediction-challenge (2020-07-25). The Data Description is word for word the UmojaHack one: "The data contains details for 119,549 trips (train and test are split by date)". It has the same files (Train, Test, Weather.csv, starter notebook) and the same target ETA. It says "This is a learning competition. Aside from knowledge, there are no prizes". The task is the same, so it is a duplicate of yassir_eta_prediction_challenge, whose source_links already hold the Algeria page; add this link there. https://zindi.africa/competitions/yassir-eta-prediction-challenge-for-azubian + This is a Yassir ETA prediction challenge variant (the 'for Azubian' edition), the same trip-duration regression task as the other Yassir ETA challenge. As a regression with a clear continuous target it is in scope, but it is very likely a re-run on overlapping or identical data, so it may duplicate yassir_eta_prediction_challenge. A human must verify whether the underlying data differs from the other Yassir record and, if not, keep only one. Provisionally Yes if unique, otherwise drop as duplicate. diff --git a/curation/records/ycbench.md b/curation/records/ycbench.md index 6ac96cf1..13bb9973 100644 --- a/curation/records/ycbench.md +++ b/curation/records/ycbench.md @@ -4,16 +4,21 @@ name: ycbench checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: -- Missing source information - TBD +- Too Small tags: - AI-Filled (Verify) +original_source: Github +year: '2026' required_split: - '?' problem_type: TBD source_links: - https://www.ycbench.com/ +- https://arxiv.org/abs/2604.02378 +- https://github.com/benstaf/ycbench source_row: 1041 needs_review: - ai_unverified @@ -24,4 +29,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the source is now known (an arXiv paper and a GitHub repo), but it is a live forecasting leaderboard over one Y Combinator batch of 196 startups, scored as a top-k ranking on a composite web-traction score; tiny and of doubtful target. +- Source: Mostapha Benhenda, "YC Bench: a Live Benchmark for Forecasting Startup Outperformance in Y Combinator Batches", arXiv 2604.02378 (April 2026), code and data at https://github.com/benstaf/ycbench; the site (https://www.ycbench.com/) says "YC Bench is a live benchmark to forecast the top 10% Y Combinator startups at Demo Day". As published: the YC W26 batch, 196 startups; files `yc_w26_startups.csv`, `yc_mentions.csv`, `yc_mentions_early.csv`, `yc_w26_traction.csv`, `yc_w26_pre_demo_scores.csv`, `yc_all_companies.json`; the target is the "Pre-Demo Day Score", "a short-term proxy metric that combines public traction signals and Google web mentions"; the baseline ranks by pre-application Google mentions. +- Use case: investors "Identify and access hot deals before Demo Day" (site); fit approximate (the label is a web-visibility proxy measured over 90 days, not an investment outcome); regime: cohort-relative, one batch, so a grouped-by-batch or temporal setup only once several batches exist. +- Siblings and duplicates: none (grep ycbench, Y Combinator). External collections: none found. +- Papers and pages: the arXiv paper (abstract read, text not), the README, the site. +- Metric: precision@20 (70%) and recall@11 (55%) of the Demo Day top performers (abstract, README): a ranking cut-off, not one of our task metrics, so the fallback if the task were kept (MET-5, MET-6); inputs: the batch, as a group. +- Risks for /add-dataset: 196 rows and about 11-20 positives (TP-5, `Too Small`); the label is a composite of the same public signals that would serve as features (SEL-5, LEAK); a top-k-within-cohort framing close to ranking (SEL-9); a live benchmark whose ground truth is published after the predictions, by design for LLM forecasters. +- Priority: gap 0, use case 0 (a proxy label), source + (paper and code), shape - (196 rows, one batch), risk - (proxy target built from the features). + The record only gives the name 'ycbench' and a link to ycbench.com, with no comments, collection, problem type, target, or size. It is not a Zindi competition and I have no reliable knowledge of what ycbench contains or whether it is a tabular predictive task. There is too little information to decide, and the criteria say not to reject solely for missing info. A human must open the source and determine the task type, target, modality, and size before any verdict; provisionally 2nd Tier. diff --git a/curation/records/zimnat_insurance_recommendation_challenge.md b/curation/records/zimnat_insurance_recommendation_challenge.md index 15718196..cf6f7641 100644 --- a/curation/records/zimnat_insurance_recommendation_challenge.md +++ b/curation/records/zimnat_insurance_recommendation_challenge.md @@ -29,6 +29,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `No` (`Out-of-scope Task (CTR/RecSys/Ranking)`, SEL-9): a hold-one-out product recommendation over a customer x product holdings matrix, which Zindi itself tags as collaborative filtering. +- Source: Zindi competition "Zimnat Insurance Recommendation Challenge", 2020-07-01 to 2020-09-13, Zimnat (Zimbabwe) (https://zindi.africa/competitions/zimnat-insurance-recommendation-challenge, read through Zindi's page API); as published: "data on nearly 40,000 customers who have purchased two or more insurance products"; one row per customer with "some information on the customer (when they joined, birth year etc)", the occupation, occupation category and branch codes, then "The final 21 columns correspond to the 21 products on offer"; in Test "for each customer ONE product has been removed (the 1 replaced with a 0). Your goal is to build a model to predict the missing product". +- Use case: "Zimnat wants an ML model to use customer data to predict which kinds of insurance products to recommend to customers ... This same model can then be applied to any customer to identify insurance products that might be useful to them". +- Siblings and duplicates: the second link, `indabax-sa-2022-intermediate` ("Insurance Recommendation Challenge", Dec 2022), is a rerun (API parent `zimnat-insurance-recommendation-challenge`, the same text); other Zimnat records are other tasks (`zimnat_loan_default_predictio`, `the_zimnat_insurance_assurance_challenge`); same pattern: `farm_to_feed_shopping_basket_recommendation_challenge`, `xente_purchase_prediction_challenge`. +- Metric: log loss over every customer x product pair ("For every customer ID in the test set, for each product code, you must submit a prediction", Evaluation tab). +- Steelman: insurance is a thin domain, and a per-product propensity model on the customer attributes is a classification; but the source's task is to recover a masked item from the other items a customer holds (the holdings are the main input), a recommendation set-up, and no observed later purchase is the label. +- Why it fails: a recommendation task by the source's own words and design (SEL-9). +- Reject check (AI, 2026-10-22): stands. SEL-9 (Must) in the source's own words and design. "Zimnat wants an ML model to use customer data to predict which kinds of insurance products to recommend to customers". Test: "for each customer ONE product has been removed (the 1 replaced with a 0). Your goal is to build a model to predict the missing product". Zindi type_of_problem: ["Prediction", "Collaborative Filtering"]. The label is an artificially masked held item, not an observed later purchase, so a per-product propensity reframing would be our own task (SEL-8). It is consistent with the curator's RecSys rejects (kasandr, reasoner_recommendation, anime_planet; Lennart). The one counter-precedent, Andrej's 'Yes' on farm_to_feed, is still open, and the AI recommended 'No' there. https://zindi.africa/competitions/zimnat-insurance-recommendation-challenge + Zimnat insurance recommendation challenge (Zindi) asks to predict which insurance product a client wants next, an explicit recommendation/next-product task. This is an out-of-scope RecSys/ranking problem rather than a standard tabular classification/regression with a single fixed target. Even though the underlying customer table is tabular, the task framing is recommendation. The name itself flags it; suggestion is No. --- diff --git a/curation/records/zimnat_loan_default_predictio.md b/curation/records/zimnat_loan_default_predictio.md index 29079d84..10437266 100644 --- a/curation/records/zimnat_loan_default_predictio.md +++ b/curation/records/zimnat_loan_default_predictio.md @@ -4,6 +4,7 @@ name: Zimnat Loan Default Predictio checked_by: - AI (UNVERIFIED) suggestion: TBD -> Yes +priority: 1 decision_markers: - TBD tags: @@ -11,6 +12,7 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2020' domain: finance required_split: - '?' @@ -18,6 +20,7 @@ problem_type: Binary Classification original_data_state: One Table source_links: - https://zindi.africa/competitions/umojahack-zimbabwe/data +- https://zindi.africa/competitions/umojahack-zimbabwe source_row: 1018 needs_review: - ai_unverified @@ -28,6 +31,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 1: a real lender's loan table with an overdue flag; in scope, but the flag is a status at an unknown snapshot over loans already running, so the prediction point needs work. +- Source: Zindi private hackathon "UmojaHack Zimbabwe: Zimnat Loan Default Prediction", 2020-10-17 (https://zindi.africa/competitions/umojahack-zimbabwe, read through Zindi's page API); as published: "a sample dataset of over 12,000 loans, many of them with money overdue"; "Each row in the dataset corresponds to a loan ... information about the loan (starting dates, maturity date, amount, interest rate etc) as well as some information on the customer"; target `Target`, "1 indicates that the loan has an unpaid payment due, a 0 indicates that all is well"; Train.csv, Test.csv, VariableDescription.csv. Rows and columns not counted (login). +- Use case: "Your challenge is to predict which loans will end in a missed payment" so the lender can "manage that risk"; fit approximate (no decision moment named; if the loans are scored at origination, the columns must be the application fields only); regime temporal by start date (to confirm from the columns). +- Siblings and duplicates: no other record of this data (grep zimnat: `zimnat_insurance_recommendation_challenge` and `the_zimnat_insurance_assurance_challenge` are other Zimnat tasks); other default records (`loan_default_prediction_challenge`, `african_credit_scoring_challenge`) are other lenders. External collections: none found. +- Papers and pages: the competition page; no paper. +- Metric: ROC AUC ("The evaluation metric for this competition is the Area Under the ROC curve (AUC) ... This will allow the clients to set thresholds to their own needs", Evaluation tab): a deliberate leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the overdue status is measured at one unstated date while loans have different ages and maturities, so loan age drives the label (SEL-6 to weigh, USE); columns set after origination (balance, days overdue, payments) would leak (LEAK); confirm the start dates for a temporal split. +- Priority: gap 0 (finance, binary not thin), use case 0 (no decision moment), source 0 (a hackathon page; the variable file not read), shape + (12,000 loans, dates and amounts), risk - (snapshot label, possible post-origination columns). + Zimnat loan default prediction (UmojaHack Zimbabwe, Zindi) asks to predict which customers will default on a loan, a clean binary classification with a clear target and standard credit/customer tabular features. This is a representative real-world finance task. As a UmojaHack/hackathon dataset it may be modest in size and could carry imbalance. A human must verify row count after cleaning, the exact target and feature set, and the appropriate split; provisionally Yes. --- diff --git a/curation/records/zindi_user_behaviour_birthday_challenge.md b/curation/records/zindi_user_behaviour_birthday_challenge.md index 7ad75524..79e4b48c 100644 --- a/curation/records/zindi_user_behaviour_birthday_challenge.md +++ b/curation/records/zindi_user_behaviour_birthday_challenge.md @@ -4,17 +4,20 @@ name: zindi-user-behaviour-birthday-challenge checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - TBD +- Needs extensive data wrangling tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2021' domain: technology & internet required_split: - '?' -problem_type: TBD +problem_type: Binary Classification source_links: - https://zindi.africa/competitions/zindi-user-behaviour-birthday-challenge/data source_row: 913 @@ -27,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: a churn-like task on Zindi's own users (active next month or not), real and binary, but masked, spread over six activity tables and framed as a user-month panel, so a forecasting-like check comes first. +- Source: Zindi's own competition "Zindi User Behaviour Birthday Challenge", 2021-09-24 to 2022-01-23 (https://zindi.africa/competitions/zindi-user-behaviour-birthday-challenge, read through Zindi's page API); as published: "a subset of Zindi user activity. All variables have been masked"; `Users.csv`, `Competitions.csv`, `CompetitionPartipation.csv`, `Submissions.csv`, `Discussions.csv`, `Comments.csv`, `VariableDefinitions.csv`, and `Train.csv` ("a summarized table of the above activities ... it is recommended you pull features from the above tables"), `Test.csv` (user, month, year); target: "1 indicates that the user was active on Zindi in that month" (entered a competition, submitted, or posted in the forums). +- Use case: "help us establish and track our performance as a data science community and platform" (Description): Zindi would target inactive users; no decision named, fit approximate; regime temporal (user-months), users recur (grouped structure). +- Siblings and duplicates: none (grep birthday, user behaviour). External collections: none found. +- Papers and pages: the competition page; no paper. +- Metric: ROC AUC ("The evaluation metric for this competition is Area Under the Curve (AUC)", Evaluation tab), a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: a user-month panel where last month's activity is the obvious predictor: the TP-10 naive-profile test decides between SEL-10 and SEL-11; activity tables to aggregate up to each prediction month (joins, LEAK on dates); masked columns. +- Priority: gap 0 (technology & internet, binary, temporal not thin), use case 0 (no decision named), source 0 (masked, a competition page), shape + (six tables with dates), risk - (forecasting-like panel, heavy joins). + This Zindi 'user behaviour birthday challenge' record has only a name and data link, with no comments describing the target. Based on the name it concerns user-behaviour modeling, which could be a legitimate classification/regression but might also be a recommendation or engagement-prediction task that is out of scope. There is too little information to decide the task type, target, modality, or size. A human must open the source to determine the target and protocol; provisionally 2nd Tier. diff --git a/curation/records/zindi_weekendz_laduma_analytics_football_league_winners_prediction_challenge.md b/curation/records/zindi_weekendz_laduma_analytics_football_league_winners_prediction_challenge.md index a4bb4d12..13c00d78 100644 --- a/curation/records/zindi_weekendz_laduma_analytics_football_league_winners_prediction_challenge.md +++ b/curation/records/zindi_weekendz_laduma_analytics_football_league_winners_prediction_challenge.md @@ -4,6 +4,7 @@ name: zindi-weekendz-laduma-analytics-football-league-winners-prediction-challen checked_by: - AI (UNVERIFIED) suggestion: TBD -> 2nd Tier +priority: 1 decision_markers: - Too Small - TBD @@ -12,10 +13,11 @@ tags: collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: Other required_split: - Temporal (NON-IID) -problem_type: TBD +problem_type: Multiclass Classification source_links: - https://zindi.africa/competitions/zindi-weekendz-laduma-analytics-football-league-winners-prediction-challenge/data source_row: 989 @@ -28,4 +30,13 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> 2nd Tier`, priority 1: the first release of the Laduma Analytics match-outcome data, the same task as the later rerun `laduma_analytics_football_league_winners_prediction_challenge`; small, built from in-match events, and to be kept in one record only. +- Source: #ZindiWeekendz hackathon "Laduma Analytics Football League Winners Prediction Challenge" with Laduma Analytics and Layer, 2022-06-03 to 2022-06-05 (https://zindi.africa/competitions/zindi-weekendz-laduma-analytics-football-league-winners-prediction-challenge, read through Zindi's page API); as published: "collected over 3 years, following 22 teams in an African country. The player and team's names have been masked"; event data ("shots, passes, tackles, and dribbles ... X and Y coordinates"); "The train set is two seasons and the test one season"; `Train.csv`, `Test.csv`, `game_statistics.csv`, `VariableDefinitions.csv`; target per `Game_ID`: Home Win, Draw, Away Win. +- Use case: "predict the outcome of a football match, based on historical match and player data ... useful to Laduma Analytics to widen the types of services they offer"; no decision named, fit approximate; regime temporal (the source splits by season). +- Siblings and duplicates: `laduma_analytics_football_league_winners_prediction_challenge` (the rerun of 2022-06-06 to 2022-09-04, log loss, "23 teams", separate train and test game statistics; `TBD -> 2nd Tier`, priority 1): the same publisher and data, so keep one record (PROV-4); this page is the first publication (PROV-8), the rerun has the deliberate log-loss metric and the clearer files; the curator picks. Other sport records: fifa, football_manager_important_matches, profb. External collections: none found. +- Papers and pages: the competition page; no paper. +- Metric: accuracy ("The evaluation metric for this challenge is Accuracy", Evaluation tab) on hard labels; the rerun moved to log loss, which MET-8 would prefer; inputs none. +- Risks for /add-dataset: `game_statistics.csv` holds the match's own events (shots, goals), so the outcome may be read off the game rather than predicted before kick-off (LEAK, USE); event tables to aggregate per game (joins); about three seasons of one league, a few hundred games (TP-5, TP-9). +- Priority: gap 0 (sport, under Other), use case 0 (no decision), source 0 (masked, a weekend hackathon), shape - (few hundred games), risk - (in-match statistics, duplicate with the rerun). + This Zindi Weekendz challenge asks to predict football league match/winners outcomes, a sports-results prediction task. It could be framed as a genuine classification (win/draw/loss) but Zindi Weekendz datasets are typically very small and the task usually requires a temporal split since matches occur in sequence. The record provides no size, features, or exact target. A human must inspect dataset size, the target definition, and the split regime; provisionally 2nd Tier given likely small size. diff --git a/curation/records/zindi_weekendz_layerai_air_quality_prediction_challenge.md b/curation/records/zindi_weekendz_layerai_air_quality_prediction_challenge.md index 9327ff80..c21b5725 100644 --- a/curation/records/zindi_weekendz_layerai_air_quality_prediction_challenge.md +++ b/curation/records/zindi_weekendz_layerai_air_quality_prediction_challenge.md @@ -3,14 +3,16 @@ unique_name: zindi_weekendz_layerai_air_quality_prediction_challenge name: zindi-weekendz-layerai-air-quality-prediction-challenge checked_by: - AI (UNVERIFIED) -suggestion: 'No' +suggestion: TBD -> Yes +priority: 2 decision_markers: -- Time-series (Forecasting) +- TBD tags: - AI-Filled (Verify) collections: - New (BeyondArena) original_source: Zindi +year: '2022' domain: environmental science & climate required_split: - Temporal (NON-IID) @@ -28,6 +30,15 @@ type_adapter_id: curation-record-v1 **⚠️ AI-FILLED — UNVERIFIED. The suggestion, metadata, and notes below were drafted by an AI assistant from public knowledge of this competition/dataset, NOT from inspecting the data. A human must verify everything before relying on it.** +**Screening (AI, 2026-10-22):** `TBD -> Yes`, priority 2: estimating PM2.5 at AirQo sensor sites from same-day Sentinel-5P retrievals, so not a forecast (the old `No` with `Time-series (Forecasting)` is refuted by the page); the first release of the data that the rerun record `layerai_air_quality_prediction_challenge` also carries. +- Source: #ZindiWeekendz hackathon "Layer.ai Air Quality Prediction Challenge" with AirQo (Makerere University) and Layer, 2022-07-01 to 2022-07-03 (https://zindi.africa/competitions/zindi-weekendz-layerai-air-quality-prediction-challenge, read through Zindi's page API); as published: `train.csv` (with the target), `test.csv` ("without the target-related columns"); target `pm2_5` from AirQo's sensors; features "extracted from google earth engine": SO2, CO, NO2, HCHO, UV aerosol index, O3 (NRTI L3) and cloud (OFFL L3). +- Use case: "The use of satellite data could expand air quality predictions to areas without air quality sensor devices"; "predict air quality readings from AirQo's sensors using Sentinel 5P data" (Description): AirQo estimates PM2.5 where it has no sensor; fit direct; regime grouped by site for new places (the use case), or temporal as the release split; one regime to pick (USE-10). +- Siblings and duplicates: `layerai_air_quality_prediction_challenge` is the rerun (2022-07-04 to 2022-10-02, MAE, `TBD -> Yes`, priority 3): the same task and publisher, keep one record (PROV-4); this page is the first publication (PROV-8). Related but distinct: `airqo_african_air_quality_prediction_challenge`, `urban_air_pollution_challenge`, `airqo_ugandan_air_quality_forecast_challenge` (a forecast). External collections: none found. +- Papers and pages: the competition page; the rerun's 1st-place write-up (Zindi discussion 13525) treated it as a time series. +- Metric: MAE ("The evaluation metric for this challenge is Mean Absolute Error", Evaluation tab), a leaderboard metric (MET-4); inputs none. +- Risks for /add-dataset: the Sentinel-5P columns are retrieved atmospheric quantities (SEL-15), not image descriptors (SEL-13), to confirm; heavy missingness in the satellite columns; any lagged PM2.5 of the device would be a forecasting input (TP-10); duplicate with the rerun record. +- Priority: gap 0 (environmental not thin; grouped only if chosen), use case + (AirQo and the gap it fills named), source + (AirQo, the GEE products listed), shape 0 (size not checked), risk - (duplicate with the rerun, forecasting-like solutions). + The LayerAI air quality challenge (Zindi Weekendz) concerns PM2.5 air-quality prediction, typically forecasting pollutant concentrations over time/locations from satellite (e.g. Sentinel-5P) and meteorological time-series. This is a spatio-temporal forecasting task over a horizon, which is excluded from scope, and the features are largely satellite-derived rather than a natural cross-sectional tabular table. A human could confirm framing, but as known this is forecasting, so No. ---