ACEL-Eval: A Benchmark Dataset for End-to-end Entity Linking in Classical Chinese

Overview

The ACEL-Eval dataset is constructed from two sections of Shiji, defines six entity types, and links them to a structured knowledge base. The benchmark includes two subtasks. The example in the subtasks means: "On the bingwu day of the eighth lunar month, the King of Qi intended to have the chancellor executed."

Datasets

Based on the data collection, annotation, and quality control procedures described above, we constructed the final ACEL-Eval dataset for entity linking in Classical Chinese. ACEL-Eval consists of annotated text records and a person-oriented knowledge base, and it is divided into training and test splits for standardized evaluation. An overview of the dataset composition is presented in Table 1, which summarizes the number of annotated entities and characters in each split across the six entity categories: person, location, state, official title, book, and time.

Table 1: Numbers of annotated entities and characters in the training and test splits.

Split	#Person	#Location	#State	#Official Title	#Book	#Time	#Character
Training Set	9,211	2,282	3,380	1,318	61	888	160,002
Test Set	812	322	96	250	6	358	18,565

Environment

All experiments were conducted using Python 3.8 and PyTorch 2.0.1.

How to Use Your Data

To run the code with your own data, open the corresponding settings/settings.json file and fill in the data paths there.

For example, in named_entity_recognition/settings/settings.json, set:

{
  "train_data_path": "/path/to/your/train_data.txt",
  "valid_data_path": "/path/to/your/valid_data.txt",
  "test_data_path": "/path/to/your/test_data.txt"
}

In entity_disambiguation/settings/settings.json, set the JSONL file paths in the same way:

{
  "train_data_path": "/path/to/your/train_data.jsonl",
  "valid_data_path": "/path/to/your/valid_data.jsonl",
  "test_data_path": "/path/to/your/test_data.jsonl"
}

After updating the paths, run the corresponding main.py file to start training.