Stopwords

This repository is intended as an example of how the stopwords are produced and to provide the data used in the paper (Stop Words for Processing Software Engineering Documents: Do they Matter?)

Folder Structure

Stopwords
    |_ data_for_replications (contains all the required data for replicating software engineering tools)
       |_ Maalej_Dataset (original data for app review tool)
       |_ queries (queries used for RACKTool)
    |_ stackoverflow_questions (more than 10k top reviewed questions on stackoverflow)
    |_ stopwords_lists (all the stoplists)
    |_ replications
    |_ stackoverflow (code for creating the domain-specific corpus)

Expected Result

The result may vary by a small fraction depending on the trial, but it should be approximately the same as the tables below.

Tool 1 (App Review)

	PD  (bug report)	RT  (rating)	FR  (feature request)	UE  (user experience)
	Pre  Rec  F1	Pre  Rec  F1	Pre  Rec  F1	Pre  Rec  F1
SE domain (Poisson)	10.0% 37.5% 15.8%	72.1% 78.0% 74.9%	7.1% 29.8% 11.5%	11.6% 32.0% 17.0%
SE domain (TF-IDF)	10.7% 40.2% 16.9%	72.2% 78.2% 75.1%	7.9% 33.3% 12.8%	11.7% 32.5% 17.2%

Tool 2 (RACK)

	Top-10	MRR@10	MAP@10	MR@K
SE domain (Poisson)	83.85%	52.29%	43.27%	54.47%
SE domain (TF-IDF)	84.17%	53.20%	45.82%	56.8%

Tool 3 (Requirements Change Impact Analysis)

	SE domain (Poisson)	SE domain (TF-IDF)
Query 2	0.588	0.588
Query 4	0.981	0.981
Query 5	0.602	0.602

Overall Performance

Stop word list	Better	Worse	Same
No Stop Words	4	12	3
Very Small	10	7	2
Small	13	5	1
Medium	11	7	1
Large	11	8	0
Technology Domain	9	9	1
SE Domain (Poisson)	12	5	2
SE Domain (TF-IDF)	17	1	1