Building from git main generating errors it didn't before

[brett--anderson]

Problem Description

Hi, thanks for this great library! I used it a couple of months ago successfully but on returning my code no longer worked. I was installing the library as:

pip install git+https://github.com/midas-research/dlkp.git

The first thing I noticed is that when I loaded a model I had trained with the library months ago it started warning about a whole heap of roberta layers that were unused and recommend fine tuning before getting any use out of the model, which I didn't see before.

Then when I tried to use the library I received the error:

---> 81 all_kps = KEDatasets.extract_kp_from_tags(token_ids, tags)
     83 extracted_kps = self.tokenizer.batch_decode(
     84     all_kps,
     85     skip_special_tokens=True,
     86     clean_up_tokenization_spaces=True,
     87 )
     88 examples["extracted_keyphrase"] = extracted_kps

TypeError: extract_kp_from_tags() missing 1 required positional argument: 'tokenizer'

Which was very strange because looking at the code on github in the main branch that method did not expect a tokenizer in its signature.

I tried uninstalling and reinstalling but still the same strange code that I couldn't locate on git.

My Hack Solution

I managed to resolve the issue by installing from an earlier branch (ldkp):

pip install git+https://github.com/midas-research/dlkp.git@ldkp

I have no idea where or why it was pulling in code with method signatures that didn't match the code I could see in the repo. But this fixed it.

Any idea what's going on with the main branch?

Thanks!