When adding the benepar component to spacy ('3.8.13') using nlp.add_pipe("benepar", config={"model": "benepar_en3"}) I get an attribution error.
It appears that retokenization.py is calling a deprecated function on line 114, build_inputs_with_special_tokens(). In the current version of T5 (unsure when this was changed) this function has moved.
As a super quick fix, replacing
dummy_ids = self.tokenizer.build_inputs_with_special_tokens([-100])
with
dummy_ids = [self.tokenizer.pad_token_id, self.tokenizer.bos_token_id, -100, self.tokenizer.eos_token_id]
appears to work.
I've tested with _en3, _fr2, and _de2 and can now pipe in each successfully. With @Tannock's patch from Error loading German model
#103
When adding the benepar component to spacy ('3.8.13') using
nlp.add_pipe("benepar", config={"model": "benepar_en3"})I get an attribution error.It appears that retokenization.py is calling a deprecated function on line 114,
build_inputs_with_special_tokens(). In the current version of T5 (unsure when this was changed) this function has moved.As a super quick fix, replacing
dummy_ids = self.tokenizer.build_inputs_with_special_tokens([-100])with
dummy_ids = [self.tokenizer.pad_token_id, self.tokenizer.bos_token_id, -100, self.tokenizer.eos_token_id]appears to work.
I've tested with _en3, _fr2, and _de2 and can now pipe in each successfully. With @Tannock's patch from Error loading German model
#103