tokenizer = AutoTokenizer.from_pretrained("camembert-base", use_fast=True)
fb_emotion = Fabula(scorer=TransformersScorer(model="astrosbd/french_emotion_camembert"),
segmenter=SlidingWindowTokenSegmenter(tokenizer = tokenizer,window_tokens=512, stride_tokens=256),
analysis="emotion")
Output :
Token indices sequence length is longer than the specified maximum sequence length for this model (14046 > 512). Running this sequence through the model will result in indexing errors
Je suppose un problème au niveau du self.tokeize dans segment.py, j'investigue
Output :
Token indices sequence length is longer than the specified maximum sequence length for this model (14046 > 512). Running this sequence through the model will result in indexing errorsJe suppose un problème au niveau du self.tokeize dans segment.py, j'investigue