Fix set_annotation_for_token function, because it always releases the tokens!
set_annotation_for_token
function is based on AnnotatedSentence class from Corpus2
. The function
uses AnnotatedSentence_wrap_sentence
method of to annotate sentences, but this method ALWAYS
releases the tokens! This means that it deletes existing tokens from sentences being annotated. To correct
this behaviour we have to change AnnotatedSentence_wrap_sentence
to AnnotatedSentence_wrap_sentence_clone
in cclutils
.
Code: https://gitlab.clarin-pl.eu/analysers/corpus2/-/blob/master/src/libcorpus2/ann/annotatedsentence.cpp
, lines 45, 46.