Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
A
anonymizer
Manage
Activity
Members
Labels
Plan
Issues
0
Issue boards
Milestones
Wiki
Redmine
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
nlpworkers
anonymizer
Commits
ebfa99fc
Commit
ebfa99fc
authored
1 year ago
by
Paweł Walkowiak
Browse files
Options
Downloads
Patches
Plain Diff
Add checks
parent
0af138c0
1 merge request
!13
Fix anonymizer errors
Pipeline
#15187
passed with stages
in 1 minute and 38 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
src/dictionaries/morphosyntactic/ner_file.py
+19
-5
19 additions, 5 deletions
src/dictionaries/morphosyntactic/ner_file.py
with
19 additions
and
5 deletions
src/dictionaries/morphosyntactic/ner_file.py
+
19
−
5
View file @
ebfa99fc
...
...
@@ -191,11 +191,19 @@ class NERFileMorphosyntacticDictionary(MorphosyntacticDictionary):
original_entries
)
possible_lemmas
=
set
(
self
.
_dictionary
[
detection_type
][
required_tags
[
0
]].
keys
())
possible_lemmas
=
set
(
self
.
_dictionary
[
detection_type
][
required_tags
[
0
]].
keys
()
)
\
if
detection_type
in
self
.
_dictionary
\
and
required_tags
[
0
]
in
self
.
_dictionary
[
detection_type
]
\
else
set
()
for
tag
in
required_tags
[
1
:]:
possible_lemmas
.
intersection_update
(
self
.
_dictionary
[
detection_type
][
tag
].
keys
()
)
keys
=
self
.
_dictionary
[
detection_type
][
tag
].
keys
()
\
if
detection_type
in
self
.
_dictionary
\
and
tag
in
self
.
_dictionary
[
detection_type
]
\
else
set
()
if
keys
:
possible_lemmas
.
intersection_update
(
keys
)
if
len
(
possible_lemmas
)
==
0
:
return
[
self
.
get_random_replacement
(
original_entries
[
0
])]
*
len
(
...
...
@@ -208,7 +216,13 @@ class NERFileMorphosyntacticDictionary(MorphosyntacticDictionary):
for
entry
in
original_entries
:
if
isinstance
(
entry
,
MorphosyntacticInfoMixin
):
morpho_tag
=
entry
.
morpho_tag
word
=
self
.
_dictionary
[
detection_type
][
morpho_tag
][
lemma
]
if
detection_type
in
self
.
_dictionary
\
and
morpho_tag
in
self
.
_dictionary
[
detection_type
]
\
and
lemma
in
\
self
.
_dictionary
[
detection_type
][
morpho_tag
]:
word
=
self
.
_dictionary
[
detection_type
][
morpho_tag
][
lemma
]
else
:
word
=
lemma
else
:
word
=
lemma
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment