Skip to content
Snippets Groups Projects
Commit 2d8783a5 authored by Michał Pogoda's avatar Michał Pogoda
Browse files

Working MVP of pipeline

parent 8bc5de43
No related branches found
No related tags found
2 merge requests!10Anonimizer v2,!7Better coverage
Pipeline #7500 failed
Showing
with 61 additions and 1 deletion
defaults:
- detectors: all
- replacers: tag
- suppressor: order_based
- input_parser: ccl
- pipeline: default
- _self_
language: "pl"
\ No newline at end of file
defaults:
- date
- email
- ner
- phone
- url
- user
\ No newline at end of file
date:
_target_: src.detectors.date.DateDetector
language: $language
\ No newline at end of file
email:
_target_: src.detectors.email.EmailDetector
\ No newline at end of file
ner:
_target_: src.detectors.ner.NerDetector
language: ${language}
\ No newline at end of file
phone:
_target_: src.detectors.phone.PhoneNumberDetector
\ No newline at end of file
url:
_target_: src.detectors.url.UrlDetector
language: $language
\ No newline at end of file
user:
_target_: src.detectors.user.UserDetector
\ No newline at end of file
_target_: src.input_parsers.ccl.CCLInputParser
\ No newline at end of file
dictionaries_path: dictionaries
\ No newline at end of file
_target_: src.pipeline.default.DefaultPipeline
input_parser: ${input_parser}
detectors: ${detectors}
suppressor: ${suppressor}
replacers: ${replacers}
\ No newline at end of file
date:
_target_: src.replacers.date_replacer.DateReplacer
\ No newline at end of file
delete:
_target_: src.replacers.delete_replacer.DeleteReplacer
\ No newline at end of file
email:
_target_: src.replacers.email_replacer.EmailReplacer
\ No newline at end of file
ner:
_target_: src.replacers.ner_replacer.NERReplacer
dictionary:
_target_: src.dictionaries.morphosyntactic.pl_ner.PlNERMorphosyntacticDictionary
dictionary_path: ${paths.dictionaries_path}/pl_dict.txt
\ No newline at end of file
defaults:
- date
- email
- ner
- user
- tag # Fallback to tag replacement if no other replacement is found
\ No newline at end of file
tag:
_target_: src.replacers.tag_replacer.TagReplacer
\ No newline at end of file
user:
_target_: src.replacers.user_replacer.UserReplacer
\ No newline at end of file
_target_: src.suppressors.order_based.OrderBasedSuppressor
\ No newline at end of file
"""Implementation of anonymizer service."""
import argparse
import nlp_ws
from src.worker import Worker
from src.worker_old import Worker
def get_args():
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment