Skip to content
Snippets Groups Projects
Commit 2d8783a5 authored by Michał Pogoda's avatar Michał Pogoda
Browse files

Working MVP of pipeline

parent 8bc5de43
Branches
2 merge requests!10Anonimizer v2,!7Better coverage
Pipeline #7500 failed with stage
in 19 seconds
Showing
with 61 additions and 1 deletion
defaults:
- detectors: all
- replacers: tag
- suppressor: order_based
- input_parser: ccl
- pipeline: default
- _self_
language: "pl"
\ No newline at end of file
defaults:
- date
- email
- ner
- phone
- url
- user
\ No newline at end of file
date:
_target_: src.detectors.date.DateDetector
language: $language
\ No newline at end of file
email:
_target_: src.detectors.email.EmailDetector
\ No newline at end of file
ner:
_target_: src.detectors.ner.NerDetector
language: ${language}
\ No newline at end of file
phone:
_target_: src.detectors.phone.PhoneNumberDetector
\ No newline at end of file
url:
_target_: src.detectors.url.UrlDetector
language: $language
\ No newline at end of file
user:
_target_: src.detectors.user.UserDetector
\ No newline at end of file
_target_: src.input_parsers.ccl.CCLInputParser
\ No newline at end of file
dictionaries_path: dictionaries
\ No newline at end of file
_target_: src.pipeline.default.DefaultPipeline
input_parser: ${input_parser}
detectors: ${detectors}
suppressor: ${suppressor}
replacers: ${replacers}
\ No newline at end of file
date:
_target_: src.replacers.date_replacer.DateReplacer
\ No newline at end of file
delete:
_target_: src.replacers.delete_replacer.DeleteReplacer
\ No newline at end of file
email:
_target_: src.replacers.email_replacer.EmailReplacer
\ No newline at end of file
ner:
_target_: src.replacers.ner_replacer.NERReplacer
dictionary:
_target_: src.dictionaries.morphosyntactic.pl_ner.PlNERMorphosyntacticDictionary
dictionary_path: ${paths.dictionaries_path}/pl_dict.txt
\ No newline at end of file
defaults:
- date
- email
- ner
- user
- tag # Fallback to tag replacement if no other replacement is found
\ No newline at end of file
tag:
_target_: src.replacers.tag_replacer.TagReplacer
\ No newline at end of file
user:
_target_: src.replacers.user_replacer.UserReplacer
\ No newline at end of file
_target_: src.suppressors.order_based.OrderBasedSuppressor
\ No newline at end of file
"""Implementation of anonymizer service."""
import argparse
import nlp_ws
from src.worker import Worker
from src.worker_old import Worker
def get_args():
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment