Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
C
combo
Manage
Activity
Members
Labels
Plan
Issues
20
Issue boards
Milestones
Wiki
Redmine
Code
Merge requests
2
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Syntactic Tools
combo
Commits
9218e33c
Commit
9218e33c
authored
4 years ago
by
Mateusz Klimaszewski
Committed by
Mateusz Klimaszewski
4 years ago
Browse files
Options
Downloads
Patches
Plain Diff
Extend training configuration.
parent
1b958e29
2 merge requests
!31
Release 1.0.2.
,
!30
Release 1.0.2.
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
scripts/train.py
+4
-0
4 additions, 0 deletions
scripts/train.py
scripts/train_eud.py
+9
-2
9 additions, 2 deletions
scripts/train_eud.py
scripts/utils.py
+7
-0
7 additions, 0 deletions
scripts/utils.py
with
20 additions
and
2 deletions
scripts/train.py
+
4
−
0
View file @
9218e33c
...
...
@@ -229,6 +229,10 @@ def run(_):
"
UD_Marathi-UFAL
"
,
"
UD_Norwegian-Bokmaal
"
}:
command
=
command
+
"
--targets deprel,head,upostag,lemma,feats
"
# Datasets without FEATS
if
treebank
in
{
"
UD_Japanese-GSD
"
,
"
UD_Korean-Kaist
"
}:
command
=
command
+
"
--targets deprel,head,upostag,xpostag,lemma
"
# Datasets without LEMMA and FEATS
if
treebank
in
{
"
UD_Maltese-MUDT
"
}:
command
=
command
+
"
--targets deprel,head,upostag,xpostag
"
...
...
This diff is collapsed.
Click to expand it.
scripts/train_eud.py
+
9
−
2
View file @
9218e33c
...
...
@@ -105,7 +105,8 @@ def run(_):
serialization_dir
=
pathlib
.
Path
(
FLAGS
.
serialization_dir
)
/
lang
serialization_dir
.
mkdir
(
exist_ok
=
True
,
parents
=
True
)
utils
.
execute_command
(
""
.
join
(
f
"""
combo --mode train
command
=
f
"""
combo --mode train
--training_data
{
train_path
}
--validation_data
{
dev_path
}
--targets feats,upostag,xpostag,head,deprel,lemma,deps
...
...
@@ -115,7 +116,13 @@ def run(_):
--word_batch_size 2500
--config_path
{
pathlib
.
Path
.
cwd
()
/
'
config.graph.template.jsonnet
'
}
--notensorboard
"""
.
splitlines
()))
"""
# Datasets without XPOS
if
lang
in
{
"
fr
"
}:
command
=
command
+
"
--targets deprel,head,upostag,lemma,feats
"
utils
.
execute_command
(
""
.
join
(
command
.
splitlines
()))
def
main
():
...
...
This diff is collapsed.
Click to expand it.
scripts/utils.py
+
7
−
0
View file @
9218e33c
...
...
@@ -4,6 +4,13 @@ import subprocess
LANG2TRANSFORMER
=
{
"
en
"
:
"
bert-base-cased
"
,
"
pl
"
:
"
allegro/herbert-base-cased
"
,
"
zh
"
:
"
bert-base-chinese
"
,
"
fi
"
:
"
TurkuNLP/bert-base-finnish-cased-v1
"
,
"
ja
"
:
"
cl-tohoku/bert-base-japanese
"
,
"
ko
"
:
"
kykim/bert-kor-base
"
,
"
de
"
:
"
dbmdz/bert-base-german-cased
"
,
"
ar
"
:
"
aubmindlab/bert-base-arabertv2
"
,
"
eu
"
:
"
ixa-ehu/berteus-base-cased
"
}
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment