Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
C
combo
Manage
Activity
Members
Labels
Plan
Issues
20
Issue boards
Milestones
Wiki
Redmine
Code
Merge requests
2
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Syntactic Tools
combo
Commits
0eec74e1
Commit
0eec74e1
authored
1 year ago
by
Lukasz Pszenny
Browse files
Options
Downloads
Patches
Plain Diff
Change of lambo.py name to lambo_tokenizer.py
parent
ed7f3103
No related merge requests found
Pipeline
#10237
passed with stages
in 6 minutes and 59 seconds
Changes
3
Pipelines
2
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
combo/predict.py
+3
-3
3 additions, 3 deletions
combo/predict.py
combo/utils/lambo_tokenizer.py
+0
-0
0 additions, 0 deletions
combo/utils/lambo_tokenizer.py
docs/prediction.md
+2
-2
2 additions, 2 deletions
docs/prediction.md
with
5 additions
and
5 deletions
combo/predict.py
+
3
−
3
View file @
0eec74e1
...
...
@@ -12,7 +12,7 @@ from overrides import overrides
from
combo
import
data
from
combo.data
import
sentence2conllu
,
tokens2conllu
,
conllu2sentence
from
combo.utils
import
download
,
graph
,
lambo
from
combo.utils
import
download
,
graph
,
lambo
_tokenizer
logger
=
logging
.
getLogger
(
__name__
)
...
...
@@ -59,7 +59,7 @@ class COMBO(predictor.Predictor):
def
predict
(
self
,
sentence
:
Union
[
str
,
List
[
str
],
List
[
List
[
str
]],
List
[
data
.
Sentence
]]):
if
isinstance
(
sentence
,
str
):
if
isinstance
(
self
.
_tokenizer
,
lambo
.
LamboTokenizer
):
if
isinstance
(
self
.
_tokenizer
,
lambo
_tokenizer
.
LamboTokenizer
):
segmented
=
self
.
_tokenizer
.
segment
(
sentence
)
return
self
.
predict
(
segmented
)
else
:
...
...
@@ -239,7 +239,7 @@ class COMBO(predictor.Predictor):
@classmethod
def
with_lambo_tokenizer
(
cls
,
model
:
models
.
Model
,
dataset_reader
:
allen_data
.
DatasetReader
,
lambo_model_name
:
str
=
'
en
'
):
return
cls
(
model
,
dataset_reader
,
lambo
.
LamboTokenizer
(
lambo_model_name
))
return
cls
(
model
,
dataset_reader
,
lambo
_tokenizer
.
LamboTokenizer
(
lambo_model_name
))
@classmethod
def
from_pretrained
(
cls
,
path
:
str
,
tokenizer
=
tokenizers
.
SpacyTokenizer
(),
...
...
This diff is collapsed.
Click to expand it.
combo/utils/lambo.py
→
combo/utils/lambo
_tokenizer
.py
+
0
−
0
View file @
0eec74e1
File moved
This diff is collapsed.
Click to expand it.
docs/prediction.md
+
2
−
2
View file @
0eec74e1
...
...
@@ -34,10 +34,10 @@ You can use COMBO with the [LAMBO](https://gitlab.clarin-pl.eu/syntactic-tools/l
```
python
# Import COMBO and lambo
from
combo.predict
import
COMBO
from
combo.utils
import
lambo
from
combo.utils
import
lambo
_tokenizer
# Download models
nlp
=
COMBO
.
from_pretrained
(
"
english-bert-base-ud29
"
,
tokenizer
=
lambo
.
LamboTokenizer
(
"
en
"
))
nlp
=
COMBO
.
from_pretrained
(
"
english-bert-base-ud29
"
,
tokenizer
=
lambo
_tokenizer
.
LamboTokenizer
(
"
en
"
))
sentences
=
nlp
(
"
This is the first sentence. This is the second sentence to parse.
"
)
```
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment