Add LAMBO as a tokenizer (18627a23) · Commits · Syntactic Tools / combo · GitLab

Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

C

combo
- Activity
- Members
- Labels
- Issues
  20
- Issue boards
- Milestones
- Wiki
- Redmine
- Releases
- Container Registry
- Environments
- Incidents

Snippets Groups Projects

18627a23

Commit 18627a23 authored 1 year ago by

Maja Jablonska

Downloads
- Patches
- Plain Diff

Add LAMBO as a tokenizer

parent 82d70415

1 merge request!46Merge COMBO 3.0 into master

Changes 10

Hide whitespace changes

Inline Side-by-side

Showing

combo/data/__init__.py 1 addition, 1 deletion

combo/data/__init__.py
combo/data/dataset_loaders/__init__.py 0 additions, 0 deletions

combo/data/dataset_loaders/__init__.py
combo/data/dataset_loaders/dataset_loader.py 54 additions, 0 deletions

combo/data/dataset_loaders/dataset_loader.py
combo/data/dataset_loaders/simple_data_loader.py 102 additions, 0 deletions

combo/data/dataset_loaders/simple_data_loader.py
combo/data/tokenizers/__init__.py 1 addition, 0 deletions

combo/data/tokenizers/__init__.py
combo/data/tokenizers/lambo_tokenizer.py 53 additions, 0 deletions

combo/data/tokenizers/lambo_tokenizer.py
combo/data/tokenizers/token.py 5 additions, 0 deletions

combo/data/tokenizers/token.py
requirements.txt 1 addition, 0 deletions

requirements.txt
tests/data/data_readers/test_conll.py 6 additions, 0 deletions

tests/data/data_readers/test_conll.py
tests/data/tokenizers/test_lambo_tokenizer.py 21 additions, 0 deletions

tests/data/tokenizers/test_lambo_tokenizer.py

with 244 additions and 1 deletion

0% or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment