Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

C

combo
- Activity
- Members
- Labels
- Issues
- Issue boards
- Milestones
- Wiki
- Redmine
- Environments
- Incidents

Snippets Groups Projects

3f714e88

Commit 3f714e88 authored Mar 28, 2023 by Maja Jabłońska

Downloads
- Patches
- Plain Diff

Add AllenNLP's Token

parent b2ae0e1d

No related branches found

No related tags found

1 merge request!46Merge COMBO 3.0 into master

Changes 4

Show whitespace changes

Inline Side-by-side

Showing

combo/data/tokenizers/__init__.py 1 addition, 1 deletion

combo/data/tokenizers/__init__.py
combo/data/tokenizers/character_tokenizer.py 82 additions, 7 deletions

combo/data/tokenizers/character_tokenizer.py
combo/data/tokenizers/pretrained_transformer_tokenizer.py 13 additions, 14 deletions

combo/data/tokenizers/pretrained_transformer_tokenizer.py
combo/data/tokenizers/tokenizer.py 187 additions, 6 deletions

combo/data/tokenizers/tokenizer.py

with 283 additions and 28 deletions

Loading

0% Loading or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment