mirrors/ComfyUI
1
0
Fork 0
mirror of https://github.com/comfyanonymous/ComfyUI.git synced 2026-05-04 21:36:14 +02:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity

Update qwen tokenizer to add qwen 3 tokens. (#11029)
Some checks failed
Python Linting / Run Ruff (push) Has been cancelled
Details
Python Linting / Run Pylint (push) Has been cancelled
Details
Build package / Build Test (3.10) (push) Has been cancelled
Details
Build package / Build Test (3.11) (push) Has been cancelled
Details
Build package / Build Test (3.12) (push) Has been cancelled
Details
Build package / Build Test (3.13) (push) Has been cancelled
Details
Build package / Build Test (3.9) (push) Has been cancelled
Details
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.10, [self-hosted Linux], stable) (push) Has been cancelled
Details
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.11, [self-hosted Linux], stable) (push) Has been cancelled
Details
Full Comfy CI Workflow Runs / test-stable (12.1, , linux, 3.12, [self-hosted Linux], stable) (push) Has been cancelled
Details
Full Comfy CI Workflow Runs / test-unix-nightly (12.1, , linux, 3.11, [self-hosted Linux], nightly) (push) Has been cancelled
Details
Execution Tests / test (macos-latest) (push) Has been cancelled
Details
Execution Tests / test (ubuntu-latest) (push) Has been cancelled
Details
Execution Tests / test (windows-latest) (push) Has been cancelled
Details
Test server launches without errors / test (push) Has been cancelled
Details
Unit Tests / test (macos-latest) (push) Has been cancelled
Details
Unit Tests / test (ubuntu-latest) (push) Has been cancelled
Details
Unit Tests / test (windows-2022) (push) Has been cancelled
Details

Browse Source 
Doesn't actually change anything for current workflows because none of the
current models have a template with the think tokens.
This commit is contained in:
comfyanonymous 2025-12-01 14:13:48 -08:00 committed by GitHub
parent 7dbd5dfe91
commit 2640acb31c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 8 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
comfy/text_encoders/qwen25_tokenizer/tokenizer_config.json
View File

@ -179,36 +179,36 @@
"special": false
},
"151665": {
"content": "<|img|>",
"content": "<tool_response>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
"special": false
},
"151666": {
"content": "<|endofimg|>",
"content": "</tool_response>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
"special": false
},
"151667": {
"content": "<|meta|>",
"content": "<think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
"special": false
},
"151668": {
"content": "<|endofmeta|>",
"content": "</think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
"special": false
}
},
"additional_special_tokens": [