ver4a/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

mtmd : support InternVL 2.5 and 3 (#13422)

Browse source 
* convert : internvl support

* InternVL3-1B working

* fix regression

* rm mobilevlm from test

* fix conversion

* add test for internvl

* add to list of pre-quant

* restore boi/eoi check

* add clarify comment for norm eps
This commit is contained in:
Xuan-Son Nguyen 2025-05-10 16:26:42 +02:00 • committed by GitHub
parent d8919424f1
commit 053367d149
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 243 additions and 25 deletions
Download patch file Download diff file Expand all files Collapse all files

7
tools/mtmd/mtmd.cpp
View file

@ -252,6 +252,13 @@ int32_t mtmd_tokenize(mtmd_context * ctx,
}
else if (proj_type == PROJECTOR_TYPE_INTERNVL) {
// <img> ... (image embeddings) ... </img>
marker_modified = "<img>" + ctx->image_marker + "</img>";
string_replace_all(prompt_modified, ctx->image_marker, marker_modified);
}
// llava-1.5, llava-1.6, Yi-VL, Yi-34B, granite: don't need to add prefix and suffix
// for glm-edge, BOI and EOI token's embeddings are not present in the text model