ver4a/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

graph : normalize Q, K, V shapes + sync cross attention (#12449)

Browse source 
* graph : normalize Q, K, V shapes and add comments

ggml-ci

* context : synchronize before getting cross attention data

* model : fix command-r attention norm check
This commit is contained in:
Georgi Gerganov 2025-03-18 21:35:19 +02:00 • committed by GitHub
parent bb115d2bf7
commit 75422e8bc4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 433 additions and 277 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/llama-graph.cpp
View file

@ -1378,7 +1378,7 @@ ggml_tensor * llm_graph_context::build_attn(
// note: storing RoPE-ed version of K in the KV cache
ggml_build_forward_expand(gf, ggml_cpy(ctx0, k_cur, k_cache_view));
assert(v_cur->ne[0] == n_embd_v_gqa && v_cur->ne[1] == n_tokens);
v_cur = ggml_reshape_2d(ctx0, v_cur, n_embd_v_gqa, n_tokens);
ggml_tensor * v_cache_view = nullptr;