/start.sh: line 3: 10 Illegal instruction (core dumped) /llm/llama-server "$@"

[raphaelallain]

error trying to use the ampere on neoverse n1

_llama-server_logs (1).txt

[raphaelallain]

root@208fe3fdccae:/llm# gdb --args /llm/llama-server -m /models/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf --host 0.0.0.0 --port 8081
GNU gdb (Ubuntu 16.2-8ubuntu1) 16.2
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
https://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /llm/llama-server...
(No debugging symbols found in /llm/llama-server)
(gdb) run
Starting program: /llm/llama-server -m /models/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf --host 0.0.0.0 --port 8081
warning: Error disabling address space randomization: Operation not permitted
Function(s) ^std::(move|forward|as_const|(__)?addressof) will be skipped when stepping.
Function(s) ^std::(shared|unique)ptr<.*>::(get|operator) will be skipped when stepping.
Function(s) ^std::(basic_string|vector|array|deque|(forward)?list|(unordered_|flat_)?(multi)?(map|set)|span)<.>::(c?r?(begin|end)|front|back|data|size|empty) will be skipped when stepping.
Function(s) ^std::(basic_string|vector|array|deque|span)<.>::operator.] will be skipped when stepping.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
warn: LLAMA_ARG_HOST environment variable is set, but will be overwritten by command line argument --host
[New Thread 0xe34058feb400 (LWP 465)]
[Thread 0xe34058feb400 (LWP 465) exited]
[New Thread 0xe34058feb400 (LWP 466)]
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build_info: b8896-905ccccf2
system_info: n_threads = 4 (n_threads_batch = 4) / 4 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | LLAMAFILE = 1 | REPACK = 1 | AMPERE = 1 |
Running without SSL
init: using 8 threads for HTTP server
start: binding port with default address family
[New Thread 0xe34053ffb400 (LWP 467)]
main: loading model
srv load_model: loading model '/models/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf'
common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on
[New Thread 0xe340537eb400 (LWP 468)]
[New Thread 0xe340527da400 (LWP 469)]
[New Thread 0xe34051fca400 (LWP 470)]
[New Thread 0xe340517ba400 (LWP 471)]
[New Thread 0xe34050faa400 (LWP 472)]
[New Thread 0xe3405079a400 (LWP 473)]
[New Thread 0xe3404ff8a400 (LWP 474)]
[New Thread 0xe3404f77a400 (LWP 475)]
llama_params_fit_impl: no devices with dedicated memory found
llama_params_fit: successfully fit params to free device memory
llama_params_fit: fitting params to free memory took 1.10 seconds
llama_model_loader: loaded meta data with 48 key-value pairs and 720 tensors from /models/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gemma4
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.sampling.top_k i32 = 64
llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
llama_model_loader: - kv 5: general.name str = Gemma-4-E4B-Uncensored-HauhauCS-Aggre...
llama_model_loader: - kv 6: general.finetune str = 5Ref
llama_model_loader: - kv 7: general.basename str = KL0.0378
llama_model_loader: - kv 8: general.size_label str = 7.5B
llama_model_loader: - kv 9: gemma4.block_count u32 = 42
llama_model_loader: - kv 10: gemma4.context_length u32 = 131072
llama_model_loader: - kv 11: gemma4.embedding_length u32 = 2560
llama_model_loader: - kv 12: gemma4.feed_forward_length u32 = 10240
llama_model_loader: - kv 13: gemma4.attention.head_count u32 = 8
llama_model_loader: - kv 14: gemma4.attention.head_count_kv u32 = 2
llama_model_loader: - kv 15: gemma4.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 16: gemma4.rope.freq_base_swa f32 = 10000.000000
llama_model_loader: - kv 17: gemma4.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 18: gemma4.attention.key_length u32 = 512
llama_model_loader: - kv 19: gemma4.attention.value_length u32 = 512
llama_model_loader: - kv 20: gemma4.final_logit_softcapping f32 = 30.000000
llama_model_loader: - kv 21: gemma4.attention.sliding_window u32 = 512
llama_model_loader: - kv 22: gemma4.attention.shared_kv_layers u32 = 18
llama_model_loader: - kv 23: gemma4.embedding_length_per_layer_input u32 = 256
llama_model_loader: - kv 24: gemma4.attention.sliding_window_pattern arr[bool,42] = [true, true, true, true, true, false,...
llama_model_loader: - kv 25: gemma4.attention.key_length_swa u32 = 256
llama_model_loader: - kv 26: gemma4.attention.value_length_swa u32 = 256
llama_model_loader: - kv 27: gemma4.rope.dimension_count u32 = 512
llama_model_loader: - kv 28: gemma4.rope.dimension_count_swa u32 = 256
llama_model_loader: - kv 29: tokenizer.ggml.model str = gemma4
llama_model_loader: - kv 30: tokenizer.ggml.tokens arr[str,262144] = ["", "", "", "", ...
llama_model_loader: - kv 31: tokenizer.ggml.scores arr[f32,262144] = [-1000.000000, -1000.000000, -1000.00...
llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,262144] = [3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,514906] = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ...
llama_model_loader: - kv 34: tokenizer.ggml.bos_token_id u32 = 2
llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 1
llama_model_loader: - kv 36: tokenizer.ggml.unknown_token_id u32 = 3
llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 0
llama_model_loader: - kv 38: tokenizer.ggml.mask_token_id u32 = 4
llama_model_loader: - kv 39: tokenizer.chat_template str = {%- macro format_parameters(propertie...
llama_model_loader: - kv 40: tokenizer.ggml.add_space_prefix bool = false
llama_model_loader: - kv 41: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 42: general.quantization_version u32 = 2
llama_model_loader: - kv 43: general.file_type u32 = 15
llama_model_loader: - kv 44: quantize.imatrix.file str = /media/hauhaucs/New Volume/LM Models/...
llama_model_loader: - kv 45: quantize.imatrix.dataset str = /home/hauhaucs/llama.cpp/groups_merge...
llama_model_loader: - kv 46: quantize.imatrix.entries_count u32 = 342
llama_model_loader: - kv 47: quantize.imatrix.chunks_count u32 = 93
llama_model_loader: - type f32: 339 tensors
llama_model_loader: - type q4_K: 336 tensors
llama_model_loader: - type q6_K: 44 tensors
llama_model_loader: - type bf16: 1 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_K - Medium
print_info: file size = 4.95 GiB (5.66 BPW)
load: override 'tokenizer.ggml.add_bos_token' to 'true' for Gemma4
load: 0 unused tokens
load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: printing all EOG tokens:
load: - 1 ('')
load: - 50 ('<|tool_response>')
load: - 106 ('<turn|>')
load: - 212 ('')
load: special tokens cache size = 25
load: token to piece cache size = 1.9445 MB
print_info: arch = gemma4
print_info: vocab_only = 0
print_info: no_alloc = 0
print_info: n_ctx_train = 131072
print_info: n_embd = 2560
print_info: n_embd_inp = 2560
print_info: n_layer = 42
print_info: n_head = 8
print_info: n_head_kv = 2
print_info: n_rot = 512
print_info: n_swa = 512
print_info: is_swa_any = 1
print_info: n_embd_head_k = 512
print_info: n_embd_head_v = 512
print_info: n_gqa = 4
print_info: n_embd_k_gqa = [512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024]
print_info: n_embd_v_gqa = [512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024, 512, 512, 512, 512, 512, 1024]
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 1.0e+00
print_info: n_ff = 10240
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: n_expert_groups = 0
print_info: n_group_used = 0
print_info: causal attn = 1
print_info: pooling type = -1
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 1000000.0
print_info: freq_scale_train = 1
print_info: freq_base_swa = 10000.0
print_info: freq_scale_swa = 1
print_info: n_embd_head_k_swa = 256
print_info: n_embd_head_v_swa = 256
print_info: n_rot_swa = 256
print_info: n_ctx_orig_yarn = 131072
print_info: rope_yarn_log_mul = 0.0000
print_info: rope_finetuned = unknown
print_info: model type = E4B
print_info: model params = 7.52 B
print_info: general.name = Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
print_info: vocab type = BPE
print_info: n_vocab = 262144
print_info: n_merges = 514906
print_info: BOS token = 2 ''
print_info: EOS token = 1 ''
print_info: UNK token = 3 ''
print_info: PAD token = 0 ''
print_info: MASK token = 4 ''
print_info: LF token = 107 '
'
print_info: EOG token = 1 ''
print_info: EOG token = 50 '<|tool_response>'
print_info: EOG token = 106 '<turn|>'
print_info: EOG token = 212 ''
print_info: max token length = 93
load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
str: cannot properly format tensor name output with suffix=weight bid=-1 xid=-1
load_tensors: CPU_Mapped model buffer size = 5073.04 MiB
.............................................
common_init_result: added logit bias = -inf
common_init_result: added <|tool_response> logit bias = -inf
common_init_result: added <turn|> logit bias = -inf
common_init_result: added logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max = 4
llama_context: n_ctx = 131072
llama_context: n_ctx_seq = 131072
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = true
llama_context: freq_base = 1000000.0
llama_context: freq_scale = 1
llama_context: CPU output buffer size = 4.00 MiB
llama_kv_cache_iswa: creating non-SWA KV cache, size = 131072 cells
llama_kv_cache: CPU KV buffer size = 3072.00 MiB
llama_kv_cache: size = 3072.00 MiB (131072 cells, 4 layers, 4/1 seqs), K (f16): 1024.00 MiB, V (f16): 1024.00 MiB, V (transposed) (f16): 1024.00 MiB
llama_kv_cache: attn_rot_k = 0
llama_kv_cache: attn_rot_v = 0
llama_kv_cache_iswa: creating SWA KV cache, size = 2560 cells
llama_kv_cache: CPU KV buffer size = 200.00 MiB
llama_kv_cache: size = 200.00 MiB ( 2560 cells, 20 layers, 4/1 seqs), K (f16): 50.00 MiB, V (f16): 50.00 MiB, V (transposed) (f16): 100.00 MiB
llama_kv_cache: attn_rot_k = 0
llama_kv_cache: attn_rot_v = 0
sched_reserve: reserving ...
sched_reserve: Flash Attention was auto, set to enabled
sched_reserve: resolving fused Gated Delta Net support:
sched_reserve: fused Gated Delta Net (autoregressive) enabled
sched_reserve: fused Gated Delta Net (chunked) enabled
sched_reserve: CPU compute buffer size = 537.00 MiB
sched_reserve: graph nodes = 2132
sched_reserve: graph splits = 1
sched_reserve: reserve took 7.65 ms, sched copies = 1
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)

Thread 1 "llama-server" received signal SIGILL, Illegal instruction.
0x0000e3405975a4c0 in ggml_threadpool_new_impl () from /llm/libggml-cpu.so.0

(gdb) bt
#0 0x0000e3405975a4c0 in ggml_threadpool_new_impl () from /llm/libggml-cpu.so.0
#1 0x0000e3405975ae10 in ggml_graph_compute () from /llm/libggml-cpu.so.0
#2 0x0000e3405975ff00 in ggml_backend_cpu_graph_compute(ggml_backend*, ggml_cgraph*) () from /llm/libggml-cpu.so.0
#3 0x0000e340596939e8 in ggml_backend_sched_graph_compute_async () from /llm/libggml-base.so.0
#4 0x0000e34059ca0224 in llama_context::graph_compute(ggml_cgraph*, bool) () from /llm/libllama.so.0
#5 0x0000e34059c9fdec in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&)
() from /llm/libllama.so.0
#6 0x0000e34059ca1508 in llama_context::decode(llama_batch const&) () from /llm/libllama.so.0
#7 0x0000e34059ca57dc in llama_decode () from /llm/libllama.so.0
#8 0x0000add4ac4daa64 in common_init_from_params(common_params&) ()
#9 0x0000add4ac3853b0 in server_context_impl::load_model(common_params const&) ()
#10 0x0000add4ac2e1ca8 in main ()
(gdb) disassemble
Dump of assembler code for function ggml_threadpool_new_impl:
0x0000e3405975a480 <+0>: sub sp, sp, #0xe0
0x0000e3405975a484 <+4>: stp x29, x30, [sp, #128]
0x0000e3405975a488 <+8>: str x27, [sp, #144]
0x0000e3405975a48c <+12>: stp x26, x25, [sp, #160]
0x0000e3405975a490 <+16>: stp x24, x23, [sp, #176]
0x0000e3405975a494 <+20>: stp x22, x21, [sp, #192]
0x0000e3405975a498 <+24>: stp x20, x19, [sp, #208]
0x0000e3405975a49c <+28>: add x29, sp, #0x80
0x0000e3405975a4a0 <+32>: mov x20, x0
0x0000e3405975a4a4 <+36>: mov w0, #0x180 // #384
0x0000e3405975a4a8 <+40>: movk w0, #0x2, lsl #16
0x0000e3405975a4ac <+44>: mov x21, x2
0x0000e3405975a4b0 <+48>: mov x22, x1
0x0000e3405975a4b4 <+52>: bl 0xe340597547f0 <ggml_aligned_malloc@plt>
0x0000e3405975a4b8 <+56>: add x8, x0, #0x100
0x0000e3405975a4bc <+60>: stp x22, x21, [x0, #96]
=> 0x0000e3405975a4c0 <+64>: stlur wzr, [x0, #112]
0x0000e3405975a4c4 <+68>: stlur wzr, [x0, #128]
0x0000e3405975a4c8 <+72>: stlur wzr, [x0, #192]
0x0000e3405975a4cc <+76>: add x9, x0, #0x105
0x0000e3405975a4d0 <+80>: mov x19, x0
0x0000e3405975a4d4 <+84>: stlr wzr, [x8]
0x0000e3405975a4d8 <+88>: add x8, x0, #0x104
0x0000e3405975a4dc <+92>: stlrb wzr, [x8]
0x0000e3405975a4e0 <+96>: mov w1, wzr
0x0000e3405975a4e4 <+100>: ldrb w8, [x20, #525]
0x0000e3405975a4e8 <+104>: mov w2, #0x20000 // #131072
0x0000e3405975a4ec <+108>: stlrb w8, [x9]
--Type for more, q to quit, c to continue without paging--

[dewet22]

Assuming you're using the latest image -- this bit me too on my Altra Max last week. Rolling back to amperecomputingai/llama.cpp:3.4.2 fixed it for my use case.