Skip to content

gguf_to_q4nx: an unknown quantisation type refuses by file and number, not from inside an enum #2019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bong-water-water-bong wants to merge 1 commit into from
+13 −1
Closed

gguf_to_q4nx: an unknown quantisation type refuses by file and number, not from inside an enum #2019

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion tools/gguf_to_q4nx.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,19 @@ def moe_meta(r, items, table):


def pack(args):
r = gguf.GGUFReader(args.gguf)
# engine#2013: a quantisation type this gguf cannot read fails INSIDE GGUFReader's _build_tensors, as
# `ValueError: np.uint32(143) is not a valid GGMLQuantizationType` from the enum -- naming neither the file nor a way
# forward, and looking like a caller bug rather than "this pack uses a type this tree does not support". Refuse by
# file and number instead.
try:
r = gguf.GGUFReader(args.gguf)
except ValueError as e:
if "GGMLQuantizationType" not in str(e):
raise
raise SystemExit(
f"{args.gguf}: {e}. A tensor uses a quantisation type this gguf library does not know (it knows "
f"{', '.join(q.name for q in gguf.GGMLQuantizationType)}); a newer or vendor-specific type needs a "
"dequantiser before this pack can be packed, and nothing was written")
refuse_unsupported_rope(r) # before the output directory exists: no half-written container
os.makedirs(args.out, exist_ok=True)
items = planned(r)
Expand Down
Loading