implement f16 - #10
Closed
AJ0070 wants to merge 5 commits into
Closed
Conversation
The lexer enumerated sm_10 through sm_35, so every newer .target was a lexical error and ocelot aborted. That pinned callers to Kepler, which CUDA 12 no longer supports at all. Nothing reads the target string back: targetElement pushes it into statement.targets and ir/Module.cpp hardcodes "sm_21" for internally built modules. So one text-carrying token replaces the eight hardcoded ones and never needs updating for a new architecture.
PTX types bfi's pos and len operands .u32 regardless of whether the instruction is .b32 or .b64, but the parser types immediates from the instruction type, so `bfi.b64 d, a, b, 32, 32` yielded b64 immediates and was rejected. Operand 1 already exempts immediates; operands 3 and 4 did not. Operand 4 also tested b.type instead of c.type. eval_Bfi already reads both as operandAsU32 for .b32 and .b64, so only the validator disagreed. nvcc emits bfi.b64 from sm_50 on.
regression test for the two fixes: the lexer stopped at sm_35, and bfi with immediate pos/len was rejected for .b64.
Author
cvt to and from f16 threw "conversion not implemented", and add, sub,
mul, abs, neg, min, max, fma, set, setp, mov and ex2 rejected f16 as an
instruction type, so nothing targeting sm_53 or newer could use half.
For a CUDA 12 toolchain that is every target, since sm_35 is gone.
f32 represents every f16 exactly, so operandAsHalf widens an operand and
the existing f32 paths are reused: cvt takes f16 as a source by falling
through to the f32 case, and set and setp likewise. Destinations narrow
with floatToHalf. The exact result of an f16 add or mul also fits in f32
(11 bit significands, so a product needs 22 of the 24 available), so
computing in f32 and rounding once to f16 is correctly rounded rather
than double rounded. ex2 is .approx in PTX, so the f32 approximation is
in spec.
Reading an operand cannot key off the declared type. tinygrad's PTX
renderer declares halves .f16; nvrtc declares them .b16 and puts the op
in inline asm. The half is in the low 16 bits either way. Immediates
keep the f32 handling: the parser types an immediate from the
instruction and the lexer has no 0H half literal, so a constant reaching
an .f16 instruction was written as a float and lives in imm_single, and
reading it as bits picks up the wrong half of the union.
cvt needed one more thing: relaxedConvert stores the mnemonic's type in
a.relaxedType and leaves a.type as the register declaration, so
operandAsF32 has to honour either. Without that, half loads on the nvrtc
path came back as denormals like 2.15e-41, which is 0x3C00 sitting in a
float. set also accepts a .b16 destination, since that is the register
nvrtc writes the 1.0h/0.0h into.
halfToFloat and floatToHalf were checked against numpy: all 65536 half
bit patterns widen to the exact f32 bit pattern, and 531082 narrowing
cases including every exact tie midpoint round identically, ties to even.
mad keeps rejecting f16, which is correct: PTX has no mad.f16, half fma
is spelled fma.rn.f16 and that is implemented. ocelot has no tanh opcode
for any type, so tanh.approx.f16 still does not parse, and half atomics
are unimplemented and abort. Neither is reached by tinygrad or by the
parts of cuda_fp16.hpp the tests exercise.
tinygrad's emulated CI, test/backend, both emulator consumers:
ptx cell nv cell
master 1738 pass 453 skip 1669 pass 458 skip
sm_35 1738 pass 453 skip 1669 pass 458 skip
sm_53 1738 pass 443 skip 1669 pass 449 skip
Zero failures everywhere. sm_35 is unchanged on both cells, so this is a
no-op at the current target. The freed skips at sm_53 are the fp16 tests,
which have never been able to run: tinygrad disables half below sm_53, so
a Kepler-reporting mock meant the NV emulator never tested fp16 at all.
The nv cell at sm_53 runs on nvrtc 12.9 with matching cudart headers.
Author
|
closing so there is one small PR in front of you instead of two overlapping ones. the diff here includes #8's three commits because I cannot set a fork branch as the base, which is what made this unreviewable. I will reopen against master once #8 lands, so it shows only its own diff, and can split it into cvt and the ALU ops as separate PRs if that reads better. branch stays at AJ0070/tinygrad ptx-fp16, nothing is lost. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
stacked on #8, diff shows its commits since I cannot base on a fork branch.
f16 was rejected or unimplemented across cvt and the ALU, so nothing from sm_53 on could use half.
f32 holds every f16 exactly, so operandAsHalf widens and the f32 paths are reused, destinations narrow. add and mul of halves are exact in f32, so one rounding is correct.
reads go through the bits, not the declared type: tinygrad declares halves .f16, nvrtc declares .b16 with the op in inline asm. immediates stay on imm_single. cvt also needs relaxedType honoured, without it half loads came back as 2.15e-41, which is 0x3C00 in a float.
halfToFloat/floatToHalf match numpy on all 65536 widenings and 531082 narrowings including every tie midpoint.
test_f16 fails without the implementation.
tinygrad test/backend, 0 fail everywhere:
sm_35 ptx 1738 pass 453 skip, nv 1669 pass 458 skip, same as master
sm_53 ptx 1738 pass 443 skip, nv 1669 pass 449 skip
not implemented: tanh, ocelot has no tanh opcode for any type. half atomics abort. mad has no f16 form in PTX so rejecting it is right.