Skip to content

ClearOTag() off-by-one: writes past the array and clobbers its own terminator #12

Description

@tobyrg

Summary

ClearOTag(ot, n) writes one entry past the end of the ot array and, in doing so, overwrites the terminator entry it had just set up two lines above — leaving the resulting "empty" ordering table with no valid terminator at all. Walking it with DrawOTag (or anything else that follows the linked list) segfaults.

Where

src/psx/LIBGPU.C, ClearOTag:

u_long* ClearOTag(u_long* ot, int n)
{
	OT_TAG* ptag_list;

	if (n == 0)
		return NULL;
	ptag_list = (OT_TAG*)ot;

	// last is  aspecial terminator
	termPrim(&ptag_list[n-1]);
	setlen(&ptag_list[n-1], 0);

	// make a linked list with it's next items
	for (int i = (n-1); i >= 0; --i)
	{
		setaddr(&ptag_list[i], &ptag_list[i+1]);
		setlen(&ptag_list[i], 0);
	}

	return NULL;
}

The valid index range for an n-entry table is 0..n-1. The loop runs i from n-1 down to 0 inclusive. At i = n-1 this executes:

setaddr(&ptag_list[n-1], &ptag_list[n]);

&ptag_list[n] is one entry past the array — out of bounds — and this write also clobbers the termPrim/setlen(..., 0) terminator that had just been set up at index n-1 two lines earlier, replacing it with a pointer into whatever memory happens to follow the array.

Repro

u_long ot[4];
ClearOTag(ot, 4);
DrawOTag(ot); /* segfaults */

Confirmed with gdb (SDL2 + software GL / llvmpipe on Linux):

Thread 1 "..." received signal SIGSEGV, Segmentation fault.
0x000055555555bff5 in ParsePrimitivesLinkedList ()
#0  0x000055555555bff5 in ParsePrimitivesLinkedList ()
#1  0x000055555556cc4a in DrawOTag ()
#2  0x00005555555565fc in main ()

ParsePrimitivesLinkedList is walking the garbage pointer left in ptag_list[n-1] by the out-of-bounds write.

Suggested fix

Bound the loop at n-2 instead of n-1, so it only touches indices 0..n-2 and leaves the terminator set up for index n-1 untouched:

for (int i = (n-2); i >= 0; --i)
{
	setaddr(&ptag_list[i], &ptag_list[i+1]);
	setlen(&ptag_list[i], 0);
}

(ClearOTagR, right below it, doesn't have the equivalent bug — its loop runs i from 1 to n-1, staying in bounds.)

Impact

ClearOTag is about as basic/universal a PS1 ordering-table primitive as there is — any consumer building a fresh OT with it and then flushing via DrawOTag/DrawOTagEnv would hit this, not just an unusual code path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions