iOS 4.2.1 on the iPod touch 2G draws GL through the MBX Lite GPU. The 3DS has a
PICA200 instead, and Apple's MBXGLEngine can't drive it. tools/ios3ds-gl is a
replacement engine that plugs into the same slot inside OpenGLES.framework.
It draws with a CPU rasterizer by default. A second back end renders on the
PICA200 itself (pica200.md); it is opt-in until it has been
verified on real hardware.
Apps, and later QuartzCore, keep using Apple's OpenGLES.framework unmodified.
They call gl* and EAGLContext as usual. Only the engine behind those calls is
ours.
_eagl_init (cache VA 0x34ff870c) runs once, from the first
-[EAGLSharegroup initWithAPI:]:
- It tries
OpenGLES.framework/GLEngine.bundle/GLEngine, the SGX/GLI engine. That bundle doesn't exist on the iPod, so the call fails. IOServiceGetMatchingServices(IOServiceMatching("AppleMBXDevice"))must return at least one service. Otherwise EAGL has no engine, andinitWithAPI:returns nil.CFBundleCopyResourceURL(com.apple.opengles, "MBXGLEngine", "bundle")andCFBundleCreaterun. ThenCFBundleGetDataPointerForName("GLESGetEGLInterface")is called, and the result is saved in_mbx_dispatch(0x3e7d84c8).
Then:
-[EAGLSharegroup initWithAPI:]uses the MBX engine only forapi == 1(ES1). It callsCreateSharegroup(&S[0x44])and sets the engine idS+4 = 2.-[EAGLContext initWithAPI:properties:]callsCreateGC(sg, X+0x10, X+0xD34, X+0xC). HereXis the context's private block:- +0xC holds the engine GC;
- +0x10 and +0xD34 are two 841-slot
__GLIFunctionDispatchRectables. The second starts at 0x10 + 841·4.
Each exported gl* is a trampoline:
mrc p15,0,rT,c13,c0,3; ldr rX,[rT,#0x78] (the current context's X), then
r0 = [X,#0xC], fn = [X,#0x10+4·slot], followed by a tail call with the GL
arguments shifted up by one. tools/ios3ds-gl/genslots.py reads each export's
slot from your own dyld cache and writes gl_slots.h. That file holds only names
and numbers.
| change | why |
|---|---|
file patch, cache file offset 0x4ff9458 (OpenGLES __cstring, VA 0x34ff9458): "AppleMBXDevice\0" → "IOResources\0\0\0\0" |
Step 2 now matches the always-registered IOResources nub. No kernel or device-tree change is needed. Only OpenGLES's copy of the string changes; the copies in MBX2D and MBXGLEngine are untouched. |
write /System/Library/Caches/com.apple.dyld/enable-dylibs-to-override-cache |
MBXGLEngine is in the shared cache. With this switch, dyld (sDylibsOverrideCache) loads the on-disk file at the same path instead. The rootfs keeps no other cached dylibs on disk (only 12 symlinks into frameworks that exist only in the cache), so nothing else changes. |
file patch, cache file offset 0x46de6a8 (QuartzCore CA::WindowServer::M2Display::M2Display literal pool, VA 0x346de6a8): 'L565' → 'BGRA' |
The display constructor probes eagl_context(2), then eagl_context(1); on success it stores the API at +0x17c and the display surface format at +0x180: 'BGRA' for API 2, 'L565' (16 bpp) for API 1. With this patch an ES1 compositor renders the 32 bpp surfaces the CLCD scans out and the screen mirror reads. (Until 2026-11-18 a patch at 0x46de620 skipped the ES1 probe instead, keeping CA's software compositor; §7.) |
add OpenGLES.framework/MBXGLEngine.bundle/MBXGLEngine (0755) |
Our engine, ad-hoc signed. The bundle's stock Info.plist already names MBXGLEngine as its executable. |
add /usr/libexec/ios3ds-gltest and com.ios3ds.gltest.plist |
The test program (§5). |
A kernel-side alternative to the string patch is to publish an IOService of
class AppleMBXDevice from the shim, or to keep the /arm-io/mbx device-tree
node with a stub driver. Neither is needed while userland is the only consumer.
GLESGetEGLInterface() returns a table of 11 function pointers. The 4.2.1 table
is the 3.x table of nine plus Set/GetProperty. Every caller was checked in
8C148's OpenGLES; the stock MBXGLEngine's _GLESBindView, _GLESPresentView and
_GLESCreateBuffer confirmed the drawable protocol.
| off | entry | called from | ios3ds |
|---|---|---|---|
| 0x00 | CreateSharegroup(void **sg) → ≠0 ok |
-[EAGLSharegroup initWithAPI:] |
gl_share_new |
| 0x04 | DestroySharegroup(sg) |
-[EAGLSharegroup dealloc] |
refcounted release |
| 0x08 | CreateGC(sg, front, back, &gc) → ≠0 ok |
-[EAGLContext initWithAPI:properties:] |
new GC; fills both tables |
| 0x0C | DestroyGC(gc) |
-[EAGLContext dealloc] |
|
| 0x10 | BindCoreSurface(gc, target, IOSurfaceRef) |
attachImage:toCoreSurface:invertedRender: (OpenGLES allows GL_RENDERBUFFER only with invertedRender=YES, and 0x84F5 only with NO) |
renderbuffer: draw straight into the surface. 0x84F5 or GL_TEXTURE_2D: the bound texture samples the surface. 32 bpp 'BGRA' surfaces only. |
| 0x14 | BindView(gc, drawable, ifmt, retained) |
renderbufferStorage:fromDrawable: ([layer nativeWindow]; ifmt GL_RGBA8_OES/GL_RGB565_OES) |
see below |
| 0x18 | FinishTexture |
never called | returns 1 |
| 0x1C | PresentView(gc) |
presentRenderbuffer: |
drawable->present(d, 1) |
| 0x20 | SwapNotification(gc, fb_id, transaction, layer) |
swapNotification:forTransaction:onLayer: (CA's GL renderer) |
IOMobileFramebufferSwapSignal(main display, transaction, layer). 4.2.1's IOMobileFramebufferGetID is no longer a port, so a 3.x-style signal by id would never complete CA's swaps. |
| 0x24 | SetProperty(gc, pname, const void *) |
setParameter:to:, profiler (pname 100) |
ignored |
| 0x28 | GetProperty(gc, pname, void *) |
getParameter:to:, profiler |
writes 0 |
QuartzCore drawable ([CAEAGLLayer nativeWindow]). This is a block of function
pointers, each called with the drawable as its first argument:
| off | call | stock engine use |
|---|---|---|
| +0x04 | bind(d, fourcc, CallbackBlock *) → ≠0 ok |
BindView: fourcc 'BGRA' (0x42475241) for RGBA8/RGB8, 'L565' for RGB565 |
| +0x08 | unbind(d) |
rebinding, or failure |
| +0x0C | dequeue(d) → IOSurfaceRef |
before the first draw of each frame (_ViewTextureBeginIfNeeded) |
| +0x10 | present(d, 1) → ≠0 ok |
PresentView, only if a frame was begun |
CallbackBlock {ctx, create(ctx, IOSurfaceRef) → 1 ok, destroy(ctx, IOSurfaceRef)}
is how CA announces each buffer it allocates for the layer. ios3ds always asks for
'BGRA', because the rasterizer draws 0xAARRGGBB words. RGB565/RGB8
renderbuffers keep alpha at 1 instead (RT.opaque). The drawable dequeues at
bind time, so GL_RENDERBUFFER_WIDTH/HEIGHT are valid straight after
renderbufferStorage:fromDrawable:. GL row 0 is the surface's last row. With a
retained backing, the last presented frame is copied into a newly dequeued
buffer.
| file | role |
|---|---|
api.c |
The ES 1.1 state machine: 177 entry points by slot (gl_install_dispatch), objects (textures, buffers, renderbuffers, framebuffers, fences), matrices, glGet*, pixel transfer. |
draw.c |
Front end: vertex fetch (all array types, VBOs, indices), modelview/projection and texture matrices, 8 lights, colour material, fog, primitive assembly. Points and lines become quads. glDrawTex*OES is handled here too. Its output is clip-space triangles plus the GC state. |
pica_be.c |
The PICA200 Backend (pica200.md): command lists, GPU-side render targets and textures, read-back. |
raster.c |
The software Backend: clipping, viewport, face culling, perspective-correct spans, 2 texture units (nearest/linear, repeat/clamp/mirror; full GL_COMBINE), alpha test, depth test (16-bit), blending (all factors and equations), colour mask, scissor, fog, polygon offset. |
iface.c |
EAGL glue: the 11-entry table, IOSurface-backed renderbuffers and textures, the CA drawable protocol. IOSurface and IOMobileFramebuffer are resolved with dlsym. |
thunks.S |
841 12-byte stubs. Each unimplemented slot logs [gl] unimplemented GL call: slot N to the console once, then returns 0. |
genslots.py → gl_slots.h |
Maps API names to slot numbers, read from your dyld cache. |
test/ |
gltest (§5) and the host harness. |
API coverage. All 44 gl* functions QuartzCore imports are implemented
(checked against QuartzCore's import list). So is the rest of ES 1.1 common
profile, plus OES_framebuffer_object, OES_draw_texture, OES_point_size_array, OES_mapbuffer, blend equation/func separate,
APPLE_fence, the APPLE multisample entry points (single-sampled) and
EXT_discard_framebuffer (a no-op). Paletted compressed textures are decoded.
Not done:
- PVRTC compressed textures (
glCompressedTexImage2Dlogs and fails); - mipmap levels > 0: minification samples the base level;
- stencil: the state is kept, but there is no stencil buffer;
- user clip planes (the state is kept, but nothing is clipped);
- logic op, multisampling, smooth points and lines;
matrix_paletteandweight_pointer.
The rasterizer sits behind struct Backend (engine.h):
typedef struct Backend {
const char *name;
void (*clear)(GC *, RT *, GLbitfield mask); /* honours scissor and masks */
void (*triangles)(GC *, RT *, const Vtx *v, int n); /* n/3 clip-space triangles */
void (*tex_changed)(GC *, Tex *); /* optional: texture image changed */
void (*tex_deleted)(Tex *); /* optional */
void (*finish)(GC *, RT *); /* optional: make rt->color current */
} Backend;Vtxcarries the lit colour, texture coordinates after the texture matrix, and the fog factor.- Every per-fragment setting is in the GC: blend, alpha and depth test, texenv and combiners, scissor, cull, colour mask, and polygon offset.
RTnames the target: its colour words, a 16-bit depth buffer, the pitch, andflip(GL row 0 is the last memory row).Tex.be_priv/be_seqlet a back end cache its own copy of a texture.gl_be_forget(mem)tells back ends that renderbuffer or texture storage is about to be freed.
There are two:
| back end | file | GL_RENDERER |
|---|---|---|
| CPU rasterizer (default, and the fallback) | raster.c |
ios3ds software rasterizer |
| PICA200 GPU | pica_be.c + tools/ios3ds-pica |
ios3ds PICA200 |
gl_gc_new asks gl_pick_backend(): with IOS3DS_GL=pica (or auto) in the
process environment and a usable GPU the context draws on the PICA200,
otherwise on the CPU. The GPU back end, the kernel service behind it and the
software GPU model used to test it are described in pica200.md.
The test is tools/ios3ds-gl/test/gltest.c. It uses only the public API: an
EAGLContext through the ObjC runtime, plus gl* from OpenGLES.framework.
- It renders the test scene into an offscreen FBO (RGBA8 colour + DEPTH16). The
scene has:
- a smooth triangle;
- a GL_LINEAR checkerboard texture;
glDrawTexiOES;- a 50 % blended quad;
- a scissored clear;
- a lit, culled, depth-tested cube in a frustum.
- It reads back 10 probe pixels and prints
ok/FAILfor each, thenPASS/FAIL, to the console. It then repeats this on a second context created withIOS3DS_GL=picaand compares the two frames pixel by pixel (skipped, with a log line, when the GPU is not usable). - After
delayseconds it binds the main display's framebuffer IOSurface (IOMobileFramebufferGetLayerDefaultSurface) as a renderbuffer withattachImage:toCoreSurface:invertedRender:YES. Forsecondsit then draws the animated scene straight onto the screen, which the ARM9 mirrors to the top LCD.
The launch daemon runs it with 360 60: the probe test runs at boot, and the on-screen demo starts at about 425 s guest time, after the home screen is up. To keep only the boot self-test, set both arguments to 0.
To build and run:
make -C tools/ios3ds-gl # engine bundle + gltest into build/rootfs-add
make -C tools/ios3ds-gl host-test # same core + scene on the build host (offscreen, PPM)
make -C boot && python3 scripts/mksd.py --image build/sd.img
python3 emu/run.py --time 450 --shots 415,449 # home screen, then the GL scene
mtype -i build/sd.img@@1048576 ::/ios3ds/log.txt | grep -a 'gl'
Verified in QEMU (-M nintendo3ds, 450 s boot, gltest args 360 60):
[gl] context created (software rasterizer) (pid 8): OpenGLES loads our bundle from disk past the cache copy, andCreateGCfills the dispatch tables.gltest: PASS (0 failed): the 10 probes andglGetErrormatch the host build value for value. This covers smooth shading, a LINEAR texture, DrawTex, blending, scissor, and depth/lighting/culling.[gl] first BindCoreSurface, thendrawing into the display surface 320x480: from about 425 s the top LCD shows the animated scene, rotated 90° like everything else on the portrait display. It runs at ~13 fps including a 50 ms sleep per frame, i.e. about 25 ms per 320x480 frame in the emulator.- In the same boot, the 415 s screenshot shows the SpringBoard home screen as before. With the QuartzCore probe patch, SpringBoard no longer creates a GL context. Without the patch the screen stayed black (§1).
- No
unimplemented GL callwas logged. make host-testgives the same pixels on the build host.
Screenshots aren't committed (iOS artwork); rerun the commands above to reproduce them.
Verified only by RE, not exercised yet: BindView/PresentView (the CAEAGLLayer path of a UIKit GL app). Texture-from-IOSurface and SwapNotification now run in CA's compositor (§7).
Next:
- CA compositor without the per-frame read-back of the display surface (§7);
- a UIKit GL app through CAEAGLLayer;
- PVRTC decode and mipmaps;
- the PICA200 back end on real hardware (pica200.md §8).
SpringBoard is the render server: QuartzCore's CA::WindowServer composites
every app's layers into the display surface. Without an EAGL engine it does
that in software on the ARM11; with one it uses CA::OGL::GLESContext, i.e.
OpenGL ES 1.1 through our engine, and with the PICA back end the per-pixel
work (blending, texture sampling, transforms) moves to the GPU.
- Display probe (
M2Display::M2Display, 0x346de5b8):eagl_context(2), theneagl_context(1)(=-[EAGLContext initWithAPI:], i.e. ourCreateSharegroup/CreateGC). The first that succeeds selects GL compositing; API 1 picks the surface format'L565', patched to'BGRA'. - Display surface as render target (
GLESContext::make_buffer_current, 0x3470d274):glGenTextures,glBindTexture(0x84F5, …)(0x84F5 = GL_TEXTURE_RECTANGLE), NEAREST/CLAMP parameters, then-[EAGLContext attachImage:toCoreSurface:invertedRender:NO]with target 0x84F5 = ourBindCoreSurface(the texture's pixels are the IOSurface), then an FBO with that texture asGL_COLOR_ATTACHMENT0. - GL usage: the 44 imported
gl*functions (§3), all implemented. Per call site (disassembly of the whole__text, imports resolved through__picsymbolstub4+ the indirect symbol table): textures bound with a per-image target (2D or 0x84F5),glTexEnvfv(GL_TEXTURE_ENV_COLOR)andGL_COMBINEtables,glTexImage2D/SubImage2Duploads,glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE)(only with APPLE_client_storage, not advertised),glGetString(0x8A1E)afterGL_EXTENSIONS(an Apple-private second list),glGetIntegerv(0x84F8)(max rectangle texture size), andglFlushwith FBO 0 bound before the swap. - Present:
SwapNotification→IOMobileFramebufferSwapSignal.
| first run's symptom | fix |
|---|---|
glBindTexture(0x84F5) → GL_INVALID_ENUM |
0x84F5 accepted wherever GL_TEXTURE_2D is (bind, parameters, images, enable); a texture last bound as 0x84F5 takes texel coordinates (host test: same pixels as 2D with 0..1) |
[gl] Get: unknown pname 0x84f8 |
max rectangle size = 1024 |
GL error 0x500 in GetString → CoreAnimation: rendering error 500 |
glGetString(0x8A1E) returns "" |
| screen black except for one icon | glFlush did nothing and glFinish only synced the bound framebuffer; CA flushes with FBO 0 bound. Now glFlush/glFinish/SwapNotification write every GPU target back (Backend.flush_all) |
photo grids empty (Settings > Wallpaper, re-log 62): [gl] IOSurface pixel format 'L555' not supported |
PhotoLibrary's album rows are IOSurface CGImages in 'L555' (x1R5G5B5, MusicLibrary's thumbnail formats); BindCoreSurface on a texture target converts 'L555'/'L565' surfaces to a 32 bpp copy at bind (CA binds again for a new texture). Render targets stay 32 bpp only |
IOS3DS_GL_TRACE=1 (set for SpringBoard) logs the first GL error of every
call site as [gl] trace: GL error 0x… at engine+0x… (look it up with nm).
SpringBoard's environment (rootfs/recipe.json) has CA_ENABLE_OGL=1 and
IOS3DS_CA=1. In a process with IOS3DS_CA, the first CreateGC decides
once (tools/ios3ds-gl/ca.c, pica_ca_gate):
- boot.cfg
ca_gpu(andgpu) through the GPU service (PICA_OP_CONFIG); - the PICA200 back end's bring-up and self-check;
- a CA-like scene (clear, premultiplied textured quad, rotated translucent fill) drawn on the CPU rasterizer and on the GPU and compared (≤ 2 % of pixels may differ by more than 24, polygon edges).
If any step fails the context is refused: QuartzCore's probe fails and CA
composites in software exactly as before (one [gl] CA compositor: software (…) line). CA on our CPU rasterizer is never used, it would be slower than
CA's own software path. A GPU failure later in the run turns the back end
into the CPU rasterizer for the rest of the run ([gl] pica: … GPU rendering is OFF): the screen keeps working, slower. Kill switch: boot.cfg ca_gpu=0
(or gpu=0).
- Home screen, Spotlight + keyboard, page swipes, and app launches and exits
(
emu/soak.py: PASS) render through the PICA200 model; screenshots against the software compositor at the same moments: 2181 of 192000 pixels differ, by at most 1 (rounding), on still frames; animation frames differ where the two runs are at different animation times. - Cost: see re-log 46 (with the read-back) and 51 (without).
- GPU memory is bounded by eviction. Texture copies are dropped least
recently used when the arena is full and re-uploaded on their next use;
render targets no longer keep a linear staging buffer. If even that cannot
satisfy an allocation, the process writes its GPU images back and draws on
our CPU rasterizer from then on (slower, never black; one line). The
[gl] CA:line reports held memory, free arena and evictions. - Depth is stored reversed (citro3d's convention); a probe at bring-up picks the convention that matches the CPU rasterizer (docs/pica200.md §16).
- The scene check now includes NPOT layers (a repeated 2D texture and a rectangle texture), with a 1 % limit.
Until re-log 51 every CA frame was rendered into an arena target and then
copied back: PPF tiled→linear into arena scratch, READ into the IOSurface,
and a per-pixel byte rotation (RGBA8 in the PICA200's byte order A,B,G,R →
iOS BGRA) on the ARM11, ~616 KiB per frame. Now the final frame never passes
through the CPU:
- Which targets.
gl_get_rtmarks a render targetsurfacewhen its pixels are an IOSurface (a texture bound withBindCoreSurface, or a renderbuffer backed by a drawable). If such a target has a multiple-of-8 width and height, pitch = width and 8-byte alignment, the back end gives it a power-of-two GPU colour buffer (run 13: NPOT textures sample garbage) plus a w×h "swizzle" buffer. - Swizzle pass. At sync time (
glFlush/glFinish/SwapNotification) the back end draws one quad into the swizzle buffer, sampling the target's colour buffer as a texture (nearest, clamp, texcoords 0..w/pw, 0..h/ph). Three TEV stages build the output:tex.a·(1,0,0),+ tex.r·(0,1,0),+ tex.g·(0,0,1), alpha =tex.b(or the constant 255 for an opaque target). Read back in the PICA200's byte order this is B,G,R,A in memory, i.e. iOS BGRA, with no CPU work. - Present. The engine pre-writes one word per destination page (see
below), then asks the shim (
PICA_OP_PRESENT, protocol 3) to PPF-transfer the swizzle buffer (tiled RGBA8 → linear RGBA8, row order flipped unless the target is flipped) straight to the IOSurface. - Shim checks (
kernel/shim/pica.c,present): size = w·h·4, 8-byte aligned, plain RGBA8 → RGBA8 transfer (no texture copy, no tiled output, no crop). Then it walks the caller's page tables (TTBR0/TTBCR, L1 section or coarse table, L2 small/large page) for every page of the range: each must be mapped user read/write (AP = 3, APX = 0), the pages must be physically contiguous, and the whole range must lie in FCRAM or VRAM. So the GPU writes only memory the caller could write itself. Then the range's D-cache lines are cleaned and invalidated by VA (c7,c14,1; the caller's VAs are live because the sysctl runs in its context) and the transfer runs. A refusal is logged once with the descriptors and CP15 state. - Present check. Until one non-uniform frame has matched, every present
is followed by the old CPU read-back of the same frame into a scratch
buffer and a comparison with what the GPU wrote (±2 per channel for TEV
rounding): the swizzle pass and the transfer are not verified on hardware,
and the CA scene check only covers the render target, not the presented
surface. A mismatch logs
[gl] pica: GPU present check FAILED: …, puts the read-back into the surface for that frame and turns presenting off for the process; success logsGPU present check passedonce. Host test: a third gltest run with presents that swap R and B (IOS3DS_HOST_PRESENT=swap) must hit the check and still PASS. - Fallback. If
PRESENTfails once (refusal, older shim, GPU error) the engine logs it, stops presenting for the rest of the process and uses the read-back as before; from then on one read-back staging buffer stays allocated instead of 1 MiB being taken from the arena (and textures being evicted for it) on every frame (re-log 75). Presenting only the refused surface through the CPU and the VRAM display through the GPU was tried and left a stale home screen in the QEMU soak (re-log 75).IOS3DS_GL_NO_PRESENT=1forces the read-back;IOS3DS_GL_PRESENT_ALL=1presents every eligible target (host test).
XNU's modified-bit emulation. The first attempt was refused: not user-writable (… L1 257ca001 L2 18000823 … TTBCR 2 DACR 00000001). L2
0x18000823 is a small page at PA 0x18000000 with AP = 2 (user read-only):
XNU's ARM pmap maps writable pages read-only until the first write, takes the
permission fault, sets the page's modified bit and only then maps it AP = 3.
CA had not written the surface since it was mapped (the GPU had). The engine
now writes each destination page once (pg[k] = pg[k], one word per 4 KiB
and the last word) right before PRESENT; the walk then sees AP = 3.
What CA's display surface is. The log of the first successful present:
user VA 0x00325000 = PA 0x18000000 (VRAM, 600 KiB) — the IOSurface CA
composites into is AppleM2CLCD's scan-out framebuffer at the start of /vram,
physically contiguous. The GPU writes the picture the LCD path (ARM9 copy or
GPU mirror) reads, so a frame costs one P3D quad and one PPF transfer.
Results (QEMU, re-log 51): CA frames have 0 read-backs; render thread CPU per
frame under --icount 2 dropped from 19–27 ms to about 8 ms; screenshots
match the software compositor except for the status-bar clock and moments
where the runs are at different animation times.