diff --git a/docs/ios27/CAMERA.md b/docs/ios27/CAMERA.md index b61fd1960..ffa3291bc 100644 --- a/docs/ios27/CAMERA.md +++ b/docs/ios27/CAMERA.md @@ -436,6 +436,58 @@ What phone 1 showed with dylib `-28` (18 November, 24B156, rear camera, `cam_tra property as Apple's display does (`{PlaneBaseAlignmentLinear = 64}`); not done here, because every process reads that property (QuartzCore among them) and it is not the camera's to change. **`-30` not run on a phone** when this was written. +### Dylib `-30` on the phone, and where the picture is lost (20 November, desk work) + +Phone 1 booted dylib `-30` on 18 November (15:39, rear camera, the line `c272m`: `cam_trace=1 cam_metal=1`; console +`scion-phone1/evidence/cd-cam30-metal`, on the build Mac). Read from that console on 20 November, with no phone: + +- **Verified in the console.** cameracaptured did not abort. The preview pipeline was built and connected (`Preview Governor` + to `BWImageQueueSinkNode`), and every node on the path emitted: the source node's `Preview(PrimaryScaler)` 1504 x 1128 + `420f`, the stitcher, `Preview Scaler`, `Preview Splitter`, `BWStreamingFilterNode` and `Assistive Overlay` 828 x 1702 `420f`, + each an IOSurface, about 8 a second, none dropped. The image queue sink rendered them (300 by 15:42:06) into its image queue + (``, fences on). At the sink's first sample backboardd made a texture of format 500 (`YCBCR8_420_2P`, + usage ShaderRead) of 828 x 1702 over the surface: `MTL2[57 backboardd]: native-metal: texture memory fmt 500 ...`. Neither + process logged a Metal refusal while the camera ran (backboardd had used 18 of its 64 lines, so a refusal would have + shown), and no `MISSING ... newSharedEventHandle` (a cross-process shared event, which the bridge does not have). Camera.app + logged `DidDisplayFirstPreviewFrame`. The screenshot of the viewfinder is pure black, (0, 0, 0) at every point. +- **Inferred, not seen.** No second display layer: the framebuffer driver (`SweetFramebuffer.cpp`, the xnu fork) takes up to + four layers in a swap but draws only 4-byte (BGRA) surfaces, so a `420f` surface on a layer of its own would be dropped + without a line. The console has no `SWEET FB: layer N surface ... at` line (the driver says each position of a layer above + the first) and no new framebuffer call after the camera began, so the window server most likely composited the preview + with the GPU (hence the format 500 texture), not on a layer of the display. That swap log was used up before the camera + ran, so this is not verified. +- **Inferred: the secure indicator does not blacken it.** iOS 27's CMCapture blackens or counts frames that its ISP marks + (`BWInvalidFramesChecker`, the frame metadata's `InvalidDueToNoSensorAccess` and `SensorAccess`). `camera_device.c`'s + frames carry neither, so the check sees valid frames. The console has no `blackened` line (CAMLOG takes these only when + they name the camera). +- **Not checked by any trace: what the pictures hold.** The flow trace counts samples and says their size and format. Nothing + read a pixel. A zeroed `420f` picture would show green, not black (Y 0 with Cb and Cr 0 converts to green). A black one + is Y 0 or 16 with Cb and Cr 128. A YCbCr texture whose conversion never ran reads (0, 0, 0) with alpha 1. So pure black fits + three places equally well, and the console cannot tell them apart: + 1. the pictures handed to the image queue are black (a node in cameracaptured, or its GPU writes not reaching the surfaces); + 2. backboardd draws the image queue but its format 500 texture reads black (the conversion in metal2vulkan on PanVK, or the + surface's bytes as backboardd's GPU reads them); + 3. backboardd never draws the image queue's surfaces (the layer not drawn, or not by a texture over them). + +**The trace that decides it** (this change; trace only, nothing else changes): +- `camera_pixeltrace.c` (cam.dylib, cameracaptured, with `cam_trace=1`): for the 1st, 30th and 300th sample of every node + output, and of the image queue sink, the processor reads the picture through CoreVideo's read lock: a 16 x 16 grid of luma + and an 8 x 8 grid of chroma. It reads them once as the cache has them, and once after cleaning and invalidating the + picture's lines. Said: `trace: pixels: 's output , sample N: 828x1702 '420f' surface S: Y mean M least L most H, + K of 256 above black; Cb B Cr R` (and `; after the cache's clean: ...` where the two differ). +- `tools/ios27/metal2/YCbCrTrace.h`, in `NativeBridgeMetalFrontend.mm` (StarbiteMetal, every Metal client, always on, at most + 24 lines a process): the same grid, read in that process, when a two-plane YCbCr texture over an IOSurface is made and + when an encoder first binds a texture over each surface: `native-metal: ycbcr texture made: fmt 500 over surface S '420f' + 828x1702 plane 0: Y mean ...` and + `native-metal: ycbcr texture bound (NBRenderEncoder, stage 2, slot N): ...`. + +How to read one boot with both: the first `trace: pixels:` line on the path whose grid is black names the node in +cameracaptured (1). If the sink's pictures hold the room, backboardd's `ycbcr texture made` line for the same surface ID says +whether its pages hold it too. No `ycbcr texture bound` line means the surface is never sampled (3). Bound, holding the +picture, and still black on the screen means the conversion or the draw (2). Then the runtime's telemetry +(`/var/local/mtl2-telemetry` present, the rows in `/var/local/mtl2.log`) is the next thing to read. `TEST-ON-PHONE.md` of +the desk work (`~/s9/agents/camera-desk-1120` on the build Mac) has the steps. **Built, not run on a phone.** + ### SpringBoard's main-display layout (18 November) `camera_sbtrace.c` on phone 1 (dylib `-16`, `-17`) shows that SpringBoard does publish its layout: the home screen, the lock diff --git a/tools/arch-size.allow b/tools/arch-size.allow index 1862ce4d9..a4fd1505c 100644 --- a/tools/arch-size.allow +++ b/tools/arch-size.allow @@ -69,7 +69,7 @@ tools/ios27/fullroot/pipeline/cache/aot/fp16_stub.py 1424 tools/ios27/fullroot/pipeline/cache/aot/rewrite-grow.py 1047 tools/ios27/lse/lse-rewrite.py 826 tools/ios27/mesa-ios/starbite-drm/starbite_drm.c 985 -tools/ios27/metal2/NativeBridgeMetalFrontend.mm 2764 +tools/ios27/metal2/NativeBridgeMetalFrontend.mm 2767 tools/ios27/metal2/bridge/NativeMetalObjects.rs 2440 tools/mesa/air/air_intrinsics.c 2716 tools/mesa/air/air_metal.c 7072 diff --git a/tools/ios27/camera/build.sh b/tools/ios27/camera/build.sh index 6630b0c2e..1bf3dcf00 100755 --- a/tools/ios27/camera/build.sh +++ b/tools/ios27/camera/build.sh @@ -13,8 +13,8 @@ SDK=$(xcrun --sdk iphoneos --show-sdk-path) xcrun --sdk iphoneos clang -arch arm64 -isysroot "$SDK" -miphoneos-version-min=16.0 -O2 -Wall -Wextra -Wno-unused-parameter -Wno-deprecated-declarations \ -dynamiclib -install_name /usr/lib/cam.dylib \ -DHAKIOS_CAMERA_N104 -DHAKIOS_CAMERA_CONSOLE '-DHAKIOS_CAMERA_FW="/private/var/hakios/camera"' -I"$SRC" -I"$HERE" \ - -framework CoreFoundation -framework CoreMedia -framework CoreVideo -lobjc \ - -o "$OUT" "$HERE/camera27.c" "$HERE/camera_trace.c" "$HERE/camera_oslog.c" "$HERE/camera_formats.c" "$HERE/camera_apptrace.c" "$HERE/camera_statetrace.c" "$HERE/camera_layout.c" "$HERE/camera_refuse.c" "$HERE/camera_flowtrace.c" "$HERE/camera_colour.c" "$HERE/camera_sbtrace.c" "$HERE/camera_tenbit.c" "$HERE/camera_outputs.c" "$HERE/camera_flownext.c" "$HERE/camera_previewtrace.c" "$HERE/camera_stills.c" "$HERE/camera_connecttrace.c" "$HERE/camera_metal.c" \ + -framework CoreFoundation -framework CoreMedia -framework CoreVideo -framework IOSurface -lobjc \ + -o "$OUT" "$HERE/camera27.c" "$HERE/camera_trace.c" "$HERE/camera_oslog.c" "$HERE/camera_formats.c" "$HERE/camera_apptrace.c" "$HERE/camera_statetrace.c" "$HERE/camera_layout.c" "$HERE/camera_refuse.c" "$HERE/camera_flowtrace.c" "$HERE/camera_colour.c" "$HERE/camera_sbtrace.c" "$HERE/camera_tenbit.c" "$HERE/camera_outputs.c" "$HERE/camera_flownext.c" "$HERE/camera_previewtrace.c" "$HERE/camera_stills.c" "$HERE/camera_connecttrace.c" "$HERE/camera_metal.c" "$HERE/camera_pixeltrace.c" \ "$SRC/camera_device.c" "$SRC/camera_say.c" "$SRC/camera_isp.c" codesign -f -s - "$OUT" 2>/dev/null [ "$(lipo -archs "$OUT")" = arm64 ] || { echo "build.sh: $(basename "$OUT") is not plain arm64" >&2; exit 1; } diff --git a/tools/ios27/camera/camera_flowtrace.c b/tools/ios27/camera/camera_flowtrace.c index fd7ae2a8c..3ec243c09 100644 --- a/tools/ios27/camera/camera_flowtrace.c +++ b/tools/ios27/camera/camera_flowtrace.c @@ -13,7 +13,8 @@ * stamp is from the host clock's now), and every 10 seconds how many each has emitted; * - every node's dropped samples (the first five of each node, then every 300th), with their reason; * - the preview stitcher's and the image queue sink's first sample, the sink's image queue (nil: nothing to show), and - * the sink's own dropped samples. + * the sink's own dropped samples; + * - what the pictures hold (camera_pixeltrace.c): each output's 1st, 30th and 300th sample and the image queue sink's. * Each method is wrapped only if the class itself has it with the number of arguments and return type of iOS 27.0.1's * CMCapture (24A446's shared cache, read 18 November 2026: names and types only); one that differs is left alone and said. * The source node's own output methods (_outputConfigurationForStreamingOutputID:nodeOutput: and its kin) are direct @@ -168,10 +169,16 @@ scioncam_flow_emit(void *output_object, void *sample_buffer) if (!scioncam_flow_on) { return; } + char where[160] = ""; + pthread_mutex_lock(&flow_lock); f = flow_of(output, &fresh); if (f != NULL) { f->emitted++; + if (scioncam_pixels_due(f->emitted)) { + snprintf(where, sizeof(where), "%s's output %s, sample %llu", f->node ? class_getName(f->node) : "?", + f->output, (unsigned long long)f->emitted); + } } if (fresh) { sample_of(sample_buffer, what, sizeof(what)); @@ -180,6 +187,9 @@ scioncam_flow_emit(void *output_object, void *sample_buffer) } summary(mach_absolute_time()); pthread_mutex_unlock(&flow_lock); + if (where[0] != '\0') { + scioncam_pixels(where, sample_buffer); /* camera_pixeltrace.c: what the picture holds */ + } } /* ---- the methods wrapped ------------------------------------------------------------------------------------------- */ @@ -229,6 +239,12 @@ w_iq_render(id self, SEL sel, void *sb, id input) bool first = count == 0; first_render(self, "the preview's image queue sink", sb, &count); + if (scioncam_pixels_due(count)) { + char where[80]; + + snprintf(where, sizeof(where), "the image queue sink's sample %llu", (unsigned long long)count); + scioncam_pixels(where, sb); /* what the picture handed to the image queue holds */ + } ((void (*)(id, SEL, void *, id))o_iq_render)(self, sel, sb, input); if (first) { char q[160]; diff --git a/tools/ios27/camera/camera_pixeltrace.c b/tools/ios27/camera/camera_pixeltrace.c new file mode 100644 index 000000000..0ceed3dc4 --- /dev/null +++ b/tools/ios27/camera/camera_pixeltrace.c @@ -0,0 +1,162 @@ +/* + * SPDX-License-Identifier: MIT-0 + * + * What the preview's pictures hold, traced (cam_trace=1; docs/ios27/CAMERA.md, "What the pictures hold"): so that one boot + * with a black viewfinder names the first step whose picture is black, not only the steps that pass a sample on. + * + * Phone 1's last traced boot (18 November, dylib -30, cam_metal=1) had every node of the preview emit its samples, the + * image queue sink hand 828 x 1702 '420f' IOSurfaces to its image queue and backboardd make a YCBCR8_420_2P (500) texture + * over one of them, and the viewfinder pure black. The flow trace counts samples; nothing said what they held. Here, for the + * pictures of the preview's path (each node output's 1st, 30th and 300th sample, and the image queue sink's), the processor + * reads the picture through CoreVideo's read lock and says, from a 16 x 16 grid of luma and an 8 x 8 grid of chroma: + * the luma's mean, least and most, the share of the grid above black, and the chroma's means (128 is grey: a black 420f + * picture is Y 0 or 16, Cb Cr 128; a picture of zeros is Y 0, Cb Cr 0, which the screen would show green). It reads them + * twice: as the processor's cache has them, and again after cleaning and invalidating the picture's lines + * (sys_dcache_flush), so a picture that the GPU wrote and the processor's cache still hides is said as such ("after the + * cache's clean: ..."). The IOSurface's ID is said, so that backboardd's line about the same surface (StarbiteMetal's + * "ycbcr texture ... over surface N") can be put beside it. + * + * Reading is all it does: nothing is written, no picture is held past the call. NOT RUN ON A PHONE. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include "camera_trace.h" + +void hakios_camera_say(const char *format, ...) __attribute__((format(printf, 1, 2))); /* camera_say.c */ +void sys_dcache_flush(void *start, size_t length); /* libSystem */ + +struct grid { + unsigned mean, least, most, lit; /* luma (or for BGRA, the mean of R, G and B); lit: of 256 points, above 24 */ + unsigned cb, cr; /* chroma means (BGRA: 0) */ +}; + +static void +read_grid(CVPixelBufferRef pb, uint32_t pf, struct grid *g) +{ + size_t w = CVPixelBufferGetWidth(pb), h = CVPixelBufferGetHeight(pb); + unsigned long sum = 0, n = 0; + + memset(g, 0, sizeof(*g)); + g->least = 255; + if (pf == kCVPixelFormatType_32BGRA) { + const uint8_t *base = CVPixelBufferGetBaseAddress(pb); + size_t row = CVPixelBufferGetBytesPerRow(pb); + + for (unsigned y = 0; base != NULL && y < 16; y++) { + for (unsigned x = 0; x < 16; x++) { + const uint8_t *p = base + (h * (2 * y + 1) / 32) * row + (w * (2 * x + 1) / 32) * 4; + unsigned v = ((unsigned)p[0] + p[1] + p[2]) / 3; + + sum += v, n++; + g->least = v < g->least ? v : g->least; + g->most = v > g->most ? v : g->most; + g->lit += v > 24; + } + } + g->mean = n ? (unsigned)(sum / n) : 0; + return; + } + const uint8_t *luma = CVPixelBufferGetBaseAddressOfPlane(pb, 0), *chroma = CVPixelBufferGetBaseAddressOfPlane(pb, 1); + size_t lrow = CVPixelBufferGetBytesPerRowOfPlane(pb, 0), crow = CVPixelBufferGetBytesPerRowOfPlane(pb, 1); + size_t cw = CVPixelBufferGetWidthOfPlane(pb, 1), ch = CVPixelBufferGetHeightOfPlane(pb, 1); + unsigned long cbs = 0, crs = 0, cn = 0; + + for (unsigned y = 0; luma != NULL && y < 16; y++) { + for (unsigned x = 0; x < 16; x++) { + unsigned v = luma[(h * (2 * y + 1) / 32) * lrow + w * (2 * x + 1) / 32]; + + sum += v, n++; + g->least = v < g->least ? v : g->least; + g->most = v > g->most ? v : g->most; + g->lit += v > 24; + } + } + for (unsigned y = 0; chroma != NULL && y < 8; y++) { + for (unsigned x = 0; x < 8; x++) { + const uint8_t *p = chroma + (ch * (2 * y + 1) / 16) * crow + (cw * (2 * x + 1) / 16) * 2; + + cbs += p[0], crs += p[1], cn++; + } + } + g->mean = n ? (unsigned)(sum / n) : 0; + g->cb = cn ? (unsigned)(cbs / cn) : 0; + g->cr = cn ? (unsigned)(crs / cn) : 0; +} + +static void +grid_text(const struct grid *g, bool bgra, char *out, size_t cap) +{ + if (bgra) { + snprintf(out, cap, "RGB mean %u least %u most %u, %u of 256 above black", g->mean, g->least, g->most, g->lit); + } else { + snprintf(out, cap, "Y mean %u least %u most %u, %u of 256 above black; Cb %u Cr %u", g->mean, g->least, g->most, + g->lit, g->cb, g->cr); + } +} + +/* the picture of `sample_buffer`, said with `where` (for example "BWPreviewStitcherNode's output ?, sample 30") */ +void +scioncam_pixels(const char *where, void *sample_buffer) +{ + CVImageBufferRef ib = sample_buffer != NULL ? CMSampleBufferGetImageBuffer((CMSampleBufferRef)sample_buffer) : NULL; + struct grid before, after; + char a[160], b[160]; + + if (ib == NULL || CFGetTypeID(ib) != CVPixelBufferGetTypeID()) { + return; + } + CVPixelBufferRef pb = (CVPixelBufferRef)ib; + uint32_t pf = CVPixelBufferGetPixelFormatType(pb); + bool bgra = pf == kCVPixelFormatType_32BGRA; + + if (!bgra && pf != kCVPixelFormatType_420YpCbCr8BiPlanarFullRange && pf != kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) { + hakios_camera_say("trace: pixels: %s: '%c%c%c%c' is not read here", where, pf >> 24, pf >> 16 & 255, pf >> 8 & 255, + pf & 255); + return; + } + if (CVPixelBufferLockBaseAddress(pb, kCVPixelBufferLock_ReadOnly) != kCVReturnSuccess) { + hakios_camera_say("trace: pixels: %s: the picture could not be locked for reading", where); + return; + } + IOSurfaceRef surface = CVPixelBufferGetIOSurface(pb); + + read_grid(pb, pf, &before); + /* the picture's lines cleaned and invalidated: what the GPU wrote, where the processor's cache held older lines */ + if (CVPixelBufferIsPlanar(pb)) { + for (size_t p = 0; p < CVPixelBufferGetPlaneCount(pb); p++) { + void *base = CVPixelBufferGetBaseAddressOfPlane(pb, p); + + if (base != NULL) { + sys_dcache_flush(base, CVPixelBufferGetBytesPerRowOfPlane(pb, p) * CVPixelBufferGetHeightOfPlane(pb, p)); + } + } + } else if (CVPixelBufferGetBaseAddress(pb) != NULL) { + sys_dcache_flush(CVPixelBufferGetBaseAddress(pb), CVPixelBufferGetBytesPerRow(pb) * CVPixelBufferGetHeight(pb)); + } + read_grid(pb, pf, &after); + CVPixelBufferUnlockBaseAddress(pb, kCVPixelBufferLock_ReadOnly); + grid_text(&before, bgra, a, sizeof(a)); + if (memcmp(&before, &after, sizeof(before)) == 0) { + hakios_camera_say("trace: pixels: %s: %zux%zu '%c%c%c%c' surface %u: %s", where, CVPixelBufferGetWidth(pb), + CVPixelBufferGetHeight(pb), pf >> 24, pf >> 16 & 255, pf >> 8 & 255, pf & 255, + surface != NULL ? IOSurfaceGetID(surface) : 0, a); + } else { + grid_text(&after, bgra, b, sizeof(b)); + hakios_camera_say("trace: pixels: %s: %zux%zu '%c%c%c%c' surface %u: %s; after the cache's clean: %s", where, + CVPixelBufferGetWidth(pb), CVPixelBufferGetHeight(pb), pf >> 24, pf >> 16 & 255, pf >> 8 & 255, pf & 255, + surface != NULL ? IOSurfaceGetID(surface) : 0, a, b); + } +} + +/* whether the `count`th sample (from 1) of a picture-carrying output is one to read: the 1st, 30th, 300th, then every 3000th */ +bool +scioncam_pixels_due(unsigned long long count) +{ + return count == 1 || count == 30 || count == 300 || (count > 300 && count % 3000 == 0); +} diff --git a/tools/ios27/camera/camera_trace.h b/tools/ios27/camera/camera_trace.h index 8933ddce4..27d830fa1 100644 --- a/tools/ios27/camera/camera_trace.h +++ b/tools/ios27/camera/camera_trace.h @@ -20,6 +20,10 @@ void scioncam_trace_flow(void); void scioncam_flow_emit(void *output, void *sample_buffer); /* camera_flownext.c: at an output's first sample, the node it leads to, said, and that node's render method traced */ void scioncam_flow_next(void *output, const char *node_class, const char *output_name); +/* camera_pixeltrace.c (with the flow trace): what a sample's picture holds, read by the processor from a grid, said with + `where`; scioncam_pixels_due says which samples of an output are read (the 1st, 30th, 300th, then every 3000th) */ +void scioncam_pixels(const char *where, void *sample_buffer); +_Bool scioncam_pixels_due(unsigned long long count); /* camera_previewtrace.c: the session configuration's preview sinks, said (with the flow trace) */ void scioncam_trace_preview(void); diff --git a/tools/ios27/metal2/NativeBridgeMetalFrontend.mm b/tools/ios27/metal2/NativeBridgeMetalFrontend.mm index ff5f6c1ad..9e4f4bcf6 100644 --- a/tools/ios27/metal2/NativeBridgeMetalFrontend.mm +++ b/tools/ios27/metal2/NativeBridgeMetalFrontend.mm @@ -74,6 +74,7 @@ static void noteFormat(uint32_t format,bool made) { for(unsigned i=0;i)texture index:(NSUInteger)index stage:(uint32_t) [_commandBuffer.surfaces addObject:t]; const MTLTextureUsage u=t.usage; if(!u || (u&MTLTextureUsageShaderWrite))[_commandBuffer.writtenSurfaces addObject:t]; + if(sweetYCbCrFormat(t.descriptor.format))sweetYCbCrBound(object_getClassName(self),stage,index,t.descriptor.format,t.surface,t.surfacePlane); } [self check:nmb_encoder_set_texture(_handle,stage,texture?[(NBTexture *)texture handle]:nullptr,(uint32_t)index) what:"setTexture"]; } @@ -2260,6 +2262,7 @@ - (id)newTextureLayoutWithDescriptor:(MTLTextureDescriptor *)d isHeapOrBufferBac NBTexture *texture=[NBTexture new];texture.handle=handle;nmb_texture_describe(handle,&descriptor);texture.descriptor=descriptor; texture.buffer=buffer;texture.offset=(NSUInteger)offset;texture.rowBytes=(NSUInteger)rowBytes; texture.surface=(IOSurfaceRef)CFRetain(surface);texture.surfacePlane=plane; + if(sweetYCbCrFormat(descriptor.format))sweetYCbCrMade(descriptor.format,surface,plane); return (id)texture; } - (id)newLibraryWithData:(dispatch_data_t)data error:(NSError **)error { diff --git a/tools/ios27/metal2/YCbCrTrace.h b/tools/ios27/metal2/YCbCrTrace.h new file mode 100644 index 000000000..8a39bd9b0 --- /dev/null +++ b/tools/ios27/metal2/YCbCrTrace.h @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// The camera's preview (docs/ios27/CAMERA.md, "Dylib -30 on the phone, and where the picture is lost"), part of +// NativeBridgeMetalFrontend.mm (included there; a file of its own for the size rule): what a two-plane YCbCr texture over an +// IOSurface holds as this process's processor reads the surface (a 16 x 16 grid of 8-bit luma, an 8 x 8 grid of chroma, after +// cleaning and invalidating the surface's lines), said when the texture is made and when an encoder first binds a texture over +// each surface: 12 lines of each a process, always on. A picture is in the surface where the luma's grid is lit and Cb Cr are not +// 0; a black one reads Y 0 or 16 and Cb Cr 128. Reading only: nothing is written and no surface is held. NOT RUN ON A PHONE. +static bool sweetYCbCrFormat(uint64_t f) {return (f>=500 && f<=510) || (f>=520 && f<=530) || (f>=540 && f<=545);} +static void sweetYCbCrSay(const char *what,uint64_t format,IOSurfaceRef surface,unsigned long plane) { + char held[160]="not read (not two planes of 8 bits)"; + const OSType os=IOSurfaceGetPixelFormat(surface); + if(IOSurfaceGetPlaneCount(surface)==2 && IOSurfaceGetBytesPerElementOfPlane(surface,0)==1 && IOSurfaceGetBytesPerElementOfPlane(surface,1)==2 && + IOSurfaceLock(surface,kIOSurfaceLockReadOnly,nullptr)==0) { + const uint8_t *l=(const uint8_t *)IOSurfaceGetBaseAddressOfPlane(surface,0),*c=(const uint8_t *)IOSurfaceGetBaseAddressOfPlane(surface,1); + const size_t w=IOSurfaceGetWidthOfPlane(surface,0),h=IOSurfaceGetHeightOfPlane(surface,0),lr=IOSurfaceGetBytesPerRowOfPlane(surface,0); + const size_t cw=IOSurfaceGetWidthOfPlane(surface,1),ch=IOSurfaceGetHeightOfPlane(surface,1),cr=IOSurfaceGetBytesPerRowOfPlane(surface,1); + if(l && c && w && h && cw && ch) { + sys_dcache_flush((void *)l,lr*h);sys_dcache_flush((void *)c,cr*ch); + unsigned long sum=0,cbs=0,crs=0;unsigned least=255,most=0,lit=0; + for(unsigned y=0;y<16;y++)for(unsigned x=0;x<16;x++) { + const unsigned v=l[(h*(2*y+1)/32)*lr+w*(2*x+1)/32];sum+=v;least=vmost?v:most;lit+=v>24; + } + for(unsigned y=0;y<8;y++)for(unsigned x=0;x<8;x++) { + const uint8_t *p=c+(ch*(2*y+1)/16)*cr+(cw*(2*x+1)/16)*2;cbs+=p[0];crs+=p[1]; + } + snprintf(held,sizeof held,"Y mean %lu least %u most %u, %u of 256 above black; Cb %lu Cr %lu",sum/256,least,most,lit,cbs/64,crs/64); + } + IOSurfaceUnlock(surface,kIOSurfaceLockReadOnly,nullptr); + } + char line[400]; + snprintf(line,sizeof line,"native-metal: ycbcr %s: fmt %llu over surface %u '%c%c%c%c' %zux%zu plane %lu: %s",what,(unsigned long long)format, + IOSurfaceGetID(surface),(char)(os>>24),(char)(os>>16),(char)(os>>8),(char)os,IOSurfaceGetWidth(surface),IOSurfaceGetHeight(surface),plane,held); + sm2_logs(line); +} +static void sweetYCbCrMade(uint64_t format,IOSurfaceRef surface,NSUInteger plane) { + static std::atomic said{0}; + if(said.fetch_add(1)<12)sweetYCbCrSay("texture made",format,surface,(unsigned long)plane); +} +// The first binding of a YCbCr texture over each surface (12 surfaces a process): the camera's preview, drawn or not. +static void sweetYCbCrBound(const char *encoder,uint32_t stage,NSUInteger index,uint64_t format,IOSurfaceRef surface,NSUInteger plane) { + static std::mutex lock;static uint32_t bound[12];static unsigned n; + bool fresh=false; + { + std::lock_guard hold(lock);const uint32_t id=IOSurfaceGetID(surface);bool seen=false; + for(unsigned k=0;k