Four Clicks to a Crash: Phocus 4.2.2 and Adaptive CA

Phocus for Mac strikes again! Version 4.2.2 crashes, almost every time, when you turn on Adaptive CA correction. Click an image, let the preview render, expand Lens Corrections, open the Chromatic Aberration menu, pick Adaptive. Four clicks, and Phocus is gone within a second.

It has been like this since Hasselblad released 4.2.2 on August 20, seven weeks ago. The release notes say nothing about chromatic aberration.¹ An automated UI test that opens an image and switches on Adaptive CA would almost certainly have failed on this build. Either no such test exists, or one failed and 4.2.2 shipped anyway.

Key finding: In Phocus 4.2.2 for Mac, setting Chromatic Aberration (under Lens Corrections) to Adaptive after a preview has rendered crashes the app almost every time. I reproduced it on demand with three images on two volumes, captured three crash reports and a screen recording, and reported it to Hasselblad on October 8. Avoid Adaptive CA in 4.2.2.

How the Phocus 4.2.2 Adaptive CA crash happens

To reproduce it:

1. Start Phocus 4.2.2 and open a folder of images. The viewer opens empty.
2. Click an image. Phocus renders the preview.
3. Wait until the render finishes.
4. In the Adjust tab, expand Lens Corrections and set the Chromatic Aberration menu to Adaptive.

Phocus quits within a second. I hit it on an X2D II 100C .3FR from my internal drive, then on two more files on a second volume, each time after a fresh start of Phocus. Each time, Phocus.log shows the Adaptive CA calculation starting, and the crash follows 0.5 to 0.8 seconds later.

It does not crash on every attempt. One retry ran without a crash. Following the four steps above, I could make it crash on demand.

My log also holds an earlier session, from September 8 in New Mexico, that ends the same way: the calculation starts, the session stops with no clean quit, and I started Phocus again seven seconds later. No crash report survives for that one, so I count it as likely, not confirmed.

Why it crashes, in plain language

Phocus runs the Adaptive CA calculation on a background worker so the interface stays responsive. When the worker finishes, it leaves a note for the part of Phocus that runs the viewer: update the display, the details are on my desk. Then the worker clocks out and its desk is cleared, because its job is over. Half a second to a second later the viewer picks up the note, goes to the desk, and finds it empty or occupied by someone else.

Reading the empty desk is the memory error in two of the three crash reports. Reading someone else's desk is the third, where Phocus sends a message to the wrong object. The fix is for the note to carry its own copy of the details. For anyone who writes software, the exact mechanism is at the end of the post.

A test would have caught this. Which kind?

It is tempting to say a unit test would have caught it. It probably would not. The bug lives in the hand-off between a background thread and the main thread, and a unit test of either half on its own passes. What catches this class of bug is boring:

• An automated UI test that opens an image, waits for the render and switches on Adaptive CA, run against every build before it ships. Nobody clicks through every flow of an app this size by hand. A test suite does that on every build, and this flow would have crashed it.
• A sanitizer build. Xcode's Address Sanitizer adds its memory checks when the code is compiled, so it covers any code built with it, C++ and Objective-C alike, however old. Apple lists "use of stack memory after function return" among the errors it detects, which is exactly the read in these crash reports.³ Instead of a crash that depends on timing, you get a report of the bad read and where it happened.
• Code review. Capturing variables by reference in a block that runs later, on another thread, is a well-known C++ hazard. The C++ Core Guidelines have a rule for it, F.53: avoid capturing by reference in lambdas that will be used non-locally, "including returned, stored on the heap, or passed to another thread."⁴ It is the kind of line a reviewer is there to question.

The first two would very likely have caught it before release, and the third should have. I can't see inside Hasselblad's release process. I can see that none of them stopped 4.2.2 from shipping, and that the release notes mention no change to chromatic aberration.

I wrote in the Phocus 4.1.2 and 4.2 timeline that pulling a broken release, writing release notes that say what changed, and keeping the documentation current are "small habits for a company whose hardware deserves them." This crash is that gap at its smallest scale.

The hardware is the other half of the story

The stark difference in quality between the hardware and the software Hasselblad produces is perhaps the thing that annoys me most.

Hasselblad rates the in-body stabilization of the X2D II at 10 stops at the centre of the frame and 8 at the edges, measured under the CIPA standard.² It holds up in practice: I hold two-second exposures handheld with that body, as I described in my X2D II vs X2D 100C comparison. Someone at Hasselblad measured that system against a published standard before it shipped.

The same company shipped a raw converter that crashes, almost every time, when you switch on one of its lens corrections. The camera holds a two-second exposure steady. Phocus loses track of a piece of memory it handed off less than a second earlier.

Hasselblad clearly employs engineers who can do careful work. The hardware gets measured against a standard before release. Phocus, on the evidence of every release I have filed bugs against, is held to a much lower standard, if it is held to one at all.

What to do if you use Adaptive CA

Don't set Chromatic Aberration to Adaptive in Phocus 4.2.2 until Hasselblad ships a fix. What I tested is narrow: switching to Adaptive after a preview has rendered crashes. I have not tested whether images that already have Adaptive set crash when you open them, or whether switching before the render finishes is safe. Treat both as unknown.

In my Phocus 4.2.2 release post I leaned toward installing the update. That still holds if you do not use Adaptive CA. If you do, 4.2.2 is not ready for you.

I sent Hasselblad the steps, three crash reports, the screen recording and the matching log lines on October 8. They assigned ticket 10527221 the next day and forwarded everything to their senior engineering team, with an answer promised within two to four working days. I will update this post when it arrives, and the crash is listed on my Phocus known issues page.

For engineers: what the crash reports show

All three crash reports point at the same small piece of code, running on the main thread right after the CA calculation finishes.

Two of them are the clean case: Phocus tries to read memory that no longer exists, and macOS stops it. In both, the address it reaches for sits at exactly the same distance from the end of a thread's memory region, 552,737 bytes.

The third report fails differently, in the same code: Phocus sends a message to an object that does not understand it. Same bug, two ways to die.

My reading of them, which is my interpretation and not Hasselblad's:

• The viewer registers a callback (a C++ lambda holding a flag and a pointer to the viewer) with the image object.
• The Adaptive CA calculation runs in a std::async task. When it finishes, the task copies the stored callback into a local std::function and calls the copy. The copy is small enough to live on the async thread's stack.
• The callback builds a second lambda that captures the flag and the viewer pointer by reference, and posts it to the main queue with dispatch_async. The capture points into the local copy.
• The async task returns, the copy is destroyed and the thread exits. When the main queue runs the posted lambda, its references point into a stack that is gone (SIGSEGV) or has been reused (an unrecognized-selector abort).

Capturing by value, with a strong reference to the viewer, would close it. The crashing path appears to have changed in 4.2.2. I did not test 4.2 or 4.2.1.

References

1. Phocus for Mac 4.2.2 read-me (Hasselblad CDN)
2. Hasselblad X2D II 100C product page
3. Diagnosing memory, thread, and crash issues early (Apple Developer Documentation)
4. C++ Core Guidelines, F.53 (isocpp.github.io)