Apple to Change How It Tests Software After Buggy iOS 13 Rollout

Apple is making changes to its process for testing software following the buggy release of iOS 13, reports Bloomberg.

The change was announced by SVP Craig Federighi and lieutenants including Stacey Lysik at an internal 'kickoff' meeting with software developers. Apple's new approach will have dev teams disable unfinished or buggy features by default in daily builds. Testers will then be able to selectively enable those features via a new internal process and settings menu dubbed Flags. This should help isolate the impact of a specific feature to the system.

The new process will also help make early internal builds more usable or 'livable'. Prior to iOS 14, some teams would add untested features everyday while other teams would make weekly changes.


“Daily builds were like a recipe with lots of cooks adding ingredients,” said a source.

This would make builds difficult to use.

“Testers would go days without a livable build, so they wouldn’t really have a handle on what’s working and not working,” the person said.

This also made it difficult for engineers to check how the operating system was responding to new features.


Bloomberg reports that Apple's software quality is measured and ranked using a scale of 1 to 100 based on a 'white glove' test. Buggy releases get a score in the low 60s and stable releases score above 80. iOS 13 scored lower than iOS 12. New features are assigned green, yellow, and red color codes to indicate their quality during development. Bugs get a priority of 0 through 5 with 0 being critical and 5 being minor.

Apple is reportedly applying its new strategy to the development of iOS 14, codenamed 'Azul'. It's also considering delaying some features into 2021 - in a 'Azul +1' update that will likely become iOS 15. Despite this, iOS 14 is expected to rival iOS 13 in terms of new capabilities.

Other operating systems including iPadOS, watchOS, tvOS, and macOS will also use the same testing process.

More details in the full report linked below.

Read More