Mobile Release Readiness Criteria: A Go/No-Go Scorecard for Mobile Teams

Mobile software releases are complex with dozens of dimensions and moving parts involving several stakeholders across the value stream.

Mobile release readiness can never be determined solely by the successful completion of a CI build and a sign-off from QA. There are other important dimensions like coverage across devices and various OS versions, open defects, performance, security, accessibility, analytics, permissions, crash rates, crash recovery, and use cases.

A certain issue might be cosmetically unappealing, and the team may decide to address it in a future release. A crash on 2% of devices or a significant security flaw, however, requires immediate attention. Each of these issues must be assessed based on the level of risk associated with it. The relevant stakeholders must reach a decision.

Release readiness is a decision-making framework that defines the criteria for shipping a product. 

A Go decision is made when each of the following is true: major product flows or user journeys are completed, release-blocking defects have been resolved, agreed coverage is in place, quality metrics have not been breached, and the team has assessed the remaining product risks.

A No-Go decision is made when one or more of the criteria is not met.

Mobile teams often do not strive to release software free from defects. A release, however, should be made only after defined criteria have been met.

This framework describes the key dimensions of mobile releases and combines them into a Go/No-Go scorecard for mobile teams to help make consistent release decisions.

1. Confirm the release scope

Your testing scope should align to the change risk. A minor copy update will require minimal regression coverage as compared to a new payment flow, authentication system, or database migration.

Things to review prior to the start of testing:

• Features included in the build
• Features excluded from the release
• iOS and Android versions supported
• Device classes and supported screen sizes
• Required versions of backend and APIs
• Feature flags and remote configuration
• Planned rollout percentage
• Rollback or kill-switch

The release owner should review and confirm the final scope prior to the team reviewing readiness. Last minute changes introduce untested paths and ambiguous ownership.

2. Validate critical user journeys

There are user flows that directly contribute to the value proposition and user engagement of the mobile application. These flows should be validated on the supported platforms.

These flows include:

• Account creation and sign-in
• Password reset and biometric authentication
• Search, browse, and filter
• Checkout, payment, and payment retry
• Notifications and deep links
• File upload and download
• Offline use and network recovery
• Subscription management and omnichannel use
• Deletion of account and retrieval of data

These flows should be tested on real devices with a fresh installation of the application from the previous production release. Test both successful and failed cases. For example, a payment flow should be tested for a variety of causes of failure including lost network, interrupted authentication, backgrounding the application, expired payment sessions, and repeated button clicks.

Automated testing can provide rapid feedback for repetitive user flows. Exploratory testing should be performed to uncover unexpected user flows, behaviors, and issues with the application.

A release should not get a Go decision if a business-critical journey fails, produces bad data, or does not provide a clear recovery path for the user.

3. Set a sensible device and OS matrix

There needs to be a defined foundation for device coverage. Testing on one recent device does not cover the full user base.

Build a device matrix using production analytics, crash data, operating system adoption, revenue concentration, and hardware-specific risk. Include:

• OS version(s) currently and previously supported
• Most utilized phone and tablet models
• Screens of different sizes (small and large)
• Devices with low memory
• Older processors
• Different pixel densities
• Devices with changes to Android made by the manufacturers
• Accessibility configurations that are most commonly used.

Prioritize high-risk combinations rather than treating each device equally. A banking app might focus on biometric hardware, screen security, and OS permission behavior. A media app might care more about playback, memory use, and orientation changes.

Real device testing reveals issues that emulators miss, including camera behavior, Bluetooth connections, push notifications, background execution, battery restrictions, and vendor-specific permissions.

A good release decision describes the coverage gaps. If an older device is excluded, note the reason and the impact on the user.

4. Complete regression and upgrade testing

Regression testing ensures existing functionality is not impacted by the new code. These areas should take priority: recently changed code, shared services, authentication, the navigation framework, storage, payments, and notifications.

Mobile teams should perform both:

• Fresh installation
• Upgrade from the current production version

Testing the upgrade should take special consideration. Users maintain local data and cached content, and may have authentication tokens, permissions, or feature settings. An issue with the migration may only affect existing users and thus may be undetected with a clean installation.

Consider interruption based scenarios such as:

• application update in progress and terminated
• application synchronization in progress and network loses connection
• application installed and device rebooted
• application in background while a transaction is being processed
• data entry and orientation is changed
• insufficient storage during application installation
• session expired and user is in the middle of performing a critical action

The release should stop if there are failures during migration, and if there is a loss of critical data, repetitive application crashes, authentication protocols are compromised, or there are blocked features.

5. Review performance, stability, and battery impact

Functional correctness should not be the only measurement for the success of a mobile application. Review performance against agreed thresholds for startup, screen transitions, API responses, scrolling, image loading and displaying large data.

Relevant metrics of a release include:

• time to cold and warm startup
• application not responding events
• crash-free users and sessions
• error rates of crashes and non-fatal errors
• memory, network, and battery consumption
• frame drops and slow rendering
• time to complete a network request

Compare the release candidate to the current production version and a sharp increase in the crash rate, especially during sign in, should be reviewed before release.

Test under realistic conditions, including background activity, low memory, slow connections, and poor overall connectivity as well as a low battery.

Set thresholds before final review. Without established limits, teams tend to argue about metrics after an issue occurs.

6. Verify security and privacy

Security reviews should be included in the release gate for authentication, payments, personal, location, health and/or other sensitive data, and file access.

Verify:

• Sensitive logs should be masked
• Secure storage should be used for tokens and secrets
• TLS and certificate policy compliance should be addressed
• Permissions should be aligned with the required feature
• Sensitive content should be blurred in screenshots and previews
• Protected access should be logout-revoked or invalidated
• Deep links should not circumvent authorization
• Analytics should not include private data
• Third-party SDK versions should meet the security policy
• Release builds should not contain debug settings

Review privacy and consent for new installs and upgrades. Ensure the user is clearly informed of the reason for requested access to camera, microphone, location, contact list, and notification permissions.

A security issue that does not have a visible user impact still requires risk assessment. The security owner should evaluate if the issue should be considered a blocker for release, if a workaround should be implemented, or if it should be addressed in a follow-up.

7. Confirm accessibility and analytics

Accessibility should be tested for the primary user flows. It should not be limited to a single audit screen.

Check VoiceOver and TalkBack behavior, focus order, labels, touch targets and text scales, color contrast, reduced motion, keyboard access, and error announcements. Test forms, modals, bottom sheets, payment cards and screens, and other custom controls along with assistive technologies enabled.

Analytics also should be evaluated for release. Confirm names of events, properties, screen views, attribution data, and conversion steps, and that events occur at the correct time, once, and are not sensitive.

Analytics breakdowns in releases create an informational gap for product and support teams and create uncertainty around post-release monitoring.

8. Identify releases with open defects by risk classification

Readiness cannot be assessed by defects in total. Each impacted user must be analyzed by factors of user impact, frequency, impacted product domains, workarounds, data risk, and business impact.

Practical examples of each of these factors include the following:

A defect waiver should include impacted users, risk, workaround, owner, targeted defect fix, and approval and it would warrant more discussion and accountability than verbal approval during a release call.

9. Have a Go/No-Go scorecard

A scorecard provides a repeatable release decision criterion and will drive the review process of Go/No-Go for each of the following areas:

A single Fail in a release-blocking area should result in a No-Go decision. Conditional items must be owned and accepted by name prior to release.

10. Evidence shouldn't need to be created; it should be automated.

Release evidence from automated tests becomes useful when they become predictable. Consider a system that saves test results, screenshots, videos, device logs and performance data, and defect links.

Drizz lets mobile teams set up automated real device testing across multiple devices. It helps organize user flows and provide instantaneous test result reporting for efficient release decision making. This tool becomes even more advantageous to the user when automated executions are associated with the release scorecard. A passing automated test gets much better value if it includes the app version, device, OS, time of execution, failure evidence, and benchmarks.

Automation should focus on stable, high-value checks, like:

• Sign in and session management
• Core application navigation
• Checkout and payment
• Notifications and deep links
• Offline recovery
• Upgrade paths
• Permission flows
• Crash prone areas

Keep exploratory testing, visual review, accessibility validation, and risk-based investigation in the process. Automation supports release decision-making, but cannot replace it.

Final release decision

The release owner should capture the decision along with evidence of the unresolved risks and remaining approvals, along with the release plan and rollback criteria. Post release, continually monitor crash rates, error trends, support requests, conversion, payment failures, and various device issues.

A strong Go decision is not a guarantee of a defect free release. Instead, it is a documented decision informed by agreed upon criteria and known risk, along with a plan of action to respond to unforeseen changes when real world data hits.

This process sets a standard for mobile teams on how safe releases can occur regardless of build status, as many of the final decisions involve different forms of informed judgment.