•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
Payment tests fail in CI but payment flows work when checking out. The SDK, UPI app selector, callback came 2 seconds too late, or the keyboard covered a field. Your payment provider may also decide to update their checkout screen with no code changes made by you. The engineer takes a closer look and is now able to make the payment. The payment test failed. The payment did not.
This is a testing problem that developers run into when testing mobile payments. Just because a payment test failed doesn't mean that the payment checkout failed. Payments need to cross multiple apps, rely on OS intents, wait for provider callbacks, and use externally networked resources. The APIs of payment providers change many times as the product matures.
To understand the true nature of a payment failure, test the payment logic before your payment provider's UI. Validate payment screens and avoid fragile elements. Test UPI across multiple apps. Emulate payment failures to check the robustness of your payment flows. Your payment test framework needs to be able to tell a broken payment from a payment test that needs to be fixed.
It's not about failing payments. It's about failing payment tests.
This article discusses how to develop mobile payment tests for UPI, card payments, wallets, COD payments, and split payment cases. Additionally, it covers how to build test cases for mobile payment integrations with third party SDKs under intermittent network and payment callback conditions. You’ll also learn how vision AI can validate what users see to avoid false positives when testing mobile payment systems.
Eager to find out, let’s dive in.
False positives occur when something that is actually working correctly, gets wrongly identified as something that is not working. Payment tests have several indicators for false positives and these are usually caused by dependencies that are external to your code.
Some examples are:
Trust in our CI (continuous integration) systems is diminished when engineers spend a considerable amount of time trying to recreate failures. This creates a loss of confidence in the payment testing system. Developers begin to run payment tests repeatedly and avoid trying to address the test failures. Real defects in the payments can easily be missed.
Payment testing is a great idea but we have to distinguish among product failures, payment failures, test and environment failures.
Checkout flow includes many systems:
Each system of the checkout flow requires specific needs and tests to fulfill a successful end-to-end checkout flow. Consequently, we require a multi-layer solution for each testing case.
Different layers need different test coverage. One end-to-end test is not a failsafe.
These 4 layers cover most of the complexity:
This approach simplifies failure isolation for the areas you are responsible for, and reduces external dependencies that block pull requests.
Payment logic testing must be added to each pull request to validate the correct business logic. This testing ensures the UI will not have to be launched.
Tests should include the following functionalities:
These tests do not confirm the checkout screen functionality, however, they reinforce the guidelines of the payment methods.
Each main-branch build should contain a mobile test suite focused on the visible experience. This means not relying on brittle components of a specific Payment Method Provider.
A basic checkout test must accomplish the following:
Visual checks are necessary. Just because something is within the element tree, does not mean it will display on the screen. The screen can contain hidden elements and even success values that a user would consider an error can be hidden. The UPI selector that is provided by the OS is an example of a selector that may be in your app, but can still be out of the scope of your app's process.
Vision AI would be an excellent choice for these situations. Drizz uses vision AI to automate the validation of payments, app selectors, and payment method confirmations, which results in an overall decrease in reliance on custom payment method provider IDs.
A test is necessary to ensure that a payment input is visible and can be operated. It is then necessary to confirm that the 'Pay' button displays the correct payment amount, and the confirmation screen has a valid payment reference.
When making visual assertions, be specific about the Payment Method; the Payment Amount; the Action; the Error Message, and the Final Status. Avoid saying “the page looks good” as this is often not an adequate failure explanation when a test fails.
Quite frequently, run tests for provider integrations. This ensures the connection between your application and each payment service is working as expected.
Support for:
Use provider sandbox credentials and test in a vacuum. In case the external sandbox has a longer response time, keep these tests separate from pull requests.
When a sandbox test fails, capture the provider name, SDK version, device, OS, request ID, response code, time, and timestamp. This prevents the team from chasing bogus defects.
UPI tends to give false positives, as payment requests leave your app and move to the OS and then go to another app.
A UPI test should look into the following:
Test for the payment success, failure, cancellation and timeouts, and abandons, and also test the case where the UPI app indicates success and your app is not notified of the callback. Your backend has to verify the transaction with the service provider and only then consider the order as paid.
To test for your users commonly employing multiple UPI apps, devices need to be tested across multiple manufacturers using different versions of the Android Operating System. A virtual device without a UPI app also needs to be tested, and your messages must guide the user towards alternative options.
CI: For the repeatable checks, mock responses can be used or test credentials can be obtained from the provider and used to obtain mocked responses. For real intent and callback coverage, cross-app tests can be scheduled. The scheduling maintains daily build success and covers the risky handoff callback.
The most serious payment defects occur when the customer goes off the expected path.
Test the following:
Determine the outcome for each of the cases described. A payment status that is not available should not result in charging the customer twice. The app should show a payment status and should allow the user to check progress on the order later once payment is processed with the server.
The goal of these tests is to minimize false positive results. They provide the automation with a way to handle intermediate, otherwise undefined, states. Instead of treating a delay as a failure.
Your team is responsible for the UI so labeling screen elements and adding testing identifiers is straightforward. Once you have identifiers for product elements, you can use them to describe actions on the cart and order detail and navigation screens.
Avoid internally provided identifiers on third party elements used within a SDK. Third party APIs/SDKs change their interfaces unexpectedly. A selector that operates on a card field, payment button or confirmation message on a version of the SDK is still susceptible to a change.
The tests can be user centered and longer lasting:
Payment testing is always harder than testing any other flow in a mobile app. The dependency on third-party SDKs, OS-level intents, and external provider UIs makes it structurally different from testing screens you fully control.
Whenever a selector updates, Razorpay ships an SDK bump. The element ID changes every time PhonePe redesigns their checkout sheet. Therefore, coupling your tests to internal identifiers inside systems you don't own is the big bottleneck.
Thanks to vision AI, this coupling has been eliminated. Now, a card number field is tested as a card number field, not as rzp_card_input_v4_2_1. A UPI app selector is tested as a list of payment apps on screen, not as an OS-level intent that may or may not be in your element tree. The payment confirmation screen is verified as what the user actually sees, not as an element property that says "success" while the screen says nothing.
The teams that build resilient payment testing don't test more. They test differently at the API layer for logic, at the visual layer for experience, and at the provider layer for integration. That combination catches the payment bugs that cost revenue while spending a fraction of the maintenance time.
Drizz uses Vision AI to read an app screen just like a human would. It checks the UI for payment screens, tests UPI selectors, third-party screens, keyboard coverage, truncated labels, dark mode, and confirmation states without the need for any provider implementation. Get started with Drizz
An example payment pipeline with some balance:
Focus on test execution but also consider the pass rate of tests and the failure rate of tests. Also, consider the time it takes to fix a broken test, whether the provider needs to assist with the test, the flow of callbacks, duplicate orders, and defects that are found in the testing process.
Begin with the payment method that has the most transactions. Add coverage based on payment volume, customer complaints, revenue risk, and recently found bugs.
Payment automation does not need a million API tests run on every device for every method with every commit.
Modular API tests are useful to cover payment states and rules. Payment flows can be illustrated for customers with a series of payment screens. Integration tests with payment service partners should be added. Introduce failures in payment flows and test the limits of payment flow resiliency.
Good payment pipelines show each failure cause for all team members so they can trust the payment flow.
Differentiating one type of failure from another significantly enhances trust in automation and prioritizes payment issues that matter to the customer.
Read 30 Test Cases from Order to Doorstep for a full checklist of test cases on a delivery app.
©Drizz 2026 • All Rights Reserved
