•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
Where Drizz fits:
For a broader comparison of device cloud providers, see real device testing guide.
A typical E2E mobile test takes 90 seconds to 3 minutes on a real device, depending on flow complexity and network conditions. A 30 test regression suite running serially on one device takes 45 to 90 minutes.
That's longer than most teams' entire CI pipeline for build, lint, unit tests, and integration tests combined. The result: either real device tests run as a nightly job or tests get skipped on PR builds entirely.
The problem compounds with device matrix coverage. If you need to validate on 3 device configurations (Pixel 8 Android 14, Samsung Galaxy S23 Android 13, a low end device running Android 12), serial execution means 3x time.
A 30 test suite across 3 devices is 135 to 270 minutes serially. Nobody waits for that on a PR.
A team on r/devops reported cutting mobile E2E test time by 36x in CI by moving from static test to device assignment to a dynamic queue model where each device grabs next available test as it becomes idle. Static assignment means a single slow test stalls entire device lane; dynamic sharding eliminates that bottleneck.
Parallel execution changes equation. The same 30 tests across 3 devices, sharded so each device runs 10 tests, finishes in 15 to 30 minutes, fast enough to run on every PR merge or as a pre release gate.
Each model makes a different tradeoff between coverage breadth and execution speed. The right choice depends on whether you're running PR level smoke tests or full pre release regression.
Every test in suite runs on every device in matrix. A 30 test suite across 5 devices produces 150 test executions.
This is broadest coverage model: you validate that every flow works on every target device. Use it for pre release regression when you need full device coverage.
The execution time equals longest single device run (not sum) because all devices execute in parallel. The cost scales with device count: 5 devices running 30 tests each means 5x device minutes of a single device run.
Tests are distributed across devices by percentage. A 30 test suite with weights of 50/30/20 across 3 devices means device 1 runs 15 tests, device 2 runs 9, and device 3 runs 6.
Total executions: 30 (each test runs once on one device). Use this when certain devices represent a larger share of your user base.
Put 50% of tests on device your analytics show is most common. Mobile device fragmentation strategy covers how to pick right device matrix.
The suite is divided evenly across available devices. A 30 test suite across 5 devices means each device runs 6 tests, total executions 30, execution time roughly 9 to 18 minutes.
Use this for PR level gates where speed matters more than per device coverage. Combine with nightly full execution runs to cover device matrix comprehensively.
The integration has three steps: authenticate, upload build, trigger test plan. The pipeline should block merge or deploy if pass rate drops below your threshold.
Every CI run starts by requesting a fresh access token. Reusing tokens across runs risks expiration mid pipeline.
The typical pattern: call auth endpoint at start of pipeline job, store token as an environment variable, and pass it in headers for subsequent requests. The mechanism differs by provider (Auth0 for Drizz, API key pair for BrowserStack, IAM credentials for AWS Device Farm) but principle is same.
Upload APK or IPA only when a new build is produced. Skip upload if binary hasn't changed since last run, which saves 30 to 60 seconds per pipeline execution.
For Drizz: POST /apps/upload with binary as a multipart form payload. For BrowserStack: POST /app-automate/upload returns an app_url.
For AWS Device Farm: aws devicefarm create-upload. The response from each provider confirms build is staged and returns an identifier pipeline uses in next step.
This is where parallel model is configured. For Drizz, batch endpoint (POST /testplan/run/batch) accepts multiple test plan IDs in a single request, and each test plan has its parallel settings preconfigured in dashboard or overridden in API payload.
For Appium on BrowserStack, parallel sessions are controlled by parallels capability, and test sharding is handled at framework level (TestNG parallel="methods" or pytest xdist -n workers). For AWS Device Farm, ScheduleRun API accepts a devicePoolArn and platform handles sharding internally.
Poll execution status endpoint until run completes, then parse results. Fail pipeline if pass rate drops below your threshold (e.g., 90% pass, zero critical failures).
For Drizz, response includes per test plan status, execution IDs, and links to dashboard for artifact inspection. Smoke testing in CI covers which tests to include in gate.
jobs:
mobile-tests:
runs-on: ubuntu-latest
steps:
- name: Get auth token
run: |
TOKEN=$(curl -s -X POST $AUTH_URL \
-d '{"client_id":"$ID","client_secret":"$SECRET"}' \
| jq -r '.access_token')
echo "TOKEN=$TOKEN" >> $GITHUB_ENV
- name: Upload APK
run: |
curl -X POST $DRIZZ_API/apps/upload \
-H "x-api-key: $TOKEN" \
-F "file=@app-release.apk"
- name: Trigger test plans (batch)
run: |
RESULT=$(curl -s -X POST $DRIZZ_API/testplan/run/batch \
-H "x-api-key: $TOKEN" \
-H "Content-Type: application/json" \
-d '{"test_plans":[
{"test_plan_id":"smoke_login","apks":{"com.app":"1.0.0"}},
{"test_plan_id":"smoke_checkout","apks":{"com.app":"1.0.0"}}
]}')
echo "$RESULT" | jq .
- name: Check results
run: |
FAILED=$(echo "$RESULT" | jq '.failed_executions | length')
if [ "$FAILED" -gt 0 ]; then exit 1; fi
This pattern works identically for Jenkins (scripted pipeline with sh steps), GitLab CI (script blocks), and Bitbucket Pipelines. The CI/CD integration guide covers platform specific configurations.
The comparison matters because most teams evaluating parallel real device testing have either tried Appium or are considering it. The tradeoff is control vs. operational overhead.
QA teams on r/QualityAssurance consistently flag two Appium specific parallel pitfalls: you need ThreadLocal<AppiumDriver> (or equivalent thread isolating storage) to prevent parallel threads from sending commands to wrong device session, and XPath locators on iOS are extremely slow in parallel because converting XCUI DOM to XML is resource heavy per device. Both issues disappear with Vision AI approaches that don't use selectors.
You provision a Selenium Grid hub, connect Appium server nodes (one per device), and manage device allocation yourself. The test framework (TestNG, pytest, or JUnit) handles parallel thread management, with each thread sending commands to a different Appium node.
What you control: device selection, Appium server versions, network configuration, and test distribution logic. What you maintain: grid hub, node registration, device health monitoring, USB connection stability for physical devices, and CI scripts that orchestrate all of it.
A typical Appium parallel setup on a cloud provider (BrowserStack, Sauce Labs) removes infrastructure management but keeps framework level sharding. You still write Appium test code, manage desired capabilities per device, and configure TestNG/pytest parallelism.
Drizz handles device provisioning, session management, and parallel orchestration as platform level features. You configure parallel slider (1-10 concurrent devices) and execution strategy (Full, Weighted, Weighted Random) in test plan settings.
Tests are authored in plain English using Vision AI, so there's no Appium code, no desired capabilities objects, and no framework level threading configuration. The platform provisions devices, distributes tests according to strategy, runs them concurrently, and returns consolidated results.
Async validation (vendor reported) adds another speed layer: validation steps run in background while execution continues, saving 10-15% on validation heavy suites. Automated regression testing covers how to structure test suite itself.
Appium gives you full control over test code, device configuration, and execution logic. That control costs engineering time: grid setup, node management, framework threading, and ongoing maintenance.
Drizz removes infrastructure layer. The tradeoff: you can't customize at Appium API level (direct ADB commands mid test, custom Appium plugins, WebDriver protocol manipulation).
Engineers on r/QualityAssurance describe managing an Appium parallel stack as consuming 30-50% of QA team's sprint capacity, between grid maintenance, selector rot from UI changes, and environment drift across device nodes. The mobile testing community on Reddit broadly accepts a 15% flakiness rate as "normal" for Appium based suites, compared to 2% on web, which means parallel Appium runs generate proportionally more false negatives to triage.
For E2E functional flows (login, checkout, search, navigation), platform handles execution complexity. For teams that need low level device interaction, Appium remains right tool.
The cost unit that matters is device minutes, not device count. Running 5 parallel devices for 10 minutes and running 1 device for 50 minutes consume same total device minutes.
Teams on r/QualityAssurance comparing AWS Device Farm vs. BrowserStack note that real cost driver is concurrency limits: BrowserStack charges per parallel thread, AWS charges per device minute. At scale, per minute model often wins for burst parallelization, while BrowserStack's unlimited minutes per slot model works better for teams running continuous suites throughout day.
Pricing is per device minute. AWS Device Farm charges $0.17 per device minute on demand, or a flat monthly rate for unlimited minutes on a dedicated device slot.
The cost optimization: run fewer tests on more devices (sharded), not more tests on fewer devices (serial). A sharded 30 test suite across 5 devices uses same total device minutes as serial execution but finishes 5x faster.
The only cost increase is full execution (every test on every device), which multiplies device minutes by device count. Reserve full execution for nightly or pre release runs where device coverage justifies cost.
No per minute cost, but upfront investment is real: physical devices ($300-1,200 each), USB hubs, a dedicated machine running Appium grid, and someone maintaining device health. Teams maintaining in house labs of 10+ devices report that one engineer spends 15-25% of their time on device lab operations.
Drizz includes real device execution in its token pricing (one Drizz Token per executed test step). Parallel execution doesn't multiply cost linearly because device provisioning is platform managed.
The Team plan includes shared workspace and collaboration features. For enterprises, the Team plan is custom priced and covers on prem/VPC deployment for teams with data residency requirements. See Drizz pricing.
Reddit's own iOS engineering team describes their approach as a PR gateway: 15-20 P0 smoke tests run on parallel simulators for every commit, keeping developer feedback under 20 minutes. The full regression suite runs nightly against release branches, not on every PR.
Parallel execution is largest single lever, but three additional optimizations compound with it.
Run full regression suite nightly or pre release. On PR builds, run a smoke subset: 10-15 tests that cover login, core navigation, and primary transaction flow.
If those pass, PR is safe to merge. Flaky test patterns covers how to identify which tests are stable enough for PR gates.
If PR only changes backend API calls or feature flags and APK binary is identical to previous build, skip upload step. This saves 30 60 seconds per pipeline run by version checking APK (compare versionCode or versionName) before uploading.
Live API calls during parallel execution multiply flakiness: 5 devices hitting same staging endpoint simultaneously creates contention, rate limiting, and inconsistent response times. Reddit engineers on r/androiddev consistently recommend mocking network layer (MockWebServer on Android, MSW on React Native) for UI tests in CI so responses are instantaneous and deterministic across all parallel lanes.
Drizz processes validation steps (screenshots, UI assertions) in background while test execution continues to next action step. Vendor reported data shows this saves 10% on average and up to 15% on validation heavy suites (those with 20+ validation steps per flow).
The combined effect: sharded execution (5x faster) + smoke selection (3x fewer tests on PR) + build caching (30-60s saved) + async validation (10-15% savings) can bring a 90 minute serial suite down to under 8 minutes on a PR pipeline.
Shard your test suite across multiple devices so each device runs a subset of tests concurrently, then gate pipeline on aggregated pass rate. A 30 test suite sharded across 5 devices finishes in under 10 minutes instead of 45+ minutes on a single device.
Full runs every test on every device (maximum coverage, highest cost), weighted distributes tests proportionally across devices by priority, and sharded splits suite evenly so each device runs a subset (maximum speed, each test runs once). Choose based on whether you need device coverage or fast feedback.
Authenticate at start of job, upload APK if build changed, trigger test plan via API with parallel settings configured, poll for results, and fail job if pass rate drops below your threshold. The same pattern works for Jenkins, GitLab CI, and Bitbucket Pipelines.
Drizz manages device provisioning, session allocation, and parallel orchestration as platform features with a batch API and configurable parallel slider. Appium requires a Selenium Grid setup, framework level threading (TestNG or pytest xdist), and manual device management, trading infrastructure control for operational simplicity.
Total device minutes are same for sharded execution (each test runs once). Full execution (every test on every device) multiplies device minutes by device count, so reserve it for nightly or pre release runs.
Start with 3-5 devices covering your top user segments by OS version and OEM. Most teams find that 5 parallel devices with sharded execution bring PR level regression under 15 minutes.
PR pipelines should run a smoke subset (10-15 high priority tests) sharded for speed. Nightly runs should execute full regression suite with full device coverage, and pre release runs should use full execution across complete device matrix.
Validation steps (screenshot comparison, UI element assertion) are processed in background while test execution continues to next action step. This removes validation wait time, saving 10% on average and up to 15% on validation heavy suites (vendor reported).
©Drizz 2026 • All Rights Reserved
