Test Responsive Breakpoints With AI Browser Automation

Responsive breakpoint testing is one of those chores that never quite gets automated. You resize the browser, you eyeball the nav collapsing into a hamburger, you check that the pricing cards stack instead of overflowing, and then you ship. The problem is that "eyeball it" does not scale to 200 pages across four viewports, and the selector-based tests you write to replace it break the moment a designer renames a class. This guide walks through a different approach: driving a real Chromium browser with plain-English objectives, then asserting layout across a viewport matrix with deterministic checks. You describe the intent (menu should collapse, cards should stack), and an AI agent figures out how to verify it. I will also be honest about where this approach stops working and where a real-device lab is still the right call.

The tool I am using throughout is BrowserBash, a free and open-source (Apache-2.0) CLI from The Testing Academy. It positions itself as the validation layer for AI agents: you write a natural-language objective, an agent drives an actual browser, and you get back a deterministic verdict plus structured results. No page objects, no selectors, no maintaining a locator file for every breakpoint.

What "breakpoint testing" actually needs to prove

Before automating anything, get precise about what you are asserting. A responsive layout has a handful of failure modes, and most of them are visual-structural rather than pixel-perfect:

• Reflow correctness. At the mobile breakpoint the nav should collapse to a menu button, multi-column grids should stack to one column, and side-by-side cards should go vertical.
• No overflow. Content should not spill past the viewport width and force a horizontal scrollbar. This is the single most common responsive bug in the wild.
• Element presence per breakpoint. Some elements appear only at certain widths: a hamburger button on mobile, a full horizontal menu on desktop, a "back to top" affordance that only exists on long mobile pages.
• Tap target sanity. Buttons and links should stay reachable and not overlap when the layout compresses.
• Content parity. The mobile view should not silently drop a call-to-action or a legal disclaimer that the desktop view shows.

Notice that none of these require pixel diffing. They are assertions about structure and behavior at a given width, which is exactly the kind of thing you can state in English and verify deterministically. Pixel-perfect visual regression (did this button move three pixels?) is a genuinely different problem, and I will come back to why AI browser automation is not the right tool for that.

The viewport matrix: one test, many widths

The core primitive for responsive breakpoint testing in BrowserBash is the viewport matrix. Instead of writing four near-identical tests, you write one test and run it once per viewport. The run-all command takes a --matrix-viewport flag that accepts a comma-separated list of WxH sizes, runs every test in the folder once per viewport, and labels each result with the viewport it ran under, in the NDJSON events, the JUnit XML, and the human-readable results.

# Install once
npm install -g browserbash-cli

# Run every test in the folder across four breakpoints
browserbash run-all ./.browserbash/tests \
  --matrix-viewport 390x844,768x1024,1280x720,1920x1080 \
  --junit out/responsive-junit.xml

Those four sizes map to a common set of breakpoints: a modern phone (390x844, roughly an iPhone 14), a tablet in portrait (768x1024), a laptop (1280x720), and a wide desktop (1920x1080). You can pick whatever widths match your own CSS breakpoints. The point is that the same objective runs at each width and you get a labeled verdict for each combination, so a failure tells you not just "the nav test failed" but "the nav test failed at 390x844."

For a single ad-hoc run you do not need the matrix at all. Both engines honor a standalone --viewport flag:

# Check one page at one breakpoint, right now
browserbash run "Open https://acme.test/pricing and confirm the pricing cards stack in a single column" \
  --viewport 390x844 \
  --agent

The --agent flag emits NDJSON, one JSON event per line, which is what you want in CI or when an AI coding agent is orchestrating the run. Exit codes are frozen and boring on purpose: 0 passed, 1 failed, 2 error, 3 timeout. Your pipeline reads the exit code and the run_end event, not prose.

Writing responsive tests in plain English

The natural-language part is where this diverges hardest from Playwright or Cypress. You are not writing await expect(page.locator('.nav-mobile')).toBeVisible(). You are describing the outcome a human reviewer would look for. Here is a committable *_test.md file for a navigation breakpoint:

# Mobile navigation collapses

- Open https://acme.test
- Confirm the main navigation is collapsed into a menu or hamburger button
- Click the menu button
- Verify 'Pricing' link visible
- Verify 'Sign in' link visible

Run that file at a mobile width and the agent drives a real browser: it loads the page, looks at the rendered DOM and accessibility tree, decides whether the nav is collapsed, opens the menu, and checks the links. The Verify lines are special, and this is the part that makes the approach trustworthy rather than hand-wavy.

Deterministic Verify assertions

A Verify step compiles to a real Playwright check with no model judgment involved. The supported grammar covers the assertions you actually need for layout work: URL contains, title is or contains, text visible, 'name' button|link|heading visible, element counts, and stored-value equality. When a Verify passes, the condition genuinely held. When it fails, you get expected-versus-actual evidence in the run_end.assertions block and in the Result.md assertion table, so a failure is debuggable instead of a shrug.

That split matters for breakpoint testing. "Confirm the navigation is collapsed" is a judgment the agent makes, because "collapsed" is fuzzy and layout-dependent. But "Verify 'Pricing' link visible" after opening the menu is a hard, deterministic assertion. You lean on the agent for the parts that need visual reasoning and pin down the parts that need to be exact. Verify lines that fall outside the grammar still run, but agent-judged and flagged judged: true in the output, so you can always tell which checks were deterministic and which were reasoned. The features page has the full grammar if you want the exact list.

Asserting element counts across breakpoints

Element counts are underrated for responsive work. Say your desktop layout shows a four-column feature grid and your tablet layout shows two columns. You cannot assert that in CSS from a test, but you can assert the structural consequence. A Verify step that counts visible feature cards, run under two different viewports in the matrix, catches a grid that fails to reflow. Same idea for a mobile view that should collapse six footer columns into an accordion: count the visible column headers at each width.

Overflow and horizontal-scroll checks

The most common responsive bug is a stray element wider than the viewport, producing a horizontal scrollbar on mobile. This is annoying to test in traditional frameworks because you end up measuring document.documentElement.scrollWidth against window.innerWidth and hoping you accounted for scrollbar gutters. In plain English it is one line:

# No horizontal overflow on mobile

- Open https://acme.test/blog/some-long-post
- Confirm the page does not scroll horizontally and no content is cut off at the right edge

Run that across your matrix and you have a cheap, high-value smoke test for the single bug class that embarrasses teams most often. It will not tell you which element overflowed (the agent reports the symptom, not the CSS culprit), but it flags the page and the viewport, which is usually enough to reproduce and fix in devtools in a couple of minutes. For a deeper walk-through of writing objectives that hold up, the tutorials cover objective phrasing in detail.

Seeding state with testmd v2 before you assert layout

Responsive bugs love data-dependent layouts. A cart with one item looks fine at every breakpoint; a cart with nine items and a long promo code is where the mobile summary panel overflows. To test that reliably you need to control the data, and you do not want an LLM guessing its way through a seed flow.

testmd v2 handles this. Add version: 2 to the frontmatter and steps execute one at a time against a single browser session, with two deterministic step types that never touch a model. API steps seed data over HTTP, and Verify steps check the result through the UI:

---
version: 2
---
# Cart summary stays readable on mobile with many items

POST https://acme.test/api/cart with body { "sku": "WIDGET", "qty": 9 }
Expect status 201, store $.cartId as 'cart'

- Open https://acme.test/cart/{{cart}}
- Confirm the order summary panel does not overflow the viewport width
- Verify 'Checkout' button visible

The POST and Expect lines are pure HTTP with no model in the loop, so seeding is fast and reproducible. The plain-English lines run as a grouped agent block on the same page. You get a hybrid API-plus-UI test where the setup is deterministic and only the visual reasoning uses the agent. One honest caveat: testmd v2 currently drives the builtin engine, which needs an ANTHROPIC_API_KEY (or an Anthropic-compatible gateway via ANTHROPIC_BASE_URL). It does not yet run directly on Ollama or OpenRouter. If you are committed to fully local execution, keep your seed data in a plain v1 flow for now.

Where AI browser automation is honestly the wrong tool

I promised balance, so here is where you should reach for something else.

Pixel-perfect visual regression. If your acceptance criterion is "this component must render identically to the approved baseline down to the pixel," you want a screenshot-diffing tool like Percy, Chromatic, or Playwright's own toHaveScreenshot. An AI agent reasoning about layout is the wrong instrument for sub-pixel comparison. It is great at "did the layout reflow correctly" and poor at "did this shadow shift by two pixels." Use both: the agent for structural and behavioral assertions, a diff tool for pixel fidelity.

True on-device rendering. This is the big one for responsive testing, and it deserves its own honest paragraph. Setting a 390x844 viewport in desktop Chromium is not the same as rendering on a real iPhone. You are not exercising mobile Safari's rendering engine, iOS-specific font metrics, notch and safe-area insets, real touch behavior, or the way mobile browsers handle 100vh with a collapsing address bar. Emulated viewports catch the large majority of layout bugs, and they catch them cheaply, but they do not replace a device lab for the last mile. If a bug only reproduces on a physical Pixel 8, no amount of desktop viewport emulation will surface it.

BrowserBash does give you a path toward real devices through providers. The --provider flag routes the browser to local (your Chrome, the default), cdp, browserbase, lambdatest, or browserstack. LambdaTest and BrowserStack run real-device and real-browser grids, so you can push the same plain-English test onto their infrastructure. Note that those grids require the builtin engine (they cannot run the default Stagehand engine), and BrowserBash switches to builtin automatically when you target them. Even then, a cloud emulator is closer to a real device than local Chromium but is still not identical to hardware in your hand.

One-off exploratory checks. If you just need to look at a page at one width once, resizing your own browser is faster than writing a test. Automate the checks you will run more than a few times.

How this compares to selector-based responsive tests

Here is the trade-off laid out plainly.

The honest read: selector-based tests give you total determinism and control, at the cost of brittleness and per-element authoring. Screenshot diffing owns pixel fidelity and nothing else. AI browser automation wins on authoring speed and redesign resilience for structural and behavioral responsive checks, and it explicitly hands the pixel-fidelity job to a diff tool. These are complementary, not competitors. A mature responsive suite might use all three: BrowserBash for reflow and behavior across the matrix, a diff tool for a handful of pixel-critical components, and nothing at all for pages that do not matter.

If you want more real comparisons like this, the BrowserBash blog runs through several tool pairings with the same balanced framing.

A practical responsive suite you can commit

Let me put the pieces together into something you would actually check into a repo. Say you have a marketing site with a homepage, a pricing page, and a blog. You care about three breakpoints and three failure modes: nav collapse, card stacking, and overflow.

Create a folder .browserbash/tests/ with a few small *_test.md files, each testing one page and one behavior. Keep them small and single-purpose so a failure points at exactly one thing. Then run the whole folder across your matrix:

browserbash run-all ./.browserbash/tests \
  --matrix-viewport 390x844,768x1024,1440x900 \
  --junit out/responsive.xml \
  --budget-usd 1.50

A few things are happening here worth calling out. The run-all orchestrator is memory-aware: it derives concurrency from your real CPU and RAM, orders previously-failed and slowest tests first, and flags flaky tests across runs. The --budget-usd guard stops launching new tests once estimated spend crosses your limit, marks the remainder skipped, and exits 2, so a runaway suite cannot quietly burn money. On CI you can split the matrix across machines with --shard 2/4, which computes a deterministic slice from the sorted discovery order so parallel runners agree without coordinating.

For the second and later runs, the replay cache earns its keep. A green run records its actions, and the next identical run replays them with zero model calls, stepping the agent back in only when the page actually changed. For a responsive suite that runs on every push, that means most executions are effectively free after the first green baseline, which also makes running the full viewport matrix on every commit affordable rather than a luxury.

Wiring it into CI and monitoring

Two more integration points close the loop. The official GitHub Action installs the CLI, runs your suite, uploads the JUnit, NDJSON, and Result.md artifacts, supports the shard matrix and budget flags, and posts a self-updating pull-request comment with a verdict table. The GitHub Action docs show the full YAML, including how to fan out shards as matrix jobs so your breakpoint suite runs in parallel on pull requests.

For production, monitor mode watches a live page on an interval and alerts only on state changes:

browserbash monitor ./.browserbash/tests/mobile-nav_test.md \
  --every 30m \
  --notify https://hooks.slack.com/services/XXX/YYY/ZZZ

It runs the mobile-nav check every thirty minutes and pings Slack only when the verdict flips from pass to fail or back, never on every green run. Slack webhook URLs get Slack formatting automatically; any other URL gets the raw JSON payload. Because the replay cache makes each run nearly token-free, an always-on responsive monitor costs almost nothing to keep running. If a deploy breaks the mobile nav at 2 a.m., you hear about it before your users tweet about it.

Getting the model choice right

One last practical note, because it affects reliability. BrowserBash is Ollama-first: it defaults to free local models with no API keys and nothing leaving your machine, then auto-resolves to ANTHROPIC_API_KEY, OPENAI_API_KEY, or OpenRouter if no local model is present. For responsive breakpoint testing that involves multi-step flows (open menu, count cards, check overflow, verify links), model capability matters. Very small local models around 8B parameters and under can get flaky on long multi-step objectives. The sweet spot is a mid-size local model in the Qwen3 or Llama 3.3 70B class, or a capable hosted model for the harder flows.

The good news is that the Verify steps do not depend on model quality, since they compile to real Playwright checks. So even if the agent's reasoning wobbles on a fuzzy "is this collapsed" judgment, your deterministic assertions stay rock solid. You can also use cheap-model routing (--model-exec) to plan on a strong model and execute the repetitive steps on a cheaper one, which keeps a large viewport matrix affordable. The learn section goes deeper on picking and configuring models.

To pull the whole thing together: responsive breakpoint testing does not have to mean manually resizing a browser or babysitting a wall of brittle selector tests. Describe the reflow, the stacking, and the overflow behavior in plain English, pin the exact facts down with deterministic Verify steps, and run the whole thing once per breakpoint with a viewport matrix. Seed data-dependent layouts with testmd v2 API steps, gate spend with a budget, and let the replay cache make repeat runs nearly free. Then be honest with yourself about the boundary: emulated viewports catch most layout bugs cheaply, but a real-device lab, reached through the LambdaTest or BrowserStack providers, is still where you confirm the last mile, and a screenshot-diff tool still owns pixel fidelity. Use the right instrument for each job and you get broad, cheap, low-maintenance coverage where AI automation shines, plus targeted precision where it does not.

FAQ

Can AI browser automation replace real-device testing for responsive layouts?

No, and it is important to be clear about that. Setting a mobile viewport in desktop Chromium catches the large majority of layout bugs cheaply, but it does not reproduce mobile Safari's rendering engine, iOS font metrics, safe-area insets, or real touch behavior. Use emulated viewports for broad, fast coverage on every commit, and reserve a real-device lab (reachable through the LambdaTest or BrowserStack providers) for confirming the last mile before release.

How does BrowserBash test multiple breakpoints without writing separate tests?

You write one plain-English test and run it across a viewport matrix. The run-all --matrix-viewport flag takes a comma-separated list of widths and heights, runs every test in the folder once per viewport, and labels each result with the viewport it ran under in the NDJSON events, JUnit XML, and Result.md. A single run gives you a labeled pass or fail for each test-and-breakpoint combination, so a failure tells you exactly which page broke at which width.

Are the layout assertions deterministic or just an AI guessing?

Both, by design, and you always know which is which. Verify steps compile to real Playwright checks with no model judgment, covering text visibility, element counts, named button or link visibility, URL and title matches, and stored-value equality, with expected-versus-actual evidence on failure. The agent handles the fuzzy visual reasoning like whether a nav is collapsed, and any Verify line outside the deterministic grammar is flagged as judged so you can tell reasoned checks from exact ones.

Is BrowserBash free to use for responsive testing?

Yes. BrowserBash is free and open-source under the Apache-2.0 license, and it defaults to local Ollama models so you can run tests with no API keys and nothing leaving your machine. Anything that runs on your own machine, the CLI, the engines, the local dashboard, and the replay cache, stays free. You only pay if you bring your own hosted model key or opt into a third-party device grid like LambdaTest or BrowserStack.

Ready to try it on your own breakpoints? Install with npm install -g browserbash-cli, drop a couple of plain-English tests in a folder, and run them across a viewport matrix. An account is optional and everything runs locally by default, but if you want the hosted dashboard and monitoring you can sign up here.