Postman/Newman vs testmd v2 API Steps for Pre-Test Seeding

If you have ever written a UI test that needs a user account, an order in a specific status, or a record with a known ID before it can even begin, you already know the pattern: seed the data first, then drive the browser. The postman newman vs testmd api steps question comes down to one decision: do you run that seeding step as a separate Postman collection through Newman in your CI pipeline, or do you write it inline, in the same file as the UI steps, using testmd v2's deterministic API steps? Both get data into the system before the browser opens. They get there in very different ways, and the difference matters more than it looks once you have thirty test files instead of three.

This is not an "X is dead, use Y instead" piece. Postman and Newman are mature, widely deployed tools with a real place in most QA orgs, and nothing here changes that. What has changed is that BrowserBash's testmd v2 format, part of the features shipped in the 1.5.x line, now lets you write API calls and browser assertions as plain steps in the same markdown file, which removes an entire category of plumbing that used to require a second tool, a second pipeline stage, and a handoff between them. Let's look at what each approach actually does, where the seams are, and when each one is the right call.

What Pre-Test Data Seeding Actually Requires

Before comparing tools, it helps to name the actual requirements of seeding data for a UI test. You typically need to:

• Call an API endpoint (often POST) to create a record: a user, a cart, an invoice, a support ticket.
• Assert the call succeeded, usually by checking the HTTP status code.
• Extract a value from the response, most commonly an ID, a token, or a generated email address.
• Pass that value into the UI test that follows, so the browser navigates to the right record or logs in as the right user.
• Do all of this reliably in CI, where flakiness in the seeding step poisons every test downstream of it.

Every seeding strategy, whether it is a Postman collection, a bash script with curl, or an inline API step, is solving this same five-part problem. The question is how much ceremony sits between "I need a POST request" and "the browser sees the result."

The Postman/Newman Approach to Seeding

The standard pattern looks like this: you build a Postman collection with the seeding requests, write Postman test scripts (JavaScript, inside the collection) to assert on status codes and pull values out of responses into collection variables or environment variables, export the collection and environment as JSON, and then run newman run collection.json -e environment.json as an earlier step in your CI job, before the UI test runner starts. If the UI framework needs the extracted values, you either write them to a file Newman can output (--export-environment), read that file in a later CI step, and inject the values as environment variables for the UI tests, or you rely on some CI-specific variable-passing mechanism between jobs.

This works, and for API-only testing it works well. Postman's request builder, environment management, and pre-request/test scripting are genuinely good for exploring and iterating on an API by hand. Newman turns a saved collection into something a CI pipeline can execute headlessly and report on with JUnit or HTML reporters.

The friction shows up specifically in the pre-test-seeding use case, where the API call is not the test, it is a means to an end for a UI test that comes right after it:

• Two artifacts, two places to look. The seeding logic lives in a .postman_collection.json file, usually edited through the Postman GUI or Postman's cloud sync, while the UI test lives in your test framework's native format. A reviewer checking "what does this test actually set up" has to open both.
• Variable handoff is manual plumbing. Getting a value out of a Newman run and into your UI test process means writing it to disk, an environment file, or a CI artifact, then reading it back in a later step. Every new value you need to pass through is another line of glue code.
• A separate CI stage. Newman runs as its own step, with its own exit code, its own log output, and its own failure mode that is disconnected from the UI test failure that would follow it. When triage happens at 2 a.m. off a red CI run, you now have two logs to correlate instead of one.
• Version drift between the collection and the test. Because the Postman collection is a separate JSON blob (often huge, auto-generated by the Postman app, and not friendly to code review diffs), it is easy for the collection to seed a user shape the UI test no longer expects, especially after a schema change on the API side.

None of this is a fatal flaw. Teams run this pattern in production CI every day. But it is real overhead, and it exists specifically because Postman/Newman was designed as an API testing tool that happens to get pressed into service as a seeding step for browser tests, not as a unified test format.

The testmd v2 API Steps Approach

testmd v2 takes a different starting point: what if the seeding call and the UI assertion lived in the same file, executed as one ordered sequence against one browser session? A *_test.md file opts into this by adding version: 2 to its frontmatter. Once that flag is set, BrowserBash's testmd runner executes steps one at a time instead of joining them into a single agent objective, and it recognizes two deterministic step types that never touch a language model at all: API steps and Verify steps.

An API step looks like this:

POST https://api.example.com/users with body {"email": "{{unique.email}}", "plan": "pro"}
Expect status 201, store $.id as 'userId'

That is the whole seeding step. No Postman collection, no environment JSON, no Newman invocation, no file to read the output back from. The status code check and the JSONPath extraction ($.id) are compiled to deterministic code, not judged by a model, so a 201 either matches or it doesn't and userId either exists in the response or the step fails with the actual response body attached to the failure. The extracted userId becomes a {{userId}} variable immediately available to every step after it in the same file, including the plain-English UI steps and any Verify steps that follow.

Verify steps are the other deterministic primitive in v2. They compile to real Playwright checks (URL contains, title is or contains, text visible, a specific button/link/heading visible, element counts, a stored variable equaling a value) rather than being judged by the model. A Verify line that falls outside that grammar still runs, but it is agent-judged and flagged judged: true in the results, so you always know which assertions are deterministic and which are not.

A full seed-then-verify file might read:

---
version: 2
---
# New pro user sees the upgraded dashboard

POST https://api.example.com/users with body {"email": "{{unique.email}}", "plan": "pro"}
Expect status 201, store $.id as 'userId'

Log in as {{unique.email}} with password {{TEST_PASSWORD}}
Navigate to the dashboard

Verify text "Pro plan" is visible
Verify 'Upgrade' button is not visible

Consecutive plain-English steps, like "Log in as..." and "Navigate to the dashboard," are grouped into agent blocks that run on the same page and same browser session as the API step before them and the Verify steps after them. There is one file, one execution, one result. Run it with:

browserbash testmd run .browserbash/tests/new_pro_user_test.md

or fold it into a suite:

browserbash run-all .browserbash/tests --junit out/junit.xml

The run_end event carries a structured assertions block with the pass/fail detail for every Verify step, deterministic and judged alike, and a Result.md file is written after the run with the same information in a human-readable table. If you're driving BrowserBash from an AI coding agent instead of a human running commands, the same test file executes through the MCP server's run_test_file tool, and the agent reads the structured verdict directly rather than parsing terminal output. The full step grammar for API and Verify steps is documented in the learn section if you want the exact syntax before writing your first v2 file.

Postman/Newman vs testmd API Steps: Side-by-Side

Where Postman/Newman Still Wins

It would be dishonest to pretend testmd v2's API steps replace Postman for everything, because they don't try to. Postman is a full API development and testing environment. If your team's real problem is "we need a comprehensive, standalone API regression suite with hundreds of requests, mock servers, and a shared team workspace where engineers iterate on endpoints before they're even built," Postman is the better fit and testmd v2 is not designed to compete there. testmd's API step grammar covers GET/POST/PUT/DELETE/PATCH with a body, a status expectation, and a single JSONPath extraction per step. It does not have chained pre-request scripts, a mock server, response schema validation libraries, or a GUI for building requests interactively. If your API testing needs any of that depth, keep Postman for the API suite itself.

Postman/Newman also wins when the seeding data has nothing to do with a UI test at all, meaning you're not trying to get a browser to a known state, you're validating an API contract on its own. Mixing that into a UI test file would be the wrong instinct even if testmd v2 supported arbitrarily complex request chains, because it conflates two different kinds of tests with two different failure semantics.

Where Inline API Steps Win

The testmd v2 approach earns its keep specifically in the pre-test-seeding case: the API call exists only to set up a UI test, and nobody outside that one test cares about the API call in isolation. In that situation, keeping the seed and the assertion in one file removes the coordination tax entirely. There's no environment JSON to keep in sync with the UI test's expectations, no CI stage boundary to cross, and no separate log to open when triaging a failure. The variable that comes out of the API step ({{userId}}, {{unique.email}}, a token) is just there, in scope, for the rest of the file.

It also plays well with the rest of the BrowserBash workflow that most seeding scripts sit outside of: --budget-usd on run-all covers the whole suite including the API steps (which cost nothing themselves, since they're not model calls), --shard i/n slices deterministically across the same files, and saved auth profiles (browserbash auth save) can be layered on top of a file that already seeded its own test user via an API step, so you seed the account via API and then reuse a saved login for a different part of the flow if that's how your app is structured.

A Real Workflow: Seeding a User Then Verifying the UI

Here's what a team migrating a seeding step off Newman and into testmd v2 typically ends up with. Instead of a newman run seed-collection.json -e ci.postman_environment.json --export-environment out.json step followed by a script that reads out.json and exports USER_ID for the next CI stage, they collapse it into:

---
version: 2
---
# Order history shows the seeded order

POST https://api.example.com/orders with body {"userId": "{{TEST_USER_ID}}", "sku": "WIDGET-1"}
Expect status 201, store $.orderId as 'orderId'

Navigate to the orders page
Verify text {{orderId}} is visible
Verify 'Track order' button is visible

Run it directly, or as part of a CI job with agent-friendly output for a bot or dashboard to consume:

browserbash testmd run .browserbash/tests/order_history_test.md --agent

The --agent flag emits NDJSON, one JSON event per line, with step events as each part of the file executes and a run_end event carrying the pass/fail verdict and the assertions block. That's the same contract BrowserBash uses everywhere else, so a CI system or an AI coding agent parsing the output doesn't need a special case for "this test happened to seed its own data via an API call first."

CI Considerations: Two Tools vs One

Beyond the authoring experience, there's a maintenance dimension worth being honest about. A Postman/Newman-plus-UI-framework pipeline has two dependency trees, two version upgrade paths, and two failure taxonomies to teach new team members. When something breaks in CI, the first triage question is "which stage failed, the seed or the UI test," and answering it means opening two different log formats. With testmd v2 API steps, a failure in the seeding call and a failure in the UI assertion both show up in the same run_end event and the same Result.md, distinguished by which step failed, not which tool produced the log.

This isn't a purely aesthetic preference. Every extra tool boundary in a CI pipeline is a place where a variable can silently fail to pass through, where a version mismatch between a collection export and the environment it's run against can cause a false negative, and where a new team member has to learn a second system just to understand why a UI test is red. Consolidating the seed and the check into one file, one execution, and one deterministic result format is a real reduction in that surface area, not just fewer lines of YAML. If you're already running suites through the official GitHub Action, collapsing the seed step into the same testmd file also means one fewer job to configure in your workflow YAML, since the action already posts a verdict-table PR comment for whatever run-all reports.

When to Choose Postman/Newman

Reach for Postman/Newman when your seeding needs go beyond a handful of straightforward requests: multi-step request chains with conditional logic, response schema validation, a shared team workspace where non-automation engineers explore the API by hand, or a standalone API regression suite that has nothing to do with any particular UI test. If your organization already has a mature Postman collection covering the endpoints you'd be seeding through, and that collection is actively maintained as part of API development (not just test setup), there's little reason to duplicate that logic inside a testmd file. Keep Newman as the CI runner for that collection and treat it as infrastructure your UI tests can call into indirectly (seed via a script that hits the same endpoints, or seed via a lightweight API step that mirrors what the collection already validates more thoroughly elsewhere).

When to Choose testmd v2 API Steps

Reach for testmd v2 API steps when the seeding call exists purely to get a browser test into a known starting state: create a user, create a record, get an ID or token, and move straight into UI steps that use it. If you're tired of maintaining a Postman collection whose only consumers are your UI tests, if variable handoff between CI stages has caused real incidents, or if you want a single reviewable file that a teammate (technical or not) can read top to bottom and understand exactly what a test does before the browser even opens, this is the more direct path. It's also the natural choice if you're already running BrowserBash for the UI layer and want the seeding step to inherit the same reporting, budget controls, and sharding as the rest of your suite, instead of living in a parallel pipeline. The case study covers what that looks like once a team has moved a full suite over, seeding included. Note the current limit: v2 files drive the builtin engine, which needs ANTHROPIC_API_KEY or an ANTHROPIC_BASE_URL gateway, so if your setup is Ollama-only today, you'll need that key configured before the API and Verify steps in a v2 file will run.

Migrating Without a Rewrite

You don't have to convert everything at once. Existing v1 test files (no version: 2 frontmatter) keep behaving exactly as they do today, joined into a single agent objective, so there's no forced migration. If you're bringing existing Playwright specs into BrowserBash's format in the first place, browserbash import converts them heuristically, no model involved, with anything it can't translate landing in an IMPORT-REPORT.md instead of being silently dropped. From there, adding version: 2 and lifting a Newman-driven seed step into an inline API step is a per-file decision you can make as you touch each test, not a big-bang cutover. The tutorials walk through the import flow end to end if you're starting from an existing Playwright or Selenium suite rather than from scratch.

FAQ

Can testmd v2 API steps fully replace a Postman collection?

Not for everything. testmd v2's API steps cover single-request seeding with a status check and one JSONPath extraction per step, which fits pre-test setup well but does not replace a full API testing environment with chained requests, mock servers, or schema validation. For standalone API regression suites, Postman and Newman remain the stronger choice.

Does testmd v2 use an AI model to run API and Verify steps?

No. API steps and Verify steps that match BrowserBash's grammar compile to deterministic code, either an HTTP request with a status/JSONPath check or a real Playwright assertion, so they never involve model judgment. Only the plain-English UI steps in a v2 file (and any Verify line outside the deterministic grammar, flagged judged: true) go through the agent.

How do I pass a value from an API step into a UI step in the same test?

Use Expect status N, store $.path as 'name' on the API step, and the value becomes available as {{name}} in every step that follows in the same file, no environment export or separate CI step required.

Do I need an Anthropic API key to use testmd v2?

Yes, currently. testmd v2 files drive BrowserBash's builtin engine, which speaks the Anthropic API directly, so you'll need ANTHROPIC_API_KEY set or an ANTHROPIC_BASE_URL gateway pointed at a compatible model. It does not yet run directly on Ollama or OpenRouter, unlike BrowserBash's default v1 workflow.

Whichever way you land, seeding data before a UI test shouldn't require more tooling than the test itself needs. If you want to try the inline approach, install BrowserBash with npm install -g browserbash-cli and write your first version: 2 test file. Signing up at browserbash.com/sign-up is optional, everything above runs entirely from the CLI.