Web search provider — cost analysis & recommendation

Status: Decision document. Not scheduled for implementation — captures the swap target if/when we decide cost justifies action.

Context

The product-search feature is currently fronted by SerpApi, wired through server/.../service/WebProductSearchService.kt and consumed by both the POST /products/search route and the scheduled ProductPriceWatchRefreshScheduler. The integration is functional but expensive past the 250-search/month free tier:

• SerpApi free: 250 searches/mo, $0
• SerpApi paid: $25/mo for the next 1,000 searches (~$0.025/request), separate engine pricing, rigid monthly plan
• We are close to the free-tier ceiling at current growth

In 2026 the SERP-API market has multiple competitors offering Google Shopping data with the same surface area at 10×–25× lower per-request cost. This document captures the cost comparison and the recommended swap target so a future change can be picked up cheaply if/when we decide it’s worth doing.

Why a swap is feasible

The SerpApi-specific code lives behind a single seam:

WebProductSearchService.search(query, gl, hl): Result<List<WebSearchHit>>

WebSearchHit is a normalized DTO (store, title, unit price, currency, product URL, image URL). Every caller — the route handler, the scheduler tick, the manual checkWatchNow path — consumes only this normalized contract. The web_product_search_runs audit table already stamps provider per row (currently "SERPAPI"), so the schema was already designed for multi-provider coexistence.

Estimated effort to swap (per code exploration): 4–6 hours of focused work, plus testing.

Cost comparison

Google Shopping coverage, real-time / live mode, no monthly minimums where possible:

Projected monthly bill at three volumes:

The break-even is the first request past the SerpApi free tier — every request beyond that is roughly 12× cheaper on DataForSEO Live.

Recommendation

DataForSEO Merchant API (Google Shopping, Live mode) is the right target if/when we swap. Reasoning:

1. Cost: ~$0.002/query, ~12× cheaper than SerpApi at any volume past free tier.
2. No monthly minimum. Pure pay-as-you-go matches our usage profile (bursty user searches + a budget-capped scheduler) better than Serper.dev’s $50/mo floor.
3. Schema parity. Returns a tasks[].result[].items[] array with title, price, seller, link, image_url — every field our WebSearchHit mapper already needs from SerpApi. Adapter is small.
4. Colombia support via location_code=2170 + language_code=es. Currency stays COP via the existing currencyForGl helper.
5. Acceptable latency. User tolerance is “seconds, not minutes,” and DataForSEO Live runs in seconds. Standard queue (5-min delay) is available as a cheaper fallback for the scheduler if we ever want to push costs further.

Runners-up worth knowing about:

• Scrape.do if we ever want a multi-engine pool (Shopping + Maps + Flights billed from one credit balance) — slightly more expensive than DataForSEO but simpler operational model.
• Serper.dev if volume ever climbs above ~50k/month, where its flat $50 plan beats per-request providers.

Things that don’t change in a swap

• The WebProductSearchService facade and WebSearchHit contract.
• The scheduler tick, the daily-budget gate (callsInLast24h()), the watch-refresh logic — all provider-agnostic.
• The web_product_search_runs audit table — only the provider column value would change.
• The gl/hl parameter contract on search(...).
• Cloud Run deployment shape — secrets would just gain two new entries (DATAFORSEO_LOGIN, DATAFORSEO_PASSWORD) via the existing --set-secrets pattern.

Things that would need touching if we proceed

These are not specified here in detail — they’re flagged so a future implementer knows where to look:

1. A thin WebProductSearchProvider interface in front of the existing class, with two implementations (SerpApi…, DataForSeo…).
2. A WEB_SEARCH_PROVIDER env-var switch + Koin binding so swap and rollback are deploy-time (or env-update-time) decisions, not code changes.
3. Two new GCP Secret Manager secrets (DATAFORSEO_LOGIN, DATAFORSEO_PASSWORD) plus IAM grant to oter-server-runtime@, mirroring the SerpApi setup.
4. A new provider test using a captured DataForSEO Merchant response fixture. The existing ProductPriceWatchRefreshSchedulerTest doesn’t need changes — it mocks search() directly, not the upstream.

Decision

Defer. At current volume (~250 req/mo, free tier) the swap saves ~$0.50/mo and adds operational risk (new vendor, new failure modes). Revisit when any of these triggers:

• Monthly volume crosses ~500 (so we’re paying SerpApi $25 for partial use of the 1k tier).
• We add a new feature (more watches per user, broader auto-search, etc.) that’s expected to push usage above 1k/mo.
• SerpApi’s pricing or terms change.

Until then, keep this document up to date with any new alternatives that surface.

Sources

• SerpApi Pricing 
• DataForSEO Google Shopping API Pricing 
• Best SerpApi Alternatives in 2026 — Scrape.do 
• Cheapest SERP API in 2026 — PROXIES.SX 
• SerpApi Alternatives 2026 — Scrappa