# A price test should change the decision, not just the number www.first1000.co publishes a public search API over its indexed pages; the guide is at https://www.first1000.co/agent-access. Source: https://www.first1000.co/blog/pricing-test-decision-not-number Source observed: 2026-09-21T22:17:45.757Z Last checked: 2026-09-21T22:17:45.757+00:00 Indexed: 2026-09-21T22:19:52.492+00:00 --- # A price test should change the decision, not just the number **First 1000 field note · Updated September 21, 2026** **By Ali Abouelatta · Original issue by Ali Abouelatta: March 1, 2024** A price test is useless if we cannot say what customer decision should change. ## The old question is too small Our March 1, 2024 issue *Pricing Experiments* examined the top 100 grossing apps and price changes over six months. It reported an average price increase of roughly **27%**, about **20%** of apps testing an increase, roughly **one-third** of tests pushing harder toward annual plans, and about **10%** of tracked experiments failing. These are historical observations, not a benchmark for a current paywall. Before choosing a number, decide what the test should move: monthly versus annual selection, free-to-paid conversion, revenue per payer, refund risk, or willingness to commit. Changing several at once produces a result without an explanation. ## Write the decision before the treatment - Raise the monthly price while keeping the annual price near the old ten-month ratio: test a commitment nudge. - Raise both prices by the same percentage: test willingness to pay. - Add a lifetime plan: test whether some customers prefer one payment to recurring uncertainty. Write one sentence before shipping: “We expect this change to make [customer segment] choose [plan/action] because [reason].” Keep the rest of the page stable enough to read the answer. Annual-plan examples are useful because plan mix is part of the design rather than an accidental side effect. ## Treat failure as a result Not every pricing experiment wins. A failed change can indicate that the price, packaging, promise, or audience was wrong for the moment. It is not a reason to keep iterating until a higher number creates a pleasing screenshot. Set stop conditions before launch. Decide what would trigger a rollback, what movement in annual share would compensate for a conversion decline, and which customer segment needs protection. Without those decisions, we are running a page change rather than a pricing experiment. ## A five-question pricing review 1. Which customer decision is this test trying to change? 2. Which unit is primary: revenue per visitor, payer conversion, revenue per payer, annual share, or retention? 3. What tradeoff would make the test a win? 4. What segment or existing customer promise needs protection? 5. What result would make us stop, roll back, or run a different test? ## The takeaway Price tests become useful when they are built around a customer decision and a named tradeoff. The number is the treatment; the decision is the experiment. ## Sources and original research - [Original First 1000 source: 🤑Pricing Experiments, March 1, 2024](https://first1000.substack.com/p/pricing-experiments) Adapted from our original *Pricing Experiments*, published March 1, 2024. The top-100-app observations and percentages are historical source reporting; this adaptation turns them into a test-design checklist and adds no current pricing benchmark or new experiment. Updated September 21, 2026. Based on original First 1000 reporting and the source above. --- [Read this page’s summary](https://www.first1000.co/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.first1000.co%2Fblog%2Fpricing-test-decision-not-number&trace=t_dae5763819f4) ## Ask a follow-up about this page Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from www.first1000.co’s knowledge base. No account or API key required. Open this URL with your own question, URL-encoded. Only q is required. No account or setup. `GET https://www.first1000.co/agent/faq?q={question}&trace=t_dae5763819f4` [Tool guide and examples](https://www.first1000.co/agent-access) · [OpenAPI](https://www.first1000.co/api/public/v1/openapi.json) · [Page TLDR form](https://www.first1000.co/agent-tools/page-tldr?url=https%3A%2F%2Fwww.first1000.co%2Fblog%2Fpricing-test-decision-not-number). Opening a form does not execute a tool.