Regex or Parser? Choose the Representation Your Task Needs

An import script starts by finding an amount in a JSON response. The first payload contains one amount field, so a short regular expression appears to work. Later, the response includes a nested adjustment with its own amount. The script still finds a number, but it no longer finds the one the application intended to use.

The failure concerns the representation chosen for the task. Searching text found a local pattern; the application needed a value at a particular location in a structured document. Parsing the JSON would have made that relationship available to the code.

Regex and parsers often work together. The decision is which structure the application needs to recognise, preserve, or change, and which tool exposes that structure clearly enough to test.

Distinguish recognising text from interpreting its position

A regular expression can identify an agreed code, such as an uppercase prefix followed by a bounded sequence of digits. A parser applies rules for a format or language and produces a representation of the input, often a tree or a sequence of records. That representation lets code distinguish a field from text that merely resembles one.

Neither tool automatically establishes business meaning. A parser can return a syntactically valid order whose customer does not exist. A regex can correctly identify a date-shaped string that does not describe a real calendar date. Separate recognition, representation checks, and domain decisions when deciding what success means.

Avoid the absolute claim that software regex engines can never handle nesting. The classical notion of regular languages has limits, but real engines expose different features. PCRE2 documents recursive patterns, while RE2 deliberately excludes constructs that conflict with its design goals. A technical possibility in one engine does not establish portability or suitability for a document-processing task.

For a known format with a maintained parser, the practical question is usually whether recreating its interpretation in a pattern adds value. The need to select nested fields, preserve syntax, or explain errors often makes the parser’s representation useful well before regex reaches a theoretical limit.

Parse JSON before selecting the relevant field

Consider this response:

{
  "order": {
    "id": "O-104",
    "total": { "amountMinor": 4500, "currency": "NZD" }
  },
  "adjustment": { "amountMinor": 500 }
}

A text search for the first amountMinor depends on ordering rather than the requested relationship. If a producer rearranges object members, the search can change its result even though the document still conveys the same order data. Escaped strings can introduce further misleading text.

JSON’s specification, RFC 8259, defines objects, arrays, strings, and the rest of the format. A JSON parser handles that syntax so application code can select the intended path:

const payload = '{"order":{"total":{"amountMinor":4500}}}';
const parsed = JSON.parse(payload);
console.log(parsed.order.total.amountMinor); // 4500

This deliberately small example assumes the expected structure. Production code receiving external data should check that structure before using the value. Parsing establishes that the input can be represented as JSON; it does not establish that order.total exists or that amountMinor meets the business contract. JSON Schema and TypeScript types explain that next boundary.

Once a particular string field has been selected and checked, regex can be useful again. A customer reference inside the parsed order may have a simple local format that a bounded pattern describes well. Parsing the document and checking a field solve complementary parts of the same workflow.

Check how the parser handles ambiguity that the application’s contract forbids. Duplicate object member names are one example: a parsed object may not retain enough information to reveal that the source supplied the same field twice. If the boundary must reject duplicates, choose a parsing stage that can detect them. Validating the final object alone may be too late to inspect that source-level condition.

Tests should therefore cover the required interpretation as well as valid syntax. Reorder members, add an unrelated object with a similar field, and include a string containing text that looks like a field declaration. The selected order total should remain the same. These cases directly challenge the fragile text-search assumption without requiring a large collection of unrelated parser tests.

CSV requires a decision about its dialect

A comma is not always a field separator. In a quoted CSV field, it can be ordinary data:

name,notes,plan
Aroha,"prefers email, not phone",standard

Splitting the second line on every comma produces too many fields. Quoted fields can also contain line breaks or escaped quotes, so reading each physical line as one complete record is not sufficient for every CSV file.

RFC 4180 documents a common CSV format, including quoted fields and embedded line breaks. It is informational rather than a universal definition of every spreadsheet export. Select a parser and settings that match the producer’s actual delimiter, quote, encoding, and header conventions.

Make disagreement visible. If a row has the wrong field count or malformed quoting, decide whether the import should reject the file, quarantine the record, or request correction. Quietly shifting values into adjacent columns can create data that passes later type checks while referring to the wrong fields.

Preprocessing deserves similar care. Removing whitespace or line breaks before parsing can alter quoted content. Preserve the original input for the required audit period where appropriate, and perform transformations at a stage where the parser has already identified which text belongs to each field.

Structural edits need a representation that preserves what matters

HTML illustrates why extracting a fragment and transforming a document are different tasks. A controlled template may contain one known marker that an exact search or small regex can replace. Arbitrary HTML adds attributes, quoted values, nesting, comments, character references, and parsing rules that depend on context.

For a task such as changing the destination of selected links, use an HTML parser to identify elements and their attributes. Then consider what serialising the result preserves. A parsed representation can normalise quoting, whitespace, or document structure, which may be acceptable for generated output but undesirable for a minimal source edit.

Programming-language transformations have the same distinction. An abstract syntax tree can identify a function call without mistaking a similar string or comment for code. A source-preserving transformation may also need token positions, comments, or a concrete syntax representation. Choose the parser’s output according to the change the tool must make.

Parsing does not make a value safe for every destination. A parsed link can still have a disallowed URL scheme, and parsed markup can still contain active content. Apply the relevant destination policy after identifying the structure. Syntax recognition and permission to use that syntax remain separate decisions.

Tokenisation can use regex without doing all the parsing

For an expression such as (4 + 3) * 8, a tokenizer might recognise numbers, parentheses, and operators. Regex can describe several of those token forms. A parser then determines how the tokens combine, including grouping and precedence.

A tokenizer must account for the whole input. A global search that extracts recognised tokens while silently skipping unknown characters can turn an invalid expression into a different valid-looking token stream. Track positions and reject unconsumed text where the language requires complete recognition.

A structured input workflow

For a custom language, write down the intended grammar before adding features to an ad hoc reader. Quotes, comments, lists, and operator precedence each introduce interactions. If an existing format already expresses the requirements clearly, adopting it may reduce both implementation and documentation work.

Evaluate failure handling and resource use

A parser is not a general performance or security guarantee. Large inputs, deep nesting, expensive transformations, and unsuitable configuration can still exhaust resources. Set limits appropriate to the workload and examine how the chosen implementation reports malformed or excessive input.

Regex needs corresponding scrutiny in its actual engine. Ambiguous repetition in a backtracking implementation can make some near-matches expensive. Linear-time matching guarantees offered by a particular engine apply to its supported language and conditions; they should not be transferred to every regex library.

Useful tests follow the intended operation:

Error messages matter to users maintaining the input. A parser that identifies a failing record or token position may make a structured import much easier to support. Preserve those diagnostics through the application boundary instead of converting every failure to an unhelpful invalid-file message.

Choose the smallest representation that supports the actual task. A literal search is sufficient for a fixed marker, a regex can describe a bounded textual family, and a parser can expose relationships the application needs to navigate. When requirements change, review that choice alongside the tests. The regex builder workflow remains useful for the local patterns within a larger processing pipeline.