A regular expression is not a description of a string. It is a tiny search program, and the engine runs it: at each starting position it walks the pattern left to right, and every quantifier and every | is a decision it can come back and revise. Nearly everything that surprises people falls out of that one fact — why greedy and lazy return different answers, why the order of alternatives changes the result, why one pattern is instant and its lookalike hangs a server for two seconds. Learn the search and the syntax becomes a table you look up. Learn only the syntax and every failure looks like a new mystery.
Each chapter opens with the short version. Tap one to read the detail.
How a match actually happens
~2 min
A match is a span found inside the subject, not a verdict on the whole string. With no anchor the engine tries position 0, then 1, then 2, and the first start that works wins. A pattern that can match nothing succeeds at position 0, consuming nothing.
About a dozen characters carry meaning — . ^ $ * + ? ( ) [ ] { } | \, plus / inside a JavaScript regex literal. Everything else stands for itself, so cat means exactly what it looks like. A backslash strips a metacharacter of its power: /a\.b/ matches a.b and not axb.
The step beginners skip is where matching may start. /cat/ against concatenate succeeds at index 3 — the pattern never claimed "this string is a cat", only "a cat appears somewhere in here". Anchors turn the second question into the first.
And a pattern that can match nothing does exactly that, rather than failing. a* is satisfied by zero as, and position 0 already qualifies:
/cat/.exec("concatenate").index // 3 — matched inside a longer word
/a*/.exec("bbb")[0] // "" — a zero-length match at index 0
"bbb".match(/a*/g) // ["", "", "", ""]
One escaping trap belongs to patterns built at runtime. A literal /\d/ needs one backslash; the constructor takes an ordinary string first, so new RegExp("\\d") needs two — the string layer eats one before the regex ever sees it. Forget that and your hand-built pattern silently becomes a search for the letter d.
Rule: before asking what a pattern matches, ask where the engine is allowed to start — most "wrong match" bugs are a missing anchor rather than a wrong pattern.
The backtracking model, and why first beats longest
~2 min
The engine searches depth-first and rewinds. Every quantifier and every | is a decision point it can revisit, so a pattern is a tree of paths rather than a formula. The consequence that catches everyone: the winner is the first path that succeeds, never the longest available.
At one starting position the engine walks the pattern left to right. At every choice — repeat again or stop, take this branch or the next — it takes the first option and remembers the others exist. If the pattern later fails, it rewinds to the most recent remembered choice and tries the next option. Only once every path from this position is exhausted does it restart one character further along.
That is why alternation is first-wins, not longest-wins. | has the lowest precedence of anything in a pattern, and branches are tried in written order:
/a|ab/.exec("abc")[0] // "a" — not "ab"
"the catalog".match(/cat|catalog/)[0] // "cat"
"the catalog".match(/catalog|cat/)[0] // "catalog"
Nothing is broken: a matched and let the rest of the pattern succeed, so the search stopped. Reordering the branches changes the answer — a fact about the engine, not the words.
And it is a choice of engine, not a law. Automaton-based engines simulate every path at once: linear time guaranteed, but no backreferences and no lookaround, and the classic command-line tools built that way return the longest match rather than the first.
echo abc | grep -oE 'a|ab' # ab — longest wins here, whatever the branch order
Two habits follow: put the likeliest branch first, and prefer [abc] to (?:a|b|c) — one comparison rather than three branches with rewind points.
Rule: read a pattern as a search with rewind points, and when a match surprises you, ask which choice the engine made first.
Saying which characters count
~2 min
A bracketed class consumes exactly one character, so [cat] is never the word "cat". Ranges are code-point ranges, a negated class still consumes a character, and the shorthand escapes mean different things in different engines — \w is plain Latin in JavaScript and Unicode-aware in Python.
[abc] matches one character drawn from the set, and that sentence alone kills the commonest beginner pattern: "cat".match(/[cat]/g) returns three one-character matches, ["c","a","t"].
A range is a range of code points, which is why [A-z] is a bug rather than a shortcut for "both cases" — punctuation sits between Z and a:
/^[A-z]+$/.test("a^b") // true — ^ [ \ ] _ ` all fall inside A-z
Inside brackets the rules change: ^ negates only when it comes first, and [\b] is a backspace character rather than a word boundary. A negated class still consumes something, so [^a] can never match at end of input — and it accepts a newline even where . will not.
The shorthands are where flavors split, silently. \w is the unaccented Latin letters, digits and underscore in JavaScript, even under u; a Python str pattern makes it Unicode-aware, and re.A narrows it back.
/\w/u.test("é") // false — JavaScript
re.search(r"\w", "é") # matches
re.fullmatch(r"\d+", "٣٤٥") # matches — these are \d too
That last line is a live bug in validation code: the string passes a "digits only" check, and int() reads it as 345. For real Unicode categories in JavaScript, \p{L} and its family do the job — but only under u or v. Without one of those flags \p is simply the letter p, failing quietly.
Rule: a class is one character. If you need "not this word", that is a lookahead, not a class.
Repetition, and who gives characters back
~2 min
A quantifier binds to the single atom before it, takes as much as it can, then hands characters back one at a time until the rest of the pattern fits. Adding ? makes it start minimal and grow instead. A possessive *+ hands nothing back — and JavaScript has no possessive form.
*+?{n}{n,}{n,m} — and a quantifier attaches to exactly one atom, so ab* repeats only the b while (?:ab)* repeats the pair.
The give-back is the mechanism worth internalizing. <.*> takes the whole line, then walks backwards until it can still find a >. Adding ? makes the quantifier minimal-first instead. A negated class reaches the same answer with nothing to give back at all:
"<a><b>".match(/<.*>/)[0] // "<a><b>" greedy, walked back to the last >
"<a><b>".match(/<.*?>/)[0] // "<a>" lazy, grew until the first >
"<a><b>".match(/<[^>]*>/)[0] // "<a>" no backtracking involved
Laziness is local, and it is not "shortest match wins". The search is still leftmost-first, so /a*?$/.exec("aaa") returns aaa: the anchor forces the lazy quantifier to grow at the first start position rather than the engine moving to a later one.
Possessive quantifiers take the maximum and refuse to return any of it, which changes results, not just speed. \d+9 matches 199 because \d+ gives the final 9 back; \d++9 can never match anything. Availability is the part most summaries get wrong: Perl, PCRE2 and Python 3.11 upwards have it, while JavaScript rejects \d++ as a syntax error.
One last surprise: a repeated capturing group keeps only its final repetition, so /(\d)+/ on 1234 captures 4.
Rule: when a quantifier's give-back is doing real work, check whether a negated class can express the stopping condition instead — same answer, no rewinding.
Anchors and word boundaries
~2 min
^ and $ assert a position and consume nothing. The m flag moves them to every line break and changes nothing about .. \b is the seam between a word character and a non-word character, so it inherits whatever that particular engine thinks a word character is.
^…$ is how you say "the entire string", and two traps sit around it.
The first trap is precedence. | binds looser than the anchors, so ^a|b$ parses as (^a)|(b$) and cheerfully matches xb. Group the alternation — ^(?:a|b)$ — whenever you meant both branches anchored.
The second is a genuine disagreement about where a string ends:
re.search(r"c$", "abc\n") # matches — Python allows a trailing newline
re.search(r"c\Z", "abc\n") # no match — \Z is the absolute end in Python
"abc\n" =~ /c\Z/; # true — in Perl and PCRE2, \Z also allows that final newline
"abc\n" =~ /c\z/; # false — \z is the strict one here
/c$/.test("abc\n") // false — and JavaScript has no \A, \Z or \z at all
Word boundaries have the same problem. \b is zero-width and asserts that one side is a word character and the other is not, counting the subject's ends as non-word. That makes \bcat\b the correct way to rename cat without mangling concatenate. But an apostrophe is not a word character, so \b\w+\b splits don't into don and t; and JavaScript's word set is unaccented Latin, so /\bcafé\b/ fails on the word café.
Rule: anchor when you mean "the entire string", and treat \b as a claim about the engine's word set rather than about language.
Groups, names and backreferences
~2 min
Groups are numbered by their opening parenthesis, index 0 is the whole match, and a group in an untaken branch reads as undefined or None, never as an empty string. Names are an alias over that numbering — and their syntax is one of the sharpest flavor splits there is.
Count opening parentheses left to right and you have the numbering; an enclosing group takes the lower number. /(a(b(c)))/ on abc gives abc, bc, c as groups 1 to 3, with the whole match at index 0.
A group that did not participate is not an empty string — JavaScript reports undefined, Python None. Confusingly, the same group contributes an empty string in a replacement, which is why "b".replace(/(a)?b/, "[$1]") produces []. Use (?:…) for grouping alone and you sidestep both.
Named groups are an alias rather than a namespace: the group still holds its number. The syntax does not travel, and each engine rejects the other's form:
"in 1999".match(/(?<y>\d{4})/).groups.y // "1999"
re.search(r"(?P<y>\d{4})", "in 1999").group("y") # '1999'
A backreference matches the text a group captured, not the pattern again, which is how title=(["'])(.*?)\1 pairs a closing quote with the opening one. The flavors disagree once more: a reference to a group that never participated succeeds as an empty match in JavaScript and fails in Python.
Atomic groups belong here because they concern the other half of a group — its rewind points. (?>…) discards every backtracking position created inside it, so (?>\d+)9 cannot match 199 even though \d+9 can. JavaScript has no (?>…), but its engine cannot backtrack into a lookahead either, so capturing there and replaying the capture reproduces it exactly:
/(?=(?<n>\d+))\k<n>9/.test("199") // false — same result as (?>\d+)9
Rule: capture only what you are going to read, and use (?:…) for everything else.
Testing without consuming
~2 min
A lookaround tests the text at the current position and then puts the cursor back, adding a condition without eating anything. That is what lets several independent rules apply at one spot, and what makes "a line that does not contain this word" expressible at all.
(?=…) succeeds if the sub-pattern matches ahead; (?!…) succeeds precisely when it cannot. Either way the cursor does not move, so assertions stack — each tests from the same position, giving independent conditions in any order:
const pw = /^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{8,}$/;
pw.test("Passw0rd"); // true
pw.test("password"); // false — no digit, no capital
Without lookahead, that rule would have to commit to an order the requirement never had.
The negative form is the one to memorize, because the obvious alternative is not the same claim. ^(?!.*foo).*$ means "this line does not contain foo". ^[^f]*$ means "this line contains no letter f", which is far stronger:
/^(?!.*foo).*$/.test("a for b") // true
/^[^f]*$/.test("a for b") // false
Lookbehind — (?<=…) and (?<!…) — inspects the text just before the position, and its length rules diverge: JavaScript accepts a variable-length lookbehind, PCRE2 accepts a bounded one, and Python's re refuses anything but fixed width, with the error look-behind requires fixed-width pattern.
Consuming nothing also lets a lookaround mark positions rather than text — inserting thousands separators is the classic case:
Rule: reach for a lookaround when the thing you are testing must still be available to the rest of the pattern.
Catastrophic backtracking, measured
~2 min
One shape — a quantifier wrapped around something already quantified and ambiguous — turns a pattern exponential. The dangerous input is the one that almost matches, since only a failure forces every path. Measured below: 22 as take 28 ms, 28 take 1.7 seconds.
^(a+)+$ is the canonical example, and ambiguity is why it is dangerous: a run of as can be divided between the inner + and the outer + in many ways, all describing the same text. Feed it a subject that matches and it returns instantly; feed it one that fails at the very last character and the engine must prove no division works:
/^(a+)+$/.test("a".repeat(30)) // true, under 1 ms
/^(a+)+$/.test("a".repeat(22) + "!") // 28 ms
/^(a+)+$/.test("a".repeat(26) + "!") // 435 ms
/^(a+)+$/.test("a".repeat(28) + "!") // 1749 ms
Each extra character doubles the work, and on a single-threaded server one such request pins the process. That is the whole attack.
The instinct to reach for a bigger quantifier is wrong; the fix is to remove the ambiguity. In a quoted-string scanner, (?:[^"]|\\.)* lets a backslash be eaten by either branch; excluding it from the first — (?:[^"\\]|\\.)* — makes the division unique. On the same failing input that one change measured 43 ms against 0.002 ms.
Where the engine offers them, a possessive quantifier or an atomic group removes the rewind points outright. Where it does not, cap the subject length and impose a limit; PCRE2 ships match and depth limits, and the error when one trips names the cause: Check your regex for nested unlimited loops.
Escaping user input does not help here — that defends against a user writing your pattern, a different bug.
Rule: if a character can be matched by two parts of your pattern, you already have the shape — fix the ambiguity before tuning anything else.
Flags, and the regex object that remembers
~2 min
Flags change how a pattern is parsed or how the methods iterate, rarely both. m moves the anchors and does nothing to .; s does the opposite. And in JavaScript a g or y regex is a stateful object that remembers where its last match ended.
The flags that change matching: i relaxes case, m makes ^ and $ match at every line break, s lets . cross a line terminator. The last two are routinely confused: m never affects the dot, s never affects an anchor.
The flags that change parsing: u and v make a JavaScript pattern Unicode-aware and turn unknown escapes into errors, v adding set operations inside classes. Python and PCRE2 instead offer x, which ignores unescaped whitespace and allows # comments — the best readability tool here, and one JavaScript lacks. Both also take inline modifiers: (?i) for the rest of the pattern, (?i:…) for one span, with Python requiring the bare form at the very start.
Then g, which changes the API rather than the pattern — and brings state. A g or y regex stores lastIndex, so reusing one object across calls gives alternating answers:
Nothing is wrong with the pattern; the object is mid-scan. Reset lastIndex to 0, build the regex where you use it, or drop g for a plain yes or no. The trap is specific to JavaScript: Python keeps no state on a compiled pattern.
y is the useful side of the same feature: it must match exactly at lastIndex rather than scanning forward — what a tokenizer wants.
Rule: never share a g regex across calls unless you are managing lastIndex yourself.
Getting the answer out
~2 min
Choose the call by what you need back: a boolean, an index, one match with its captures, or every match. Two costly surprises — JavaScript's match with g throws the captures away, and Python's findall returns groups rather than whole matches once the pattern has any.
In JavaScript, test gives a boolean, search an index, exec one match array or null. match with g returns every matched string and silently discards the captures; matchAll, which requires g, keeps them.
Python divides it differently: re.match anchors at position 0, re.search scans, re.fullmatch demands the whole string. Its trap is findall, whose return shape depends on how many groups the pattern has — none gives whole matches, one gives that group, two or more gives tuples. finditer sidesteps that.
re.findall(r"\d+", "a1 b22") # ['1', '22']
re.findall(r"(\d)\d", "a12 b34") # ['1', '3'] — the group, not the match
A replacement string has its own small language and it is not a pattern. JavaScript reads $1, $<name>, $& for the whole match and $$ for a literal dollar; Python reads \1 and \g<name>. Backslash escapes are not interpreted there, so a replacement of "\\d" inserts those two characters.
Splitting has two. A capturing group in the separator is interleaved into the result, and the limit argument differs between the two languages:
"a,b,c,d".split(/,/, 2) // ["a", "b"] — the rest is dropped
re.split(r",", "a,b,c,d", maxsplit=2) # ['a', 'b', 'c,d'] — the rest is kept
Rule: pick the call by the shape you want back, and check what your split limit does with the remainder.
The same pattern in three engines
~2 min
"What does this pattern match?" has no answer until you name the engine. Three families dominate — JavaScript, Python's re, and the Perl-compatible family built on the PCRE2 library — and they disagree about anchors, word characters, lookbehind width, and whole feature sets.
The disagreements that bite, each checkable in a shell:
•
Word characters.\w is unaccented Latin in JavaScript, Unicode-aware for a Python str pattern, and unaccented Latin in PCRE2 until Unicode support is switched on. Matching \w+ against café with pcre2grep returns caf by default and café with -u — a difference that silently changes what a validator accepts.
•
End of subject.$ matches before a trailing newline in Python and Perl but not in JavaScript. \Z is the absolute end in Python, while in Perl and PCRE2 it also allows that final newline and \z is the strict one.
•
Lookbehind width. JavaScript takes any length, PCRE2 a bounded one, Python nothing but fixed width.
•
Named groups.(?<n>…) in JavaScript and PCRE2, (?P<n>…) in Python.
The Perl-compatible family then adds a tier the other two lack: \K drops everything matched so far from the reported match, (?R) recurses the entire pattern, (?1) and (?&name) call a group as a subroutine, (?(1)yes|no) branches on whether a group participated, (?|…|…) gives each alternative the same group numbers, and (*SKIP) and (*FAIL) steer the backtracker by hand. Two worth seeing run:
That second matches balanced nesting — the honest tell that "regular expression" stopped describing a regular language long ago.
Rule: never state what a pattern does without naming the engine, and when you port one, re-test the anchors and the shorthand escapes first.
When not to reach for a regex
~2 min
A regex is the right tool for a small, flat, well-defined shape and the wrong one for anything nested or quoted. Write down what the pattern must reject before you write the pattern, and never build one by pasting user input straight into it.
Three habits separate a pattern that survives from one that gets patched forever.
Prove the negatives. A pattern is only as good as the strings you have shown it rejects, and almost every regex bug that reaches review is a case nobody tried. Keep the rejections beside the accepted cases, in a test.
Escape anything you interpolate. A user-supplied string dropped into a pattern is not a search term, it is source code: a . silently becomes "any character" and a stray ( is a syntax error.
re.search("a.b", "axb") # matches — the dot was a wildcard
re.search(re.escape("a.b"), "axb") # None — now it is a literal dot
JavaScript's RegExp.escape is recent and missing from runtimes only a version or two old, so check before relying on it. And mind its limit — it stops the user writing your pattern, not a slow pattern.
Know when to stop. Nesting, comments and quoting are decisions one pattern cannot make, which is why Python's own documentation says outright that parsing markup this way is painful and points you at a parser. The same holds for anything with a real grammar: comma-separated data with quoted commas, dates, web addresses, email. For email, match something permissive and send a confirmation — the address either receives mail or it does not.
Two smaller ones. Use your language's verbose mode if it has one, and remember a plain includes or split is often clearer and faster than starting the engine.
Rule: write down what the pattern must reject first — and if that list mentions nesting or quoting, you want a parser.
Written by Keentune. We are not affiliated with or endorsed by the organizations whose documentation informs this guide, and any linked sources belong to their respective owners.
All exam, test, and product names and trademarks are the property of their respective owners and are used here for identification and reference only. Keentune is independent study practice — not affiliated with, authorized, or endorsed by any of these organizations.