lesson.md

layout	default
title	Regular Expressions
parent	Lessons
nav_order	10
permalink	/lessons/regular-expressions/
course_lesson	true
course_index	10
previous_page	/lessons/errors-logging-testing/
previous_title	Errors, Logging, and Testing
next_page	/lessons/apis-http/
next_title	APIs and HTTP

10 - Regular Expressions

Regular expressions, often called regex, describe text patterns. Use them when normal string methods are not enough, and keep patterns readable.

A simple picture

A regex is like a reusable stencil placed over text. The stencil describes shapes such as “three digits, a hyphen, then four digits.” It does not understand meaning; it only checks character patterns.

Before using regex, ask whether in, .startswith(), .endswith(), .split(), or .replace() would be clearer.

First search

import re

match = re.search(r"\d+", "Order 123")
if match:
    print(match.group())

re.search() looks anywhere in the text and returns a match object or None. match.group() returns the matched text.

The pattern r"\d+" means:

• \d — one digit;
• + — repeat the previous piece one or more times.

Use a raw string such as r"\d+" for patterns so Python does not process backslashes before the regex engine sees them.

Core pattern pieces

Pattern	Meaning
.	almost any one character
\d	digit
\s	whitespace
\w	Unicode word character
[abc]	one listed character
[^abc]	one character not listed
*	zero or more
+	one or more
?	zero or one; also makes a quantifier lazy
{2,4}	between two and four repeats
^	start of text (or line in multiline mode)
$	end of text (or line in multiline mode)
\A / \Z	absolute start / end of the string

To match a symbol that has special meaning, escape it. For example, r"\." matches a literal full stop.

Search, match, and full match

text = "Student ID: ST-204"
pattern = r"ST-\d{3}"

print(re.search(pattern, text))
print(re.match(pattern, text))
print(re.fullmatch(pattern, "ST-204"))

• search() scans for the first match anywhere;
• match() checks only at the beginning;
• fullmatch() requires the entire string to match.

Use fullmatch() for a format validator. Use search() for finding something inside larger text.

Find, split, and replace

text = "red, green; blue"

colors = re.split(r"\s*[,;]\s*", text)
numbers = re.findall(r"\d+", "A12 B7 C305")
cleaned = re.sub(r"\s+", " ", "too    many\tspaces").strip()

print(colors)
print(numbers)
print(cleaned)

findall() returns strings when there are no capturing groups. Capturing groups can change its result shape, so choose groups deliberately.

Groups

Parentheses group pattern pieces and capture text:

pattern = re.compile(r"(?P<level>INFO|WARNING|ERROR): (?P<message>.+)")
match = pattern.fullmatch("ERROR: Disk full")

if match:
    print(match.group("level"))
    print(match.group("message"))
    print(match.groupdict())

Use (?:...) for a non-capturing group when grouping is needed but the text does not need a numbered capture.

Greedy and lazy repetition

text = "<b>one</b><b>two</b>"

print(re.findall(r"<b>.*</b>", text))
print(re.findall(r"<b>.*?</b>", text))

* and + are greedy: they consume as much as possible while allowing the pattern to finish. Adding ? makes them lazy. For real HTML, use an HTML parser rather than regex.

Flags and readable patterns

pattern = re.compile(
    r"""
    ^
    (?P<code>[A-Z]{2})
    -
    (?P<number>\d{4})
    $
    """,
    re.VERBOSE,
)

re.VERBOSE permits whitespace and comments in a complex pattern. Other common flags include re.IGNORECASE and re.MULTILINE.

Bug Hunter

Bug 1: backslashes are processed twice

pattern = "\bcat\b"

Bug 2: validation accepts extra text

valid = re.search(r"\d{4}", "code 1234 invalid")

Bug 3: a match object is used without checking

match = re.search(r"\d+", "no number")
print(match.group())

Show Bug Hunter fixes

1. Use a raw string: r"\bcat\b". In an ordinary string, \b becomes a backspace character.
2. Use re.fullmatch(r"\d{4}", text) when the whole value must follow the format.
3. Check if match: before accessing groups.

Practice

1. Find the first number in text.
2. Find all numbers.
3. Validate a simple phone number.
4. Extract email-like values.
5. Split on commas or semicolons.
6. Replace repeated whitespace.
7. Match a string from the beginning and end.
8. Use named groups.
9. Explain a pattern in plain English.
10. Build a log-line parser.

Show hints

1. Use re.search.
2. Use re.findall.
3. Define the accepted format before writing the pattern.
4. Keep validation rules realistic; regex is not full email validation.
5. Use a character class such as [;,].
6. Match \s+.
7. Use ^ and $.
8. Use (?P<name>...).
9. Describe each symbol before testing.
10. Capture timestamp, level, and message separately.

Show solution ideas

1. re.search(r"\d+", text).
2. re.findall(r"\d+", text).
3. Use groups for area and local number according to your chosen format.
4. re.findall(r"[\w.-]+@[\w.-]+", text) is only a simple extractor.
5. re.split(r"[;,]", text).
6. re.sub(r"\s+", " ", text).strip().
7. re.fullmatch(pattern, text) is often clearer for full validation.
8. Read with match.group("name").
9. Keep a comment beside non-obvious patterns.
10. Use named groups and return a dictionary.

Homework

Build a log analyzer that extracts levels and counts errors by date.

Checkpoint

Explain the difference between search, match, full match, findall, and substitution.