Python Regex Replace: How to Replace Strings Using re Module

Regular expressions, or regex, are a powerful tool in Python for working with text data. They enable you to search, match, and manipulate strings based on specific patterns. Whether you are cleaning data, validating input, or performing complex string replacements, regex can help you accomplish tasks with efficiency and precision.

In Python, the re module provides a suite of functions to work with regex. Among these, re.sub() is particularly useful for replacing substrings within a text based on a regex pattern. This article will guide you through the essentials of using re.sub() for advanced string replacement, showcasing various techniques and examples.

By mastering regex-based string replacement, you can streamline your text processing tasks, automate repetitive text modifications, and handle complex patterns with ease. Let's dive into the basics before exploring more advanced use cases.
 

Understanding the Basics of re.sub()

The re.sub() function is the primary tool for performing string replacements with regex in Python. It allows you to specify a regex pattern to search for, a replacement string, and the target string where the replacement will occur. The basic syntax of re.sub() is as follows:

import re

result = re.sub(pattern, replacement, string, count=0, flags=0)

• pattern: The regex pattern you want to search for.
• replacement: The string that will replace the matched pattern.
• string: The original string where the replacement occurs.
• count: An optional parameter that specifies the maximum number of replacements (default is 0, meaning all occurrences).
• flags: Optional flags that modify the behavior of the regex engine (e.g., re.IGNORECASE for case-insensitive matching).

Here’s a simple example:

text = "Hello, World!"
result = re.sub(r"World", "Universe", text)
print(result)  # Output: Hello, Universe!

In this example, the word "World" is replaced with "Universe" using the regex pattern r"World". This is a straightforward use of re.sub(), but the true power of regex becomes apparent when you start using more complex patterns.

Advanced Pattern Matching with Regex

Regex provides a rich syntax for matching patterns, allowing you to identify and replace complex sequences of characters in a string. Some advanced features include:

• Character Classes: Match specific sets of characters, e.g., [0-9] matches any digit, [A-Za-z] matches any letter.
• Quantifiers: Specify how many times a character or group should be matched, e.g., * (zero or more), + (one or more), {n} (exactly n times).
• Groups and Backreferences: Capture parts of the matched pattern and reuse them, e.g., (abc)\1 matches "abcabc".
• Assertions: Conditions that must be true for the match, but do not consume characters, e.g., \b for word boundaries.

Let’s explore these features with an example:

text = "Contact me at [email protected] or [email protected]."
pattern = r"\b[\w.-]+@[\w.-]+\.\w+\b"
replacement = "[email]"

result = re.sub(pattern, replacement, text)
print(result)  # Output: Contact me at [email] or [email].

Here, the regex pattern matches email addresses in the text. The pattern \b[\w.-]+@[\w.-]+\.\w+\b breaks down as follows:

• \b: Word boundary to ensure we're matching whole email addresses.
• [\w.-]+: Matches the username part (letters, digits, underscores, dots, hyphens).
• @: Matches the "@" symbol.
• [\w.-]+: Matches the domain part.
• \.\w+: Matches the top-level domain (e.g., ".com").
• \b: Another word boundary.

The matched emails are then replaced with "[email]", demonstrating how regex can be used for advanced pattern matching and replacement.

Using Functions for Dynamic Replacements

One of the powerful features of re.sub() is the ability to pass a function as the replacement argument. This allows you to perform dynamic replacements based on the match content. The function receives a match object and can return a customized replacement string.

Here’s an example where we replace all numbers in a string with their squared values:

text = "The numbers are 2, 4, and 6."

def square(match):
    num = int(match.group(0))
    return str(num ** 2)

result = re.sub(r"\d+", square, text)
print(result)  # Output: The numbers are 4, 16, and 36.

In this example, the regex r"\d+" matches any sequence of digits. The square() function is called for each match, converting the matched number to an integer, squaring it, and returning the result as a string. This technique is particularly useful when the replacement value depends on the matched content.

Handling Case Sensitivity and Multiline Strings

By default, regex in Python is case-sensitive. If you want to perform case-insensitive replacements, you can use the re.IGNORECASE flag. Additionally, regex can work across multiple lines if the string contains line breaks. The re.MULTILINE flag allows you to match patterns at the start or end of each line within the string.

Consider the following example:

text = """Hello World!
hello world!
HELLO WORLD!"""

pattern = r"hello world"
replacement = "hi universe"

result = re.sub(pattern, replacement, text, flags=re.IGNORECASE)
print(result)

Output:

Copy code

hi universe!
hi universe!
hi universe!

Here, the re.IGNORECASE flag ensures that all variations of "hello world" (regardless of case) are replaced with "hi universe".

For multiline strings, consider using re.MULTILINE to match patterns at the beginning or end of each line:

text = """First line
Second line
Third line"""

pattern = r"^\w+"
replacement = "Line"

result = re.sub(pattern, replacement, text, flags=re.MULTILINE)
print(result)

Output:

Line line
Line line
Line line

This example demonstrates how the re.MULTILINE flag allows the ^ anchor to match the start of each line, replacing the first word on every line with "Line".

Using Backreferences for Complex Replacements

Backreferences in regex allow you to refer back to captured groups in your pattern, which can be useful for more sophisticated replacements. Backreferences are denoted by \1, \2, etc., corresponding to the order of the capturing groups.

Here’s an example where we swap the first and last name in a string:

text = "Doe, John"
pattern = r"(\w+), (\w+)"
replacement = r"\2 \1"

result = re.sub(pattern, replacement, text)
print(result)  # Output: John Doe

In this example, the pattern r"(\w+), (\w+)" captures the last name and first name separately. The replacement string r"\2 \1" refers to these captured groups, swapping their order.

This technique is especially useful for reformatting text data or performing complex text transformations where the new format depends on the structure of the original text.

Conclusion

Regex in Python is a really good tool for advanced string replacement tasks. By leveraging the power of re.sub(), you can efficiently handle complex patterns, dynamic replacements, and multi-line strings. Whether you’re working with simple substitutions or intricate text manipulations, mastering regex will enable you to process and transform text data with precision.

Understanding how to use regex effectively requires practice and familiarity with its syntax. Start with basic patterns and gradually explore more advanced features like groups, backreferences, and flags. With time, you’ll find regex to be an indispensable tool in your Python programming toolkit, capable of simplifying even the most challenging text processing tasks.

Join Index.dev, the remote work platform connecting senior Python developers with remote tech companies. Begin working remotely on innovative projects across the US, UK, and EU!