# Regex Special Characters – Examples in Python Re

Regular expressions are a strange animal. Many students find them difficult to understand – do you?

I realized that a major reason for this is simply that they don’t understand the special regex characters. To put it differently: understand the special characters and everything else in the regex space will come much easier to you.

Related article: Python Regex Superpower – The Ultimate Guide

Do you want to master the regex superpower? Check out my new book The Smartest Way to Learn Regular Expressions in Python with the innovative 3-step approach for active learning: (1) study a book chapter, (2) solve a code puzzle, and (3) watch an educational chapter video.

Regular expressions are built from characters. There are two types of characters: literal characters and special characters.

## Literal Characters

Let’s start with the absolute first thing you need to know with regular expressions: a regular expression (short: regex) searches for a given pattern in a given string.

What’s a pattern? In its most basic form, a pattern can be a literal character. So the literal characters `'a'`, `'b'`, and `'c'` are all valid regex patterns.

For example, you can search for the regex pattern `'a'` in the string `'hello world'` but it won’t find a match. You can also search for the pattern `'a'` in the string `'hello woman'` and there is a match: the second last character in the string.

Based on the simple insight that a literal character is a valid regex pattern, you’ll find that a combination of literal characters is also a valid regex pattern. For example, the regex pattern `'an'` matches the last two characters in the string `'hello woman'`.

Summary: Regular expressions are built from characters. An important class of characters are the literal characters. In principle, you can use all Unicode literal characters in your regex pattern.

## Special Characters

However, the power of regular expressions come from their abstraction capability. Instead of writing the character set `[abcdefghijklmnopqrstuvwxyz]`, you’d write `[a-z]` or even `\w`. The latter is a special regex character—and pros know them by heart. In fact, regex experts seldomly match literal characters. In most cases, they use more advanced constructs or special characters for various reasons such as brevity, expressiveness, or generality.

So what are the special characters you can use in your regex patterns?

Let’s have a look at the following table that contains all special characters in Python’s `re` package for regular expression processing.

But these are not all characters you can use in a regular expression.

There are also meta characters for the regex engine that allow you to do much more powerful stuff.

A good example is the asterisk operator that matches “zero or more” occurrences of the preceding regex. For example, the pattern `.*txt` matches an arbitrary number of arbitrary characters followed by the suffix `'txt'`. This pattern has two special regex meta characters: the dot `.` and the asterisk operator `*`. You’ll now learn about those meta characters:

## Regex Meta Characters

Feel free to watch the short video about the most important regex meta characters:

Next, you’ll get a quick and dirty overview of the most important regex operations and how to use them in Python.

Here are the most important regex operators:

Note that I gave the above operators some more meaningful names (in bold) so that you can immediately grasp the purpose of each regex. For example, the `‘^’` operator is usually denoted as the ‘caret’ operator. Those names are not descriptive so I came up with more kindergarten-like words such as the “start-of-string” operator.

Let’s dive into some examples!

## Examples

```import re

text = '''
Ha! let me see her: out, alas! he's cold:
Her blood is settled, and her joints are stiff;
Life and these lips have long been separated:
Death lies on her like an untimely frost
Upon the sweetest flower of all the field.
'''

print(re.findall('.a!', text))
'''
Finds all occurrences of an arbitrary character that is
followed by the character sequence 'a!'.
['Ha!']
'''

print(re.findall('is.*and', text))
'''
Finds all occurrences of the word 'is',
followed by an arbitrary number of characters
and the word 'and'.
['is settled, and']
'''

print(re.findall('her:?', text))
'''
Finds all occurrences of the word 'her',
followed by zero or one occurrences of the colon ':'.
['her:', 'her', 'her']
'''

print(re.findall('her:+', text))
'''
Finds all occurrences of the word 'her',
followed by one or more occurrences of the colon ':'.
['her:']
'''

print(re.findall('^Ha.*', text))
'''
Finds all occurrences where the string starts with
the character sequence 'Ha', followed by an arbitrary
number of characters except for the new-line character.
Can you figure out why Python doesn't find any?
[]
'''

print(re.findall('\n\$', text))
'''
Finds all occurrences where the new-line character '\n'
occurs at the end of the string.
['\n']
'''

print(re.findall('(Life|Death)', text))
'''
Finds all occurrences of either the word 'Life' or the
word 'Death'.
['Life', 'Death']
'''
```

In these examples, you’ve already seen the special symbol `\n` which denotes the new-line character in Python (and most other languages). There are many special characters, specifically designed for regular expressions.

## Which Special Python Regex Characters Must Be Escaped?

Short answer: Here’s an exhaustive list of all special characters that need to be escaped:

````.`      ---->     `\.`
`*`      ---->     `\*`
`?`      ---->     `\?`
`+`      ---->     `\+`
`^`      ---->     `\^`
`\$`      ---->     `\\$`
`|`      ---->     `\|````

Question: Is there a comprehensive list of which special characters must be escaped in order to remove the special meaning within the regex?

Example: Say you search for those symbols in a given string and you wonder which of them you must escape:

`|^&+-%*/=!>`

Answer: Differentiate between using the special symbols within or outside a character class.

• Within the character class, you need to escape only the minus symbol replacing `[-]` with `[\-]` as this has a special meaning within the character class (the “range” character).
• Outside the character class in a normal regex pattern, you need to escape only the regex chars with special meaning. Here’s an exhaustive list of all special characters that need to be escaped: `.*?+^\$|`
```import re

text = '|^&+-%*/=!>'

# WITHIN CHARACTER CLASS --> ESCAPE '-'
print(re.findall('[|^&+\-%*/=!>]', text))
# ['|', '^', '&', '+', '-', '%', '*', '/', '=', '!', '>']

# WITHOUT CHARACTER CLASS --> ESCAPE ALL SPECIAL CHARS '.*?+^\$|'
pattern = '|^&+\$-%*/=!>'
print(re.findall('\|', text))
print(re.findall('\^', text))
print(re.findall('\\$', text))
print(re.findall('\+', text))
print(re.findall('-', text))
print(re.findall('%', text))
print(re.findall('\*', text))
print(re.findall('/', text))
print(re.findall('=', text))
print(re.findall('!', text))
'''
['|']
['^']
['\$']
['+']
['-']
['%']
['*']
['/']
['=']
['!']
'''```

By escaping the special regex symbols, they lose their special meaning and you can find the symbols in the original text.

## Where to Go From Here

You’ve learned all special characters of regular expressions, as well as meta characters. This will give you a strong basis for improving your regex skills.

If you want to accelerate your skills, you need a good foundation. Check out my brand-new Python book “Python One-Liners (Amazon Link)” which boosts your skills from zero to hero—in a single line of Python code!