RegularExpressionPrimer < Archive < Vanderbilt Biostatistics Wiki

You are here: Vanderbilt Biostatistics Wiki>Archive Web>RegularExpressionPrimer (18 Dec 2015, JeremyStephens) (raw view)EditAttach
---+ Regular Expression Primer

%TOC%

%SLIDESHOWSTART{template="BiostatSlideTemplate"}%
---++ Regular expressions
<div style="text-align: center">
%ATTACHURL%/email-regexp.png
<br/>
[[Jeremy Stephens]] - Thursday, September 29, 2011
</div>
---+++ Comments
   1 Text-version: <code>^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$</code>
---++ What are they?
<div style="text-align: center">
Regular expressions are a way to describe patterns in text.
</div>
---+++ Comments
   1 The regular expression shown in the previous slide is for matching valid e-mail addresses. Hopefully by the end of the talk, what this does will be more clear.
---++ When to use them
   * Finding things
   * Replacing things: <code>c("123", "123x", "abc123")</code>
   * Extracting data: <br/> <code>"Tom Brady passed for 5 touchdowns and 456 yards, but they lost anyway."</code>

---++ Barriers to using regular expressions
<div style="text-align: center">
%ATTACHURL%/scary-regexp.jpg
<br/>
<span style="font-size: x-small">Photo: [[http://commons.wikimedia.org/wiki/File:Halloween.JPG][Carole Pasquier]]; Regular expression: [[CharlesDupont][Thomas Dupont]]</span>
</div>
---+++ Comments
   1 They're scary! The above regular expression was written by Thomas in order to parse court case documents.
   1 Some good advice...
---++ Advice
<div style="text-align: center;">
%ATTACHURL%/Dont_panic.jpg
<br/>
<span style="font-size: x-small">GFX credit: <a href="http://commons.wikimedia.org/wiki/File:Dont_panic.jpg">Seddon</a></span>
</div>
---+++ Comments
   1 All regular expressions are made up of the same building blocks.
---++ Things you (might) need to know
There are 3 main regular expression standards:

   * POSIX Basic Regular Expressions
   * POSIX Extended Regular Expressions (R uses this by default)
   * Perl-based Regular Expressions (R has support for this as well)
---+++ Comments
   1 Different programming languages support different standards. R supports ERE by default, but it also has support for Perl-based RE. (Using perl=TRUE)
   1 There is a caveat when using regular expressions in R, but that will appear later on in this talk.
   1 If you use regular expressions in a language and find that it doesn't behave as you expect, it may be because the language doesn't support certain features.
   1 Examples in this talk will be using R for demonstration.
   1 See http://en.wikipedia.org/wiki/Regular_expression#Syntax for more information
---++ The simplest regular expression
   * Plain text!<br/><pre>
> grep("test", c("foo", "bar", "test"))
[1] 3
</pre>
   * "test" is a perfectly valid regular expression
   * Works just like CTRL+F in your browser

<div style="text-align: center">
%ATTACHURL%/browser-find.png
</div>
---+++ Comments
   1 The R function =grep= can be used to find matches in a character vector. It returns the indices of matching strings
   1 You may have already used regular expressions without your knowledge!
   1 "test" is a pattern, but not a very complicated one. How do you describe more complex patterns?

---++ Metacharacters
   * Characters that have special meaning
   * Meta, meaning "about"
---+++ Comments
   1 Characters with special meaning in regular expressions are called "metacharacters". We add "meta", which means "about", because we're describing information about other characters.

---++ Your first metacharacter: Dot
   * Dot (or period) matches any character<br/><pre>
> grep("foo.bar", c("fooxbar", "foo bar", "foobar", "foo12bar"))
[1] 1 2
</pre>
   * "foo.bar" is the regular expression
---+++ Comments
   1 You'll notice that *dot* matches a space
   1 What happens when you want to match two wildcard characters?
---++ Matching two wildcards
<pre style="line-height: 1.5em">
> grep("foo..bar", c("fooxbar", "foo bar", "foobar", "foo12bar"))
[1] 4
</pre>
---+++ Comments
   1 But what if you want to match three wildcard characters? There's an easier way.

---++ Variable length matching, using "plus"
<pre style="line-height: 1.5em">
> grep("foo.+bar", c("fooxbar", "foo bar", "foobar", "foo12bar"))
[1] 1 2 4
</pre>
   * "foo.+bar" is the regular expression
   * The *plus* metacharacter means: "one or more times"
---+++ Comments
   1 You can use the plus metacharacter on other characters, too.

---++ More "plus" usage
<pre style="line-height: 1.5em">
> grep("bar+", c("bar", "barr", "barrrrrrrr", "ba"))
[1] 1 2 3
</pre>

---++ Quantifiers
   * Metacharacters that describe "how many"
   * The "plus" metacharacter is a quantifier
---+++ Comments
   1 There are a couple of other quantifiers you can use

---++ Star metacharacter: "Zero or more times"
<pre style="line-height: 1.5em">
> grep("foo.*bar", c("fooxbar", "foo bar", "foobar", "foo12bar"))
[1] 1 2 3 4
</pre>

<pre style="line-height: 1.5em">
> grep("bar*", c("bar", "barr", "barrrr", "ba"))
[1] 1 2 3 4
</pre>
---+++ Comments
   1 You'll notice that both "foobar" and "ba" match now, because we're matching zero or more times instead of one or more times.

---++ Question mark metacharacter: "Zero or one times"
<pre style="line-height: 1.5em">
> grep("abc?def", c("abdef", "abcdef", "abccdef"))
[1] 1 2
</pre>
---+++ Comments
   1 The first two strings match, but the third one doesn't. If the question mark had been a star, all three strings would match.

---++ Brace metacharacter: "Exactly n times"
<pre style="line-height: 1.5em">
> grep("10{6}", c("1000", "1000000"))
[1] 2
</pre>

---++ Brace metacharacter improved
<pre style="line-height: 1.5em">
> grep("10{2,4}1", c("101", "1001", "10001", "100001", "1000001"))
[1] 2 3 4
</pre>
   * <code>{n, m}</code> - Match at least *n* times, but not more than *m* times

---++ Intermission
Questions?

---++ Anchors!
<pre style="line-height: 1.5em">
> grep("10{3}", c("100", "1000", "10000"))
[1] 2 3
</pre>
   * The third string matches!
---+++ Comments
   1 The third string matches because this regular expression is "floating". It matches *anywhere* in the string.
   1 Solution: use an anchor.

---++ Anchor to the end with a dollar sign
   * Dollar sign metacharacter:<br/><pre>
> grep("10{3}$", c("100", "1000", "10000"))
[1] 2
</pre>
---+++ Comments
   1 The dollar sign "anchors" the regular expression to the end of the string. Now, the third string doesn't match.
   1 Anchor metacharacters are special because they only specify string positions.

---++ Anchor to the beginning with a caret
<pre style="line-height: 1.5em">
> grep("10{3}", c("1000", "abc1000"))
[1] 1 2
</pre>
   * Caret metacharacter: <br/><pre>
> grep("^10{3}", c("1000", "abc1000"))
[1] 1
</pre>
---+++ Comments
   1 The caret "anchors" the regular expression to the beginning of the string.

---++ Using both anchors
<pre style="line-height: 1.5em">
> grep("10{3}", c("abc1000", "1000", "10000"))
[1] 1 2 3
</pre>

<pre style="line-height: 1.5em">
> grep("10{3}$", c("abc1000", "1000", "10000"))
[1] 1 2
</pre>

<pre style="line-height: 1.5em">
> grep("^10{3}$", c("abc1000", "1000", "10000"))
[1] 2
</pre>

---++ Character classes with brackets
   * In case you don't want to match *any* character by using dot: <br/><pre>
> grep("ab[cdef]", c("abc", "abd", "abe", "abf", "abg")) 
[1] 1 2 3 4
</pre>
   * Bracket metacharacter

---++ Character class ranges
   * Instead of typing out all of those letters: <br/><pre>
> grep("ab[c-f]", c("abc", "abd", "abe", "abf", "abg")) 
[1] 1 2 3 4
</pre>
   * Dash metacharacter

---++ Multiple ranges
<pre style="line-height: 1.5em">
> grep("ab[c-fC-F]", c("abc", "abC", "abf", "abF", "abg")) 
[1] 1 2 3 4
</pre>

---++ Mixing ranges and characters
<pre style="line-height: 1.5em">
> grep("ab[c-fC-F123]", c("abc", "abF", "ab1", "ab2")) 
[1] 1 2 3 4
</pre>

---++ Context matters
   * Because regular expressions aren't hard enough
   * Some metacharacters mean different things depending on *context*

<img src="%ATTACHURL%/office-sign-brain.jpg" width="300" height="389" />

---++ Context in character classes
   * Most metacharacters lose their meaning inside brackets: <br/><pre>
> grep("[.+*]", c(".", "+", "*", "x"))
[1] 1 2 3
> grep("[a-c-]", c("a", "b", "c", "-"))
[1] 1 2 3 4
</pre>

---++ Full circle
<div style="text-align: center">
%ATTACHURL%/email-regexp.png
</div>
---+++ Comments
   1 Now, hopefully this is less scary.
   1 Let's go through this step by step.

---++ Better e-mail regular expression
<div style="text-align: center">
%ATTACHURL%/email-regexp-2.png
</div>
---+++ Comments
   1 Complete with Titans colors!
   1 <code>^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}$</code>

%SLIDESHOWEND%

-- Main.JeremyStephens - 26 Sep 2011
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
jpg	Dont_panic.jpg	manage	635.7 K	28 Sep 2011 - 15:58	JeremyStephens	Don't panic! (From Wikimedia Commons)
png	browser-find.png	manage	6.5 K	28 Sep 2011 - 15:32	JeremyStephens	Find from Firefox
png	email-regexp-2.png	manage	50.0 K	29 Sep 2011 - 12:16	JeremyStephens	Better e-mail regular expression
png	email-regexp.png	manage	45.3 K	26 Sep 2011 - 17:31	JeremyStephens	E-mail regular expression
jpg	office-sign-brain.jpg	manage	70.7 K	29 Sep 2011 - 11:12	JeremyStephens	Via http://www.happyworker.com/magazine/fun/april-fools-day-in-the-stressed-office
jpg	scary-regexp.jpg	manage	409.1 K	26 Sep 2011 - 17:31	JeremyStephens	Scary regular expression
Topic revision: r7 - 18 Dec 2015, JeremyStephens