library(fast.string)2 Quickstart
2.1 Installation
remotes::install_github("KyleHaynes/fast.string")This compiles C++ source on install, so you need a working toolchain: Rtools on Windows, Xcode command-line tools on macOS, or build-essential + libpcre2-dev on Linux. See Chapter 7 if the install fails.
Loading the package prints a one-time startup banner listing every exported function; suppress it with options(fast.string.verbose = FALSE) set before library(), or with suppressPackageStartupMessages().
The rest of this chapter is a guided tour through every function group, using small inputs so it renders fast — Chapter 6 is where the same functions get pushed to 3 million rows.
2.2 Matching & substitution
fgrepl(), fgrep(), fsub(), and fgsub() are argument-compatible with base::grepl()/grep()/sub()/gsub() — same defaults, same NA handling — just parallelised.
[1] TRUE TRUE FALSE NAfgrep("^invoice", x, value = TRUE)[1] "invoice-2024-001" "invoice-2024-002"fsub("-([0-9]+)$", "_\\1", x)[1] "invoice-2024_001" "invoice-2024_002" "credit-note_001" NA fgsub("[0-9]", "#", x)[1] "invoice-####-###" "invoice-####-###" "credit-note-###" NA fcount() counts non-overlapping matches per element — the counting counterpart to fgrepl(), replacing the vapply(gregexpr(...), length, ...) idiom, which allocates a vector of match positions per row only to take its length.
fcount("[0-9]", x)[1] 7 7 3 NAfcount("-", x, fixed = TRUE)[1] 2 2 2 NA# Column-level quality rules in one call each
names_in <- c("JOHN O'BRIEN", "UNKNOWN 00000000", "Kyle John Haynes")
fcount("[0-9]", names_in) # digits in a name field: placeholder rows[1] 0 8 0fcount(" +", names_in) + 1L # token count[1] 2 2 3gsub_all() has no base R equivalent: it applies several patterns in one pass, either sequentially (chained, like calling gsub() repeatedly) or as a single combined left-to-right scan where the first matching pattern at each position wins.
2.3 String utilities
ftrimws(), fsubstr(), fnchar(), and fchartr() mirror their base equivalents exactly, including names(x) preservation.
[1] "John Smith" "Mary Jones" NA [1] "NSW" "VIC"fsubstr(records, 4, 6) # category code[1] "REC" "POL"[1] 4 NA 0[1] "O Brien" "Smith Jones"2.4 Dates
format_date()/fas.Date() trade base::as.Date()’s locale-aware, multi-format flexibility for speed on one fixed format at a time. date_parts() decomposes a Date vector into year/month/day columns in a single pass — useful for building blocking keys.
dates <- as.Date(c("2024-06-18", "1999-12-31", NA))
format_date(dates, "compact")[1] "20240618" "19991231" NA date_parts(dates) year month day
1 2024 6 18
2 1999 12 31
3 NA NA NA[1] "2024-06-18" "1999-12-31"fas.POSIXct() and format_datetime() are the timestamp equivalents, covering four fixed shapes (iso, rfc3339, compact, iso_offset). Everything is handled as UTC seconds since the epoch — no timezone-database lookup — and "iso_offset" writes one fixed numeric offset rather than a real timezone (no daylight-saving rules).
stamps <- fas.POSIXct(c("2024-06-18 09:15:00", "2024-06-18 23:59:59", "not a time"))
stamps[1] "2024-06-18 09:15:00 UTC" "2024-06-18 23:59:59 UTC" NA format_datetime(stamps, "rfc3339")[1] "2024-06-18T09:15:00Z" "2024-06-18T23:59:59Z" NA format_datetime(stamps, "compact")[1] "20240618091500" "20240618235959" NA format_datetime(stamps, "iso_offset", offset = "+10:00")[1] "2024-06-18T19:15:00+10:00" "2024-06-19T09:59:59+10:00" NA Unlike fas.Date(), which validates field ranges only, fas.POSIXct() validates the whole calendar — month lengths and leap years included:
fas.Date("29/02/2023", "dmy") # rolls over -- 2023 is not a leap year[1] "2023-03-01"fas.POSIXct("2023-02-29 08:00:00") # NA[1] NA2.5 Phonetic blocking keys
soundex(), nysiis(), refined_soundex(), cologne(), double_metaphone(), and caverphone() have no base R equivalent. They’re typically used to group records that sound alike before running a more expensive pairwise comparison only within each group, rather than comparing every record against every other record. Different rulesets group different near-miss spellings together, so trying more than one can catch matches a single phonetic key misses.
names_x <- c("Robert", "Rupert", "Ashcraft", "Ashcroft", "Smith", "Smyth")
data.frame(
name = names_x,
soundex = soundex(names_x),
nysiis = nysiis(names_x),
refined = refined_soundex(names_x),
caverphone = caverphone(names_x)
) name soundex nysiis refined caverphone
1 Robert R163 RABAD R901096 RPT1111111
2 Rupert R163 RAPAD R901096 RPT1111111
3 Ashcraft A261 ASCRAF A03039026 ASKRFT1111
4 Ashcroft A261 ASCRAF A03039026 ASKRFT1111
5 Smith S530 SNAT S38060 SMT1111111
6 Smyth S530 SNYT S38060 SMT1111111double_metaphone(c("Smith", "Schmidt", "Catherine", "Kathryn")) primary secondary
1 SM0 XMT
2 XMT SMT
3 K0RN KTRN
4 K0RN KTRN"Ashcraft"/"Ashcroft" sharing a Soundex code (A261) despite differing spelling is the textbook example of why Soundex exists — and a known point of disagreement between implementations of Soundex, covered in Chapter 3.
double_metaphone() returns two codes per word — primary and secondary — because some spellings have more than one plausible pronunciation (a C that could be a hard K or a soft S sound, for instance); two names are usually considered a phonetic match if any of one side’s codes matches any of the other’s. "Catherine" and "Kathryn" sharing the code K0RN despite no letters lining up is the kind of match Soundex/NYSIIS — designed around English/American surnames — typically miss.
The two newer schemes sit at opposite ends of the precision/recall trade-off. refined_soundex() encodes vowels instead of discarding them, so it produces a longer, more discriminating code — smaller blocks, fewer false candidates. cologne() targets German pronunciation and folds umlauts to their base vowels, so spelling variants that ASCII-only schemes split are grouped together:
german <- c("Müller", "Mueller", "Miller", "Meier", "Meyer", "Schmidt", "Schmitt")
data.frame(
name = german,
soundex = soundex(german),
refined = refined_soundex(german),
cologne = cologne(german)
) name soundex refined cologne
1 Müller M460 M8709 657
2 Mueller M460 M80709 657
3 Miller M460 M80709 657
4 Meier M600 M809 67
5 Meyer M600 M809 67
6 Schmidt S530 S30806 862
7 Schmitt S530 S30806 862refined_soundex() splits "Müller" off from "Mueller"/"Miller", because the umlaut isn’t an ASCII vowel it can encode; cologne() folds it and puts all three in one block. Neither is “right” — they’re different recall/precision settings, and Chapter 4 uses both at once and takes the union of the candidate blocks.
2.6 Fuzzy string similarity
Three functions, three shapes of comparison:
-
jaro_winkler(a, b)— pairwise, equal-length vectors.a[i]vs.b[i]for everyi. -
jaro_winkler_matrix(a, b)— all-pairs. Every element ofaagainst every element ofb, returned as ann × mmatrix. -
jaro_winkler_tokens(a, b)— pairwise like the first, but tolerant of token reordering and punctuation/spacing differences within a token.
jaro_winkler(c("SMITH", "DWAYNE"), c("SMYTH", "DUANE"))[1] 0.8933333 0.8400000candidates_a <- c("John Smith", "Mary Jones")
candidates_b <- c("Jon Smith", "Mary-Anne Jones", "Someone Else")
jaro_winkler_matrix(candidates_a, candidates_b) [,1] [,2] [,3]
[1,] 0.9733333 0.4444444 0.4722222
[2,] 0.4037037 0.8833333 0.5388889# Token reordering — two of three tokens match exactly, the third is close
jaro_winkler_tokens("Kyle John Haynes", "John Kylie Haynes")[1] 0.9844444# Punctuation/spacing variants of the same name
jaro_winkler_tokens(c("OBrien", "OBrien"), c("O Brien", "O'Brien"))[1] 1 1The full mechanics of how jaro_winkler_tokens() resolves reordered tokens — and the extra_penalty argument for handling a stray extra token — are in Chapter 3 and the function reference (Chapter 5).
2.7 Edit distance
levenshtein(), osa_distance(), damerau_levenshtein(), and hamming() cover plain edits, restricted adjacent transpositions, unrestricted Damerau-Levenshtein, and position-wise differences. Existing APIs compare bytes by default; set use_bytes = FALSE for UTF-8 code points.
levenshtein("kitten", "sitting")[1] 3osa_distance("ca", "abc") # 3 under the OSA restriction[1] 3damerau_levenshtein("ca", "abc") # 2 under unrestricted DL[1] 2levenshtein_similarity("kitten", "sitting")[1] 0.5714286levenshtein_within("kitten", "sitting", max_distance = 3)[1] TRUEhamming("karolin", "kathrin")[1] 3hamming("abc", "ab") # Inf for unequal length, matching stringdist[1] Inf2.8 Fuzzy lookup without a full matrix
fuzzy_match() and fuzzy_top_n() scan a candidate table while retaining only the requested matches. They default to Unicode code-point comparison and avoid the potentially enormous all-pairs matrix.
2.9 Q-gram similarity
jaccard_index(), dice_coefficient(), tversky_index(), and cosine_similarity() compare two strings’ overlapping length-q substrings (“q-grams”) rather than their character-by-character alignment — insensitive to where a difference occurs, which makes them a useful second signal alongside Jaro-Winkler/edit-distance.
jaccard_index("night", "nacht")[1] 0.1428571dice_coefficient("night", "nacht") # weights shared q-grams more heavily[1] 0.25tversky_index("night", "nacht", alpha = 1, beta = 1) # == jaccard_index()[1] 0.1428571tversky_index() generalises the first two: alpha = beta = 1 is Jaccard, alpha = beta = 0.5 is Dice, and asymmetric weights are useful when one side is a query and the other a reference and the two kinds of mismatch shouldn’t be penalised equally.
cosine_similarity() is the one that isn’t a set metric. The other three reduce each string to a set of distinct q-grams, so a repeated q-gram counts once; cosine keeps the frequency profile and compares it as a vector, so repetition carries weight:
a <- c("aaaa", "BANANA", "night")
b <- c("aaab", "BANANAS", "nacht")
data.frame(a, b,
jaccard = jaccard_index(a, b),
dice = dice_coefficient(a, b),
cosine = cosine_similarity(a, b)) a b jaccard dice cosine
1 aaaa aaab 0.5000000 0.6666667 0.8944272
2 BANANA BANANAS 0.7500000 0.8571429 0.9486833
3 night nacht 0.1428571 0.2500000 0.2500000"aaaa"/"aaab" share only the bigram "aa" out of two distinct bigrams — Jaccard 0.5 — but that bigram occurs three times on one side and twice on the other, which cosine sees and Jaccard cannot. Repetition like this is common in the numeric and code-like fields ("0000", "AAA-111") that sit next to names in a linkage file.
2.10 fuzzywuzzy-style ratios
fuzz_ratio(), fuzz_partial_ratio(), fuzz_token_sort_ratio(), and fuzz_token_set_ratio() are vectorised ports of Python’s fuzzywuzzy package — useful if you’re porting a matching pipeline from Python, or just want its specific flavour of fuzzy scoring (Ratcliff/Obershelp matching blocks, not edit distance). All four return 0-100 like the originals.
fuzz_ratio("this is a test", "this is a test!")[1] 100fuzz_partial_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")[1] 91fuzz_token_sort_ratio("fuzzy was a bear", "bear was a fuzzy")[1] 100fuzz_token_set_ratio("fuzzy was a bear", "fuzzy fuzzy bear was a bear")[1] 1002.11 Putting it together
A typical pipeline profiles the input with fcount(), parses dates and load timestamps with fas.Date()/fas.POSIXct(), normalises with fchartr()/ftrimws(), builds blocking keys with refined_soundex()/cologne()/format_date(), scores candidates within each block with jaro_winkler_tokens() and cosine_similarity(), and stamps the result with format_datetime(). Chapter 4 walks through exactly that, end to end, on a small linkage example.