8  Troubleshooting & performance

8.1 Working with results

Extracting the best match per input

best <- results[match_rank == 1]

Filtering by confidence

# Only high-confidence matches for automated processing
high_conf <- results[match_rank == 1 & total_score >= 80]

# Flag low-confidence matches for manual review instead of dropping them
results[, needs_review := total_score < 70]

Finding inputs that didn’t match

gnaf_match() keeps every input in its output — even unmatched ones — with NA score/address columns and a match_status explaining why:

library(gnafr)
con <- gnaf_connect(":memory:")
gnaf_init(con)
gnaf_add(con, data.table::data.table(
  address_label = "10 MUSGRAVE ROAD, RED HILL QLD 4059",
  number_first = 10L, street_name = "MUSGRAVE", street_type = "ROAD",
  locality_name = "RED HILL", state = "QLD", postcode = 4059L
))

results <- gnaf_match(
  c("10 Musgrave Road, Red Hill QLD 4059", "1 Nowhere Street, Fakesuburb QLD 9999"),
  con, verbose = FALSE
)
results[, .(input_raw, matched, match_status, total_score)]

                               input_raw matched match_status total_score
                                  <char>  <lgcl>       <char>       <int>
1:   10 Musgrave Road, Red Hill QLD 4059    TRUE      matched         100
2: 1 Nowhere Street, Fakesuburb QLD 9999   FALSE no_candidate          NA

match_status takes one of four values:

Value	Meaning
matched	At least one candidate scored at or above min_score.
below_min_score	Candidates existed but none cleared min_score — try lowering it, or inspect with min_score = 0 to see what was found.
insufficient_parse	The parser couldn’t extract a street name, or found none of postcode/state/locality to search by. Check with address_parse().
no_candidate	Parsing succeeded but no database row satisfied even the coarse pre-filters (most commonly: the street number genuinely isn’t in the database — see the number filter).

Joining match results back to your input data

dt_in[, input_id := .I]

geo <- results[match_rank == 1, .(input_id, total_score, longitude, latitude,
                                   address_label, address_detail_pid)]

dt_out <- geo[dt_in, on = "input_id"]

Diagnosing a specific low score

parsed <- address_parse("my problem address")
print(parsed)

result <- gnaf_match("my problem address", con, max_results = 5, min_score = 0)
print(result[, .(match_rank, total_score, score_postcode, score_suburb,
                  score_street_name, score_number, address_label)])

Common causes of an unexpectedly low (or missing) score, in roughly descending order of how often they come up:

• Street number not in the database for that street. Returns no_candidate, not a low score — see the callout in Scoring. Confirm with address_parse() that the number parsed as expected, then check DBI::dbGetQuery(con, "SELECT COUNT(*) FROM gnaf_addresses WHERE postcode = ... AND street_name = '...'").

• Postcode not loaded. If you loaded one state’s extract and the address is from another, the postcode genuinely isn’t in the database:

  DBI::dbGetQuery(con, "SELECT COUNT(*) FROM gnaf_addresses WHERE postcode = 4001")

  A count of 0 means the postcode isn’t in your loaded data, not a matching bug.

• Suburb spelling diverges sharply ("MT GRAVATT" vs "MOUNT GRAVATT EAST"). Jaro-Winkler handles small differences and the abbreviations normalize = TRUE expands, but not radical abbreviations or missing qualifying words.

• Street type not recognised. If score_street_type is consistently the 40%-of-weight “disagreement” tier instead of full credit, the input’s abbreviation may not be in the lookup table:

  data.table::fread(system.file("extdata", "street_types.csv", package = "gnafr"))

  Add the missing abbreviation/canonical pair if it’s genuinely absent.

• A long building name is confusing the parser. Building names before the flat/number can occasionally prevent number extraction — check with address_parse() whether in_number_first actually came through.

8.2 Performance

Typical throughput

Input size	Estimated time
1,000	< 1 second
10,000	2–5 seconds
100,000	15–45 seconds
500,000	2–5 minutes (chunking recommended)

Times assume a laptop with an SSD and a few million G-NAF rows loaded for the relevant state(s). Results vary with CPU, how concentrated the input postcodes are, and the proportion of addresses with no parseable postcode.

What actually drives runtime

• Parsing. For inputs that miss every vectorised fast pattern (Section 3.1), address_parse() falls back to a row-by-row parser, which is the dominant cost for large batches of unusual input. It parallelises trivially since each row is independent:

  library(parallel)
  n_cores <- detectCores() - 1L
  chunks  <- split(addresses, cut(seq_along(addresses), n_cores, labels = FALSE))
  parsed_list <- mclapply(chunks, address_parse, mc.cores = n_cores)
  parsed <- data.table::rbindlist(parsed_list)

• Postcode concentration. If a huge share of input addresses share one postcode, the candidate set for that postcode’s join gets large — the (postcode, number_first) index keeps the join itself cheap, but a very postcode-concentrated batch with a wide spread of house numbers still touches more rows than a postcode-diverse one.

• The locality fallback firing often. It only runs for inputs scoring at or below fallback_threshold (default 90), but if a large fraction of your input has bad postcodes, you’re paying for two passes (postcode path, then locality fallback) instead of one. Worth checking gnaf_cache_history() / a sample of match_status values if a batch feels slow — a high proportion of below_min_score or unmatched rows usually means the locality fallback is doing a lot of work for not much payoff, and the input data quality is the actual problem.

• Re-establishing the connection per call. Reuse a single con across many gnaf_match() calls — for Shiny apps or API services, keep it in a global or module-level variable rather than reconnecting per request.

Chunking very large batches

For inputs exceeding ~500k rows or spanning many postcodes, splitting into chunks caps peak memory:

chunk_size <- 50000L
ids <- seq_len(nrow(dt_in))
chunks <- split(ids, ceiling(ids / chunk_size))

results_list <- lapply(chunks, function(idx) {
  gnaf_match(dt_in$address_string[idx], con, max_results = 1, min_score = 60)
})

results <- data.table::rbindlist(results_list)

The match cache makes a second run over previously-seen input close to free, regardless of chunking — high-confidence results from any earlier chunk (or earlier run) are served from gnaf_match_cache rather than re-matched.

8.3 Common errors

“File not found” from gnaf_load()

Use forward slashes or doubled backslashes — a single backslash is an escape character in an R string literal:

gnaf_load(con, "C:/temp/gnaf.qld.csv")      # OK
gnaf_load(con, "C:\\temp\\gnaf.qld.csv")    # also OK
gnaf_load(con, "C:\temp\gnaf.qld.csv")      # wrong — \t is a tab escape

DuckDB version errors after upgrading the duckdb package

DuckDB database files are tied to the format version of the engine that created them. If you see an incompatible-version error after upgrading the duckdb R package, the database file needs rebuilding from source data — there’s no in-place upgrade path:

file.remove("C:/temp/gnaf.duckdb")
con <- gnaf_connect("C:/temp/gnaf.duckdb")
gnaf_init(con)
gnaf_load(con, "C:/temp/gnaf.qld.csv")

gnaf_cache_clear() / gnaf_cache_rollback() error in a script

Both prompt for confirmation by default (ask = TRUE), which requires an interactive session. Pass ask = FALSE for scripted or scheduled use — you’re opting out of the safety prompt deliberately, so make sure the call is scoped correctly first (see the cache walkthrough).

Weights error: 'weights' must sum to 100

gnaf_match(..., weights = list(...)) validates that every required component is present, numeric, non-negative, and sums to exactly 100 (within floating-point tolerance). The required names are postcode, suburb, street_name, street_type, number, flat — see current default weights for a sensible starting point to adjust from.