pub fn tokenize(text: &str) -> Vec<String>Expand description
Lowercase alphanumeric terms, splitting snake_case, kebab-case,
dotted paths and camelCase. No stemming: experiment vocabulary is
identifiers and acronyms, which stemmers only damage.
Two camel boundaries, both needed by real filter names: aB
(valF1 → val f1) and ABc, the end of an acronym (MoSHead →
mo s head).