Instance Methodgrdb 7.3.0GRDB

    tokenize(document:)

    Tokenizes the string argument as a document that would be inserted into an FTS5 table.

    FTS5Tokenizer.swift:116
    func tokenize(document string: String) throws -> [(token: String, flags: FTS5TokenFlags)]
    Mangled symbol
    s4GRDB13FTS5TokenizerPAAE8tokenize8documentSaySS5token_AA0B10TokenFlagsV5flagstGSS_tKF What are these?
    FNV24 hash
    7RL49

    Parameters

    string

    The string to tokenize.

    Returns

    An array of tokens and flags.

    Throws

    An error if tokenization fails.

    For example:

    let tokenizer = try db.makeTokenizer(.ascii())
try tokenizer.tokenize(document: "foo bar") // [("foo", flags), ("bar", flags)]

    See also tokenize(query:).