Skip to main content

Interface: FoldedText

Defined in: packages/agentos/src/cognition/library/tokens.ts:151

A text folded for matching, with the source interval of each of its UTF-16 units.

Properties​

map​

map: readonly readonly [number, number][]

Defined in: packages/agentos/src/cognition/library/tokens.ts:163

For each UTF-16 unit of text, the half-open interval [start, end) of the source, in UTF-16 units, it came from. The units one source character yields share its interval (a Hangul syllable yields its three jamo, a character beyond the Basic Multilingual Plane its two units), and a combining mark dropped after a character widens that character's interval over it, so a match placed through the map never splits a letter from its marks.

text​

text: string

Defined in: packages/agentos/src/cognition/library/tokens.ts:156

The folded text: each character decomposed (NFD), its combining marks dropped and a curly apostrophe made straight, then the whole lower-cased, so a capital sigma takes its final form by its neighbours, as toLowerCase gives it.