The same query, run through every chunking strategy right now.
Top-k is the protocol everyone uses; it is confounded, because a strategy
emitting more, smaller chunks gets more shots at the same passage
(corr with chunks/doc = −0.820). Equal budget gives every strategy the
same 400 words of context — correlation drops to −0.152 and the ranking
inverts.
Every token-level metric in QA runs text through
re.sub(r"[^\w\s]", " ", s). In Python, \w
is str.isalnum(), which is False for Unicode
Mn/Mc — the combining marks carrying the vowels in every
Brahmic script. Paste any Indic sentence and watch it shatter.