Skip to Content
  • Website
Codsen
  • Home
  • Open Source
  • Articles
  • About

backOpen Source→extract-search-index→Examples

Permalink to Table of ContentsTable of Contents

  • Quick Take
  • Keep the first occurrence of each case-insensitive word
  • Parse attribute boundaries before decoding retained text
  • Strip HTML tags and decode entities
  • Normalise fancy punctuation and omit digits and astral characters
  • Exclude code, preformatted text, and CSS from the index
  • Remove URLs and stop words from search-index text
  • Preserve BMP letters while separating words around surrogate code units
↑ back to top

Permalink to Quick TakeQuick Take

↑ back to top

Permalink to Keep the first occurrence of each case-insensitive wordKeep the first occurrence of each case-insensitive word

↑ back to top

Permalink to Parse attribute boundaries before decoding retained textParse attribute boundaries before decoding retained text

↑ back to top

Permalink to Strip HTML tags and decode entitiesStrip HTML tags and decode entities

↑ back to top

Permalink to Normalise fancy punctuation and omit digits and astral charactersNormalise fancy punctuation and omit digits and astral characters

↑ back to top

Permalink to Exclude code, preformatted text, and CSS from the indexExclude code, preformatted text, and CSS from the index

↑ back to top

Permalink to Remove URLs and stop words from search-index textRemove URLs and stop words from search-index text

↑ back to top

Permalink to Preserve BMP letters while separating words around surrogate code unitsPreserve BMP letters while separating words around surrogate code units

↑ back to top
back 

Copyright

All rights reserved © Roy Revelt 2026
All our open source packages are under MIT licenceopens in a new tab

Activities

🐛 See a bug? Raise an issueopens in a new tab
💘 Check out the Indiewebopens in a new tab and Libera manifestoopens in a new tab