codsen-tokenizer^4.3.0

HTML and CSS lexer aimed at code with fatal errors, accepts mixed coding languages

npm i codsen-tokenizer

The default is exported, so instead of tokenizer below, you can name the consumed function however you want.

Consume via a require():

const tokenizer = require("codsen-tokenizer");

or as an ES Module:

import tokenizer from "codsen-tokenizer";

Put this in your HTML:

<script src="https://cdn.jsdelivr.net/npm/codsen-tokenizer/dist/codsen-tokenizer.umd.js">

Then, you’ll get a global variable codsenTokenizer in your Global object and you consume it like this:

const tokenizer = codsenTokenizer;

This tokenizer is your main function. See its API in the main section.

This program has 497 assertions in its unit tests. That's more than all unit test assertions' median (50) and more than geometric mean (70).

In monorepo, 2 other packages consume this package.

§ Notice

codsen-tokenizer is currently being developed. Please come back later.

See it in the monorepo , on Sourcehut.

Parser aiming at broken or mixed code, especially HTML & CSS