string-fix-broken-named-entities^7.2.5

No 3rd party dependencies. All dependencies and devDependencies, checked recursively, are Codsen packages.

Permalink to InstallationInstallation

Permalink to Quick TakeQuick Take

Permalink to ExamplesExamples

• Apply the returned ranges to repair the source string
• Report healthy named-entity ranges
• Decode healthy named entities
• Decode decimal and hexadecimal numeric entities
• Map findings into custom callback results
• Repair repeatedly encoded entities
• Observe scan progress
• Access the complete public rule-name list
• Sift raw ampersands in a string from broken character references

Purpose

This program detects and fixes broken named HTML entities (like  ). Its typo stage uses string-typo-match with a one-event policy for omissions, extra or repeated characters, adjacent swaps, and substitutions. It declines competing typo suggestions instead of guessing; case handling and curated fixes apply separately.

In practice, this means it can catch errors like: &nbp; (mistyped  ).

This program also works as a healthy entities catcher — broken entities are fed to one callback (opts.cb), healthy entities are fed to another callback (opts.entityCatcherCb).

There is a decoding function; the algorithm is aware of numeric HTML entities as well.

API — fixEnt()

The main function fixEnt() is imported like this:

It’s a function which takes two input arguments:

The Optional Options Object has the following shape:

By default, fixEnt() returns an array of ranges describing proposed fixes. An empty array means there are no changes. Set cb: null to receive diagnostic objects, or provide a callback to map each diagnostic into your own output type. The default call uses the shared Ranges type:

For example, four fixed nbsp’s:

[
  [6, 11, " "],
  [11, 18, " "],
  [27, 34, " "],
  [34, 41, " "],
];

The output can be further processed by other range libraries: cropping, sorting, merging can be done on range arrays, instead of mutating the input string each time.

Repair behavior

Whitespace within a suspected named entity is ignored during exact, curated, and typo matching, including long gaps. For example, &nbsq ; and &n b s q; both repair to  . Returned ranges always use the original input indices. Ambiguous or unrecognized candidates retain the deletion-range convention.

Multiple encoding is reduced to one named reference, or to its decoded value when decode is enabled. Both amp and AMP are supported as encoding layers:   becomes  . The final entity name remains case sensitive. Matching uses complete names and actual source endpoints, preserving following text and markup.

Uncertain names without a semicolon follow the same prose-preservation rules after an encoded ampersand. For example, male & female employees stays unchanged by default and becomes male & female employees with decode: true. An explicit reference such as ♀ still repairs to ♀ or decodes to ♀. Combined defects such as &;NBSP; and &;nbsq; consume the opening ampersand and repair to one  .

opts.decode

Set decode: true to return decoded characters for repaired named entities and healthy named or numeric references. Numeric decoding uses Unicode code points, so both 😀 and 😀 decode to 😀.

Numeric references require a complete decimal body or a hexadecimal body introduced by x or X. Leading zeroes are accepted without a padding-length limit. Missing-ampersand hexadecimal candidates such as #x26;, #xA3;, and #XFF; receive malformed-numeric deletion ranges; they are not repaired into numeric references. Invalid digits, empty bodies, null values, surrogates, and values above U+10FFFF produce malformed-numeric deletion ranges in either mode. Valid control characters retain their code points; this package does not apply HTML control-character remapping.

For example:

import { fixEnt } from "string-fix-broken-named-entities";
const result = fixEnt("zz nbsp;zz nbsp;", { decode: true });
console.log(JSON.stringify(result, null, 4));
// => [[3, 8, "\xA0"], [11, 16, "\xA0"]]

opts.cb

So, normally, the output of this library is an array of zero or more arrays (each meaning string index ranges), for example:

[
  [1, 2],
  [3, 4]
]

Above means, delete the string from index 1 to 2 and from 3 to 4.

However, for example, in emlint, we need slightly different format, not only ranges but also issue titles:

[
  {
    "name": "tag-generic-error",
    "position": [[1, 2]]
  },
  {
    "name": "tag-generic-error",
    "position": [[3, 4]]
  }
]

Callback function via opts.cb allows you to change the output of this library.

The concept is, you pass a function in options object’s key cb. That function will receive a plain object with all “ingredients” under various keys. Whatever you return, will be pushed into a results array. For each result application is about to push, it will call your function with findings, all neatly put in the plain object.

For example, to solve the example above, you would do:

import { fixEnt } from "string-fix-broken-named-entities";
const res = fixEnt("zzznbsp;zzznbsp;", {
  cb: (oodles) => {
    // "oodles" or whatever you name it, is a plain object.
    // Grab any content from any of its keys, for example:
    // {
    //   ruleName: "bad-html-entity-malformed-pi",
    //   entityName: "pi",
    //   rangeFrom: 3,
    //   rangeTo: 4,
    //   rangeValEncoded: "π",
    //   rangeValDecoded: "\u03C0"
    // }
    return {
      name: oodles.ruleName,
      position: oodles.rangeValEncoded != null ? [oodles.rangeFrom, oodles.rangeTo, oodles.rangeValEncoded] : [oodles.rangeFrom, oodles.rangeTo],
    };
  },
});
console.log(JSON.stringify(res, null, 4));
// => [
//      {
//        name: "bad-html-entity-malformed-nbsp",
//        position: [3, 8, " "]
//      },
//      {
//        name: "bad-html-entity-malformed-nbsp",
//        position: [11, 16, " "]
//      }
//    ]

Here’s the detailed description of all the keys, values and their types:

Set cb: null to return the diagnostic objects directly. A custom callback maps those same objects into its return values; callback selection does not change which repairs are reported. TypeScript infers the output element type from the callback. Opts<T> lets you describe a reusable callback’s return type explicitly.

import { fixEnt } from "string-fix-broken-named-entities";

const raw = fixEnt("&nsp;", { cb: null });
const ruleName: string = raw[0].ruleName;

const mapped = fixEnt("&nsp;", {
  cb: (finding) => ({ rule: finding.ruleName }),
});
const rule: string = mapped[0].rule;

A callback that returns nothing produces an array of undefined values, one per diagnostic. Use the default callback when passing the result to range-processing libraries.

opts.decode in relation to opts.cb

decode determines which entities need a result. Repairs are reported in both modes. Healthy named and numeric references are also reported when decode: true. Every diagnostic includes both encoded and decoded replacement values, so a custom callback chooses which value to use.

import { fixEnt } from "string-fix-broken-named-entities";

const input = "  &nsp;";
const rules = (options) => fixEnt(input, {
  ...options,
  cb: ({ ruleName }) => ruleName,
});

rules({ decode: false });
// => ["bad-html-entity-malformed-nbsp"]
rules({ decode: true });
// => ["bad-html-entity-encoded-nbsp", "bad-html-entity-malformed-nbsp"]

opts.entityCatcherCb

entityCatcherCb(from, to) receives healthy entity ranges. With the default decode: false, it catches healthy named and numeric references. With decode: true, named references are reported through cb, while healthy numeric references also reach entityCatcherCb. Malformed numeric references produce diagnostics through cb and are excluded from the healthy-entity callback.

import { fixEnt } from "string-fix-broken-named-entities";

const inp1 = "y   z &nsp;";
const gatheredEntityRanges = [];
fixEnt(inp1, {
  entityCatcherCb: (from, to) => gatheredEntityRanges.push([from, to]),
});
console.log(`${`\u001b[${33}m${`gatheredEntityRanges`}\u001b[${39}m`} = ${JSON.stringify(gatheredEntityRanges, null, 4)}`);
// => [[2, 8]]

opts.textAmpersandCatcherCb

Sometimes input string can contain ampersands-as-text and ampersands-as-part-of-entities.

For example, consider a string abc& &xyz. What do you see here? There’s one named HTML entity,  , surrounded by two raw text ampersands. This callback, opts.textAmpersandCatcherCb can be used to catch raw ampersands, probably with aim to HTML-encode them. Specifically, this option would call your function twice, with numbers 3 and 10, positions of raw ampersands. Indices are reported once, in ascending input order. Ampersands consumed by healthy references or repairs are excluded.

See the supplied example where broken entity is fixed and raw text ampersands are encoded, all within the same string, all done using this program.

opts.progressFn

progressFn(percentageDone) receives increasing integer percentages, starting at 0 and ending at 100 on successful completion, including empty input. Intermediate values can be skipped. The final update occurs after range cleanup and all result callbacks have finished. You can forward these numbers from a web worker to update a progress indicator.

It’s hard to show a minimal worker application here but at least here’s how the pinging progress works from the side of this npm package:

// let's define a variable on a higher scope:
let count = 0;

// call application as normal, pass opts.progressFn:
const result = fixEnt("text &ang text&ang text text &ang text&ang text text &ang text&ang text", {
  progressFn: (percentageDone) => {
    // console.log(`percentageDone = ${percentageDone}`);
    count++;
  },
});
// each time percentage is reported, "count" is incremented

// now imagine if instead of incrementing the count, you pinged the
// value out of the worker

API — allRules

Every rule name this program can report, as an array of strings:

import { allRules } from "string-fix-broken-named-entities";

console.log(allRules.length);
// => 4255

console.log(allRules.slice(0, 3));
// => [
//      "bad-html-entity-malformed-Aacute",
//      "bad-html-entity-malformed-aacute",
//      "bad-html-entity-malformed-Abreve",
//    ]

The list is assembled from all named HTML entities, twice over, plus five catch-all names:

The ruleName key of the opts.cb callback object is always one of these. emlint consumes allRules to register a rule per name, which is why the list is exported rather than kept private.

API — version

You can import version:

API — types

This package is written in TypeScript and exports the following types:

import type { Obj, Opts, Ranges, cbObj } from "string-fix-broken-named-entities";

Permalink to changelogChangelog