detergent^9.4.7

Permalink to InstallationInstallation

Permalink to Quick TakeQuick Take

Permalink to ExamplesExamples

• Insert a line break where a selected HTML tag is stripped
• Add missing spaces after sentence punctuation
• Discover which cleanup rules apply to an input
• Transform text outside HTML tags with a callback
• Normalise typographic apostrophes
• Normalise a dash according to the conversion setting
• Convert three dots to an ellipsis entity
• Choose whether to encode HTML entities
• Inspect the exported default options
• Set the end-of-line sequence inserted after HTML break tags
• Fix misspelled named HTML entities
• Choose HTML or XHTML break-tag syntax
• Keep selected tags while stripping the rest
• Preserve or encode non-Latin characters
• Preserve CSS in retained HTML
• Collapse line breaks to spaces
• Keep the last two words together
• Replace line breaks with HTML break tags
• Choose whether to strip HTML tags

Purpose

detergent prepares text for copy-pasting into HTML, especially; the email-template HTML:

• deletes invisible Unicode characters (like ETX)
• collapses whitespace in prose
• trims
• prevents widow words
• recursively decodes entities in prose and encodes the text again, preferring named HTML entities over numeric-ones, switching to numeric for entities which don’t render correctly across common email clients
• optionally strips HTML (with optional granular control over which tags exactly)
• improves English grammar style: converts M- and N-dashes (spec), apostrophes (spec) and curly quotes (spec)

Extra features are:

• You can skip the HTML encoding of non-Latin language letters. Useful when you are deploying Japanese or Chinese emails because otherwise, everything would be HTML-encoded.
• Detergent is both XHTML and HTML-friendly. You can set which way you want your <br>’s to appear: with a closing slash, <br/> (XHTML) or without (HTML), <br> — that’s to reduce code validator errors.

Retained HTML and CSS

Prose cleanup preserves the source contents of retained tag attributes. HTML character references, CSS escapes, whitespace, quotes, and line continuations inside an attribute keep their original spelling. This applies from the initial entity and whitespace cleanup through typography, widow handling, callbacks, and final line-break cleanup:

import { det } from "detergent";

const html = String.raw`<b style="font-family:'\31  a';content:'&amp;'">x</b>`;
console.log(det(html).res === html);
// => true

To keep a style element and its raw CSS text, include "style" in stripHtmlButIgnoreTags, or set stripHtml: false. A literal closing style tag still ends the CSS region. With stripping disabled, closing tags such as </style>, </script>, and </xml> are normalized once, preserving a single leading slash. Its text is excluded from prose callbacks and transformations. HTML introduced by a callback receives the same protection during subsequent cleanup.

Ordinary text around retained markup still receives the requested cleanup, including widow handling across inline tags. Prose-specific applicableOpts fields describe eligible text; quotes, entities, or line breaks found only in protected markup do not activate those controls. Explicit tag normalization, such as useXHTML adding a slash to <br>, remains available without rewriting attribute values. Backslashes in ordinary prose are not interpreted as CSS.

API — det()

The main function det() is imported like this:

It takes two input arguments:

The Optional Options Object has the following shape:

Here are all defaults in one place for copying:

The default set is a wise choice for the most common scenario — preparing text to be pasted into HTML.

You can also set the options to numeric 0 or 1, that’s shorter than Boolean true or false.

Returns

Function returns a plain object (marked type Res above):

API — defaults

You can import defaults from opts:

It's a plain object:

The main function calculates the options to be used by merging the options you passed with these defaults.

We also use these for testing purposes, in test-mixer, to generate all possible combinations of this options object.

API — version

You can import version:

applicableOpts

Next generation web applications are designed to show only the options that are applicable to the given input. This saves user’s time and also conserves mental resources — you don’t even need to read all the labels of the options if they are not applicable.

At the moment, detergent currently has 14 option keys, 12 of them boolean. That’s not a lot but if you use the tool every day, every optimisation counts.

We got inspiration for this feature while visiting competitor application typograf — it has 110 checkboxes grouped into 12 groups and options are hidden twice — first sidebar is hidden when you visit the page, second, option groups are collapsed.

Another example of overwhelming options set — Kangax minifier — html-minifier — it’s got 26 options with lots of descriptions.

Detergent tackles this challenge differently. While it processes the given input, it makes a note, is particular option applicable or not. This is done independently from the actual options settings.

For example, detergent’s output might look like this — all options not applicable because there’s nothing to do on “abc”:

{
  res: "abc",
  applicableOpts: {
    fixBrokenEntities: false,
    removeWidows: false,
    convertEntities: false,
    convertDashes: false,
    convertApostrophes: false,
    replaceLineBreaks: false,
    removeLineBreaks: false,
    useXHTML: false,
    dontEncodeNonLatin: false,
    addMissingSpaces: false,
    convertDotsToEllipsis: false,
    stripHtml: false
  }
}

Now, the UI, driven by detergent, could grey-out those toggles.

The options keys which have values of a type array (for example, stripHtmlButIgnoreTags and stripHtmlAddNewLine) are omitted from applicableOpts report.

opts.cb

One of the unique (and complex) features of this program is the HTML tag recognition. The program processes only the text and doesn’t touch the tags and their attributes. For example, widow word removal won’t add non-breaking spaces within your tags if you choose not to strip the HTML.

opts.cb runs once per eligible prose slice. It excludes recognized tags, their attributes, comments, CDATA, and style-element contents, including CSS that will later be stripped. For example, here’s how to upper-case all non-HTML-tag characters:

import { det } from "detergent";
const { res } = det(`aAa\n\nbBb\n\ncCc`, {
  cb: (str) => str.toUpperCase(),
});
console.log(res);
// => "AAA<br/>\n<br/>\nBBB<br/>\n<br/>\nCCC"

Permalink to changelogChangelog