esbuild is faster but the built programs on average are slower

by — posted on

Programs built by esbuild v0.11.5 appear to perform on average 27% slower than the same code built by Rollup. Some packages perform more than 90% slower though! In this context, I can't use it in production.

Here are the details of the benchmarks. I hope it's me, not esbuild.

Setup

  • MacOS 11.2.3 Big Sur
  • node 15.13
  • npm 7.7.6
  • esbuild 0.11.5
  • rollup 2.44.0
  • @babel/core 7.13.14
  • benchmark 2.1.4

also, the particulars:

Results

CJS built using Rollup, ops/s. CJS built using esbuild, ops/s. difference

📦 object-no-new-keys 4.0.1

Check, does a plain object (AST/JSON) has any unique keys, not present in a reference object (another AST/JSON)

604,665 880,577 45%

📦 util-array-object-or-both 4.0.1

Validate and normalise user choice: array, object or both?

1,417,168 1,944,428 37%

📦 ranges-is-index-within 3.0.1

Checks if index is within any of the given string index ranges

1,245,405 1,687,385 35%

📦 ast-get-values-by-key 4.0.1

Extract values and paths from AST by keys OR set them by keys

52,666 68,463 29%

📦 ast-delete-object 3.0.1

Delete all plain objects in AST if they contain a certain key/value pair

21,558 27,404 27%

📦 ranges-sort 5.0.1

Sort string index ranges

1,206,225 1,498,702 24%

📦 string-overlap-one-on-another 3.0.1

Lay one string on top of another, with an optional offset

939,264 1,164,131 23%

📦 string-split-by-whitespace 3.0.1

Split string into array by chunks of whitespace

28,761 35,358 22%

📦 string-convert-indexes 5.0.1

Convert between native JS string character indexes and grapheme-count-based indexes

49,520 60,141 21%

📦 ast-deep-contains 4.0.1

Like t.same assert on array of objects, where element order doesn't matter

46,517 55,476 19%

📦 edit-package-json 0.5.1

Edit package.json without parsing, as string, to keep the formatting intact

17,628 20,664 17%

📦 helga 2.0.1

Your next best friend when editing complex nested code

808,293 946,780 17%

📦 object-set-all-values-to 5.0.1

Recursively walk the input and set all found values in plain objects to something

93,886 108,901 15%

📦 ast-monkey-traverse 3.0.1

Utility library to traverse AST

7,331 8,429 14%

📦 charcode-is-valid-xml-name-character 2.0.1

Does a given character belong to XML spec's "Production 4 OR 4a" type (is acceptable for XML element's name)

616,548 708,691 14%

📦 ast-monkey-util 2.0.1

Utility library of AST helper functions

5,567,715 6,258,121 12%

📦 easy-replace 5.0.1

Replace strings with optional lookarounds, but without regexes

343,214 385,237 12%

📦 string-range-expander 3.0.1

Expands string index ranges within whitespace boundaries until letters are met

401,309 441,705 10%

📦 ast-loose-compare 3.0.1

Compare anything: AST, objects, arrays and strings

1,119,977 1,231,384 9%

📦 object-flatten-all-arrays 6.0.1

Merge and flatten any arrays found in all values within plain objects

112,809 123,730 9%

📦 object-all-values-equal-to 3.0.1

Does the AST/nested-plain-object/array/whatever contain only one kind of value?

755,624 817,277 8%

📦 ranges-offset 3.0.1

Increment or decrement each index in every range

5,431,865 5,913,447 8%

📦 csv-split-easy 6.0.1

Splits the CSV string into array of arrays, each representing a row of columns

29,342 31,505 7%

📦 str-indexes-of-plus 4.0.1

Like indexOf but returns array and counts per-grapheme

5,612,898 6,057,985 7%

📦 detect-is-it-html-or-xhtml 5.0.1

Answers, is the string input string more an HTML or XHTML (or neither)

4,104,618 4,336,869 5%

📦 is-char-suitable-for-html-attr-name 3.0.1

Is given character suitable to be in an HTML attribute's name?

138,053,165 144,933,037 4%

📦 string-left-right 5.0.1

Looks up the first non-whitespace character to the left/right of a given index

2,796,142 2,909,465 4%

📦 string-unfancy 5.0.1

Replace all n/m dashes, curly quotes with their simpler equivalents

184,901 193,216 4%

📦 bitbucket-slug 3.0.1

Generate BitBucket readme header anchor slug URLs. Unofficial, covers whole ASCII and a bit beyond

463,477 471,869 1%

📦 all-named-html-entities 2.0.1

List of all named HTML entities

12,988,337 13,076,600 0%

📦 arrayiffy-if-string 4.0.1

Put non-empty strings into arrays, turn empty-ones into empty arrays. Bypass everything else

851,031,198 852,625,728 0%

📦 color-shorthand-hex-to-six-digit 4.0.1

Convert shorthand hex color codes into full

991,925 995,975 0%

📦 object-flatten-referencing 6.0.1

Flatten complex nested objects according to a reference objects

46,439 46,879 0%

📦 array-includes-with-glob 4.0.1

Like _.includes but with wildcards

389,175 388,184 -1%

📦 csv-sort 6.0.1

Sorts double-entry bookkeeping CSV coming from internet banking

828 822 -1%

📦 object-boolean-combinations 5.0.1

Consumes a defaults object with booleans, generates all possible variations of it

26,872 26,850 -1%

📦 regex-is-jsp 3.0.1

Regular expression for detecting JSP (Java Server Pages) code

857,259,446 841,373,993 -2%

📦 string-character-is-astral-surrogate 2.0.1

Tells, is given character a part of astral character, specifically, a high and low surrogate

386,957,104 380,800,575 -2%

📦 util-nonempty 4.0.1

Is the input (plain object, array, string or whatever) not empty?

758,700 748,129 -2%

📦 html-all-known-attributes 5.0.1

All HTML attributes known to the Humanity

872,646,305 853,734,857 -3%

📦 html-entities-not-email-friendly 0.6.1

All HTML entities which are not email template friendly

8,152 7,922 -3%

📦 object-delete-key 3.0.1

Delete keys from all arrays or plain objects, nested within anything, by key or by value or by both, and clean up afterwards. Accepts wildcards

487 476 -3%

📦 ast-is-empty 3.0.1

Find out, is nested array/object/string/AST tree is empty

842,803 816,877 -4%

📦 regex-empty-conditional-comments 2.0.1

Regular expression for matching HTML empty conditional comments

879,313,074 844,474,669 -4%

📦 regex-jinja-specific 3.0.1

Regular expression for detecting Python-specific Jinja code

879,049,231 848,836,389 -4%

📦 array-pull-all-with-glob 6.0.1

Like _.pullAll but with globs (wildcards)

80,492 76,541 -5%

📦 is-html-tag-opening 3.0.1

Does an HTML tag start at given position?

208,444 199,699 -5%

📦 regex-is-jinja-nunjucks 3.0.1

Regular expression for detecting Jinja or Nunjucks code

927,186,953 879,108,894 -6%

📦 detect-templating-language 3.0.1

Detects various templating languages present in string

3,733,385 3,473,999 -7%

📦 ast-contains-only-empty-space 3.0.1

Does AST contain only empty space?

28,905 26,653 -8%

📦 is-language-code 4.0.1

Is given string a language code (as per IANA)

1,795,426 1,632,861 -10%

📦 ast-compare 3.0.1

Compare anything: AST, objects, arrays, strings and nested thereof

45,942 37,713 -18%

📦 object-fill-missing-keys 9.0.1

Add missing keys into plain objects, according to a reference object

83,730 68,346 -19%

📦 ast-monkey 8.0.1

Traverse and edit AST

10,914 8,680 -21%

📦 ast-get-object 3.0.1

Getter/setter for nested parsed HTML AST's, querying objects by key/value pairs

84,227 65,060 -23%

📦 is-media-descriptor 4.0.1

Is given string a valid media descriptor (including media query)?

681,155 528,020 -23%

📦 test-mixer 3.0.1

Test helper to generate function opts object variations

128,661 90,184 -30%

📦 is-relative-uri 4.0.1

Is given string a relative URI?

965,326 664,065 -32%

📦 json-comb-core 7.0.1

The inner core of json-comb

5,194 3,554 -32%

📦 line-column-mini 2.0.1

Convert string index to line-column position

536,985 352,397 -35%

📦 lerna-clean-changelogs 3.0.1

Removes frivolous entries from commitizen generated changelogs

174,708 108,303 -39%

📦 check-types-mini 7.0.1

Validate options object

24,786 14,888 -40%

📦 ranges-apply 6.0.1

Take an array of string index ranges, delete/replace the string according to them

402,362 240,594 -41%

📦 ranges-ent-decode 5.0.1

Recursive HTML entity decoding for Ranges workflow

313,404 176,076 -44%

📦 object-merge-advanced 13.0.1

Recursively, deeply merge of anything (objects, arrays, strings or nested thereof), which weighs contents by type hierarchy to ensure the maximum content is retained

10,820 5,918 -46%

📦 string-trim-spaces-only 4.0.1

Like String.trim() but you can choose granularly what to trim

1,122,252 598,728 -47%

📦 ranges-regex 5.0.1

Integrate regex operations into Ranges workflow

634,846 325,307 -49%

📦 json-variables 11.0.1

Resolves custom-marked, cross-referenced paths in parsed JSON

4,663 2,334 -50%

📦 string-uglify 2.0.1

Shorten sets of strings deterministically, to be git-friendly

162,093 75,494 -54%

📦 detergent 8.0.1

Extracts, cleans and encodes text

1,003 447 -56%

📦 ast-monkey-traverse-with-lookahead 3.0.1

Utility library to traverse AST, reports upcoming values

8,446 3,664 -57%

📦 array-of-arrays-into-ast 3.0.1

Turns an array of arrays of data into a nested tree of plain objects

21,030 9,028 -58%

📦 generate-atomic-css 2.0.1

Generate Atomic CSS

40,356 17,296 -58%

📦 string-remove-widows 3.0.1

Helps to prevent widow words in a text

4,967 2,083 -59%

📦 tap-parse-string-to-object 3.0.1

Parses raw Tap: string-to-object or stream-to-a-promise-of-an-object

72,457 30,365 -59%

📦 array-group-str-omit-num-char 5.0.1

Groups array of strings by omitting number characters

123,562 48,255 -61%

📦 email-all-chars-within-ascii 4.0.1

Scans all characters within a string and checks are they within ASCII range

304,707 116,948 -62%

📦 ranges-process-outside 5.0.1

Iterate string considering ranges, as if they were already applied

139,148 52,921 -62%

📦 ranges-merge 8.0.1

Merge and sort string index ranges

402,352 129,858 -68%

📦 string-find-heads-tails 5.0.1

Finds where are arbitrary templating marker heads and tails located

83,697 27,322 -68%

📦 string-match-left-right 8.0.1

Match substrings on the left or right of a given index, ignoring whitespace

89,936 29,353 -68%

📦 html-img-alt 3.0.1

Adds missing alt attributes to img tags. Non-parsing

38,241 11,878 -69%

📦 string-remove-thousand-separators 6.0.1

Detects and removes thousand separators (dot/comma/quote/space) from string-type digits

76,826 24,111 -69%

📦 html-table-patcher 5.0.1

Visual helper to place templating code around table tags into correct places

422 129 -70%

📦 ranges-invert 5.0.1

Invert string index ranges

266,478 82,308 -70%

📦 string-process-comma-separated 3.0.1

Extracts chunks from possibly comma or whatever-separated string

359,421 105,242 -71%

📦 string-remove-duplicate-heads-tails 6.0.1

Detect and (recursively) remove head and tail wrappings around the input string

5,214 1,528 -71%

📦 stristri 4.0.1

Extracts or deletes HTML, CSS, text and/or templating tags from string

535 157 -71%

📦 string-collapse-white-space 10.0.1

Replace chunks of whitespace with a single spaces

82,405 23,875 -72%

📦 ranges-crop 5.0.1

Crop array of ranges when they go beyond the reference string's length

533,039 148,791 -73%

📦 codsen-tokenizer 6.0.1

HTML and CSS lexer aimed at code with fatal errors, accepts mixed coding languages

10,075 2,674 -74%

📦 is-html-attribute-closing 3.0.1

Is a character on a given index a closing of an HTML attribute?

295,486 63,758 -79%

📦 codsen-parser 0.12.1

Parser aiming at broken or mixed code, especially HTML & CSS

6,672 1,381 -80%

📦 string-fix-broken-named-entities 6.0.1

Finds and fixes common and not so common broken named HTML entities, returns ranges array of fixes

82,165 15,341 -82%

📦 string-extract-sass-vars 3.0.1

Parse SASS variables file into a plain object of CSS key-value pairs

40,932 6,498 -85%

📦 string-collapse-leading-whitespace 6.0.1

Collapse the leading and trailing whitespace of a string

1,741,567 203,157 -89%

📦 string-extract-class-names 7.0.1

Extracts CSS class/id names from a string

268,807 24,855 -91%

📦 array-of-arrays-sort-by-col 4.0.1

Sort array of arrays by column, rippling the sorting outwards from that column

2,631,276 215,987 -92%

📦 ranges-push 6.0.1

Gather string index ranges

1,369,476 109,753 -92%

📦 js-row-num 5.0.1

Update all row numbers in all console.logs in JS code

12,136 918 -93%

📦 string-apostrophes 2.0.1

Comprehensive, HTML-entities-aware tool to typographically-correct the apostrophes and single/double quotes

48,811 3,215 -94%

📦 ranges-iterate 3.0.1

Iterate a string and any changes within given string index ranges

12,707,835 670,891 -95%

📦 string-find-malformed 3.0.1

Search for a malformed string. Think of Levenshtein distance but in search

324,137 18,415 -95%

📦 string-strip-html 9.0.1

Strips HTML tags from strings. No parser, accepts mixed sources

8,835 213 -98%
Average: -27%
Median: -18%

Maybe my esbuild settings are wrong?

Programs benchmarked above were built using esbuild on the following settings opens in a new tab:

  • format as "cjs"
  • bundle on, but exclude any dependencies or peer dependencies (ends up bundling local imports only)
  • minify on (to strip console.log and console.time instances which we keep in the source to help the maintainability)
  • target as node10.4

There shouldn't be any surprises, though?

You can try yourself

  1. Clone codsen monorepo, git clone https://github.com/codsen/codsen.git
  2. cd codsen
  3. npm run bootstrap (it's a monorepo, you don't npm i, Lerna does it)
  4. cd into any package's root, cd packages/string-strip-html
  5. for Rollup, build and benchmark npm run build && npm run perf
  6. for esbuild, build and benchmark npm run esbuild && npm run perf

Benchmarks run on a cjs build and vary depending on thermal throttling, machine's load and other factors.

Takeaway

For me, esbuild still feels not production-ready yet, even the perf issues aside:

Fingers crossed, maybe esbuild will mature in a year or so. After all, it took three years and seven months for Rollup to reach v.1, and esbuild is still one-year-old!

Related packages:

📦 esbuild opens in a new tab
An extremely fast JavaScript bundler and minifier
📦 rollup opens in a new tab
Next-generation ES module bundler