csv-split-easy^7.2.5

No 3rd party dependencies. All dependencies and devDependencies, checked recursively, are Codsen packages.

Permalink to InstallationInstallation

Permalink to Quick TakeQuick Take

Permalink to ExamplesExamples

• Convert decimal commas while retaining grouping separators
• Use options created in another JavaScript realm
• Leave single decimal place numbers unpadded
• Unescape doubled-up double quotes
• Keep the thousand separators in numbers
• Preserve line breaks inside quoted fields
• opts.delimiter
• Pad single-digit fractions without changing longer decimal values
• Read quoted fields after record breaks
• Skip blank rows
• Convert continental number style into UK style

Purpose

Split a string representing a CSV file into an array of arrays, so that you can traverse it later.

Acceptance Criteria:

• It should accept CSV’s with or without a header row
• Header row might have different amount of columns than the rest of the rows
• Content (not header) rows might be offset and have different amount of columns from the rest
• There can be various line break types (\n, \r, \n\r or even \n\n\n\n\n\n\r\r\r)
• It accepts quoted and unquoted fields in the same row
• Delimiters and line breaks inside double quotes remain part of the field value
• Doubled quotes inside a quoted field decode to a single quote
• It should accept empty fields and output them as empty strings
• It should automatically detect (dot/comma) and remove thousand separators from digit-only cells
• Minimal dependencies and 100% unit test code coverage in all ways: per-branch, per-statement, per-function and per-line.

Outside of the scope:

• Parsing numeric values. Parse them yourself. It’s outside of the scope of this program.
• Sorting rows of double-entry, accounting CSV’s. See csv-sort.

API — splitEasy()

The main function splitEasy() is imported like this:

It’s a function which takes two input arguments:

Options must be a plain object: an object literal, a value created with new Object(), or a direct null-prototype object. Plain objects from another JavaScript realm, such as a same-origin iframe or a Node.js vm context, work too. Arrays, class instances, and objects with custom prototypes are rejected.

The optional options object has the following shape:

Here are all defaults in one place for copying:

The function returns an array of rows containing string values. Empty input and input containing only blank rows return [['']].

The result always contains at least one row and one cell.

Multiline fields and whitespace

A quoted field can contain LF (\n), CRLF (\r\n), or CR (\r) line breaks. These characters stay in the field value and do not start another row:

import { splitEasy } from "csv-split-easy";

splitEasy('id,notes\r\n1,"hello\r\nworld"');
// => [["id", "notes"], ["1", "hello\r\nworld"]]

Quoted fields containing line breaks or doubled quotes retain their surrounding whitespace. Doubled quotes decode to a single quote. Other text fields have surrounding whitespace trimmed, including when all number-formatting options are disabled. Rows containing only empty or whitespace-only fields are skipped.

Escaped quotes are decoded in the first field of every row, including immediately after a record break. A field containing a literal quote is nonempty, so its row is retained. Rows containing only quoted empty fields are skipped:

splitEasy('before\r\n"""hello"\r\n""""\r\n""\r\nafter');
// => [["before"], ['"hello'], ['"'], ["after"]]

Convert decimal commas while keeping grouping

Decimal conversion works independently of thousand-separator removal. Use a delimiter such as ; when unquoted cells contain decimal commas:

splitEasy('item;"1 234,50"', {
  delimiter: ";",
  removeThousandSeparatorsFromNumbers: false,
  forceUKStyle: true,
});
// => [["item", "1 234.50"]]

Pad positive fractions below one

With default options, 0.5 becomes 0.50 and .5 becomes .50. Set padSingleDecimalPlaceNumbers: false to keep a single fractional digit. Fractions with more digits retain their precision, regardless of the grouping-removal option. forceUKStyle independently converts their decimal comma to a dot:

import { splitEasy } from "csv-split-easy";

splitEasy('item;"0,075"', { delimiter: ";", forceUKStyle: true });
// => [["item", "0.075"]]

splitEasy("item,0.5");
// => [["item", "0.50"]]

Use a delimiter other than the decimal comma, or quote a comma-containing field. These recognized fractions are trimmed before formatting, including when padding is disabled. Leading zeros and all fractional digits are retained for unsigned positive fractions below one.

API — defaults

You can import defaults:

It's a plain object:

The main function calculates the options to be used by merging the options you passed with these defaults.

API — version

You can import version:

The algorithm

CSV files, especially accounting-ones, are different from just any files. The assumption here is that empty rows are not wanted in the parsed arrays. It means, conventional string splitting libraries would be inefficient here because after splitting, you’d have to clean up any empty rows.

The second requirement is that any of the column values in CSV can be wrapped with double quotes. We have to support mixed, wrapped and not wrapped-value CSV’s because Metro bank used to produce these when we banked with them back in 2015.

The third requirement is that any of the values can be wrapped with double quotes and have commas within as values.

The requirements mentioned above pretty much rule out the conventional regex-based split algorithms. You can just split by /\r?\n/ but later you’ll need to clean up possible empty rows. You can’t string.split each row by comma because that comma might be a value, you need to check for wrapping double quotes first!

So, the best algorithm is a single for-loop traversal on the input string, detecting and array.pushing the values one by one. It worked very well on email-comb where we remove unused CSS from an HTML template within around 2.5 times more characters “travelled” than there are in the file. Traversing as a string also worked well on html-img-alt which needs only a single traversal through the string to fix all the img tag alt attributes and clean all the crap in/around them.

Permalink to changelogChangelog