unified

Learn/Guide/Create an editor

Creating an editor

This guide shows how to create an interactive online editor with unified. In it we’ll visualize syntactic properties of text by “syntax highlighting” them. The editor will run in a browser. It’ll be fast as we’re using virtual-dom (but you could use React and the like too).

For this example we’ll create an app that visualizes sentence length. It’s based on a tip by Gary Provost, and the visualization is based on a tweet by @gregoryciotti.

You can also view this project with some more features online.

Stuck? Have an idea for another guide? See support.md.

Contents

Case

Before we start, let’s first outline what we want to make. We want to highlight sentences in text based on how many words they have. The user should be able to change text, and it should highlight live.

We’ll use xo as a linter, and browserify as a bundler to compile our JavaScript with require calls to JavaScript that works in the browser (you can swap those out for your favorite linter and bundler).

Project structure

Let’s first outline our project structure:

demo/
├─ index.js
├─ build.js
├─ index.html
├─ index.css
└─ package.json

…where demo/ is our directory, and build.js is the JavaScript generated by compiling index.js.

Keep index.js, index.html, and index.css empty for now, and fill package.json with the following.

{
  "name": "demo",
  "private": true,
  "type": "module",
  "devDependencies": {
    "esbuild": "^0.13.0",
    "prettier": "^2.0.0",
    "xo": "^0.44.0"
  },
  "scripts": {
    "build": "esbuild index.js --bundle --minify --target=es2020 --format=esm --outfile=build.js",
    "lint": "prettier . -w && xo",
    "test": "npm run build && npm run lint"
  },
  "prettier": {
    "tabWidth": 2,
    "useTabs": false,
    "singleQuote": true,
    "bracketSpacing": false,
    "semi": false,
    "trailingComma": "none"
  },
  "xo": {
    "envs": [
      "browser"
    ],
    "prettier": true,
    "ignore": [
      "build.js"
    ]
  }
}

private: true means you can’t accidentally publish your package to npm.

Now, after running npm install and npm test you’ll see build.js appear too. The above package sets up xo as the linter and browserify as the bundler.

Now, fill index.html with the following:

<!DOCTYPE html>
<meta charset="utf8" />
<title>demo</title>
<link rel="stylesheet" href="index.css" />
<div id="root"></div>
<script type="module" src="build.js"></script>

This links index.css and build.js, and adds an element (#root) which we’ll add our editor to later. Oh, did you know that <html>, <head>, and <body> are optional? For this example we’ll keep the HTML minimal, but feel free to add them if you prefer them.

Also add .prettierignore file to not format our build:

build.js

Setting up JavaScript

Alright! Now, let’s set up our JavaScript. Start by adding the following to index.js:

import h from 'virtual-dom/h.js'
import createElement from 'virtual-dom/create-element.js'
import diff from 'virtual-dom/diff.js'
import patch from 'virtual-dom/patch.js'

const root = document.querySelector('#root')
let tree = render('The initial text.')
let dom = root.append(createElement(tree))

function onchange(ev) {
  const next = render(ev.target.value)
  dom = patch(dom, diff(tree, next))
  tree = next
}

function render(text) {
  const node = parse(text)

  return h('div', {className: 'editor'}, [
    h('div', {key: 'draw', className: 'draw'}, highlight(node)),
    h('textarea', {
      key: 'area',
      value: text,
      oninput: onchange
    })
  ])

  function parse() {}

  function highlight() {}
}

Don’t forget to npm install virtual-dom.

That’s going a bit fast, I can imagine, if you’ve never seen virtual-dom in use before. If that’s the case, please take some time to peruse the virtual-dom docs at your leisure. This guide will wait!

To summarize what all these things in the code mean:

In render, we’re creating two elements: a <div> that we’ll draw our syntax highlighting in, and a <textarea> that the user can edit. Both are wrapped in a parent <div>. We’ll style the text area and the drawing area exactly the same, and position the text above the drawing area, with the following styles.

html {
  font-size: 16px;
  line-height: 1.5;
}

.editor {
  position: relative;
  max-width: 37em;
  margin: auto;
  overflow: hidden;
}

textarea,
.draw {
  margin: 0;
  padding: 0;
  width: 100%;
  border: none;
  outline: none;
  resize: none;
  overflow: hidden;
  /* Can’t use a nice font: kerning renders differently in textareas. */
  font-family: monospace;
  line-height: inherit;
  font-size: inherit;
  background: transparent;
  white-space: pre-wrap;
  word-wrap: break-word;
  font-size: inherit;
  line-height: inherit;
}

textarea {
  color: inherit;
  position: absolute;
  top: 0;
}

.draw {
  min-height: 100vh;
}

That’s quite a bit of code: mainly to enforce the same styles on our text and drawing areas.

Natural language syntax tree

Now, let’s set up our natural language syntax tree parsing. We’ll use unified (d’oh), and retext-english to parse English natural language.

Change index.js like so:

--- a/index.js
+++ b/index.js
@@ -2,7 +2,10 @@ import h from 'virtual-dom/h.js'
 import createElement from 'virtual-dom/create-element.js'
 import diff from 'virtual-dom/diff.js'
 import patch from 'virtual-dom/patch.js'
+import {unified} from 'unified'
+import retextEnglish from 'retext-english'

+const processor = unified().use(retextEnglish)
 const root = document.querySelector('#root')
 let tree = render('The initial text.')
 let dom = root.append(createElement(tree))
@@ -25,7 +28,9 @@ function render(text) {
     })
   ])

-  function parse() {}
+  function parse(value) {
+    return processor.runSync(processor.parse(value))
+  }

   function highlight() {}
 }

Don’t forget to npm install unified retext-english.

Sweet, now we have access to a lot of info on the text. It still doesn’t do anything yet though. Let’s add some usefulness.

Virtual DOM

Our next task is to go from a natural language syntax tree to a virtual DOM. We already have highlight for that, but it’s empty, so let’s add code to fill it:

--- a/index.js
+++ b/index.js
@@ -32,5 +32,19 @@ function render(text) {
     return processor.runSync(processor.parse(value))
   }

-  function highlight() {}
+  function highlight(node) {
+    const results = []
+    let index = -1
+
+    while (++index < node.children.length) {
+      results.push(...one(node.children[index]))
+    }
+
+    return results
+  }
+
+  function one(node) {
+    const result = 'value' in node ? [node.value] : highlight(node)
+    return result
+  }
 }

highlight searches all children in the given node, and one returns either the “text content” of a node, or the result of searching its children for text content.

If you’d now run npm test again, and open index.html in your browser, you’ll see that the drawing area already has our text (it’s hidden with styles, but you should be able to see it in your web inspector).

We need one more thing before we can start highlighting: we need to detect sentences, and apply styles to them. Change index.js like so:

--- a/index.js
+++ b/index.js
@@ -18,6 +18,7 @@ function onchange(ev) {

 function render(text) {
   const node = parse(text)
+  let key = 0

   return h('div', {className: 'editor'}, [
     h('div', {key: 'draw', className: 'draw'}, highlight(node)),
@@ -45,6 +46,22 @@ function render(text) {

   function one(node) {
     const result = 'value' in node ? [node.value] : highlight(node)
+
+    if (node.type === 'SentenceNode') {
+      key++
+      return [
+        h(
+          'span',
+          {key: 's-' + key, style: {backgroundColor: color(count(node))}},
+          result
+        )
+      ]
+    }
+
     return result
   }
+
+  function count() {}
+
+  function color() {}
 }

key is needed for virtual-dom to be performant.

We don’t color sentences yet, but there’s <span> elements wrapping them now. You can see that in action by running npm test again and using your web inspector to inspect the drawing area.

We’ve also set up two functions to highlight sentences. count will count the number of words of a given sentence, and color will pick a corresponding color.

Highlight

Now, let’s add colors. Update index.js like so:

--- a/index.js
+++ b/index.js
@@ -4,6 +4,9 @@ import diff from 'virtual-dom/diff.js'
 import patch from 'virtual-dom/patch.js'
 import {unified} from 'unified'
 import retextEnglish from 'retext-english'
+import {visit} from 'unist-util-visit'
+
+const hues = [0]

 const processor = unified().use(retextEnglish)
 const root = document.querySelector('#root')
@@ -61,7 +64,18 @@ function render(text) {
     return result
   }

-  function count() {}
+  function count(node) {
+    let value = 0
+
+    visit(node, 'WordNode', () => {
+      value++
+    })
+
+    return value
+  }

-  function color() {}
+  function color(count) {
+    const value = count < hues.length ? hues[count] : hues[hues.length - 1]
+    return 'hsl(' + [value, '93%', '85%'].join(', ') + ')'
+  }
 }

Don’t forget to npm install unist-util-visit.

The count function searches node for all occurrences of words, through unist-util-visit, and returns that count.

color takes a number, and returns a nice color in HSL for it. It does so based on if there’s a corresponding hue for it in hues (now only one value). If there’s no corresponding hue, it uses the last specified hue.

It’s not much, but it’s something. Try it out by running npm test again, and viewing index.html in your browser. If everything went okay, you should see each sentence highlighted in red.

Color

One color isn’t that cool, and we’re trying to recreate that visual by @gregoryciotti. We need some more colors. From that image, I deducted the following hues. But you could use any hues you like!

To match that image, change hues like so:

--- a/index.js
+++ b/index.js
@@ -6,7 +6,7 @@ import {unified} from 'unified'
 import retextEnglish from 'retext-english'
 import {visit} from 'unist-util-visit'

-const hues = [0]
+const hues = [60, 60, 60, 300, 300, 0, 0, 120, 120, 120, 120, 120, 120, 180]

 const processor = unified().use(retextEnglish)
 const root = document.querySelector('#root')

Squashing bugs

💃 After running npm test again, and reopening index.html in your browser, you should now see The initial text in purple! If you add more sentences, they each should receive colors based on how many words they have.

If you add more text, you’ll notice that our drawing area grows nicely, but our text area does not. That’s because this example positions the <textarea> absolutely on top of the drawing area. The easiest way to get both areas the same height, is with the following slightly hacky code:

--- a/index.js
+++ b/index.js
@@ -13,10 +13,13 @@ const root = document.querySelector('#root')
 let tree = render('The initial text.')
 let dom = root.append(createElement(tree))

+setTimeout(resize, 4)
+
 function onchange(ev) {
   const next = render(ev.target.value)
   dom = patch(dom, diff(tree, next))
   tree = next
+  setTimeout(resize, 4)
 }

 function render(text) {
@@ -79,3 +82,11 @@ function render(text) {
     return 'hsl(' + [value, '93%', '85%'].join(', ') + ')'
   }
 }
+
+function resize() {
+  dom.lastChild.rows =
+    Math.ceil(
+      dom.firstChild.getBoundingClientRect().height /
+        Number.parseInt(window.getComputedStyle(dom.firstChild).lineHeight, 10)
+    ) + 1
+}

This updates the rows attribute on the text area to correspondent with the size of the drawing area.

Further exercises

The above code has a few issues:

…maybe you could solve some? Other than those issues, it’s a pretty cool little demo.

If you haven’t already, check out the other articles in the learn section!