Youdao Writing Browser extension, as an auxiliary tool to add English grammar correction for web pages, allows users to edit English text in most rich text editors and multi-line text input boxes on any web page. Users can get feedback on the correction results in real time, and accept suggestions for automatic modification to achieve perfect writing.

Source/Youdao Technology team official account

By Jingwen Li

Editor/Liu Zhenyu

I. Background introduction

Youdao Writing service is an intelligent writing correction product produced by Youdao, providing users with high-quality writing spelling, grammar, style correction services. Youdao Writing not only supports the browser extension form, but also supports the use of other platforms: for example, Youdao Dictionary APP-composition correction, Web online terminal, Word plug-in, PC dictionary. Welcome to experience.

write.youdao.com/

Browser plug-in in the Browser is called Browser Extension, also known as the Browser Extension, is a plug-in to extend the function of the web Browser. It is mainly based on HTML, JavaScript, CSS development, at the same time because it is an extension feature, can use the browser itself provided by the bottom API interface development, can be used to the page or a specific page to add some special functions without affecting the original page logic.

Each browser that supports extensions has its own app store where extensions can be downloaded directly. Some products provide their own.crx files for browser extensions that users can download and install.

Two, adapt the browser

Youdao Writing can be installed on both Windows and Mac systems, with Chrome, 360 Security Browser, 360 Speed Browser, Edge new version browser, etc. Search Youdao Writing in the browser store above and click the install button.

Function introduction & effect display

Before introducing the development ideas and practice, let’s have a visual look at the actual effect of youdao Writing browser extension, and briefly introduce its functions.

3.1 Presentation Mode

The visual effect is to draw a line under the wrong text character, and to add a highlight to the text if it hover. When choosing to accept the suggestion, we can replace it with the text data we want.

3.2 Application Scenarios

>>> Online Mail editing:

163 email

Outlook email

Gmail

>>> Social news, comments:

Facebook

Weibo dynamic

comments

>>> Tools, Note Classes:

Youdao translation

Google translate

Graphite document

3.3 Functions

>>> Real-time correction:

Support real-time correction feedback and display the number of correction errors.

>>> Syntax check:

>>> Enhanced edit box:

You can view the details of each error feedback and filter the results by error type.

>>> Accept suggestions:

Replace the correct text when clicking accept suggestions.

Iv. Development ideas

Requirements: The extension needs to be able to make corrections to edit boxes for input text on the page

4.1 Adaptation editor

So, what are the edit boxes for entering text in a web page?

Usually we can input the edit box is:

  • Web-based forms can enter text controls: input, Textarea
<input value="123"/>
<textarea>123456</textarea>
Copy the code
  • Elements with editable properties: Contenteditable
<blockquote contenteditable="true">
    <p>Edit this content to add your own quote</p>
</blockquote> 
Copy the code

The Input element is usually a single line with a short Input range. Considering the capability of the correction interaction, our extension is compatible with the following editors that can enter more text:

  • Contenteditable rich text editor
  • textarea
  • Other document editors

4.2 Rich text editor

Common rich text editors based on contenteditable implementations are Baidu Editor, draft.js, Youdao Cloud Note (old version), and so on.

Compared to textarea, rich text editors can contain many different tags that can be used to render text, images, attachments, video, audio, and other elements in different font colors.

Four elements for implementing a browser-based rich text editor

Technical selection of four generation editor

  • The first generation of editors operated primarily on HTML documents with limited execCommand instructions.
  • The second generation is based on execCommand, adding more custom instructions and even implementing their own instructions to modify HTML documents.
  • The third generation introduces data models (JSON/XML) that bind custom implementation instructions to render HTML documents.
  • The fourth generation mainly abandons the entire contenteditbale directly, making separate selections and listening for input events.

For more information about the editor, please refer to a previous post by the Youdao Technology team:

  • Architecture Design of the new version of Youdao Cloud Note Editor (PART 1)
  • Architecture Design of the new version of Youdao Cloud Note Editor (Part 2)

The reason for introducing rich text editor content is that knowing more about these editor implementations and saving mechanisms can help you later implement and optimize the functionality of the extension.

At the beginning of 4.3 to

The initial idea was to extract the plain text content of the original editor and send it to the server, then reconcatenate it based on the data returned by the server, using special tag tags at the wrong node locations.

Take Youdao Writing Web terminal as an example:

Use this method to achieve the effect of correcting 163 mailbox English intelligent check, Gmail built-in letter grammar check function, etc. This method is suitable for our ** custom editor, ** can control their own text rendering and instructions.

However, because the browser extension is based on an auxiliary tool in an editor written by someone else, the text format and style cannot be modified at will. Copying and pasting underlined text, for example, will result in redundant underlined text (unless the original editor has a label filter for pasted text), but don’t expect editors written by others to have this feature.

4.4 implementation

There are two parts to consider:

  1. How to position the line drawing
  2. How do I accept suggestions to replace the correct text

How do I position the lines and give them a highlight?

The problem is that the underlined sections need to be added to the interface without affecting the format and functionality of the original editor.

> > > contenteditabe:

  • Step 1: Virtual helper borders

Create an element (of the same size, opposite each other) on top of the original editor and underline the result on top of the element, which I call the helper.

Because it is overridden on the original editor, the mouse response behavior of the helper needs to be prohibited, and the mouse position needs to be synchronized to the helper when hover.

The helper needs to be the same as the original editor to be accurate, and the following attributes of the original editor need to be obtained.

  • Step 2: Find the correct positioning

Question: * * * * if simple extraction innerHTML/corrects the InnerText/textContent as request parameters, unable to realize accurate positioning.

** Cause: ** The server returns the result based on plain text positioning, the editor format on the web page is HTML document format, including different font tags with different formats.

Here’s an example:There are two block-level elements in HTML, each displaying two sentences, the only difference being thatFont size is different.

The contents extracted from element.textContent are the same, and the error flags returned by the verification are the same, as follows:

Because you can’t see the difference between the two cases from a plain text perspective, the difficulty is that you need to parse the HTML format and convert the server-side data into the actual format location.

Remember, this is equivalent to adding multiple absolutely positioned highlighting elements to a blank whiteboard element. You need to know the relative displacement of each error relative to the original edit box, as well as its width and height.

Here’s a reverse process:

  1. HightlightElement: {top, letf, width, height};
  2. Range.getclientrects () gets the data we want.
  3. So you need to know how to get the range of an error node.

  1. You need to find the corresponding starting node and its relative displacement, and the ending node and its relative displacement. Range: {startNode, stratOffest, endNode, endOffest}. The method is calculated by traversing the data length of each textNodes (textNodes.nodevalue) of the full text through the error node at the start (fixedposStart) and end (fixedposEnd) of plain text.

  2. FixedposStart and fixedposEnd can be calculated according to the data returned by the server. Full text each textNodes needs to be obtained by filtering certain tags.

  3. So you need to think about how to deal with the various tags in HTML.

** Therefore, the principle of underline is as follows: ** Extracts its pure text TextNode node, matches the start node and displacement, end node and displacement according to the result position, obtains the x, y, height and width corresponding to the editor of its text fragment range, and draws the highlighted area.

The specific steps are as follows:

A. Extract pure text and processed text node set according to all HTML tags of the original text:

HTML contains various tag nodes, which need to be processed according to the different meanings of these tags. For example, for the P tag, which usually represents a paragraph, you need to add a newline character to the end of the content it wraps.

P tag processing example: Problem: The newline character is sent to the server together, and the data location returned by the server is also counted by the newline character. Solution: Filter the label while recording the position of the text processing, and reverse the processing in the later calculation. At the same time, we also need to pay attention to the problem of escaping characters, especially zero-width characters.

B. Extract plain text nodes:

(The text content in the figure above is divided into 5 plain text nodes according to the label content)

C. Calculate each error based on server data

For example, error node information corresponding to has error.

D. Get text fragment of each error node according to location:

E. Obtain the position of the relative viewport through the text fragment:

Scribing step diagram

  • Step 3: Underline and highlight the assist range

Flowchart of the work of a Contenteditable collection helper

> > > textarea:

The Textarea itself cannot get its TextNode, which is equivalent to having only one node. Consider converting it to a text node:

  • Create an invisible mirror with the same border size and editable area as the original editor.

  • Textarea synchronizes any text changes to the mirror
this.textarea.addEventListener('input',this.mirror.update);
Copy the code
  • Create an assist for the mirror, in line with the process above for contenteditbale.

>>> About mutations:

An editor is a common element, and the following editor interactions cause changes to the text nodes in our pages:

  • Text content change
  • Size changes (Windows become larger and smaller)
  • Position change
  • Font size changes (bold, centered)
  • rolling

These changes also affect changes in our location, called mutations. Each mutation causes a relocation problem that needs to be addressed.

At the same time, you need to listen for input from the original editor, font changes, editor size changes, and so on to trigger assist’s repositioning method.

ObjResizeObserver = new ResizeObserver((entries) => {
    var entry = entries[0];
    this.elementResizeHandler(entry.target)
}); 
Copy the code

ResizeObserver compatibility issues need to be addressed through the Polyfill library file.

Relocating method (mutation) :

  • By comparing the old and new textnode arrays, forward and reverse traversal of the node set can be obtained, which segment textnode textnode set is the modified textnode.
  • It only needs to process the set of error nodes corresponding to the affected TextNode to calculate the node displacement of subsequent influence according to the moving offEST.

  • The displacement of other errors relative to their TextNode will not change.

How to accept a suggestion, replace the text:

Replacing text means changing the data and even the format of the original editor, which can cause formatting errors and save failures in some editors.

** Difficulty: ** Does not affect the original data storage format, does not affect the original editor recall operation, and can trigger the original editor save mechanism.

** Solution: ** Instead of directly scripting dom nodes, simulate the way users modify data: select text and replace content.

Take the new Version of Youdao Cloud Note as an example:

  1. The result fragment is obtained through the previous complex calculation. According to the result fragment, the position of the visual window is calculated and {top, left, height, width} is obtained.
  2. An action event that simulates mouse sliding from left to right is added to the content area.

  1. Find a custom draw area.

  2. Different styles may be involved in an error result, so we just get the font style of the first fragment of the current node, simulating a paste event.

  1. Triggers a custom paste event in the paint area.

4.5 Enhanced edit box

Entry click on the bottom right button or hover results card, click on detailed suggestions to enter

>>> Enhance the role of the edit box:

  • Provide more editing space
  • View detailed correction results

The Enhanced edit box is a special contenteditable editor.

>>> Initialize and close assignment:

When initializing the enhanced editor, get the original editor data directly, ignoring some styles and images from the original editor and using only the HTML data part.

When you return to the original editor after editing in the enhanced editor, you need to return the assignment to the new data.

> > > communication:

The enhanced edit box is an iframe embedded in the page that appears only on the top page. Communication with the original page is via postMessage.

(Note: postMessage cannot pass HTML elements and overly complex JSON objects.)

If the original editor is iframe, you need to find the top layer window.top and use window.top to communicate with the enhanced edit box.

Five, the overall process

The picture above shows how the Youdao Writing browser extension runs from injection to the browser page.

In order not to affect the user’s operation, the extension script is only loaded when the current page is idle, and the correction function is only activated in part of the editor when the user clicks focus.

The above is the development ideas and some technical implementation details in the process of developing Youdao writing browser extension, take this opportunity to share with you, welcome to discuss more knowledge about front-end and browser extension with Youdao technical team.