Boa: Use Python in Node.js

The text/Yorkie

I haven’t seen you for a while. This time, I’m going to bring you a nice thing, which is basically a way to use Python’s interfaces and functions in Node.js. Why use Python when I’m so good at Node.js? If you’ve tried some machine-learning JavaScript in the past, you’ll know why. The reality is that the machine learning ecosystem is almost tied to Python iterating at a high speed, while JavaScript is not, and if we want to go from zero to the scale that Python is today, the amount of work is enormous. I already felt this way a few years ago when I wrote Tensorflow-nodejs. Therefore, we have to change the way of thinking, since we can not surpass you, then use you. For scripting language developers, it doesn’t really matter how the bottom layer is implemented, as long as the top layer language and interface are familiar to me, so Boa is a Node.js library that Bridges CPython to give JavaScript access to the Python ecosystem. In addition, with the help of new ES6 features, to provide users with a seamless development experience, so exactly what is the experience? Here’s a simple example:

const boa = require('@pipcook/boa');
const os = boa.import('os');
console.log(os.getpid()); // prints the pid from python.

// using keyword arguments namely `kwargs`
os.makedirs('.. ', boa.kwargs({
  mode: 0x777.exist_ok: false,}));// using bult-in functions
const { range, len } = boa.builtins();
const list = range(0.10); // create a range array
console.log(len(list)); / / 10
console.log(list[2]); / / 2
Copy the code

After loading Python objects through boa. Import, the rest of the object access, function calls, and array access are pretty much the same as if we were using JavaScript.

const boa = require('@pipcook/boa');
const { len, tuple, enumerate } = boa.builtins();
const torch = boa.import('torch');
const torchtext = boa.import('torchtext');
const { nn, optim } = torch;

class TextSentiment extends nn.Module {
  constructor(sizeOfVocab, dimOfEmbed, numOfClass) {
    super(a);this.embedding = nn.EmbeddingBag(sizeOfVocab, dimOfEmbed, boa.kwargs({
      sparse: true,}));this.fc = nn.Linear(dimOfEmbed, numOfClass);
    this.init_weights();
  }
  init_weights() {
    const initrange = 0.5
    this.embedding.weight.data.uniform_(-initrange, initrange);
    this.fc.weight.data.uniform_(-initrange, initrange);
    this.fc.bias.data.zero_();
  }
  forward(text, offsets) {
    const embedded = this.embedding(text, offsets);
    return this.fc(embedded); }}Copy the code

In addition to showing how to inherit a Python class from JavaScript, the above example also shows how we can use PyTorch to create a model. Isn’t that JavaScript? It is worth noting that there is no encapsulation of PyTorch in the Boa code, as long as you install the corresponding package locally in Python, you can use the same code as above, so in theory you can do the same for any Python package. Next, we introduce some of the main methods.

builtins()

Python will build some commonly used methods in the builtin, specific API list in: docs.python.org/3.7/library… , Boa also provides corresponding methods:

const { len, list, range } = boa.builtins();
Copy the code

const np = boa.import('numpy');
Copy the code

kwargs(map)

Next, there are Keyword Arguments in Python, which provide a way to represent Arguments using a Map, such as:

foobar(100, x=10, y=20)
Copy the code

It does a better job of helping callers understand what each parameter means, and for this reason, the kwargs method has been added to Boa to support this usage:

foobar(100, boa.kwargs({ x: 10.y: 20 }));
Copy the code

with(localcontext()) {
  # balabala
}
Copy the code

boa.with(torch.no_grad(), () => {
  const output = model(text, offsets);
  const loss = criterion(output, cls);
  validLoss += loss.item();
  validAcc += boa.eval` (${output.argmax(1)}= =${cls}).sum().item()`;
});
Copy the code

eval(str)

Finally, why provide a method to dynamically execute Python expressions (single line)? This brings back Python’s advantage. In some very complex data processing scenarios, Python expressions can be expressed very easily, which greatly reduces code complexity. Let’s look at an example:

const line = (boa.eval`'\t'.join([str(x) for x in ${vec}]) `);
Copy the code

The above code would be converted to JavaScript:

vec.map(x= > x.toString()).join('\t');
Copy the code

Looks like it’s about what? Let’s look at the following example:

boa.eval`{u:i for i, u in enumerate(${vocab})} `;
boa.eval` [${char2idx}[c] for c in ${text}] `
boa.eval`${chunk}` [: - 1]
boa.eval`${chunk}[0: - 1:2] `
Copy the code

How about this? Does it feel like the above example can’t be done with a single line of JavaScript?

It’s worth noting, however, that JavaScript is stepping up in this area as well. Here’s a summary of some of the related standards TC39 is working on, including the Slice Notation above.

Going back to eval’s position, it’s like a complement to JavaScript. It allows us to use Python expressions to express things more easily, with a little bit of low-cost learning, before standards are settled. Here’s how eval is used. It accepts a “String”, but we usually use it through Template String. Here are two examples:

boa.eval('print("foobar")');
boa.eval(`print("${txt}") `);
Copy the code

After looking at the above two lines of code, they are relatively rare uses. The most common and effective way to use eval is to use Tagged Template String. As we saw at the beginning, we use eval directly after the Template String. The advantage of this is that the eval function accepts all the Template parameters. This allows us to combine JavaScript objects with Python expressions for a smoother experience, as follows:

const chunk = range(0.10);
boa.eval`${chunk}[0: - 1:2] `
Copy the code

This is done by passing chunk into the expression, using Python’s Slice Notation syntax to fetch the corresponding value, and then returning to the JavaScript world.