Last year, Google released Google Neural Machine Translation (GNMT), a sequence-to-sequence model. It is now used in the Google Translate product system.

Although the improvement felt by consumers is not obvious, Google claims that GNMT has brought a huge leap forward in the quality of translation.

But Google clearly wants to do more than that. “GNMT’s influence is limited because outside researchers do not have access to a framework for training these models,” it said on its official blog.

How do you maximize the impact of this technology? There is only one answer – open source.

So last night, Google released TF-Seq2SEq, a Framework for Seq2Seq based on TensorFlow. Google says it has made it easier for developers to experiment with the SEQ2SEQ model and achieve top-notch results. In addition, tF-SEq2SEq’s codebase is clean and modular, preserving all test coverage and writing all functionality into files.

The framework supports various configurations of the standard SEQ2SEQ model, such as encoder/decoder depth, attention mechanism, RNN unit type, and Beam size. This versatility helps researchers find the optimal hyperparameters and makes it better than other frameworks. See The Google paper “Massive Exploration of Neural Machine Translation Architectures” for more details.

The figure above shows a translation model of SEQ2SEQ from Chinese to English. At each time step, the encoder receives a Chinese character with its previous state (black arrow) and generates an output vector (blue arrow). Next, the decoder generates the English translation word by word. At each time step, the decoder receives the previous word, the previous state, the weighted output sum of all encoders to generate the next English word. Lei feng reminds us that in Google’s execution, they use WordPieces to deal with obscure words.

According to Lei Feng, besides machine translation, TF-SEQ2SEQ can also be applied to other sequence-to-sequence tasks. Any task that needs to learn the order of outputs given the order of inputs. These include Machine Summarization, image capture, speech recognition, and dialog modeling. Google acknowledges that it has been careful to design the framework to maintain this level of versatility and provide user-friendly tutorials, pre-processing data, and other machine translation features.

In a blog post, Google said:

“We hope that you will use TF-SEQ2SEQ to accelerate (or get started) your deep learning research. We welcome your contributions to the GitHub repository. Have a list of open problems that need your help!”

GitHub address: github.com/google/seq2…

Making a repository: Google. Making. IO/seq2seq/NMT…