“Hand in Hand to Learn NLP” is a series of practical programs based on the Paddle LP. This series by baidu many senior engineers, meticulously provided from the word vector, the training of language model, the information extraction, the emotional q&a, structured data q&a, text analysis and text translation, machine with transmission and dialogue system and so on practice project of the whole process, designed to help developers more comprehensive grasp clearly baidu fly blade framework in the field of NLP usage, And can draw inferences from one example, flexibly use the fly paddle frame and paddle LP to carry out NLP deep learning practice.
In June, Baidu Feiojiao and the Natural Language Processing Department jointly launched 12 NLP video lessons, in which the practical project was explained in detail. Watch the course replay please stamp: https://aistudio.baidu.com/ai… Welcome to the QQ group of the course (group number :758287592) to communicate ~~
Chatbots’ Past Lives
Between 1964 and 1966, Joseph Weizenbaum, a German-American computer scientist at MIT’s Artificial Intelligence Laboratory, developed Eliza, the first chatbot in history. Eliza takes its name from George Bernard Shaw’s Flower Girl, in which Eliza learns to communicate with high society and becomes the envy of the Princess Royal of Hungary at an embassy ball. As the world’s first chatbot, Eliza was given a dramatic connotation by its authors. Although there were some basic digital language generators (programs that could output some coherent text), ELIZA was the first program specifically designed to interact with people. The user could use a typewriter to type in human natural language and get a response from the machine. As Mr Weissenbaum explains, Eliza makes “a conversation between a person and a computer possible”. As deep learning technology continues to evolve, chatbots are getting smarter and smarter. We can complete some mechanical questions and answers through the robot, but also in the leisure time and intelligent robot dialogue, their appearance makes life more colorful. Now a simple chatbot can be created by flying a paddle and combining it with Wechaty. The following picture shows the WeChat chatty robot demo based on PaddleHub + Wechaty. The message received by WeChat is obtained through Wechaty, and then the Plato-mini model of PaddleHub is used to generate new dialogue text according to the context of the dialogue. Finally, it is sent in the form of WeChat message to realize the interaction of small talk.
Below is a demo of WeChat emotion recognition robot based on PaddlenLP + Wechaty. The message received by WeChat is obtained through Wechaty, and then the PaddlenLP TextCNN model is used to make emotional judgment on the input text, and finally the text is returned in the form of WeChat message to realize the recognition of text emotion.
Interested students can refer to this demo in WeChat implementation on a robot emotion recognition oh ~ demo links: https://github.com/mawenjie87…
Isn’t that interesting? If that’s not enough for you, please come and sign up for the creative competition brought by PaddlePaddle in collaboration with Wechaty, an open source chatbot framework, and the designer community Mixlab. PaddlenLP provides you with a wealth of deep learning pre-training models, and Wechaty also provides you with a convenient Chatbot building SDK. You can use PaddlenLP to automatically write poems, rainbow fart, name naming, automatic couplets and other fun functions with reference to existing demos. Competition registration link: https://aistudio.baidu.com/ai… Today we’re going to use the Paddle LP to achieve poetic reparation and a simple chatty robot. Come on! A quick practice
PaddlenLP provides the generate() function for generative tasks, embedded in all of the PaddlenLP generative models. Greedy Search, Beam Search and Sampling decoding strategies are supported. Users only need to specify the decoding strategy and the corresponding parameters to complete the predictive decoding and get the Token IDS and probability scores of the generated sequence.
2.1 A small example of the GPT model using the generation API
1. Loading paddlenlp. Transformers. GPTChineseTokenizer used for data processing
Before input to the pre-training model, text data needs to be transformed into Feature through data processing. This process usually includes steps such as word segmentation, token to id, add special token, etc.
The PaddlenLP has built-in tokenizer for various pre-trained models. Specify the name of the model you want to use to load the corresponding tokenizer. Call the __call__ method of the GPTChineseTokenizer to turn what we say into input acceptable to the model.
from paddlenlp.transformers import GPTChineseTokenizer
Sets the name of the model you want to use
model_name = ‘gpt-cpm-small-cn-distill’
tokenizer = GPTChineseTokenizer.from_pretrained(model_name)
Import paddle user_input = “I have a pot of wine in the room. Raise a glass to the moon,”
Convert the text to IDS
input_ids = tokenizer(user_input)[‘input_ids’]
print(input_ids)
Change the converted ID to tensor
input_ids = paddle.to_tensor(input_ids, dtype=’int64′).unsqueeze(0)
2. Use Paddlen LP to load the pre-training model with one key
PaddlenLP provides GPT,UnifiedTransformer and other Chinese pre-trained models, which can be loaded with one key by the name of the pre-trained model.
GPT takes the encoder of Transformer Decoder as the basic component of the network and adopts one-way attention mechanism, which is suitable for long text generation tasks.
Paddlen LP currently offers a variety of Chinese and English GPT pre-training models. This time we are using a small Chinese GPT pre-training model.
from paddlenlp.transformers import GPTLMHeadModel
One-click loading of Chinese GPT model
model = GPTLMHeadModel.from_pretrained(model_name)
Call the generate API up to text
ids, scores = model.generate(
input_ids=input_ids,
max_length=16,
min_length=1,
decode_strategy='greedy_search')
generated_ids = ids[0].numpy().tolist()
Converts the generated ID to text using Tokenizer
generated_text = tokenizer.convert_ids_to_string(generated_ids)
print(generated_text)
The pair made three.
As you can see, the generation is pretty good, and the use of the generative API is very easy.
2.2 UnifiedTransformer
Models and generative APIs complete the small talk
1. Loading paddlenlp. Transformers. UnifiedTransformerTokenizer used for data processing
UnifiedTransformerTokenizer invocation style is the same as GPT, but data processing of the API is slightly different.
Call UnifiedTransformerTokenizer dialogue_encode method can into a model of acceptable input to what we say.
from paddlenlp.transformers import UnifiedTransformerTokenizer
Sets the name of the model you want to use
model_name = ‘plato-mini’
tokenizer = UnifiedTransformerTokenizer.from_pretrained(model_name)
User_input = [‘ Hello, how old are you this year ‘]
Call the dialogue_encode method to generate the input
encoded_input = tokenizer.dialogue_encode(
user_input,
add_start_token_as_response=True,
return_tensors=True,
is_split_into_words=False)
2. Use Paddlen LP to load the pre-training model with one key
As with GPT, you can invoke the UnifiedTransformer pre-trained model with one click.
UnifiedTransformer takes the encoder of Transformer as the basic component of the network, adopts flexible attention mechanism, and adds special tokens of different dialogue skills into the model input, so that the model can support small talk dialogue, recommendation dialogue and knowledge dialogue simultaneously.
PaddlenLP currently offers three Chinese pre-training models for UnifiedTransformer:
Unified_Transformer – 12L-CN This pre-training model is trained on a large Chinese session data set. Unified_Transform-12L-CN-Luge This pre-training model is obtained by fine-tuning Unified_Transform-12L-CN on the thousan-word dialogue data set. The Plato-Mini model was pre-trained using a billion levels of Chinese chat data.
from paddlenlp.transformers import UnifiedTransformerLMHeadModel
model = UnifiedTransformerLMHeadModel.from_pretrained(model_name)
Next we pass the processed input into the generate function and configure the decoding policy.
Here we use the decoding strategy of TOPK plus SAMPLING. That is to sample the k results with the highest probability.
ids, scores = model.generate(
input_ids=encoded_input['input_ids'],
token_type_ids=encoded_input['token_type_ids'],
position_ids=encoded_input['position_ids'],
attention_mask=encoded_input['attention_mask'],
max_length=64,
min_length=1,
decode_strategy='sampling',
top_k=5,
num_return_sequences=20)
from utils import select_response
Simply choose the best response based on probability
result = select_response(ids, scores, tokenizer, keep_space=False, Num_return_sequences =20) print(result) [” Sequences “, num_return_sequences=20)
PaddleNLP example provides a complete dialogue system is set up in the code (hci), are interested can go to try oh ~ human-computer interaction in terminal address: https://github.com/PaddlePadd… It’s fun to try it out. Xiaobian strongly recommends beginners refer to the above code and knock it again, because only in this way can you deepen your understanding of the code. The corresponding code of the project: https://aistudio.baidu.com/ai… More PaddleNLP information, welcome to visit after making some star collection experience: https://github.com/PaddlePadd…
Baidu AI developer community, https://ai.baidu.com/forum, for developers across the country to provide a platform of communication, share, unriddling, let developers no longer “alone” in the research and development on the road, through constant communication and discussion to find a better technical solutions. If you want to try a variety of artificial intelligence technologies and explore application scenarios, join the Baidu AI community as soon as possible. Everything you think about AI can be realized here!
Scan the QR code below, add the little assistant WeChat “JD card, small custom surrounding, mysterious gift box, suitcase” and other benefits you can get ~
