Professional search, parents good helper

Long did a picture recognition project, there was a function is the whole problem identification, namely incoming math problem screenshots, can recognize the image content of by OCR technology, but only to identify the text at that time, did not make further processing, now want to practicality is not strong, after all, we need more of is their thinking, Rather than let AI read the question dry (the question dry text, I know, connect up I don’t know how to start to do = =), recently just have time, so try to have a baby friend to do a search artifact.

In view of the good experience of using Youdao Wisdom Cloud in the development of whole question recognition before, I opened its official document again and found the open API of photo question search service, and made a simple batch question search demo with ease and familiarity. The development process will be shared below.

Preparations for calling the API

Detailed introduction of the development process

The following describes the specific code development process.

The parameters received by the API are simple:

Sign is generated as follows: signType=v2; Sign =sha256(app ID+input+salt+curtime+ app key) Input = the first 10 characters of Q + q length + the last 10 characters of Q (when Q length is greater than 20) or input=q character string (when Q length is less than or equal to 20).

Note that the API has the following requirements for the title image:

The Demo development:

This demo is developed using python3, including maindow. Py, questionclass. py, OcrQuestion.

1. Interface part:

   UI part is relatively simple, the main function is to select the picture to be subject, select the correction result storage path. The layout code is as follows:

   root=tk.Tk()
   root.title(" youdao ocr question test")
   frm = tk.Frame(root)
   frm.grid(padx='50', pady='50')
   
   # Select topic and save result button
   btn_get_file = tk.Button(frm, text='Select subject picture', command=get_files)
   btn_get_file.grid(row=0, column=0, ipadx='3', ipady='3', padx='10', pady='20')
   text1 = tk.Text(frm, width='40', height='10')
   text1.grid(row=0, column=1)
   btn_get_result_path=tk.Button(frm,text='Select search result path',command=set_result_path)
   btn_get_result_path.grid(row=1,column=0)
   text2=tk.Text(frm,width='40', height='2')
   text2.grid(row=1,column=1)
   
   
   # Search button
   btn_sure=tk.Button(frm,text="Search the topic",command=search_question_files)
   btn_sure.grid(row=4,column=1)
   
   root.mainloop()
   Copy the code

   Where, start the binding event search_question_files() of button btn_sure to search questions according to the question photos, and open the result storage path after completion:

   def search_question_files() :
       question.start_ocr()
       os.system('start '+question.result_path)
   Copy the code

2. QuestionClass.py

   Here, mainly with the logic of UI, call the search method.

   Start by defining a class Question:

   class Question() :
       def __init__(self,file_paths,result_path) :	
           self.file_paths=file_paths		# Title photo storage path
           self.result_path=result_path	# result path
   Copy the code

   The start_ocr() method calls the connect() method to search the questions in turn and save the results.

   def start_ocr(self) :
       for file_path in self.file_paths:
           result=connect(file_path)
           print(file_path)
          self.save_result_format(file_path,result)
   Copy the code

   The result obtained from ocrquestion.py’s connect method is json format, save_result_format() method parses the interface obtained from the interface, reformats, and saves the result to HTML:

       def save_result_format(self,file_path,result) :
           result_file_name=os.path.basename(file_path).split('. ') [0] +'_result.html'
           f=open(self.result_path+'/'+result_file_name,'w',encoding='utf-8')
           result_json= json.loads(result)
           if result_json['errorCode'] = ='0':
               data=result_json['data']
               questions=data["questions"]
               text=data["text"]
               f.write("
   
   "+text)
               i=0
               for answers in questions:
                   i=i+1
                   subject="Subject:"+answers["subject"] +"<br>"
                   answer="Answer:" +answers["answer"] +"<br>"
                   analysis="Analysis:"+answers["analysis"] +"<br>"
                   knowledge="Knowledge point:"+answers["knowledge"] +"<br>"
                   print(subject+answer+analysis+knowledge)
                   result_each="< H3 > Results"+str(i)+"<br></h3>"
                   result_each=result_each+subject+answer+analysis+knowledge+"
   
   ================= this is a delimiter ============
   
   "
                   f.write(result_each)
           else:
               f.write("result error code:"+result_json['errorCode'])
   Copy the code

3. OcrQuestion.py

   Ocrquestion.py encapsulates the methods that request the OCR search API, the main of which is connect() :

   def connect(pic_path) :
       f = open(pic_path, 'rb')  Open the image file in binary mode
       q = base64.b64encode(f.read()).decode('utf-8')  Read the content of the file and convert it to Base64 encoding
       f.close()
   
       data = {}
       data['q'] = q
       data['signType'] = 'v2'
       curtime = str(int(time.time()))
       data['curtime'] = curtime
       salt = str(uuid.uuid1())
       signStr = APP_KEY + truncate(q) + salt + curtime + APP_SECRET
       sign = encrypt(signStr)
       data['appKey'] = APP_KEY
       data['salt'] = salt
       data['sign'] = sign
   
       response = do_request(data)
       result=response.content.decode('utf-8')
       print(result)
       return result
   Copy the code

   Example API response results

   {
       "data": {"questions":[
               {
                   "score":0.9875."answer":"D"."subject":"History"."id":"a9db8f1252778836c99204e5cf9d7738"."analysis":""."type":""."content":"Scholars believe that the May 4th Movement marked the great awakening of Chinese national consciousness, as shown in < BR />A. Modern national democratic thought began to spread B. Opened the curtain of the struggle against imperialism and feudalism. Opened the ideological enlightenment of modern China. Broad mobilization and participation of all sectors of the population."."knowledge":""
               },
               {
                   "score":0.9875."answer":During the May 4th Movement, students, workers, and businessmen were all involved in the mobilization and participation of people from all walks of life. < BR > The idea of national democracy in China began to spread around the Hundred Days' Reform. < BR > The May 4th Movement was a patriotic movement against imperialism and feudalism, but it was not the beginning of the struggle against imperialism and feudalism. < BR > In modern Times, China spread the idea of democracy during the New Culture Movement and carried out ideological enlightenment. 
   
    D."."subject":"History"."id":"2f8b9c814f064131cc7b2bead54f15da"."analysis":The May 4th Movement. The Significance of the May 4th Movement. Test students' ability to memorize and use basic knowledge.."type":"Multiple choice"."content":"Scholars believe that the May 4th Movement marked the great awakening of Chinese national consciousness, which is manifested in () question_option< p style = "max-width: clear; clear: both; min-height: 1em; 
   
   D. The widespread mobilization and participation of people from all walks of life 
           "."knowledge":"/ The history of human political civilization/The trend of anti-aggression and democracy in modern China/the May Fourth Movement"
               },
               {
                   "score":0.9875."answer":
   D
    
    
    
   ."subject":"History"."id":"48a28969d882f20b04ffec15c8f1b94c"."analysis":< P > During the May 4th Movement of 1898, China's national democratic ideology began to spread. The May 4th Movement was a patriotic movement against imperialism and feudalism, but it was not the beginning of the struggle against imperialism and feudalism. In modern China, democracy was spread and enlightenment was carried out during the New Culture Movement. During the May 4th Movement, students, workers and businessmen were all involved. 
    
    
   "."type":"Multiple choice"."content":"
    Scholars believe that the May 4th Movement marked the great awakening of Chinese national consciousness, as shown in 
    
    question_option< p style = "max-width: clear; clear: both; min-height: 1em; 
   
   D. The widespread mobilization and participation of people from all walks of life 
           "."knowledge":"/ The history of political civilization of mankind/The trend of anti-aggression and Democracy in modern China/The First National Congress of the Communist Party of China"}]."text":"Scholars believe that the May 4th Movement marked the great awakening of China's national consciousness, manifested in (A) the spread of modern national democratic thought, (B) the prelude to the anti-imperialist and anti-feudal struggle, (C) the ideological enlightenment of modern China. Broad mobilization and participation of all sectors of the population."
       },
       "errorCode":"0"
   }
   Copy the code

   The response result is output in JSON format, including fields as shown in the following table:

Results show

Demo Operation Demonstration

Check out the results:

Math search results display:

History questions search results

conclusion