The recent HBO series Silicon Valley, which has taken the world by storm, includes a segment about the super fun iOS app Not Hotdog. You can even download it on the APP Store.

Is it a hot dog?

Inspired by this, I wanted to develop a “bot” that could do the same thing: users could upload any image and immediately get feedback telling you whether it was a hot dog or not. On top of that, my code is all in JS, and it’s time for the front end engineers to make a difference in ai/machine learning.

Implementation details

The APP is hosted on Twitter and is based on a Twitter Bot: any Twitter user can post an image, add “@isitahotdog” to the uploaded description, and get an immediate response. It’s as simple as adding a “#” descriptor to a common microblog in mainland China.

Hot Dog Detection APP

Don’t be fooled by appearances, and don’t be fooled by the “AI/machine learning” label. In fact, the implementation method and principle is very simple.

First up, I forked @Bryanebraun’s open source Twitter bot, which is based on NodeJS.

The official introduction is also very succinct:

This is a simple twitter bot, designed to retweet the contents of a twitter list.

Using this tool, my next job was to respond to tweets that mentioned “IsItAHotdog” (i.e., with the hashtag IsItAHotdog).

After installing the Tuiter NPM package, introduce the dependencies into the code and add:

 var tu = require('tuiter')(config.keys);
 function listen() {    
    tu.filter({        
        track: 'isitahotdog'    
    }, function(stream) {        
        console.log("listening to stream");
        stream.on('tweet', onTweet); })}Copy the code

Of course, we only processed tweets with images:

  if(tweet.entities.hasOwnProperty('media') && tweet.entities.media.length > 0)Copy the code

Finally, we included the results in our tweet response:

tu.update({
    status: "@" + tweet.user.screen_name + message,        
    in_reply_to_status_id: tweet.id_str    
}, onReTweet);Copy the code

Training model

The above just introduces the content of hijacking tweets and releasing replies. So how should the result of the reply be obtained? How do we know if the picture is a hot dog? This brings us to the most important step.

Those who are familiar with deep learning may know that we may need to collect pictures and use Keras to build the neural network commonly used by CNN. Keras is a neural network advanced package compatible with Theano and Tensorflow. It is highly modular and very fast and convenient to build a neural network with Keras.

These contents may not be much in Chinese, but if you are interested in them, I recommend them:

  • Comparative learning to build CNN RNN and other common neural networks with Keras
  • Basic Machine Learning and Deep Learning

But deep learning is something that most front-end engineers don’t know very well, so do we have to reinvent the wheel?

Don’t worry, we can start now! Here I will give you a little amity about AWS Rekognition. Our APP is also based on AWS Rekognition.

Amazon Rekognition is a service that lets you easily add image analysis capabilities to your applications. With Rekognition, you can detect objects, scenes, and faces, you can search for and compare faces, and you can identify inappropriate content in images. With Rekognition’s API, you can quickly add sophisticated visual search and image classification capabilities based on deep learning to your application.

In other words, “If you don’t understand machine learning, you can easily call several apis.”

Amazon Rekognition is based on proven and highly scalable deep learning technology, also developed by Amazon computer vision scientists, and is able to analyze billions of Prime Photos images per day.

This may be a little convoluted, but let’s look at the code, which is very simple:

 var params = {
     Image: { 
         Bytes: body
     },
     MaxLabels: 20.MinConfidence: 70
 };

 rekognition.detectLabels(params, function(err, data) {
    if (err) console.log(err, err.stack); // an error occurred
    else {
        console.log(data);           // successful response
        var isItAHotdog = false;
        for (var label_index in data.Labels) {
            var label = data.Labels[label_index];
            if(label['Name'] = ="Hot Dog") {
               if(label['Confidence'] > 85) {
                    isItAHotdog = true;
                    tweetBasedOnCategorization(tweet, true); }}}if(isItAHotdog == false) {
            tweetBasedOnCategorization(tweet, false); }}});Copy the code

I downloaded the picture attached to the tweet to my server machine, and then passed it to Rekognition through AWS Node SDK, and set the corresponding parameters, including confidence interval, etc. Finally, the final result is obtained in the callback.

The final result

Let’s take a look at a set of test results:

test

test

The development process for all of this is very simple, if you want to see the source code, I fork a copy and add Chinese annotations. Click here to view the source code.

This article is translated from Building Silicon Valley’s Hot Dog App in One Night.

Happy Coding!

Finally, a shameful wave of advertising:

I’ve been invited by gitChat to share. The form is similar to Zhihu Live, but I feel that this platform is less impetuous and more professional. The topic content is: how to access and apply ES6 to liberate development efficiency in the face of six-year history of front-end code.

I invited a veteran front-end expert, a community influencer@ YanHai mirrorClick with me for detailshere

Scan the QR code below on wechat to participate:

Scan this QR code

The content is introduced

PS: author Github warehouse, welcome to communicate through various forms of code.