Suck the cat with code! This paper is participating in[Cat Essay Campaign]
It all started when I took my two kittens to get a physical for their annual vaccinations
The doctor said that the RBC, HGB and HCT indexes were abnormal, and she was dehydrated. She asked if she didn’t drink much water at home, because there were two cats at home, and the water in the water dispenser was consumed all the time, but you didn’t know which cat drank it. The doctor said it was not clear, and then the doctor gave me this tool
The doctor said to water him with this if he doesn’t drink!
Go back to it filled a syringe, scene comparable to kill a cat….
This job is unworkable
Is there a way to monitor their drinking? If you don’t drink enough water, and cats can’t talk can you find out if you’re sick and don’t drink enough water? Because there are many children in the house. If the others drink normally and one does not drink, is there any way to detect it in time?
👨 : you also have the face to say you know your own worth??
Looks like it’s just a monitoring tool to monitor their drinking
Did you drink any water today? v0.0.1
The framework is as follows
Preparing hardware Devices
What does the device acquisition side use? Old cell phones? Raspberry pie? Other cheap IoT devices?
Based on cost considerations and hardware expansion, I chose the ESP32 CAM universal taobao price 25-35 large bunch looks like this
The worst thing about this board is that it has no USB port. As a garbage junker, I dug out a CP2102 connector to connect it
After the physical connection is long like this (it is recommended to buy CP2102 compatibility is better, taobao is another CH340 cheaper but worse compatibility
Server device
What does the server use? I pulled out all the gray eating suits I’ve been collecting for years
I will choose whoever the cockerel points to
Software Part scheme
Arduino bar is used on the device side, and there are abundant online resources
Pytorch for AI, MobileNet or SqueezeNet for model,
MNN not only performs well on mobile phones, but also runs on raspberry PI
The server part can use ant Eggjs front-end display or App bar, after all, I am a down and out Android development 🐶 ~
Well, the overall architecture can be realized in theory ~ open!
The device end
The embedded system uses Arduion. It is a little difficult to use the official IDE. Here, VSCode+PlatformIO is used for development
The camera configuration
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sscb_sda = SIOD_GPIO_NUM;
config.pin_sscb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
config.pixel_format = PIXFORMAT_JPEG;
config.frame_size = frameSize;
config.jpeg_quality = 10;
config.fb_count = 1;
bool ok = esp_camera_init(&config) == ESP_OK;
Copy the codeThen call the photo API to take the photo
camera_fb_t *frame_buffer = esp_camera_fb_get(a);Copy the codeHow do you know if a kitten is drinking water? Add an infrared body sensor module?
There’s a bunch of them under five on Universal Taobao that look like this
It can sense human pyroelectric, when people or animals pass by it will output a high level of output interface
Deployed in the cat door to drink water on the ground (limited funding ~ 😭 a little ugly
Actual test to find took so many photos due to the human body infrared sensing its sense Angle to 120 degree, distance of 3 ~ 7 meters, then due to the relatively small home, drink water place in the hallway when someone passed by, it will also take pictures with integration module is not adjusting circuit parameters, if plus obstructions is more troublesome. Looks like we need to adjust the plan
How to do? Or just send all photos to the server for testing? If you take a picture every 5 seconds and you have 1.7 watts of photos to check over the course of a day, it’s not going to be environmentally friendly even though the Raspberry PI only consumes 5 watts….
The Mi camera at home has the motion detection function, that is, it can record when an object moves, basically comparing the difference between the two frames before and after. If the change is large, it indicates that an object has moved.
Can we do the same? Upload the photos to raspberry PI for detection when large changes are detected in the two frames before and after.
Motion detection for ESP32 Cam
First let’s see if it can take grayscale images, which is the image format it supports
typedef enum {
PIXFORMAT_RGB565, // 2BPP/RGB565
PIXFORMAT_YUV422, // 2BPP/YUV422
PIXFORMAT_GRAYSCALE, // 1BPP/GRAYSCALE
PIXFORMAT_JPEG, // JPEG/COMPRESSED
PIXFORMAT_RGB888, // 3BPP/RGB888
PIXFORMAT_RAW, // RAW
PIXFORMAT_RGB444, // 3BP2P/RGB444
PIXFORMAT_RGB555, // 3BP2P/RGB555
} pixformat_t;
Copy the codeSo you can see that PIXFORMAT_GRAYSCALE is the format that we want, why do we want this format? Because it’s not only a small amount of data but it’s also a nice format to work with which is an image is a two-dimensional array representation that’s wide x long
How to compare whether two grayscale images have changed if all the pixels are traversed and compared to judge
- It’s a lot of calculation
- It’s a poor generalization
Therefore, we can reduce the sampling rate to reduce the amount of comparison calculation, and at the same time, perform block comparison to improve generalization ability
Reduced sampling rate
Let’s say the image resolution is 240 * 320 and in grayscale format that’s 1BPP per pixel point 1 byte that’s 76800 array length data and you can think of that as the first line of the image the first 240 bytes, The second action is the second 240 bytes. If you want to reduce sampling by 10 times, map 240 * 320 to 24 * 32. The rough code implementation is to map 10 * 10 of 240 * 320 to a point of 24 * 32
const uint16_t x = i % 240;
const uint16_t y = floor(i / 320);
const uint8_t block_x = floor(x / 10);
const uint8_t block_y = floor(y / 10);
const uint8_t pixel = frame_buffer->buf[i]
current_frame[block_y][block_x] += pixel;
Copy the codeAfter that you need to set current_frame[block_y][block_x] / 100; The value of this single pixel contains the general information of the original 10 * 10 and then you just have to compare the difference of 24 * 32. Each point can define a threshold, for example, if the difference is more than 25%, the block has changed. If the proportion of all the blocks has changed by more than 10%, the two frames have changed.
The service side
To install Nodejs, refer to Google. To install EggJS, refer to Google.
Upload and statistics interface is a simple CURD, but there are some problems in uploading pictures here to briefly record
My bodyParser was originally configured as Text something like this
config.bodyParser = {
formLimit: '8mb'.enableTypes: ['json'.'form'.'text'].extendTypes: {
text: ['* / *']}}Copy the codeThe body received was 10 bytes smaller than the content-Length of the header. The image received could not be opened on the computer. I wrote a Python Flask demo to receive the image and found it was OK. The probability of this situation should be that there are some problems with binary encoding as long as the content sent below base64 should be no problem, but after base64 memory will be about 30% larger than the device end of the memory embarrassment (ESP32 official total of about 520K, You can use this library to retrieve raw data (‘raw-body’) and adjust the upload code
async upload() {
const { ctx } = this;
const buffer = await getRawBody(ctx.req, { length: ctx.request.headers['content-length']})const time = sd.format(new Date(), 'YYYY-MM-DD_HH-mm-ss');
try {
fs.writeFileSync('./file-data/' + time + ".jpg", buffer);
} finally {
}
ctx.body = {
"code": 1001."msg": "ok"}}Copy the codeThe picture is finally working
AI
The environment
PyTorch installs my own Google, but I’m running a black Apple N card CUDA environment on my machine and it’s going to take me a long time to do that
To prepare data
Get the training data, “Two cat photos,” all over your phone! Temporarily did not find a good marking method can only be manually played each 1 ~ 200 pieces should be enough, (PS: our home cat a black a gray average color value should be ok? What if I get a new cat next year? !!!!!
Training data in the following two folders respectively into the master and child photos
‘
The surface
Each prepared about 100
Preprocessing data
The main thing is to press the image size down to 224 and then label it according to the file path. If the path has Naonao, lable is defined as 1
# coding:utf8
import os
from PIL import Image
from torch.utils import data
import numpy as np
from torchvision import transforms as T
class ChaochaoNaonao(data.Dataset) :
def __init__(self, root, transforms=None, train=True, test=False) :
self.test = test
imgs = []
for rootDir, dirs, files in os.walk(root):
for fileItem in files:
if "DS_Store" in fileItem:
continue
imgs.append(os.path.join(rootDir, fileItem))
imgs_num = len(imgs)
self.imgs = imgs
if transforms is None:
normalize = T.Normalize(mean=[0.485.0.456.0.406],
std=[0.229.0.224.0.225])
if self.test or not train:
self.transforms = T.Compose([
T.Resize(224),
T.CenterCrop(224),
T.ToTensor(),
normalize
])
else:
self.transforms = T.Compose([
T.Resize(256),
T.RandomResizedCrop(224),
T.RandomHorizontalFlip(),
T.ToTensor(),
normalize
])
def __getitem__(self, index) :
""" Return the data of an image """
img_path = self.imgs[index]
print(img_path)
if self.test:
label = int(img_path.split('/')[-1].split('. ') [0])
else:
If there is naonoa in the path, it is defined as 1. Noise is defined as 0
label = 1 if 'naonao' in img_path.split('/')[-2] else 0
data = Image.open(img_path)
data = self.transforms(data)
return data, label
def __len__(self) :
return len(self.imgs)
Copy the codeDefine the network
Here select Squeezenet is just a dichotomy should be anything 😄
from torchvision.models import squeezenet1_1
from models.basic_module import BasicModule
from torch import nn
from torch.optim import Adam
class SqueezeNet(BasicModule) :
# change to binary
def __init__(self, num_classes=2) :
super(SqueezeNet, self).__init__()
self.model_name = 'squeezenet'
self.model = squeezenet1_1(pretrained=True)
self.model.num_classes = num_classes
self.model.classifier = nn.Sequential(
nn.Dropout(p=0.5),
nn.Conv2d(512, num_classes, 1),
nn.ReLU(inplace=True),
nn.AvgPool2d(13, stride=1))def forward(self,x) :
return self.model(x)
def get_optimizer(self, lr, weight_decay) :
return Adam(self.model.classifier.parameters(), lr, weight_decay=weight_decay)
Copy the codeLoading data training
# train
for epoch in range(opt.max_epoch):
for ii,(data,label) in tqdm(enumerate(train_dataloader)):
# train model
input = data.to(opt.device)
target = label.to(opt.device)
optimizer.zero_grad()
score = model(input)
loss = criterion(score,target)
loss.backward()
optimizer.step()
loss_meter.add(loss.item())
if (ii + 1)%opt.print_freq == 0:
vis.plot('loss', loss_meter.value()[0])
Enter debug mode
if os.path.exists(opt.debug_file):
import ipdb;
ipdb.set_trace()
print("loss : %s " % loss_meter.value()[0])
model.save()
Copy the codeAfter 50 times of the epoch, the final loss was about 0.2, and the fitting effect was not tested directly
Test some data all right (after all a black a grey AI should be easy to learn 😆
The model was pretty small at 2.9 meters
Deploy to raspberry PI
Pytorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch install pyTorch Follow the steps to complete the installation
Just use the model trained above and use the validation script
App
Search online for Flutter development tutorials.
Network requests use dio pub.dev/packages/di…
Charts_flutter pub.dev/packages/ch…
Simply heap two pages
Specific code and so on the subsequent collation of open source bar
The end of the
👩 : the doctor all said this cat does not drink water you do this have what use can let it drink more water??
I:… Hospital did not misdiagnose, this cat really does not like to drink water!! Anyway, I have to go get some water
Make trouble 🐱 : meow! Meow!!! Meow!!! Meow!!! Kill the cat! Are you demons?!
reference
Github.com/fengdu78/Co…
Github.com/chenyuntc/p…
Github.com/espressif/e…
eggjs.org/zh-cn/
flutter.dev/