preface

Text and pictures from the network, only for learning, exchange, do not have any commercial purposes, copyright belongs to the original author, if you have any questions, please contact us to deal with.

Author: Luo Luo Pan

PS: If you need Python learning materials, please click on the link below to obtain them

Note.youdao.com/noteshare?i…

This paper mainly explains:

  • Tiktok video crawler
  • Video download

PS: Although many netizens scold her for not having the strength, others are really lucky ~

Tiktok video crawler

The packet is still captured by Fiddler. Now Tiktok has an encryption algorithm. Most codes on the Internet before cannot be used, so let’s take a look at the encryption fields.

https://aweme.snssdk.com/aweme/v1/aweme/post/?iid=40337863888&device_id=35102252294&os_api=18&app_name=aweme&channel=App %20Store&idfa=11926ED5-C282-4BBC-AF01-0E8C18120647&device_platform=iphone&build_number=23101&vid=177A5A79-D6F6-4A03-9352 - 57 c0681cdddc & openudid = 1 ee725d39e05794bcdc14537f8c1f4220c7d6fd5 & device_type = iPhone8, 1 & app_version = 2.3.1 & version_code = 2.3 . 1 & os_version = 11.4.1 & screen_width = 750 & aid = 1128 & ac = 21 & max_cursor WIFI&count = = 0 & min_cursor = 0 & user_id = 58554069260 & mas = bf53 01 7030d65155897d6fd1d7c97862dbca9722fea8c96d2b68de&as=a1858817de104b87435065&ts=1534297870Copy the code

As = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

import requests
import json

f = open('2.txt'.'w+',encoding='utf-8')

headers = {
    'Host': 'api.amemv.com'.'Accept': '* / *'.'Cookie': 'install_id=40337863888; login_flag=d6f29ec905af4bf1101199aa942c466f; odin_tt=a1e12dc3e4b92de77cccf6be1717377188f8aa7582f703c1391c8dc7d4a0df1b166119681af4277bd2cdc8aeb56000a7; sessionid=718df70f4e4964723cd1c8337c367b45; sid_guard=718df70f4e4964723cd1c8337c367b45%7C1534207148%7C5184000%7CSat%2C+13-Oct-2018+00%3A39%3A08+GMT; sid_tt=718df70f4e4964723cd1c8337c367b45; ttreq=1$ad10f98ec66ad6df5b86a7b1a613c77bb674236d; uid_tt=765536856bdc4f0f299b85dbc7338982'.'User-Agent': 'Aweme / 2.3.1 (iPhone; IOS 11.4.1; Scale / 2.00) '.'Accept-Language': 'zh-Hans-CN; q=1'.'Accept-Encoding': 'br, gzip, deflate'.'Connection': 'keep-alive'
}


def get_info(url):
    res = requests.get(url,headers=headers)
    json_data = json.loads(res.text)
    datas = json_data['aweme_list']
    for data in datas:
        desc = data['desc']
        download_url = data['video'] ['play_addr'] ['url_list'] [0]
        print(desc,download_url)
        f.write(desc+', '+download_url+'\n')

if __name__ == '__main__':
    fp = open('1.txt'.'r')
    for line in fp.readlines():
        get_info(line.strip())
Copy the code

Video download

On the Internet, this link is only valid for 10 minutes (no verification), so the crawler saves the video link first and then downloads it again.

import requests

def download_url(desc,url):
    global i
    res = requests.get(url)
    if len(desc) == 0:
        desc = str(i)
    f = open('video/'+desc+'.mp4'.'wb')
    f.write(res.content)
    i = i + 1

i = 1
fp = open('2.txt'.'r', encoding='utf-8')
for line in fp.readlines():
    desc = line.split(', ') [0]
    url = line.split(', ') [1].strip('\n')
    print(url)
    download_url(desc,url)
Copy the code