One, foreword

Hello everyone, as the saying goes, after learning new knowledge to apply, in the process of learning audio and video, do you have any questions, do not know what audio and video can be used to do. Here are a few examples, more familiar, blowing into the wind some of the scenes are: AI visual computing, AI face recognition. Detailed to some small areas, such as live streaming technology, camera monitoring pull flow; Other features include douyin beauty and filters, behind which digital makeup technology from the audio and video field is used. Thus, the application of audio and video technology has been applied to all aspects of our life.

Ii. Development background

I want to write this article because I have a friend who likes to check Douyin at ordinary times, and some videos are often set as undownloadable and saved by the author, so my friend will not be able to find them if he wants to watch them again next time. I have friends who want to download the works of the girl they secretly love. As a friend, OF course, I have the obligation to help him out. Finally, Two thousand years later, this small tool has been released. Due to the lack of time, I have no time to write the front page. The purpose of writing this article is purely based on learning. If it is abused by illegal elements for profit, it has nothing to do with me. Please take good care of the learning environment and remember not to involve me.

Third, core ideas

There are two core steps

1. To obtain the real address of Douyin according to the copied sharing link on Douyin, network programming technology is required to parse the real address of the video.

2. Then use FFMPEG to decode the network video stream and save it locally.

Four, the main technical points

1. I mainly use Java and some knowledge of network calls, such as Restemplate, a template method class in Spring Web, which mainly uses its two methods (headForHeaders, Exchange). You can also use other utility classes or implement network remote calls yourself.

2, JSON parsing using FastJSON, version number arbitrary, generally compatible.

StringUtils is a utility class that uses commons.lang3.

Ffmpeg pull streams use third-party dependencies javacV, version 1.4.3. For specific references depend on, follow or private message me.

5, if you are not very clear about the basic concepts of FFMPEG, basic concepts of audio and video, such as video frame, audio frame, bit rate and other basic knowledge, here I only say the application of technology, about the principle of the explanation, not to do too much redundant, a lot of online search, interested in their own to understand the following.

Get audio/video frames with the FFmpegFrameGrabber in JavacV. With this grabber, you can skip a bunch of complicated operations called by the native API, such as opening a video stream, finding a decoder, and determining audio and video frames.

An introduction/summary from the Internet

FFmpegFrameGrabber is used to capture/grab video images and audio samples. It encapsulates the functions of retrieving stream information, automatically guessing video decoding format, audio and video decoding API, and saving and returning the decoded pixel data (configurable pixel format) or audio data to Frame.

7, you can also use the FFmpeg command line to download. The command is as follows:

Ffmpeg-i https://xxx.mp4 -c copy -f FLV FLVCopy the code

However, this method requires ffMPEG to be installed on the deployer, so it is not considered for the time being.

8. Use FrameRecorder recorder in JavacV to encode the image pixels that have been decoded into the corresponding encoding and format to push out. Save to local is to push the stream to local files.

The following is a summary of the introduction of FrameRecorder by eguid

FrameRecorder is used to encapsulate, encode, stream, record, and save audio, video, and pictures. To take data out of a Frame from a FrameGrabber or FrameFilter and encode, encapsulate, and stream it. For easy comprehension and reading, FrameRecorder will be abbreviated as “recorder”.

Fifth, detailed ideas

1. Link parsing & interface parsing

(1) Java regular expressions extract urls from strings.

(2), use RestTemplate. HeadForHeaders () method to get the URI of a resource request header information, and only focus on the HTTP request header information.

(3) The URL extracted in the first step can be found in the browser simulation and will be redirected to a new address. The redirected address is obtained from the request header, that is, location is obtained from the header, and then the real ID of the video is obtained from the location.

(4) Obtain video information, such as playing information, author information and background music information, according to the real ID of the video and the interface of Douyin, and use JSON to parse out the URL of the playing address layer by layer.

2. Ffmpeg pulls and saves the stream

(1) Use FFMPEG to obtain the first frame of URL video frame and detect whether the video is empty.

(2) Create video stream recorder, set video parameters, resolution, format and output position.

(3), cycle to get video frames, using recorder frame by frame recording video frames.

Core code

1. Use re to extract url
@param text @return */ public static String pickURI(String text) {// public static String pickURI(String text) { Text = "GV: 5.1 / appearance give a person a kind of wonderful feeling of jiangnan % l % % song red horse https://v.douyin.com/e614JkV/ abdomen system Ci lian, open the Dou search, kan directly view video!" ; Pattern pattern = Pattern.compile("https?://[-A-Za-z0-9+&@#/%?=~_|!:,.;] +[-A-Za-z0-9+&@#/%=~_|]"); Matcher matcher = pattern.matcher(text); if (matcher.find()) { return matcher.group(); } return ""; }Copy the code
2. Initiate a network call and parse JSON to get the real address
public final static String DOU_YIN_WEB_API = "https://www.iesdouyin.com/web/api/v2/aweme/iteminfo/?item_ids="; * @param text * @throws FrameGrabber.Exception * @throws framerecOrder. Exception */ public static String douYin(String text) throws FrameGrabber.Exception, FrameRecorder.Exception { // String url = pickURI(text); RestTemplate client = new RestTemplate(); // HttpHeaders headers = client.headForHeaders(url); String location = headers.getLocation().toString(); String vid = StringUtils.substringBetween(location, "/video/", "/?" ); RestTemplate restTemplate = new RestTemplate(); HttpHeaders queryHeaders = new HttpHeaders(); QueryHeaders. Set (HttpHeaders.USER_AGENT, "Mozilla/5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"); HttpEntity<String> entity = new HttpEntity<>(queryHeaders); ResponseEntity<JSONObject> response = restTemplate.exchange(DOU_YIN_WEB_API + vid, HttpMethod.GET, entity, JSONObject.class); if(response.getStatusCodeValue() ! = 200) { return ""; } JSONObject body = response.getBody(); assert body ! = null; List<JSONObject> list = JSONArray.parseObject(body.getJSONArray("item_list").toJSONString(), new TypeReference<List<JSONObject>>(){}); if(list.size() == 0) { return ""; } JSONObject item = list.get(0); JSONObject video = item.getJSONObject("video"); JSONObject playAddr = video.getJSONObject("play_addr"); JSONArray urlList = playAddr.getJSONArray("url_list"); List<String> urlListArr = JSONArray.parseObject(urlList.toJSONString(), new TypeReference<List<String>>(){}); if(urlListArr.size() == 0) { return ""; } return urlListArr.get(0); // videoconvert. record(finalAddr, "/home/yinyue/upload/ hongma.flv "); // Videoconvert. record(finalAddr, "/home/yinyue/upload/ hongma.flv "); }Copy the code
3. Ffmpeg pulls the stream and saves it
/** * dump video stream * @param input * @param outFile * @throws FrameGrabber.Exception * @throws framerecOrder. Exception */ public  static void record(String input, String outFile) throws FrameGrabber.Exception, FrameRecorder.Exception { FrameGrabber grabber = new FFmpegFrameGrabber(input); grabber.start(); OpenCVFrameConverter.ToIplImage converter = new OpenCVFrameConverter.ToIplImage(); Frame frame = grabber.grab(); opencv_core.IplImage image = null; If (frame == null) {system.out.println (" the first frame is empty, please check the video source "); return; } image = converter.convert(frame); FrameRecorder recorder = FrameRecorder.createDefault(outFile, frame.imageWidth, frame.imageHeight); recorder.setVideoCodec(AV_CODEC_ID_H264); recorder.setFormat("flv"); recorder.setFrameRate(25); recorder.setGopSize(25); recorder.start(); Frame saveFrame; while((frame = grabber.grab()) ! = null) { saveFrame = converter.convert(image); // EnumSet< frame.type > videoOrAudio = saveframe.getTypes (); // Record the video recorder. Record (saveFrame); } recorder.close(); }Copy the code

7. Run screenshots

After the operation, the downloaded video file is successfully generated locally

Eight, the author’s experience

We are both lucky and sad to have been born in an age of ever-changing technology. In such a vast and endless tide of knowledge, it is difficult to guarantee omnipotence and perfection. Some people focus on the algorithm, some people focus and data processing, and the ability not line, but the strong ability of theory, such as the famous physicist Yang zhenning, some people focus on how to apply to the ground, is committed to technology was applied to social life, so, if the paper is useful to you, please feel free to appreciate, if you feel the content is too shallow or make you feel unwell, And be silent, and save respectability for one another.