This article was first published on:Walker AI

Recently, I have been working on a cloud platform for Android devices. The device cloud platform is not unfamiliar to everyone, with common functions such as remote monitoring operation of a mobile phone, uploading files, checking logs and so on. But when you want to use cloud mobile phone to watch video, play games, listen to music, you will find that now on the market of cloud mobile phone, no one is a voice transmission function.

After discovering this situation, I came up with the idea of making this feature. What should I do? Before doing this, I did a little research to find out if there are any ready-made solutions and see if I can bring some references to myself.

After several days of research, I will tell you the general results.

Voice transmission is limited, but it can still be done.

1. Comparison of existing schemes

Equipment microphone Third-party screen casting software Sndcpy
The sound source Mobile phone microphone Mobile phone microphone Application streams its own data
The effect It’s subject to external interference Will interfere, but there are also algorithms to provide noise reduction Depends on the audio data itself
Application restrictions Other applications call the microphone stream Other applications call the microphone stream Applications that have not been granted permissions
The android version any any Android 10 or above

2. Program selection

In fact, it is not hard to see, the third-party screen casting software is also the microphone of the device, just because the data through the software client first, before the audio playback, their multi-purpose algorithm processing for a noise reduction (burning goose algorithm noise reduction is spicy chicken). But have you ever thought about his practical effect? Imagine a typical usage scenario as a device cloud platform:

Figure 1. Group control picture (material from network)

Such a dense equipment, such a scene, rely on external capture sound, one is easy to say, more than a few, that effect who can stand? And if the phone itself has an application that requires the use of a microphone, the effect is self-evident. Therefore, the use of equipment microphone such a scheme, as a platform, is certainly not!

If the microphone doesn’t work, does that mean there’s no other way out? My gut tells me of course not. If there was a way to capture the app’s media stream directly, rather than recording it with an external voice, it would be much better! This led me to snDCPy, an open source project.

3. Program implementation

3.1 Project Introduction

Sndcpy making address

There are detailed instructions in the introduction of the project, so I will just make a brief overview:

(1) Only support Android 10 or above

(2) It can run across system platforms

(3) You can use VLC to listen to connected devices

For the first point, which is essentially for the SDK version that has the corresponding interface, in the Android app configuration, the switch that allows the sound stream to be listened to is turned on, so that the app can get the sound.

(Qq Music, for example, has been tested and Google Chrome has been found to be a no-no.)

3.2 Procedure

(1) Download the built zip file in the project (if there is no ADB, download the with-ADB version, and configure the environment, here will not talk about how to configure the environment);

(2) Connect the mobile device to the decompression directory./ sndCPy, with the device number in the case of multiple devices;

(3) Confirm permission on the phone;

(4) Download and install VLC;

(5) When running VLC, you may encounter an error for the first time, but it doesn’t matter, just try it again.

4. Integrate

It can be used, but as for the project of cloud platform, the customer needs to play and hear the device through web page, and this way can only be heard through VLC, which obviously does not meet the requirements of use. As streaming media real-time network communication, we should naturally think of making it into the form of Websocket, which also needs to realize the existing socket service.

However, if you look closely at the project’s documentation, you will see that what we want can actually be used. There are three files in the compressed package, among which the. Bat file, let’s open it and have a look:

#! /bin/bash set -e ADB=${ADB:-adb} VLC=${VLC:-vlc} SNDCPY_APK=${SNDCPY_APK:-sndcpy.apk} SNDCPY_PORT=${SNDCPY_PORT:-28200} serial= if [[ $# -ge 1 ]] then serial="-s $1" echo "Waiting for device $1..." else echo 'Waiting for device... ' fi "$ADB" $serial wait-for-device "$ADB" $serial install -t -r -g "$SNDCPY_APK" || { echo 'Uninstalling existing version first... ' "$ADB" $serial uninstall com.rom1v.sndcpy "$ADB" $serial install -t -g "$SNDCPY_APK" } "$ADB" $serial forward tcp:$SNDCPY_PORT localabstract:sndcpy "$ADB" $serial shell am start com.rom1v.sndcpy/.MainActivity echo "Press Enter once audio capture is authorized on the device to start playing..." read dummy "$VLC" -Idummy --demux rawaud --network-caching=0 --play-and-exit tcp://localhost:"$SNDCPY_PORT"Copy the code

It was found that the TCP service was being transferred using the ADB command. In order to facilitate direct operation, here I directly list the commands and steps, follow this step, android device transmission problem can be solved:

(1) Save sndcpy.apk to /data/local/ TMP. Adb push sndcpy.apk /data/local/ TMP is recommended

Adb forward TCP :port localAbstract :sndcpy

(3) the adb shell am start com. Rom1v. Sndcpy /. MainActivity

(4) Start a socket locally, bind the port defined by the first port, if the returned data is not null, congratulations, success! This is the basic startup, but for web projects, a WebSocket forward can be completed on the web side of the playback (of course, the front-end itself supports this format of audio streaming processing).

At this point, we’re done!

PS: more dry technology, pay attention to the public, | xingzhe_ai 】, and walker to discuss together!