This is the 11th day of my participation in the August More Text Challenge

Last week, I finished reading the beauty of design patterns and wanted to write something for practice. I happened to be reconstructing the company’s upload library recently, so I came up with this article. (Just learning and practicing, if you have any suggestions, please kindly comment)

0 x1, gossip

① Why to rebuild the upload library?

Our APP needs a series of processing before uploading pictures/videos, such as:

  • Image: Check whether the path exists → check whether rotation is needed → Check whether compression is needed → get MD5 → Check whether there is second transmission record if second transmission is enabled → return directly if there is no upload → upload the corresponding status update;
  • Video: Determine whether the path exists → determine whether compression is needed → compress if compression is needed → obtain MD5 → obtain the first frame of the video → determine whether compression is needed → obtain video MD5 again if compression is needed → also transmit verification in seconds → upload the first frame after uploading the video

Some business scenarios are more complex to handle, Talk is cheap, show you the code, it is normal to have code like this (partial) :

Thanks to the RX chain call, the above code has been simplified after a version, can imagine that before Rx even more chaotic, for the people who write, for the people who read, refactoring is imperative…

(2) Are there any other low-cost solutions without refactoring?

A: Yes, based on the above code optimization, the flatMap is extracted as a separate function and called according to the flow, of course, not very elegant. The most elegant would be to run a Kotlin coroutine, write a few suspend functions, and write asynchronous code synchronously. Of course, there are problems: a bit of learning cost and not being able to use Java.

0x2. Demand Disassembly

The original requirements

Write an image upload library that gives you a local image path and an upload interface to complete the image upload.

The small white perspective

It’s easy to write UploadPicUtils utility class and define an upload method

Speed of light type code:

object UploadPicUtils { fun uploadPic(picPath: String, serverUrl: String) {val PIC = File(picPath) if(pic.exists()) { UploadPicUtil. UploadPic (" local image path ", "upload interface ")Copy the code

It seems simple enough, but the only constant is change, and demand is often capricious

  • Because the company is not willing to buy pictures with watermarking services, so the client before transferring pictures to local watermarking;
  • BUG: Some users take photos with their mobile phones, but the pictures uploaded are rotated. Check them before uploading, and straighten them out.
  • BUG: Some users feedback that uploading pictures is too slow, one check the picture is too big, the server can not support, before uploading the picture should be compressed;
  • Some pictures have a specified size (X*Y), the wrong size can not be uploaded;
  • Second transmission function, md5 like the file is not transmitted, directly return the address;
  • Support to upload multiple pictures at the same time;
  • Now not only upload pictures, but also upload video, audio, files, folders and other scenes…

Then the code becomes the result of the above, the person who wrote saw silence, the person who received the disk saw tears.

So, when you get the original requirements, don’t start coding them. Instead, break them down, analyze them, assume them, and think about them.

  • Are you sure you only upload pictures? Will there be other things to upload, such as audio and video?
  • Do I need to check the validity of pictures? For example, whether the file exists, whether the size is 0, the file format is picture type, etc.
  • Does the library need to do anything special with uploaded images? Such as watermarking, flipping, compression, cutting, size check, etc.;
  • Whether to support multiple pictures uploaded at the same time, the maximum number of simultaneous upload;
  • Whether to support second transmission;
  • Compatible with different mobile phone system version or device file acquisition API;
  • Whether the address of the upload interface changes all the time, whether authentication is required, and whether there is a specific upload configuration;
  • Whether the uploading task is carried out asynchronously in the background or blocked synchronously in the foreground, and whether the uploading can be cancelled during the process;
  • The uploading task is interrupted (kill the APP). Do you need to retain the progress and re-upload the APP next time?
  • Whether to retry the upload failure and what is the maximum retry times?
  • Whether the upload task has priority, etc.;

Of course, don’t think to give a perfect design scheme and complete function realization all at once. Limited by designers’ architectural experience and limited schedule, first give a rough and basic usable scheme with an iterative basis, and then slowly optimize to minimize feasible products.

0x3. Architecture Design

From a macro perspective, the process of file uploading is similar to assembly on a workshop line. Take bagged potato chips for example:

Potatoes into the factory → cleaning peeling → slicing drying → 350 degrees high temperature frying → adding salt → filling by gram nitrogen → bag potato chips

From potatoes through various transitions, to the final bag of potato chips, analogies to our single upload task:

Abstracted and simplified into three parts:

The pipelining approach of task construction and completion lends itself well to the chain of responsibility model.

The traditional chain of responsibility is realized, one way back, layer upon layer of interception, until someone deals with it.

Here is a reference to the implementation of OkHttp interceptor, bidirectional chain of responsibility, the general principle:

  • The Interceptor implementation class calls Intercept (Chain) to pass down the Chain instance (including requests after this Interceptor processing);
  • The last interceptor call chain.proceed() returns an instance of Response, passed recursively up;

Specific interpretation of the visible: read the book “the thin |” “the beauty of design patterns” design patterns and paradigm – chain of responsibility pattern (behavior)”

The combination of individual tasks becomes:

Interceptors before the request → executing the upload request → interceptors after the request

Performing the upload request is left up to the user to customize, providing methods for constructing the request and sending the request. Success or failure can be informed by callback.

This is the case for a single task upload, but multiple tasks are also required: task queues, pollers, and thread pools.

When an upload task is initiated, the task is added to the queue, and the poller keeps taking tasks from the queue (until there are no more tasks) and another thread from the thread pool to execute the task.

The general principle is very clear, and then the specific code implementation, code notes are written in detail, will not explain one by one ~

0x4 use of libraries

At present, I still write to play with the state (a pile of pit, vegetable chicken while stepping on the change in ~), interested in the star, warehouse address: github.com/coder-pig/C…

Add dependencies:

allprojects {
	repositories {
		...
		maven { url 'https://jitpack.io' }
	}
}

dependencies {
        implementation 'com. Making. Coder - pig: CpLightUpload: v0.0.3'
}
Copy the code

① Customize tasks

Different scenarios may have different requirements, customize attributes as needed:

class CpImageTask : ImageTask() {
    var needRotate: Boolean? = null
    var needCompress: Boolean? = null
    var compressPercent: Int? = 80
}

class CpVideoTask : VideoTask() {
    var limitSize: Int? = -1    // Limit the size of the video
    var compressVideoPath: String? = null   // Compress the video path
    var compressVideoMD5: String? = null   // Compress video MD5
    var firstFramePath: String? = null   // The first frame path of the video
    var firstFrameMD5: String? = null    // Video frame 1 MD5
}
Copy the code

② Customize upload configurations

Is a default configuration for uploading tasks. If no configuration is configured for uploading tasks, fill in the default configuration:

class ImageUploadConfig : LightUploadConfig() {
    var needRotate: Boolean = true  // Whether rotation correction is required
    var needCompress: Boolean = true   // Whether compression is required
    var compressPercent: Int = 80   // Compression ratio, default 80
}

class VideoUploadConfig : LightUploadConfig() {
    // Customize as needed
}
Copy the code

③ Customize the front interceptor

Inherit the Interceptor interface and implement the Intercept () method:

class PictureRotateInterceptor : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Task {
        val task = chain.task()
        val config = task.config as? ImageUploadConfig
        if (task is CpImageTask) {
            if(task.needRotate == null) task.needRotate = config? .needRotate"============ Determine whether image flipping is required ============".logV()
            val degree = FileUtils.readPictureDegree(task.filePath!!)
            if(degree ! =0) {
                "Picture rotation correction".logV() FileUtils.rotateToDegrees(task.filePath!! , degree.toFloat())"Image rotation completed".logV()
            } else {
                "No rotation correction required.".logV()
            }
        }
        // pass it down
        return chain.proceed(task)
    }
}

class VideoFrameInterceptor : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Task {
        val task = chain.task()
        if (task is CpVideoTask) {
            "Generate video thumbnails...".logV()
            // Get the name of the first frame
            valtag = task.compressVideoPath!! .substring(task.compressVideoPath!! .lastIndexOf("/")) 
            val frameFile = File(getExternalVideoPath() + tag + ".jpg")
            task.firstFramePath = frameFile.absolutePath
            val mmr = MediaMetadataRetriever()
            mmr.setDataSource(task.compressVideoPath!!)
            val frameBitmap = mmr.frameAtTime
            FileUtils.compressImage(frameBitmap, frameFile, 80) task.firstFrameMD5 = FileUtils.getFileMD5ToString(frameFile) LightUpload.upload(task = CpImageTask().apply { filePath = task.firstFramePath md5 = task.firstFrameMD5 }) frameBitmap? .recycle() }return chain.proceed(task)
    }
}
Copy the code

4 Customize a request

Implement the Upload abstract class and rewrite initRequest() and sendRequest() methods to call back different request results:

class HucUpload : Upload() {
    override fun sendRequest(a) {
        "Start file upload...".logV()
        var ins: InputStream? = null
        try{ mTask.reqData? .let { req ->val conn = (URL(req.uploadUrl).openConnection() as HttpURLConnection).apply {
                    readTimeout = req.timeout!!
                    connectTimeout = req.timeout!!
                    doInput = true
                    doOutput = true
                    useCaches = false
                    requestMethod = req.requestMethod
                    // Request header Settings
                    val boundary = UUID.randomUUID()
                    req.headers["Content-Type"] = "multipart/form-data; boundary=${boundary}"
                    for ((k, v) in req.headers) setRequestProperty(k, v)
                    val dos = DataOutputStream(outputStream)
                    val sb = StringBuilder().append("--").append(boundary).append("\r\n")
                        .append("Content-Disposition: form-data; name=\"file\"; filename=\"")
                        .append(mTask.md5).append(mTask.fileName).append("\" ")
                        .append("\r\n")
                        .append("Content-Type: application/octet-stream; charset=utf-8")
                        .append("\r\n").append("\r\n") dos.write(sb.toString().toByteArray()) ins = FileInputStream(File(mTask.filePath!!) )val bytes = ByteArray(1024)
                    var len: Int
                    while(ins!! .read(bytes).also { len = it } ! = -1) {
                        dos.write(bytes, 0, len) } ins!! .close() dos.write("\r\n".toByteArray())
                    val endData: ByteArray = "--$boundary--\r\n".toByteArray()
                    dos.write(endData)
                    dos.flush()
                }
                // Get the response
                val input = BufferedReader(InputStreamReader(conn.inputStream, "UTF-8"))
                val sb1 = StringBuilder()
                var ss: Int
                while(input.read().also { ss = it } ! = -1) {
                    sb1.append(ss.toChar())
                }
                val result = sb1.toString()
                "File upload completed...".logV() mTask.response = Response(conn.responseCode, result) mTask.status = TaskStatus.DONE mCallback? .onSuccess(mTask) } }catch(e: IOException) { e.message? .logE() mTask.status = TaskStatus.FAILURE mTask.throwable = e mCallback? .onFailure(mTask) LightUpload.postTask(mTask) }finally {
            if(ins ! =null) {
                try{ ins!! .close() }catch(e: IOException) { e.message? .logE() } } } } }Copy the code

⑤ Custom post-interceptor

Processing of response data, such as string parsing display

class SimpleParsingInterceptor: Interceptor {
    override fun intercept(chain: Interceptor.Chain): Task {
        val task = chain.task()
        if(task isImageTask) { task.response? .let {var tempContent = it.content
                if(tempContent.startsWith("{")) {
                    val index: Int = tempContent.indexOf("{")
                    tempContent = tempContent.substring(index)
                }
                try {
                    val jsonObject = JSONObject(tempContent)
                    if (jsonObject.getInt("code") = =200) {
                        // Parse the content returned by the server
                        val mapJson: JSONObject = jsonObject.getJSONObject("data")
                        var key = ""
                        var image = ""
                        val ite = mapJson.keys()
                        while (ite.hasNext()) {
                            key = ite.next()
                            image = mapJson[key] asString } task.fileUrl = image task.fileUrl? .logV() }else {
                        jsonObject.toString().logV()
                    }
                } catch(e: Exception) { e.message? .logD() } } }return chain.proceed(task)
    }
}
Copy the code

6. The initialization

You don’t need to initialize it in the App class, just make sure it is init() before upload

LightUpload.init(LightUploadBuilder()
        // Pass in the default configuration, variable parameters, support multiple types of Task customization
        .config(LightUploadTask.IMAGE to ImageUploadConfig().apply {
            reqData = ReqData(
                uploadUrl = "http://127.0.0.1:5000/upload",
                requestMethod = "POST",
                headers = hashMapOf(
                    "Charset" to "utf-8"."connection" to "keep-alive"
                )
            )
        }, LightUploadTask.VIDEO to VideoUploadConfig()
            .apply {
            reqData = ReqData(
                uploadUrl = "http://127.0.0.1:5000/upload",

                requestMethod = "POST",
                headers = hashMapOf(
                    "Charset" to "utf-8"."connection" to "keep-alive"))})// Set the upload request, also a variable parameter, support a variety of types of customization
        .upload(LightUploadTask.IMAGE to HucUpload())
        // Add the front interceptor
        .addBeforeInterceptor(PictureRotateInterceptor())
        .addBeforeInterceptor(PictureCompressInterceptor())
        .addBeforeInterceptor(VideoCompressInterceptor())
        .addBeforeInterceptor(VideoFrameInterceptor())
        // Add post-interceptor
        .addDoneInterceptors(SimpleParsingInterceptor())
)
Copy the code

⑦ Call upload

LightUpload.upload(task = CpImageTask().apply {
    filePath = path
    needCompress = true
    compressPercent = (1.100.).random()
    callback = object : Upload.CallBack {
        override fun onSuccess(task: Task) {
            // Successful callback
            text = " ${task.response!! .content}\n"
        }

        override fun onFailure(task: Task) {
            // Failed callbacktask.throwable? .message? .let { it1 -> shortToast(it1) } } } })Copy the code

0x5. Demo Test

Run the following command to install python script dependencies before the first run:

pip install -r pip install requirements.txt
Copy the code

Once installed, type the following command to run the script:

python app.py
Copy the code

Then the mobile phone and computer in the same LAN, configure the agent, enter ipconfig to view the local IP

Configure the mobile phone and open the Charles packet:

The running effect is as follows:

Logcat can also see the output:

Nice~