Application construction of mobile version of forum based on Discuz X system | project review

Well, we have to talk a little bit

To be honest, I don’t know what motivated me to get into application development. It seems that it was HumanBeingXenon and a bunch of forum friends who wanted me to make an app to replace Discuz X’s pathetic mobile web view after seeing that I had developed GZMTR, a project that now has its code confused and the app abandoned.

I am actually an active forum er, ranked among the top in the forum’s 20 years of operation (although I have been registered for less than a year), and an unknown [Web front end] developer.

What’s the background?

The main problem is that

• Less rendering content (a lot of style tags are ignored on mobile)
• Fewer functions (less scoring (plus or minus points), advanced search, uploading (multiple photos))
• Slow loading (Discuz X optimization of pulling hip)
• Multiple ads (10 replies, AdBlock blocks 10 ads)
• Does not render and edit personality signatures

The functionality you want to implement

• The editor can use styles
• Upload multiple photos at once
• To advertising
• Optimized loading (lazy-loading, etc.)
• Fill in missing render content
• Render personality signatures and provide entry to edit personality signatures

Crooked path – Resource script injection

I first tried to build with my best Web technology and, in fact, built several versions of the application and packaged the entire Web project package with a WebView. But in practice, found a lot of problems, the biggest problem or focus on the speed of resource loading – the direction is wrong!

I went straight to resource script injection, With the aid of the WebView. Into the WebView evaluateJavascript injection window. Onload () = = > {} and use JavaScript to infuse the custom CSS page and improving of the script.

As it turned out, I was naive. WebView loading page speed is extremely slow, a large number of pictures are not lazy loading, blocking the process, so that the page has been in the loading state, and a large number of functions or missing, and difficult to modify, difficult to debug.

At the time, the average page load time was 16s (100 Mbps), and with cellular data, that was horrible. It just blew my head off 😂.

Now I still remember the most feedback in the forum is:

It just keeps loading. It never stops

— Laughing too hard to load.

Get on track — native

Recent want to refactor (the original is really learn while writing, thinking is a mess), the original code is afd615412f0d0679a65faa327629fa8a80c78954 hash

Or the simple and honest proposal of HumanBeingXenon, which made my already unfulfilling winter vacation life even more unfulfilling.

He asked me if I wanted to build a Java version of the app with him as the core (lol, he doesn’t know how to develop for Android) and optimize existing apps.

I hesitated first, after all, the third winter vacation, I still want to brush points, but endure his persuasion, I actually started = =.

I don’t really have any Android development experience, so I’m pretty new to Android.

What did I do? Go to developer.android.com and search for features while copying Exception to Google.

However, I am not the webmaster of the forum, and I have no authority to do something in the forum system, and I can only use the existing data to deal with it.

I began my life as a reptile. After climbing the data using URLConnection, use Jsoup to process the data. Every day is spent in debugging and modification, solved the development of the main functions within a week, basically realized the modernization of the forum mobile application (doge 🐶).

Reverse crawl handling

We tried a lot of HTTP headers, and unlike our current validation (retro, after all), it checks user-Agent, Origin Referer Host, and Cookie. So just a simple URLConnection. SetRequestProperty solved this problem, and even then I still packing into a function, by the way, return URLConnection directly:

@WorkerThread
@Throws(NetworkOnMainThreadException::class)
private fun openConn(url: String, retrieveAsDesktopPage: Boolean): HttpURLConnection {
    val urlConnection = URL(url).openConnection() as HttpURLConnection
    urlConnection.setRequestProperty("Cookie", CookieManager.getInstance().getCookie(url) ? :"")
    urlConnection.setRequestProperty("Referer"."http://www.ditiezu.com")
    urlConnection.setRequestProperty("Origin"."http://www.ditiezu.com")
    urlConnection.setRequestProperty("Host"."www.ditiezu.com")
    urlConnection.setRequestProperty("DNT"."1")
    urlConnection.setRequestProperty("Proxy-Connection"."keep-alive")
    if (retrieveAsDesktopPage)
        urlConnection.setRequestProperty("User-Agent", userAgent)
    return urlConnection
}
Copy the code

Com. Passionpenguin/NetUtils. Kt…

Individuality signature

The second big issue is personality signatures, which is one of the few features that requires you to dig deep into the API yourself. This function because the forum encountered a large area of advertising in the spring, and finally the webmaster and other administrators decided to close the personal space, and pull the gap Discuz X personal space contains password, personal information editing page, also with the ban.

However, even administrators who want to restart this feature find it difficult to find a way. Finally Posting a more recent post to the app, I took a look at the source code for Discuz X (the link below is a third-party repository) and found this:

if($_GET['signaturenew']) {
    $signaturenew = censor($_GET['signaturenew']);
    $sightmlnew = discuzcode($signaturenew.1.0.0.0.$member['allowsigbbcode'].$member['allowsigimgcode'].0.0.1);
} else {
    $sightmlnew = $signaturenew = ' ';
}
Copy the code

/upload/source/include/modcp/modcp_member.php#L39

Signaturenew for POST (see L10, GET), so we just need to send the user information + EditSubmit + SignatureNew.

The image processing

Discuz X is a very ancient system, limited by the processor and storage space, the default image upload size should be set to 16MP, exceeding this size can be uploaded, but will not be written to the server system, only return an error.

Most of the image compression solutions on the market don’t work – either the image gets bigger after compression, or it’s slow to manipulate.

Libjpeg-turbo was originally used by Skia on Android for device performance reasons (you again!). Instead of the Huffman compression, we can use JNI + NDK, import a self-compiled libjpeg or libjpeg-Turbo and modify the interface. , you can achieve the compression function.

Ref: github.com/bither/bith…

After a bit of wrangling, this is done by determining the size of the image and then by calling the NativeUtils class and JNI’s Java_net_bither_util_NativeUtil_compressBitmap.

After some torturing, set the compression ratio of 90%, pro test general pictures can be compressed to 10% or even the minimum tried 5% (Millet device screenshots are not optimized! = =).

But in fact

But it’s actually quite simple to implement:

object NativeUtil {
    fun compressBitmap(bit: Bitmap, fileName: String, optimize: Boolean, quality: Int = 75) {
        compressBitmap(bit, quality, fileName, optimize)
    }

    private fun compressBitmap(bit: Bitmap, quality: Int, fileName: String, optimize: Boolean) {
        Log.d("native"."compress of native")
        if(bit.config ! = Bitmap.Config.ARGB_8888) {val result: Bitmap = Bitmap.createBitmap(bit.width, bit.height, Bitmap.Config.ARGB_8888)
            val canvas = Canvas(result)
            val rect = Rect(0.0, bit.width, bit.height)
            canvas.drawBitmap(bit, null, rect, null)
            saveBitmap(result, quality, fileName, optimize)
            result.recycle()
        } else {
            saveBitmap(bit, quality, fileName, optimize)
        }
    }

    private fun saveBitmap(bit: Bitmap, quality: Int, fileName: String, optimize: Boolean) {
        compressBitmap(bit, bit.width, bit.height, quality, fileName.toByteArray(), optimize)
    }

    private external fun compressBitmap(bit: Bitmap, w: Int, h: Int, quality: Int, fileNameBytes: ByteArray, optimize: Boolean): String?

    init {
        System.loadLibrary("jpegbither")
        System.loadLibrary("bitherjni")}}Copy the code

And the C++ part to write an interface, the rest of the libjpeg-turbo code does not need to be modified.

Steering RecyclerView

I started with a ListView, which was extremely sluggish on a large amount of data. After a search, choose to use RecyclerView to solve the problem. Now can clearly remember, the home page of RecyclerView adapter like a dog eat:

class ThreadItemAdapter(private val activity: Activity, items: List<ThreadItem>, private val isHomeMixed: Boolean = false) : RecyclerView.Adapter<ThreadItemAdapter.ViewHolder>() {

    private val mInflater: LayoutInflater = LayoutInflater.from(activity)
    private var mItems: List<ThreadItem> = items
    override fun getItemViewType(position: Int): Int {
        return if (position == 0 && isHomeMixed) 1 else 0
    }

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ViewHolder {
        return ViewHolder(mInflater.inflate(if (viewType == 1) R.layout.item_home_mixed else R.layout.item_thread_item, parent, false))}override fun getItemCount(a): Int {
        return mItems.size + if (isHomeMixed) 1 else 0
    }

    class ViewHolder(view: View) : RecyclerView.ViewHolder(view) {
        var base: ViewGroup = if(view.findViewById<LinearLayout>(R.id.threadItem) ! =null) view.threadItem else view.homeMixed
    }

    override fun onBindViewHolder(holder: ViewHolder, position: Int) {
        if (position == 0 && isHomeMixed) {
            /* TOP BANNER SHORTCUT */
        }
        val item = mItems[position - if (isHomeMixed) 1 else 0]
        /* THREAD VIEW */}}Copy the code

This code uses a bunch of judgment statements to determine what type the current View is, so there is a specific render content, but the code readability is really “high”!

The login

This is still unclear, currently using the FastLogin function from the web side XHR, but the code is really messy, and now there are many errors:

• Verification code loading
• All information is correct, but tipsPlease enter your user name and password

I don’t understand why this error can be accessed after multiple login attempts.

But the code on this page is the messiest.

You see this indented, how scary, how scary!!

Don’t you ask me why I want Flutter reconstruction

I don’t know if it was March or April or some time when Flutter came right under my nose. It felt interesting, while reading documents (I didn’t know there was a Chinese website at that time, but the Chinese website was also full of bugs and not as comfortable to read as English) and writing code.

The idea is the same, but the end result is very different — Skia and other optimized tools have improved page animation and smoothness by orders of magnitude, while also reducing the difficulty of development by orders of magnitude.

At that time, I felt that the Flutter was not mature, many functions were too closed, but a lot of functions could not be implemented. I modified Dart and the Flutter SDK (/usr/local/bin/flutter/…). Repository source code that compiles smoothly (this really kudos to Dart and Flutter in this respect).

No one can compile the code for my repository

— I can’t either because I updated the Flutter and overwrote the SDK update

But really, the code for Flutter has several features:

• It’s neat, at least, and a lot more comfortable to write than Kotlin or Java
• It’s confusing, after all, because the layout, unlike HTML that supports CSS, results in a bunch of controls that have to be copied from each other in order to achieve a certain style

Some renderings of the Flutter version app just captured

The login

Home page

account

partition

The message

KanTie

Bonus points

It’s too hard to take screenshots…

Posting

Summary of reflection after completion

In fact, the initial idea is not impossible, there are many applications developed in this way, but should use Node.js back-end processing data, front-end rendering page, but AT that time I was new to the back-end front-end (although AT that time I had been developing experience for four or five years).

Then came the Kotlin version, the architecture is chaotic, the code is chaotic, like a beginner’s warehouse, View and Script mixed together, chaos can not be praised.

The Flutter version is the same. The architecture is chaotic and the code is chaotic, much like a beginner’s repository, with widgets and scripts mixed together.

All three approaches have something in common:

• My architecture should be optimized
• My code should have more comments
• My code should strictly separate functions, not stack them together

Harvest of things

In all of these attempts, I participated in the development as a full stack, and even HumanBeingXenon has not committed any code to date. I did all the design and drawing (vector icon + Banner +…) , design, programming, deployment, operation and maintenance.

It was this experience that enabled me to successfully transfer from a budding student who only knew HTML + CSS and lost a little C++ and JS to other fields in the third year of junior high school, which expanded my knowledge of GitHub and other tools and my understanding of different languages.

Still, I have to say this

My drawings and designs are really ugly.

Thank you to everyone who has used the app and for this opportunity.

This article is participating in the “Nuggets 2021 Spring Recruitment Campaign”, click to see the details of the campaign