Modularization practice of Douban App

2016-10-16

background

Business background

Douban converged its mobile business in 2014, launching an App called Douban. With the development of Douban App, more and more business lines of Douban have been included. Douban App has more and more code, more and more complex functions, and more and more bulky. In order to deal with this situation more calmly and make the whole project healthier, we implemented modularity. The ultimate purpose of modularization is to create several independent service modules, so that each service module does not interfere with each other and can be independently developed. However, in fact, the development of Douban App is still under the responsibility of one team, and the engineering team is not divided into business lines. Therefore, the independence of business modules is not an engineering requirement corresponding to the organizational structure of the company. It’s about the health of the project. Although, the whole process has been accompanied by some pain. But it worked out well, and in the end we got the following benefits:

  • A clearer project structure;
  • Some common components that can be reused;
  • Several business modules isolated from each other.

Results of open source

We also produced several libraries and tools during the modularization process:

  • FRDIntent, a library that handles jumps between pages.
  • FRDModuleManager, a simple module management library.
  • Rexxar, mobile hybrid development framework.

We open source these tools. On the one hand, it is to provide us with some reference direction; On the other hand, it is also to improve the quality of the project itself. We know there are still problems. Therefore, we will carefully accept your opinions and suggestions.

Engineering environment

Before describing douban’s modularization practices, let’s briefly introduce the working environment related to douban’s mobile development.

In iOS development, our version management tool is Git. All of the company’s project code is hosted on github Enterprise, which is built on the Intranet. Github Enterprise is basically the same as Github.com. This was generally welcomed by engineers.

The tool we rely on to manage our projects is Cocoapods.

Git and Cocoapods are now essentially de facto standards in the industry. In this article, these two tools are also the only ones that relate to the modularization process.

Implementation of modularization

The project structure

We first delineated the modularity and then finally the project structure we wanted. And then move one step closer to that goal.

+--------------------------------------------------------------------------+ | | | Frodo | | | | AppDelegate, HomeFeed, Mine ... | +----------+---------+-----------------------------------------------------+ | | | | | | | Leaf | | | | + -- -- -- -- -- -- -- -- -- -- -- -- -- + + -- -- -- -- -- -- -- -- -- -- + + -- -- -- -- -- -- -- -- -- -- - +... | | | | | | | | | | | | | | | Timeline | | Group | | Subjects | | | | | | | | | | | | | | | +-------------+ +----------+  +-----------+ | | | +-----------------------------------------------------+ | | | | | Fangorn | | | +--------------+ + -- -- -- -- -- -- -- -- -- -- -- -- -- + + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- +... | | | | UI | | Modole | | FrodoRexxar | | | | | | | | | | | | | +--------------+ +-------------+ +---------------+ | | +---------------------------------------------------------------+ | | | Library | | +------------+ +-------------+ + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - +... | | | Rexxar | | FRDIntent | | FRDModuleManager | | | | | | | | | | | +------------+ +-------------+ +-------------------+ | +--------------------------------------------------------------------------+Copy the code
  • Frodo is the project name of douban App. Douban has a tradition of naming projects after Lord of the Rings characters.
  • Leaf, which requires separate business modules to be placed under Frodo’s Leaf folder until they are split.
  • Timeline, Group and Subjects are three separate modules.
  • Fangorn is the project name of Frodo’s public infrastructure library. This includes business-relevant common UI components, data models, and external libraries such as FrodoRexxar.
  • Library is a Library that we split out ourselves, or a third party Library. Rexxar, FRDIntent, and FRDModuleManager belong to the Library.

The implementation steps

Modularization is a huge project, which has a significant impact on the project. After we set the goal, we then made a detailed plan. And then follow the plan step by step.

We have divided modularity into four steps, each of which is described below.

Folder isolation

We first need to change the file organization structure of the project. The previous project simply divided the files into View/Controller/Model. Now it is divided by modules first, and then each module is divided into its own View/Controller/Model/Network folders. For example, broadcast Timeline has its own View/Controller/Model folder; Group groups also have their own View/Controller/Model folders. While changing to a file structure that organizes projects by modules, at this point, all modules are still in one repository. This is really just folder isolation, the code is not really isolated. We look at the #import section of each file to reduce interdependencies between business modules. Files used by several business modules sink to the common layer.

You can’t do complete isolation at this stage. Because the process of sorting out dependencies depends on the programmer looking at the code himself. The file location appears to be separated, but since it’s still in a repository, code dependencies can’t be completely cleaned up. That’s ok, folder isolation is just a passing phase. The reason for this transition is that product development doesn’t stop just because we’re refactoring the project. Folder isolation avoids direct splitting that takes a long time to resolve compilation errors in one go. This allows the team to spread the time spent dealing with dependencies over each release. After a business module has gone through the folder isolation phase, there should be few unresolved dependencies left to deal with when it actually needs to be removed. There is much less time pressure to split into separate products.

Folder isolation also provides a buffer period for teams to change their development approach. The division of business modules first requires consensus among all team members. From then on, people began to have a modular awareness. This means that new files are put into the corresponding module; Code Review will make requirements and recommendations that other business modules should not be referenced. Folder isolation enables team members to gradually adapt to modular thinking, and subsequent product functions are grouped into corresponding modules.

Folder isolation allows product development and modularity to advance in parallel. In the case of not affecting the progress of product development, more leisurely to promote modularity.

Abstract away business-independent libraries

We also encourage the abstraction of business-neutral code into separate libraries. Such libraries should be product-agnostic and business-agnostic. This means that they can remain stable over time and do not change with the frequent changes in products and businesses. Abstracting out the underlying logic and splitting it into separate libraries has several advantages:

  • Each library has its own clear boundaries. Prior to the split, the code of each library is mixed in the project, and there is the possibility of mutual interference. Independent outbound allows more isolation in project code and improves project quality;
  • Breaking up separate libraries makes it possible to reuse them in new projects or even open source them for other developers to use. This is the case for FRDIntent, FRDModuleManager, and Rexxar;
  • This laid the foundation for the spin-off of the public underlying module Fangorn. This is because a split module must first handle its dependencies, which can only rely on the split components and third-party libraries.

Split out the common underlying modules

In Frodo, the common underlying module is called Fangorn. Fangorn includes some common code required by business modules, but it is either relevant to the business or not abstracted into a library.

After splitting out many components and libraries that are irrelevant to the business. We still have parts of our code that are common and used by multiple business modules, but that are relevant to the business. This part of the code and business related, split out of the possibility of reuse. But it is a prerequisite for unbundling business modules. After separating Fangorn as a warehouse, we were able to start splitting the business modules.

There are two ways to split a public module:

  • One is to divide Fangorn into sub-modules. Break up these sub-modules one by one;
  • One is to take Fangorn as a whole first. Dependencies between submodules within Fangorn operate as folder isolation for a period of time. If you find a submodule that can actually be removed, remove it.

The first way is more elegant. If done, individual business modules can choose their own dependencies rather than relying on Fangorn as a whole. But it’s also more difficult. It took a long time to figure out the dependencies between Fangorn’s sub-modules.

The second way is easier. To take Fangorn as a whole, it’s just a matter of dealing with the dependencies between Fangorn and other external modules. That’s a lot less work.

In order to separate the business modules earlier, we chose the second option.

Service module independence

In Frodo, individual business modules are placed in a folder called Leaf during the folder isolation phase. Our goal is to separate three modules: broadcast Timeline, Group and Subjects.

This part of the job is to remove dependencies between business modules. All three modules rely solely on Fangorn, the split-out library, and third-party libraries. The ultimate goal is for the three business modules to be independent and split into separate libraries. After a long period of folder isolation, the separation of many business-neutral libraries, and the separation of Fangorn, we began to split the business modules from easy to hard. The first business module to be split is the broadcast Timeline, followed by the Group Group, followed by the item Subjects.

Finally, we set up a running application Demo project for each of these three business modules. In this way, they can be developed and run independently of each other. Our development process changed to developing new product features in these three module libraries and using the module’s own Demo to see the results. After completion, upgrade the versions of the three business module libraries in the main project Frodo and verify the test integration effect in Frodo.

FRDIntent: Decoupling between View controllers

On iOS projects, there are a lot of page hops. However, iOS does not have a uniform way to jump to a page like Android’s Intent. In iOS, to handle page jumps, you need to rely on the class that represents the page to which you want to jump, and you need to know its initialization method. So the pages need to depend on each other. This situation is not conducive to decoupling and splitting modules. To solve this problem, we made a library that deals specifically with page jumps in iOS, FRDIntent, and made it open source.

The FRDIntent has two parts: the FRDIntent/Intent resolves in-app page jumps; FRDIntent/URLRoutes Resolve external application page calls of the application. After using FRDIntent, we decoupled the View Controller nicely. And provides a clear and unified solution for internal call and external call. Solved a big part of the coupling problem in iOS projects: the coupling between View Controllers. It has laid a solid foundation for us to promote modularization smoothly.

FRDIntent/Intent

FRDIntent/Intent is a message passing object that launches UIViewController. It can be seen as a parody of Intent in Android. Of course, FRDIntent/Intent is extremely simplistic. This is because the FRDIntent/Intent usage scenario is simpler: it only handles in-app jumps between View Controllers.

Directly use the iOS system method to complete the jump between view controller, each view controller code will be tightly coupled. When jumping, a View Controller needs to know all the details of how the next View Controller is created. This creates a dependency between view Controllers. To decouple the code between view Controllers, use FRDIntent/Intent to convey the view Controller jump information.

FRDIntent/Intent has the following advantages:

  • Fully decoupled. The caller and the called are completely isolated, and the caller only needs to rely on the protocol:FRDIntentReceivable. A UIViewControlller that conforms to this protocol can be started.
  • This provides a common solution to the more common requirement of “start a page and get results from that page.” To check the value, run startControllerForResult. This is a parody and simplification of startActivityForResult in Android.
  • Support custom transition animation.
  • Support for passing complex data objects.

FRDIntent/URLRoutes

FRDIntent/URLRoutes is a URL Router. FRDIntent/URLRoutes calls up a registered block with a URL.

The iOS system provides a URL-based processing scheme for calls between applications. That is, an application can declare that it can handle certain urls with specific schemes and hosts. Other applications can call these urls to jump to certain pages of the application.

FRDIntent/URLRoutes is designed to make handling urL-based interapp calls easier in iOS. Therefore, the functions and purposes of FRDIntent/URLRoutes are similar to those of existing URL Routers in the community. FRDIntent built the wheels again so that FRDIntent/URLRoutes can work with FRDIntent/Intent to handle in-app and out-of-app page calls.

FRDIntent/Intent and FRDIntent/URLRoutes

FRDIntent/URLRoutes and FRDIntent/Intent can be used together. Intent handles internal page jumps; URLRoutes is responsible for page calls from outside. In the implementation of FRDIntent/URLRoutes, FRDIntent/URLRoutes only plays the role of exposing the external call entry and receiving the external call. Within the application, the View Controller is still launched with FRDIntent/Intent.

This unifies the implementation of external and internal calls while separating them. External calls end up landing with internal calls as a result of naturally reusing code. Separating external and internal calls provides the following benefits:

  • The ability of iOS to call another app through a URL is itself used between apps. The isolation between apps in iOS is clear and unambiguous, and passing information between apps via urls is appropriate. However, if in-app calls also use urls to pass information, there are limitations. Intents are better suited for internal invocation scenarios. With IntEnts, you can pass complex data objects and easily customize transitions. These are difficult to do in a URL scheme.
  • By distinguishing between external and internal calls, we can choose whether to expose an internal call for external use. This avoids the problem of not being able to distinguish between internal and external calls in a URL scenario, exposing calls intended for internal use to the outside of the application.

FRDModuleManager: Reduces the load for the AppDelegate

Over the course of development, we noticed that the AppDelegate implementing the UIApplicationDelegate protocol in the project became more and more bloated. To make the AppDelegate healthier, modules can more easily learn about application life cycle events. We developed a simple module management gadget: FRDModuleManager. The gadget is surprisingly simple, with just a.m file. We’ve open-source it as well.

FRDModuleManager is a simple iOS module management tool. The FRDModuleManager can reduce the amount of code in the AppDelegate, splitting many responsibilities into modules. Make the AppDelegate easy to maintain.

If you find that the AppDelegate in your project that implements the UIApplicationDelegate protocol is getting bloated, you might want a gadget like this; Or if your project implements componentization or modularity and you need to leave hooks in UIApplicationDelegate methods for each module so that the module knows the entire application life cycle, you might also need a widget like this, To better manage the hooks left behind by modules in various METHODS of the UIApplicationDelegate protocol.

The FRDModuleManager allows hook methods left in the AppDelegate to be managed uniformly. An AppDelegate that implements the UIApplicationDelegate protocol is an important way for me to keep track of the application lifecycle. If a module needs to initialize the application startup, then we will need the AppDelegate application: didFinishLaunchingWithOptions: call an initialization method of the module. With more modules, more initialization methods are called. Finally, the AppDelegate gets bloated. The FRDModuleManager provides a unified interface for modules to know the application life cycle. This will simplify the AppDelegate.

Strictly speaking, an AppDelegate should not be responsible for anything other than notifying the application’s life cycle. One of the most common and bad uses of an AppDelegate is to hang global variables on an AppDelegate. This gives you a global variable that you can use within your application. An AppDelegate with too many global variables hanging can be a pain in the neck if you need to modularize your project. This is because the AppDelegate becomes a dependency center. It relies on many modules, which is not a problem. However, since access to global variables needs to be through an AppDelegate, this means that many modules also rely on the AppDelegate, which is a big problem. This is because the AppDelegate can rely on the module to be split; On the other hand, modules to be split out cannot rely on an AppDelegate.

To do this, first remove the global variable from the AppDelegate. Each class should provide its own global access directly, and the simplest way to do this is to implement the class as a singleton. Variables can also hang on an object that provides global access. Of course, this object should not be an AppDelegate.

Second, the AppDelegate is only responsible for providing notification of changes in the application’s life cycle. This can be solved more elegantly by using the FRDModuleManager.

After these two steps, the AppDelegate dependency problem can be nicely resolved so that the AppDelegate is no longer the dependency center of the project.

Rexxar: Mixed development

Douban has done a lot of practical work in mixed development. We open source the main output of this process: Rexxar. This article introduces Rexxar in detail.

Our main goal in promoting mixed development is to improve engineering efficiency. But there is also a byproduct: because pages implemented using Rexxar use Web technologies to implement the entire business, the Web code implementing the Rexxar page is, in effect, completely isolated from the rest of the project’s Native code. The front-end framework we use at the business layer is React. Due to React’s componentized nature, the project code is also well modularized inside the front-end code.

So, in effect, hybrid development also plays a role in our implementation of modularity. Pages developed using Rexxar do not rely on the project’s Native code except for the Rexxar library.

Decoupling between common objects

Using FRDIntent when handling page jumps has solved most of the intermodule invocation problems. However, this does not solve all coupling problems, and there may be interdependencies between modules in a project, in addition to page hops. How do we deal with dependencies on a module that exposes objects or data in addition to the interface to jump to a page in the module? For example, you want to expose a UI component, or a calculation result. Use a specific broadcast Timeline module as an example: if other modules in the project need to show a broadcast, or know how many broadcasts a user has made. This has of course been implemented in the broadcast module. So, how do we provide a single broadcast for the UI component, and how many broadcasts did the user send? We started with two slightly more simple and crude methods:

  • If they are exactly the same and multiple modules are used. We’ll sink it into the public underlying module Fangorn. So that each module depends on the common underlying module, rather than each other;
  • If they’re not exactly the same, we might make a copy of the code. Classes are renamed to prevent conflicts. Because modularity is implemented, some degree of code redundancy is a price to be paid and tolerated.

Beyond that, a possible solution is something similar to the dependency injection container commonly used in the Java community. Spring, for example, is a well-known dependency injection container. With the introduction of a dependency injection container, you can dynamically create objects and inject dependencies into them. While all dependencies are registered in the container, the business object only needs to depend on the interface, not the specific type. This is a general method of decoupling. Applies to almost any dependency management scenario.

But in our practice, to avoid complexity, we did not introduce a dependency injection container. Using a dependency injection container means that the creation of many objects in a project is entrusted to the container. This made a big difference to the way the entire project code worked. We are cautious about making fundamental changes like this. And we’re not optimistic about finding or implementing such a solid base container in mobile.

Problems encountered

Swift

We used Swift quite a bit in the project. There are some problems:

  • Swift itself is unstable at the moment. Both Swift 2 and Swift 3 have major changes from the previous version. Although Xcode provides automatic conversion tools, you still need to put in the effort to check and test. In a large project, safety and stability are generally the first requirements. Introducing a language that is unstable during development is a threat to project quality. But Swift is a slightly different story. For iOS development, we don’t have a lot of choices. There’s objective-C and Swift, but Swift is going to be the future, Objective-C is going to be the history. So we really just have to choose when and how to switch to Swift. Our rule of thumb is to experiment in small pieces and proceed cautiously. Each upgrade to the Swift version of existing code is done piece by piece, rather than converting all the code at once.
  • Using a library that contains Swift code requires iOS versions. If you want to use a library that contains Swift code, you need to introduce it as a dynamic library. The dynamic library must be iOS 8 or later. We dropped support for iOS 7 when iOS 10 was released. This allows us to use a library that contains Swift code. If the project requires support for earlier versions of iOS, the library containing Swift code cannot be used at this stage. The way to use a library as a dynamic library through Cocoapods is simple: open Cocoapods! use_frameworkIdentification.

Swift does outperform Objective-C in engineering efficiency. Swift can do the same thing in a much cleaner way with much less code than Objective-C. Of course, there are engineering costs associated with mixing Swift and Objective-C. So, there’s a tradeoff: Do you keep it simple and just use Objective-C? Or do you put up with some inconvenience and use some Swift for efficiency gains?

Our experience with Swift on the project was that there was joy, but also some inconvenience. On the whole, the inconvenience can be overcome.

conclusion

We believe that the experience gained in the whole process of modularization practice should have some reference for a growing mobile project. Some of the open source libraries and tools we produce may also be helpful or instructive. We also welcome feedback and suggestions to help us improve and improve.

Click to see the comments