As the Instagram team rewrote their new iOS Feed, they learned a lot and ran into more pitfalls than they expected, such as aggregate views, Diffing, and the dangers of long code. In this a try! In his Swift talk, Ryan Nystrom shared what it takes to do a successful refactoring and introduced us to a great open source component for Instagram: IGListKit.

An overview of the(0:00)

Hi, my name is Ryan Nystrom and I’m an engineer at Instagram In New York. We did a lot of cool work on the base frame. Over the past year, I’ve been trying to refactor Instagram’s Feed module. It was a lot of fun, and we learned a lot from it. Whenever I go to a conference, I really like to listen to the experience of other industries, because I think the organizational structure and operation form of other companies is very interesting.

So I’d like to share with you how we refactor the Instagram Feed module.

Technical debt(she)

Why did we rewrite the Feed? The answer is simple: Technical Debt.

Instagram is six and a half years old, but the underlying code is still the same. If you search git history and sort through it, you’ll find a lot of initial code submissions for Instagram. Then there’s a lot of stuff from the days of manual memory management, where the code is cluttered and messy. This makes it hard for us to do new things.

How was it initially implemented?2:05 ()

Initially, we used the collection view. When we look at a Post on Instagram, we see a large Section that we can break down into smaller cells. The decomposition results are as follows:

You can see that there’s a supplementary View at the top, the view cells in the middle, then the cells that have the action items, and finally the text cells at the bottom. This is entirely driven by a data model called “Feed Item”.

So we use FeedItem to decide how many comments there are, whether we want to show pictures, whether we want to play a video, what the username is, and so on. Our entire application is built entirely on the FeedItem data model, which includes images, videos, comments, and so on. When someone comes along and says, “We want to add this to the Feed,” we sadly tell them, “No, we can’t because it’s not in the FeedItem model.”

It’s a cell, it’s designed by a designer, and a product manager has been working on it, but the data in it is actually a group of users, not reviews, so there’s nothing we can do about it. I don’t think it’s a good thing to say “can’t do it” to our own team.

Random packing(()

Instagram was launched in 2010, and at that time it was just pictures. Over time, we wanted to add data models like videos, users, and other types. I’m pretty sure at the time we were thinking “just change it a little bit” instead of refactoring, we did it the wrong way… So we just pile it on and add it up as it works.

Well, that’s how we do it. Instead of creating a series of separate small models, we created a bloated model. Parsing this model is becoming more and more complex and slows us down dramatically. Keep in mind that this cell is mapped as FeedItem driven. If you look at the Feed on Instagram, you actually see that it’s not just a simple Post, it contains a lot of data.

Our view controller’s job is to get the corresponding data model, put it in the Section, and then configure the cells (there are many view controllers). There is also a separate view controller for the collection view that inherits from the network task view controller that executes the common feeds, as well as a view controller for the main interface of the application. This gives this kind of view controller four layers of depth. Adding new cells becomes extremely difficult.

You might be thinking, “The code is a little bit more complicated, which might slow down development at times, so is that a bad idea?”

Yes, very bad! Technical debt is starting to drag on. So we decided to take this seriously, so we launched a new version of the Feed three months ago.

Feed 2.0 (again)

One of our main goals was to solve the problem of cascading view controller inheritance, and we wanted to make feeds lighter, simpler, and allow developers to use different cells and data models. We want to get rid of the Feed Item method, which is completely unmanageable.

differentiation(7:10)

Our first thought was differentiation, which is the concept of creating an array of models. When we delete, insert, or move the array, we update the value. Differentiation is useful when building an infrastructure, but difficult to use in a collection view.

First, we have to delete the contents of the old array, then reload the data to get the contents of the array into place, and then perform an insert based on the last index. It takes a little bit of math to do this well.

The original implementation of differentiation is O(n²). When you do too much, you will find that the speed is greatly slowed down. Most implementations I’ve seen go to the background queue, do some math, and then come back to the foreground and continue, but even that’s not fast enough. Since they solve complex problems with low-priority queues, why not just do it on the main thread?

We searched for solutions, and we found a paper written in 1978 by Paul Heckel. This article uses something called least common subsequence, which allows the problem to be solved in linear time.

Based on this paper, we write an algorithm that can find everything that needs to be deleted between two data sets in linear time, reload it, and then perform inserts and moves. This allows us to do this in the main queue so that we can perform all of these updates on the collection view; This provides a much simpler model for us. In addition, we came up with the collection view to work better with this algorithm, which took an incredible amount of time.

Perform the update(and)

Let’s go back to the view controller, at this point we’ve removed a lot of stuff, we’ve circumvented it by rewriting it to shared objects, using system libraries, and so on. So you don’t have to inherit so many view controllers. Network to network, and the main Feed, such as analytics, can become a shared object. But we still need to do some processing with the Feed.

There’s a concept here that we call the world, where the view controller knows everything about the array of items, knows how those items should be added to sections, knows how those sections should be configured, knows how cells should be filled. It handles user interaction, logging, display events, and more.

Item controller(also)

In the new infrastructure we created, we decided to break these tasks down. We created an abstract concept called “Item Controller.” It’s actually a little view controller that implements Section.

Here we determine the number of items, configure the cells, return the cell size, and handle user interaction. But most importantly, this is where all the business logic is stored. We’re not using any dark arts, it’s just a collection view, but we can add any type of object to the collection view by breaking it up like this, right

All we need to do is create a new project controller that handles all the logic on its own.

We thought it was crazy, but we made it happen. The team was really excited about it, and we turned it into our base framework.

What can we give back?(11:26)

When we finished building the framework, we realized we had solved a big problem, so we asked ourselves, what can we give back to the community? We want you to get rid of this problem.

IGListKit (his)

We will open source a new framework, IGListKit (release date TBD), that will help you implement the things I’ve described above.

All of our sample applications and documentation are written entirely in Swift, along with objective-C nullability, sophisticated annotations, and generics. This is fully Swift compliant, C++ is completely masked, and you won’t see a shred of C++.

IGItemController (this)

The most important class in this framework is IGItemController. This is the “project controller” I mentioned at the beginning. There’s not a lot of code in there. It defaults to a single cell with a text label, and that’s it.

To make this class useful, we need to create a new project controller that implements the IGListItemType protocol.

class LabelItemController: IGListItemController, IGListItemType {
    ...
}
Copy the code

At compile time, this protocol ensures that you implement all the necessary methods, such as returning the total number of items:

func numberOfItems() -> UInt {
    return 1
}
Copy the code

On Instagram’s main Feed, we have a dynamic array of pictures, comments, and action bars. We’re going to return the size of this array. Also note that we have a context object:

func sizeForItemAtIndex(index: Int) -> CGSize { return CGSize(width: collectionContext! .containerSize.width, height: 55) }Copy the code

This sets the cell to the width of the screen, or to the width of its parent container, with a height of 55 points. Now comes a whole new concept, which is very different from the traditional collection view. We need to implement the didUpdateToItem method:

var item: String?

func didUpdateToItem(item: AnyObject) {
    self.item = item as? String
}
Copy the code

Here, the infrastructure will pass the required models to your project controller. By using maps, we can map all the models to the project controller. In this case, once we have an item, we can choose to convert it to a string and store it in an instance variable. We can then read the instance variable, reuse the cell in the corresponding index cell project, set the text on the label, and then return the cell, similar to the data source of the collection view.

We have removed the concept of reusing identifiers, and we have completely eliminated the need to register cells and provide complementary views. So, that’s what this controller does. So how do we use this project controller?

IGListAdapter (15:42)

Here we derive the concept of IGListAdapter:

//MARK: IgListAdapterDataSource

func itemsForListAdapter(listAdapter: IGListAdapter) -> [IGListDiffable] {
    return [
        "Foo",
        "Bar",
        "Baz"
    ]
}

func listAdapter(listAdapter: IGListAdapter, itemControllerForItem item: AnyObject) -> IGListItemController {
    return LabelItemController()
}
Copy the code

It brings together the data, all the project controllers, and your collection view so they can work together. In order to use this feature, we need to connect to the data source.

In the first method, we just return an array. Now the values among them are all strings. This could be any value. Notice that the return type is actually IGListDiffable. We’ve already provided a standard implementation for this protocol, so you don’t have to worry too much about it. However, you can still rewrite and extend this protocol to implement your own processing operations for more flexible differentiation. Then there is another method, which returns the corresponding project controller for a given project. Here we’re just returning the same base project controller, which has only one label in it.

Suppose we want to add an indicator while we are waiting for a network request to return data. This way we can create a token object (in this case just an NSObject object named spinToken) that we can put in the middle of the array. Since this is a protocol, we can put any type of model object we want. Next, when the framework asks us to provide a project controller, we need to check “is this project a spinToken?” If so, we return this new SpinnerItemController. Otherwise leave it as it is:

func listAdapter(listAdapter: IGListAdapter, 
            itemControllerForItem item: AnyObject) -> IGListItemController {
    if item === spinToken {
        return SpinnerItemController()
    } else {
        return LabelItemController()
    }
}
Copy the code

This will display an indicator in the middle of our cell:

It may not look exciting, but I’m actually very proud of the big story we made. Imagine that we provided a UISearchBar. When the user enters the text, we can update the search results in real time:

func searchBar(searchBar: UISearchBar, textDidChange searchText: String) {
    filterString = text
    adapter.performUpdatesAnimated(true, completion: nil)
}
Copy the code

So in this searchBar delegate method, we put the text that the user searched into the instance variable, and then call the performUpdatesAnimated method. This tells the framework to get new items and then perform differentiation to update the collection view.

We can also match arrays:

let words = ["Foo", "Bar", "Baz"]

func itemsForListAdapter(listAdapter: IGListAdapter) -> [IGListDiffable] {
    return words.filter { word in
        return word.containsString(filterString)
    }
}
Copy the code

We match an array of strings and return the result. This method is a data source method because it is called only when you tell the framework to update it. It automatically performs inserts, deletes, updates, and everything else that happens on the collection view. I didn’t write any code for this collection view; I simply configure the cell, project controller, and notify the adapter to update. All animation and update operations are performed within the framework.

Why use IGListKit?(19:15)

Let’s say I have a very simple application that has a very simple table view, and I call its reloadData method. What are the benefits of using IGListKit? In fact, I recommend using IGListKit when you have multiple data types in a view like a Feed. If your Feed is very complex and you hate dealing with annoying integer enumerations (which I am), IGListKit is for you.

If you want a fast, crash-free Feed with updated animations, you can also choose IGListKit. It also encourages you to write reusable functional components that separate your cell and project controllers from your view controller.

I can write a project controller somewhere and use it in other view controllers because they don’t have to worry about their parent. I’m also glad I don’t have to call those pesky performBatchUpdates or reloadData methods anymore.

You might be thinking, “Instagram wrote this framework, so should I use it?” Within 15 minutes of launch, we had handled 39 million differentiation operations worldwide, and none of them crashed, all of them were on the main thread, and there was no lag.

Where do you use it?(and)

This whole project grew out of our desire to rewrite our feeds. We now use IGListKit for our Explore page, Activity Feed, and even for complex cells and interactions in our communications module. Our Instagram Stories product, which launched a month ago, also uses this framework and is built entirely using IGListKit. We certainly applaud the use of this framework. Because that’s what we’re going to use in the future.

IGListKit is coming

I’m looking forward to sharing a few stories about working on Instagram, and I hope you’ll learn something from them and apply it to your organization’s apps. I’m really happy to see people building their own applications using IGListKit!