Original address: dteare.medium.com/behind-the-…

Dteare.medium.com/

Published: May 18, 2021 -9 minutes to read

We are officially launching 1Password for Linux today! 🎉

What makes this release even more amazing is that it was developed from scratch, using new languages and technologies that most of us on the team have never used.

We do almost everything you can think of in new ways: from team organization, programming languages and toolkits, design languages and processes, all the way to new deployment and packaging, and testing procedures. In fact, more than half of the teams contributing to 1Password for Linux have joined our family in the last 18 months.

I thought it would be interesting to get a high-level overview of how 1Password for Linux is structured, how we fell in love with Rust, and highlight some of the incredible things our team is doing to make our new Linux application a reality.

A hybrid application powered by Rust 🦀

1Password for Linux was our first hybrid application.

The back end is written in Rust, a true system programming language known for its security and performance. Rust compiles directly to native code, avoiding overhead associated with runtime or garbage collection.

On the front end, we used web technology that allowed us to create an entirely new design language for 1Password. Not only does it look great, but it allows us to iterate very quickly with great agility.

We’ve had a lot of success with TypeScript and React when developing the web interface for 1Password.com and our browser extensions, so these are natural choices here.

To tie everything together, we used Neon and a homegrown TypeShare tool that generates types on both sides of the FFI layer. Finally, we used Electron to bundle everything, allowing us to deeply integrate with the operating system.

Here’s a quick sketch of how it all fits together.

An overview of the hybrid application architecture we use in 1Password for Linux

Quick trip down memory lane

It took us a while to realize how great Rust was and how it solved many of our problems. To understand how we got here, we first need to review how we have developed applications over the past 15 years.

Before our new approach to building hybrid applications, all of our applications were written on a completely different back end on each platform. Although they are all connected to the same 1Password.com service, they are all implemented in their own handwritten code. Each platform is responsible for synchronization, conflict resolution, data storage, creating view-models for user interfaces, and just about everything else you can imagine.

This may seem surprising in this day and age when Rust exists and is largely mature, but these applications were developed in the space of 15 years, before there were any languages that allowed us to write software across operating systems in a secure and reliable way.

Sure, there was C and C++, but neither Roustem nor I were fans of C++, and even if we were, the thought of writing a multiplatform library in an insecure language scared the pants off us. In the case of cryptography, we just say no.

So each team wrote using the best languages and libraries available on each platform at the time.

As you can imagine, we try to keep features (and bugs!) consistent across each platform. But with so much code base, it’s very difficult to do that.

We all wanted to find a way to write back-end code in a secure and high-performance way and share it across all the platforms we supported. It’s a noble and exciting endeavor, but how?

How we fell in love with Rust 🥰

Back in 2017, our Windows team was the first to try Rust. They quickly fell in love and released 1Password for Windows 7.0.558 in May 2018, which did a lot of heavy lifting using Rust.

The number of crashes and error reports on Windows dropped rapidly.

While this was great, it took the rest of the team longer to see the glory of using Rust. At the time, many on the team were using Go Lang to create a shared back end for 1Password.next (our code name for the 1Password core at the time), and they had seen enough success that they weren’t in a rush to switch languages.

With GopherJS compiled into JavaScript, we have all the support for Go. At least for a while.

Over time, we’ve surpassed GopherJS. When we started experimenting with WASM, we found that Rust compiled into WASM very well and had a solid tool chain that we had to try.

So we started the experiment to transplant the brain from Go to Rust at NSNorth 2019, and then things really started at a meeting at Roustem’s house a few weeks later.

Soon, we went from Rust to a prototype populated in a browser, which marked a huge turning point in the entire 1Password development.

At this point, we realized what our Windows team had been trying to tell us. Rust is the real deal. It is fast, secure, and compiles to all platforms, including WASM.

From the initial stage to a full Linux application

To ensure that our passion for Rust survives in the real world, we made a small library to derive one-time Time-based passwords (TOTP) and use it for all of our desktop and mobile applications. We also used the new WASM brain in our browser extension.

The result was very smooth. There were no surprises other than the rollout went very smoothly.

Rust has proven itself, so now it’s time to see if we can create a complete 1Password core for all of our applications to use. By the end of the fall, we had seen enough internal success that we wanted to find a way to start sharing our progress with our users.

Linux is the perfect playground, so in late 2019, at a meeting at my house, we started creating a Linux application to focus our efforts. Disney had just premiered Mandala, so it was a natural time for us to make a movie, and the Manto project was born. 🤩

This was an interesting proof of concept, since most of our code didn’t use Rust yet, it used the 1Password CLI as a back end (Secrets Automation wasn’t there yet).

Things went so well that we decided to bring in more teams, and in January 2020, we held a larger meeting at Roustem’s home to set it up. A month later, at our annual conference, an early prototype of 1Password for Linux was demonstrated to the entire company on Navigator of the Seas.

Since then, as we’ve become more comfortable with Rust, things have continued to heat up and the team has grown. By August 2020, we soft-launched our first development preview, and then we launched beta in October.

Today, we’re officially launching 1Password for Linux! 🎉

Ironically, we now have an implementation that handles all the grunt work in the background. It’s just like the XKCD 927. Thankfully, this is only temporary, as we worked hard to create a universal 1Password core that will work on every platform in the future.

1Password Core charm 😍

“According to the credo, it’s in your care. This is the way “- Mandala Panzer Division

One of our main goals with the Release of Linux was to create a common core that could be used to interact with 1Password.com and do as much heavy lifting as possible in one place.

We wanted each customer to have a single code base that we could develop once and fix bugs in one place. We also want to structure things in such a way that this common core can’t be used incorrectly.

After some back-and-forth, we decided that the core should basically be a complete client application without a user interface. A headless client, if you will. The core will have its own runtime loop that will handle every aspect of being a 1Password client and provide a secure, hard-to-abuse API for each UI to use.

A more detailed diagram of the 1Password core and how some Rust Crates are put together.

There’s a lot to like about this picture. First, the core is completely independent and has a clearly defined API for client applications to use.

Op-app and op-Uicrates connect all the other pieces together, providing everything each app needs. Most of the state is kept entirely in-house to ensure that keys and other secrets are handled correctly, while also allowing each client UI to focus on their strengths rather than business logic.

Each platform has its own advantages, so FoundationCrate gives the core access to platform-specific services such as kernel key rings and biometrics.

My favorite part is building validation tests. Our BVT is written in Rust to automate testing by using apis and view models directly. Our GitLab CI has jobs to run these tests during merge requests, which takes a lot of the burden off our QA team.

Since we can start as many headless clients as needed, we can test concurrent scenarios or other situations that are difficult to set up manually.

Our dream is to make the entire runtime run in WASM so that we can replace our TypeScript implementation. This should be possible, but it turns out to be quite difficult because the network and WASM environments are quite different, so for now we’ll limit ourselves to using WASM’s generic libraries.

Overall, this new 1Password Core approach has been very successful. Going forward, we’ll be using it in the next generation of 1Password apps coming out next year. 🙌 🏼

1Password Number for Linux

No behind-the-scenes developer post would be complete without some numbers. So, here are some interesting numbers about 1Password for Linux and our development process.

• Number of repositories. 1. How to Play poker 🤘🏽 Well, not everyone celebrates this, but I personally like it. 🙂
• File statistics. 7,528 files totaling 1,038,929 lines, 813,514 lines of code, and 161,095 comments
• Start date: ~18 months ago for Linux; Rust’s experiment started earlier
• First mention of a Linux commit date. 17 December 2019
• Total submissions since December 17:40,575
• Merge requests. 7365
• Problems with creation. 7,529 (assuming every problem is fixed and closed 😉).
• CI/CD: 183,905 pipelines, 1,741,045 operations (65% success rate).
• How many Node package managers are required to create an application? 3.5! During development, we went from NPM to YARN to YARN 2, and then to PNPM. 😂
• Usability testing. 39 1-to-1 meetings with new and existing users
• User surveys. 3
• Release. 12 development previews with 109 changes, 27 beta versions with 460 changes, 83 nightly builds attempted, of which about 50 were released successfully.
• In terms of design, we also did a lot of experiments, from Sketch to Zeplin to Abstract, and then Figma.
• Design highlights. 2,802 Figma components, with 7 designers supporting a team of over 60 developers, engineers and project managers.
• Team members: This is harder to calculate than you might think, because we have a lot of teams outside of development. Our # Core Slack channel has 143 people, which has more than doubled since we started. This is a screenshot of our Linux launch celebration, the Zoom Call.

1Password client application team. At least those who attended the Zoom call. 🙂

Each of the points above is worthy of a blog post of your own, and hopefully one day we can make it a reality. Let me know which ones you like best in the comments and help encourage writing these posts. 🙂

I hope you enjoyed this behind-the-scenes look at our development process, and like the newest member of our 1Password family. 😘

A finished illustration of the red carpet for Linux users

Translation via www.DeepL.com/Translator (free version)