Key Lessons When Moving Towards a Modular Codebase
Note: This article can be found online at mobilesystemdesign.com
I recently had the pleasure of being on Walid Sassi’s podcast where we talked about modular architectures for mobile applications.
Walid asked me some good questions, such as when to use interface modules, how to extract modules from a legacy codebase, and the roles of modules once an app codebase grows larger.
The questions he asked were based on real-world problems and considerations Walid had in his experience.
I recommend checking out the video below. It’s packed with lessons and ideas.
Working with modules warrants its own book, but in this article we’ll share some key lessons, related to the video, that you can apply to your own modular architecture.
Note that these lessons are based on real-world experience; My track record includes leading the transformation of a large-sized, single-workspace app at ING into a large-scale, multi-modular, multi-app environment. This expanded system now serves many countries, each with their own specific variations and regulations. These apps contain a complex mix of local and remote modules, as well as new and legacy code.
I’ve also served as a feature engineer at Twitter, where I joined a company that had already done a great job of figuring out how to scale a mobile app. Thus, giving me multiple perspectives of working in large mobile codebases.
1 What is a module anyway?
First, let’s start by defining the term ‘module’, so we are talking about the same thing.
For this article’s purpose. When we refer to a module, we mean a separate piece of code that can be developed independently , tested, and maintained. A module is standalone, meaning it doesn’t need an app to function, but ultimately will be imported in one, either directly, or via another module.
We can make a module sound imposing and fancy. E.g. Devs might say “I work on a large-scale modular application that supports semantic versioning, a custom package manager, and implements third-party frameworks”.
But don’t let that scare you; A simpler way to look at a module is to think of it as “just” a folder with some files and others folders in it.
I’ll use the term ‘module’ liberally in this article. But, if you want to get into the details, a module can mean multiple things. For example:
Some modules are pure code (e.g. libraries), which can be statically compiled or dynamically linked to in a project.
Some modules contain resources, such as localization files or image assets. Some developers refer to these as frameworks.
Some modules perform a lot of heavy lifting. Such as a giant UI framework on which you build your entire app like Jetpack Compose or SwiftUI. Once a module dictates a way of working, it’s also often referred to as a framework.
When pulling in a module remotely, such as via a package manager. Then developers tend to call these packages or dependencies.
For simplicity’s sake, we’ll call each of these a module. When we talk about a ‘modular codebase’, we mean an app that depends on modules, no matter where these modules come from or what they look like.
Of course modules can make an app's structure more complicated. And, to be honest, developers are good at making things more complicated than necessary.
2 Working without modules
You may wonder if you even want a modular codebase.
For example, let’s say we have an app that allows users to keep track of their workouts. It also offers payment plans to receive support from real-life coaches.
This app may contain a lot of features. Besides the obvious Workout feature, it contains a Payments feature. But it might have a lot more; Such as an Onboarding feature that helps users register, or a Sharing feature to invite friends to do workouts together.
To start, we’ll just focus on the Workout and Payments feature.
These features have types (e.g. classes or structs) inside them. For example, the Workout feature has a WorkoutResultView that depends on the NetworkClient class to submit a workout after it’s completed. PaymentFlowdepends on User to display their name, as well as NetworkClient, and so on.
For simplicity, we are omitting a ton of classes. Both an app and its features would have dozens, if not hundreds, more types.
This app is not “modular”. All code is accessible from within the app. All developers will work together in the same workspace.
The benefit of this approach is that we don’t have to think about modules. We “just” create what we need, make it compile, and we can ship it.
A single workspace is simpler, easy, and often a suitable solution for smaller apps, since we don’t have to deal with the overhead of setting up modules, package managers, different CI or testing environments, and so on.
2.1 Pain points
But now, let’s say the company grows and we are reaching some pain points; The team might grow, and — from a technical point of view — everybody gets in each other’s way, causing frustrations.
For example, the number of merge conflicts is increasing since all developers share the same workspace. More code might break, because everybody is touching shared code. For example, one team-member might update NetworkClient so they can now upload binary files, but now another team’s POST method might accidentally break.
Last, the compile times increase. For serious apps, this can grow from just a minute to 20 minutes or longer on clean builds.
There comes a point that once the app (and team grows), it becomes increasingly more difficult to maintain speed, keep the quality high, and frustrations low.
For these reasons, you may want to consider moving towards a modular codebase, so that everybody gets their own “island” to work on. This ensures they interfere less with each other in day-to-day work.
Let’s see how we could extract modules from an existing codebase while we go over multiple approaches.
3 Moving to a modular codebase
You may wonder whether you should start by moving pre-existing features into modules, or make modules for new features first.
My answer is: Neither, you need to start with lower-level modules. Read on to learn why.
For instance, let’s say we’ll introduce a new Social feature as a module that allows users to keep track of progress pic and videos, and making it easier to share their progress with social media.
This requires a User and the ability to make network calls via NetworkClient.
Unfortunately, we end up with a circular dependency. The app depends on the Social module. But this module depends on the app for a User and NetworkClient.
3.1 Breaking the cycle
To break this cycle, we can take a shortcut, after which we’ll explore a better approach.
A tempting workaround is to think about the Social module’s dependencies. Such as NetworkClient and User, offer these as interfaces inside the module, and then have the app inject concrete types.
The app depends on the Social module. The Social module offers some interfaces, and the app passes (injects) the User and NetworkClient to the Social module.
We have succesfully broken the circular dependency, but at a high cost.
Just for this minor example, we would need to introduce two interfaces containing all used properties and methods to break the cyclical dependency.
Thinking larger scale: That means that for every dependency inside everymodule, we have to offer a way to inject things from the app to the module. It’s not too hard to imagine that if we have 10 modules with hundreds of types, we would have a similar amount of interfaces all over the place.
There would also be duplication, because every module would offer some variation of NetworkClientInterface.
Or imagine that the app has a shared library of views. Now each module needs to get all these views injected into them.
We want to scale up our team and codebase by using modules, but this solution itself doesn’t scale.
Let’s look at extracting pre-existing features and learn why that doesn’t work, either. After that, we’ll take a better approach.
3.2 Extracting current features
Let’s forget the new feature for a second. Perhaps we should start by extracting pre-existing features into their own module? Let’s take the Workout feature and move that into its own module.
With this setup, the app depends on the Workout module. But, the Workoutmodule depends on types inside the app.
Again, we have a similar situation, where we end up with a circular dependency.
Again, we would have to break this cycle, and we’ve learned that introducing tons more types to decouple isn’t the ideal solution.
Let’s try something else.
3.3 Start with lower-level modules
There are more “solutions”, such as introducing god-modules that contain allinterfaces. But to me, those are just more workarounds.
The next solution — which I prefer a lot more — would be to start “bottom-up”, meaning we start by introducing low-level foundational modules, on which feature modules can depend.
These "foundational modules" can depend on platform-specific modules, such as Foundation on iOS. But it doesn't depend on our own code.
We’ve seen how we have User and NetworkClient. Let’s introduce a module called Fundamentals, and place these types in there, because these types have no dependencies themselves.
The app will depend on this module, and both User and NetworkClient are available to any types within the app, still.
We'll fade out the arrows between types to make the modular dependencies more clear.
We have invested into extracting foundational types into their own module. This is preparation work, since there are no obvious benefits yet.
But, now we can more easily introduce a new feature module, as well as extract features into modules.
Let’s discover how.
3.4 Adding a new feature module
Let’s re-introduce the Social feature module. Notice how it’s much easier to introduce it without introducing new types or interfaces. This is possible because the Social module can now directly depend on the new Fundamentals module.
Now it becomes easier to extract a feature module, because there are no more cyclical references.
A useful side-effect is that we didn’t introduce any new types to deal with our modular set up. We “just” started from the bottom, and everything falls into place correctly.
I would argue that this codebase is vastly simpler than our solution containing interfaces, since we now require fewer types.
4 Extracting a pre-existing feature
Besides new features, it now becomes much easier to extract pre-existing features, too.
For example, we can extract the Workout feature, which will also depend on the new Fundamentals module, and it will “just” work.
The app becomes more lean, and there is now clear separation between features.
The key lesson here is: Invest by extracting foundational module(s). After that, you can build feature modules on top.
4.1 Tips for extracting a pre-existing feature
Note that extracting a pre-existing feature — such as Workout — is often more work as opposed to introducing a new feature as a module.
This is because a pre-existing feature can be tightly coupled to an app. Whereas a new feature can start from a clean slate.
To extract a feature into a module, first consider the folder structure. You may originally have one where features are sharing the same folders.
For example, before we introduced any modules, our files and folders would be categorized by models (business logic), and views (anything UI).
% tree
.
├── models
│ ├── NetworkClient.kt
│ ├── TransactionResult.kt
│ ├── User.kt
│ ├── Workout.kt
├── views
│ ├── PaymentFlow.kt
│ ├── WorkoutResultView.kt
│ ├── WorkoutView.kt
This makes it harder to “cut and paste” a feature such as Workout or Fundamentals.
Instead, aim to already “prepackage” a feature into their own respective folder, as if it already is a module.
In the next example, we group types as if they were modules, such as:
% tree
.
├── fundamentals
│ ├── NetworkClient.kt
│ ├── User.kt
├── workout
│ ├── Workout.kt
│ ├── WorkoutResultView.kt
│ ├── WorkoutView.kt
├── payment
│ ├── TransactionResult.kt
│ ├── PaymentFlow.kt
Of course, you can have more sub-folders in each folder. But the point is, the main folders are grouped as if they already are modules.
This makes it easier to extract modules, such as Fundamentals and Workout, because the folder structure already represents a module’s folder.
This is step one.
Now the next step is figuring out how much work you really have left once you move a folder to its own module.
4.2 Checking compiler errors
After creating the Fundamentals module, let’s focus on Workout.
Next, temporarily delete the Workout folder and compile the app. Now you get a good idea of how entangled your feature really is.
For example, if you see that you have 300 locations pointing to Workout’s types, you know you have a lot of work to do.
However, if you only have 1 compiler error in one location, you know it’s much more feasible to extract the feature.
This should give you an idea of how much work you have left, and perhaps, if it’s too much work, it’s better to just stop, even, and focus on other modules first.
4.3 Reviewing the public API
If extracting a module is too painful at this stage, then consider simplifying the public API.
For example, there could be the scenario that the app knows about Workout, and WorkoutResultView, and all its other related types.
Try to shrink the public API. For example, most likely all the app needs is a startWorkoutFlow() method. After which the Workout feature can handle the rest, it can initiate its own flow, views, and so on.
By shrinking the public API, it means that the app is much less coupled to Workout. This makes it a lot easier to extract Workout to its own folder and fix the compiler errors.
4.3 Move the feature
Last, move the feature to the module by physically cutting and pasting the folder into a module’s definitive location.
Next, set up the dependencies, such as ensuring that Workout points to Fundamentals, and get everything compiling again.
Last, try to shrink the public API. As we covered, this makes it easier to decouple a module, but also makes it easier to use a module.
For example, if all an app has to do is call startWorkoutFlow(), then it’s much easier to implement the Workout feature. On top of that, it also makes it much easier to move the Workout feature around in an app.
5 Deciding when to make a module
We’ve covered how to introduce modules to a non-modular setup.
Online you can read advice such as “Place new features into its own module”. This is sound advice when you already have an app that supports modular structures.
But, the problem I have with this advice is that it falls apart once there is nuance, and it doesn’t always apply to all codebases.
We’ve already seen how we ended up with circular dependencies if we don’t start by doing the prework first, such as by introducing low-level modules.
Yes, it is easier to introduce a feature module, as opposed to extracting a module. But, there is more to it than "always introduce a new feature module".
For instance, what if you’re implementing a new single-screen feature? Would you make an entire module for just that screen?
Or, imagine that this feature spans multiple screens. So it appears substantial, and you’d think it warrants its own module. However, it relies on other modules for 90% of its code. In other words, this feature has very little code. Would it make sense to create a whole new module for just a few lines of code?
And what if this feature is similar to another? Perhaps it should go in that other pre-existing module, instead?
And what if your feature isn’t really user-facing, what if it’s a background uploader for large files? It’s an important feature, just not user-facing. Does that warrant its own module?
Or, how would you place current features in a module? How does that work?
Apart from “obvious” features that can comprise their own module, it becomes increasingly more nuanced and difficult to come up with an excellent design for all your source files, folders, and modules inside an application.
Bert and Ernie squabble about this too when cleaning up their toys.
Ernie: “I’m going to store the fire truck in the red toys box.”
Bert: “Clearly, it should go inside the car box.”
If two muppet characters can’t even agree on this, imagine multiple opinionated developers.
Introducing modules is an art, and finding the right “boxes” (modules) will always be tricky.
As a rule of thumb: If you’re making a new feature, check how large the specifications are. If, for example, you’re about to implement a ten-screen onboarding feature. Then yes, it’s more obvious this can go in its own onboarding module because it will have a multitude of files.
If you’re making a user profile screen. Does this really need to be a new module? Now it’s not so clear. You don’t want to end up with 30 modules for 30 screens.
When in doubt, consider adding a feature to the app in such a way that it can be easily extracted if needed. That means: Think of API design, access levels, and place files together so you can easily cut and paste a feature folder into a module folder if needed.
6 Feature modules versus Foundational Modules
We’ve covered how there are many types of modules; Libraries, framework, static, dynamic, packages, and so on.
One mental model that works well for mobile, is to discern a module into two types:
Feature modules These modules are often user-facing and use UI. Feature modules enable a user to perform a task. Such as an
Onboardingmodule that helps a user sign up, or aPaymentsmodule that helps a user complete a financial transaction.Foundational modules These modules support other modules and the app itself. For instance, it could be a
Networkingmodule that offers networking support to features. Or anAnalyticsmodule that allows features to track their events. A user rarely, if ever, directly interacts with these modules.
One foundational module you could consider user-facing is a UIComponents module, offering reusable components as custom views.
But, these views are still unaware of features and business logic, and need to be implemented by feature modules or features inside an app. Hence why we still consider this a foundational module.
In our scenario, the Fundamentals module is considered a foundational module. As well as a newly introduced UIComponents module (depicted below). The Workout and Social modules are considered feature modules.
Looking at an app structure, we can see that feature modules live on top, making use of foundational modules.
The benefit of categorizing modules in these two types is because they have two very distinctive roles and responsibilities. As a result, they often have wildly different API requirements and even affect the organization. Let’s discover how.
6.1 API surface
First, feature modules tend to have a smaller API surface — assuming they are well-implemented. For example, consider the Workout module. It should only need to offer a startWorkoutFlow() method so that a user can start (and finalize) a workout. Perhaps it also offers a WorkoutListView, but in our example, it doesn’t have to offer much more.
Or consider a Payments module that allows a user to complete a payment transaction. Besides some other API methods (such as passing a configuration and other elements), a Payments module will only need to offer a makePayment() method. It can then, under water, render flows respective of a payment provider, such as creditcard or Paypal.
In contrast, a foundational module, such as UIComponents, is likely to have a giant API surface. Expect it to be packed with various UI components, used by allfeature modules and the app itself. That means that all these UI components need to be made public.
Or, consider a Utilities module that contains low-level shared utility classes for all other modules and the app. It may contain types related to localization, multi-threading, and helper functions to deal with arrays and strings. As a result, almost everything inside of this module will be publicly available, used as low-level tools for code in other modules. The API surface would be enormous compared to a feature module.
That means that all feature modules rely on most, if not all, foundational modules.
6.2 Implementations
In contrast, a feature module itself is only implemented in very few locations. For example, it’s highly unlikely that we need to offer an Onboarding feature in multiple locations. It’s more likely it’s only implemented once.
Or consider an Authentication feature that offers a flow where a user authenticates before an important action. This would likely only be used in a handful of locations.
In contrast, every feature that uses localization might import a string helper function from a foundational module.
A larger public API means this module will have more “connections” to an app or other modules. A large public API makes it harder to change things “underwater” inside a module. If a module is versioned, it’s harder to keep stable because the public API is so large.
In contrast, a feature module is easier to rewrite completely, as long as you can keep the small public API stable.
6.3 When something breaks
Another reason to classify modules as either a feature module or foundational module is because it helps us understand quality requirements.
Let’s say you’re in charge of the Workout feature module. This contains a screen called WorkoutResultView that shows the final results of a workout. Now imagine that your team introduces a serious bug that breaks this screen. Because of this bug, the submit button is broken, meaning users can not submit their workout to the server on smaller screens. This means all types of workouts are affected. It’s not great, but at least the bug is restricted to that one feature.
Next, imagine you’re working on a foundational module, such as a Networkingmodule. Imagine that network calls break there. Now all features break, includingthe WorkoutResultView since it can’t submit a workout to the server anymore. The same goes for a Security module or a Utility module’s multithreading helpers. If you break something there, it could be catastrophic.
Simply put: If a feature breaks, that feature breaks. But, if a foundational module breaks, then all features break.
6.4 Ensure foundational modules are stable
That means that foundational modules are often more “important” from a stability point-of-view. Or at least, we can consider them as a higher priority for quality checks and tests.
Foundational modules also tend to live longer. A feature may come and go. But a Networking module will most likely live as long as the app itself. This warrants more care and tests for foundational modules.
As an organization, that means you often want to place the most experienced engineers on foundational modules. But, consider a culture where newcomers and juniors can also contribute to these foundational modules, because having to consider the entire app and all teams can be an excellent way to get people to grow inside an organization.
Working on shared code in foundational modules can offer more perspective and forces developers to communicate more with other teams, giving them new insights and a wider range of requirements.
In contrast, feature teams are more able to work in silos, since they have to “only” consider their own feature, and of course, the app’s users.
7 Modules means silos
Speaking of silos. An enormous benefit of turning a feature into its own module is that it allows teams to create their own independent “island” to work in.
Working in solos sounds can sound detrimental, and can be a sign of Conway’s law, where the app reflects the organization, such as each tab on a tab bar representing one team.
But, the whole point of modularization is so that teams can work independentlyof each other. It can bring speed and avoid issues.
One risk when working this way is that teams can become too insular.
When companies grow larger, making teams feel united warrants its own entire book. But one problem I noticed is that developers tend to only invite their direct team-members to approve their pull requests.
One tip I want to give, is to ensure that you invite developers from other teams to review your feature. It will bring more perspective and involvement across teams.
7.1 Secondary benefits of modules
Besides creating teams that work on features independently, you will get secondary benefits from a modular design.
A module forces you to think about API design, which is always a good thing. Because it makes you conscious of what you expose to an app or other modules. It also makes you think about how someone else would implement your code.
Modules also offer more control. Because you can have internal code for testing (visible code only to the module), and public code for the consumer of your module. Whereas in an app, there is no discernable difference between internal and public code. Both public and internal code is accessible from anywhere inside an app.
Another benefit of a module is that it forces you to place relevant files together. You won’t be able to smear a feature out over 20 folders inside an app, shared by other features. A feature’s files are all grouped together in isolation. Which I think is a great way to work, regardless of whether you’re using a module, because it makes it easier to “cut and paste” a feature into its own respective folder, or to create a module out of it, as we’ve just covered.
Conclusion
These are just some key lessons when starting to work with modules. But, there is a lot more to learn.
Again, check out Walid Sassi’s podcast where I share a lot of key lessons about modular apps.
If you’re hungry for more, check out my GOTO conference presentation Scaling up an iOS codebase. Even though this presentation is older and more iOS centric, there are valuable lessons inside that still apply today, and are suited for multiple platforms.
For example, this talk covers monolithic apps versus modular apps, versus apps that rely on external packages via Semantic Versioning. In this video, I explain how interfaces (protocols, in Swift terms), can make a modular codebase much harder to manage.
Also keep an eye out for the Mobile System Design book, where I will share everything I know about modules so that you can get the most comprehensive understanding.
It will cover API design, modular design, how to handle dependencies across modules, interfaces, app architectures, versioned modules, legacy code with modules, supporting multiple targets, and much more.
This article, any many others, can be found online at mobilesystemdesign.com