Firebase: Introducing Bulk User Export and Custom Claims
Firebase team just introduced a set of new user management APIs to the Firebase Admin SDKs. Amidst a plethora of incremental updates, this release also brings the full armada of user management capabilities to the Admin Go SDK — something it has lacked until now. I wrote an article about the user management support in Admin SDK last summer. Moreover, Firebase provides detailed examples on how to use these APIs. Therefore today I will just focus on a couple of new features Firebase introduced this fall— bulk user export and custom claims support. These two features are now available in all flavors (Node.js, Java, Python and Go) of the Admin SDK.
Bulk user export (listing users)
Sometimes it is necessary to enumerate or iterate over all the user accounts associated with a Firebase project. Some example use cases include listing active user accounts on a dashboard page, locating and deleting inactive accounts, and migrating user data from one system to another. The preexisting user management APIs in the Admin SDK (getUser(), getUserByEmail()) support retrieving only one user account at a time. In contrast, the new API enables retrieving user accounts in bulk with pagination. Listing 1 shows how to use this API in Java:
The listUsersAsync() method returns a page of users. The null argument instructs the SDK to load users from the start (i.e. from the first page). The starting point is set by Firebase Auth in a deterministic way. By default the page size is 1000 users, which also happens to be the maximum allowed page size. If necessary, a smaller custom page size can be specified as the second argument to listUsersAsync(). To retrieve another page of users, simply call getNextPage() on an already retrieved page. Or call getNextPageToken() to obtain a page token string, and pass it to listUsersAsync().
If all you want is to just iterate through all the user accounts in your project without having to explicitly deal with pagination, you can simply call page.iterateAll() as shown in line 13 of listing 1. This returns an Iterable, which can be used as the target of a for-each loop. It takes care of pagination under the hood, while ensuring that no more than 1000 users are buffered in memory at any time. Therefore it can be used to efficiently iterate through millions of user accounts without blowing up the program heap.
Listing 2 shows the same API as available in the Admin Go SDK. The syntax is clearly different, but note that the familiar abstractions like pages and page tokens are present here as well.
Unlike Java and Python, Go does not have a built-in iterator type. Therefore the Admin Go SDK makes use of a custom iterator package, which implements pagination and iteration in a generic way. Developers who have been using various Go libraries related to Google Cloud Platform (GCP) should find this package familiar.
In general, all four Admin SDK implementations have similar APIs and semantics when it comes to bulk user export. You can use any of them to iterate through all the user accounts in a Firebase project, without being constrained by memory.
Custom claims
Most web and mobile apps deal with several types of users with varying levels of privileges. For example, imagine a content publishing app with writers, moderators, and guests, where access to data should be regulated based on each user’s role within the app. In this type of scenarios, developers need a simple and secure way to provision user accounts with the appropriate roles and privileges.
In the Firebase ecosystem, this usually entailed minting a custom JWT for each user where the privileges and roles are encoded into the JWT as claims. A user would then sign in using his/her custom JWT, and obtain a Firebase ID token in return. This ID token carries the claims added to the original custom JWT, and can be examined from Firebase security rules to enforce access control. If you need a refresher of this feature, here’s a quick tutorial from Jen Person.
While this feature is quite powerful, it is not really designed for access control. The purpose of custom JWTs is to integrate custom authentication systems, and third-party auth providers with Firebase. For access control, ideally we need a mechanism that is amenable to users signing in via any auth provider (Google, Facebook, Email+Password, custom etc). To that end, app developers should be able to set custom claims directly on the user accounts, as opposed to the user credentials. This is exactly what the new Admin SDK APIs aim to facilitate.
A developer can now specify a small number of custom claims on a user account using the Admin SDK. Whenever that user signs in, Firebase will include those custom claims in the issued ID token. This happens regardless of the auth provider in play, and the resulting ID token can be inspected via Firebase security rules as usual.
Listing 3 illustrates a Java method that sets two custom claims on a given user. Lets assume that by doing so it marks the user as a moderator of some hypothetical app.
Note that the new API accepts custom claims in the form of a map. Firebase serializes the specified claims into JSON, and includes them in the user’s ID token next time the user signs in. If the user is already signed into the app, the new claims take effect when the user’s current ID token expires. ID tokens automatically expire every hour (unless refreshed forcefully), and therefore the updates to claims should take effect without much delay.
ID tokens are passed alongside all the RPC calls made by Firebase apps. They get processed and validated in a number of systems along the way. Therefore a large ID token can incur a non-trivial computational and network overhead on apps. For this reason it is highly recommended that the number of custom claims set on a user is kept to a minimum. Refrain from storing arbitrary user data as claims, other than the critical bits of information necessary to implement access control. The Admin SDKs enforce this best practice by putting a 1000 bytes cap on the custom claims. That is, the JSON representation of the custom claims map is not allowed to exceed 1000 bytes.
Listing 4 shows a Firebase Realtime Database rule that examines the custom claims on the user’s ID token. This rule regulates access to the /moderators portion of the database. Specifically, it allows only those who have the claim role = moderator on their ID tokens, to read and write to the protected section of the database.
It is also possible, and useful to access custom claims on client-side Firebase apps. Listing 5 shows a Firebase JavaScript client that inspects custom claims to determine which portions of the UI should be rendered depending on the privilege level of the user.
At the moment the client-side code must parse and decode the user’s ID token to extract the claims embedded within. In the future, the Firebase client SDKs are likely to provide a simpler API for this use case.
Custom claims support is a quite versatile feature, and many developers have been eagerly waiting for it. A lot can be achieved with custom claims, especially when combined with the other user management capabilities of the Admin SDK. For example, imagine a Python-based Cron that iterates through all the user accounts in a project, and incrementally adjusts the privileges assigned to each user. Or imagine a Go web service or a Node.js Cloud Function that updates the claims on a user account in response to a certain event. Potential for bridging Firebase apps with their server-side counterparts just got a lot bigger.
Conclusion
The release on December 8th brings the different flavors of Admin SDK to a state of parity in terms of their user management capabilities. I would love to know what the Firebase community think about these new features. What would you implement with them? What challenges are you facing? What other related features would you like to see in the Admin SDK? Let me know in the comments, and also reach out to the Firebase opensource community with bug reports, feature requests and patches.
Also keep in mind that Auth is a fundamental component of Firebase that evolves rapidly, with new features and APIs being introduced regularly. With the last release barely out the door, a new set of APIs are already being implemented in the Admin Node.js SDK to support revoking refresh tokens. But that’s a story for another day.