Taking upGrad.com International
A few months back, upGrad decided to expand into international markets and thus came the need to support this expansion within the various client-facing technology products. The first one that required a substantial change was the course listing website — upgrad.com. In this post I talk about how we approached and implemented this, and the things we learnt on the way.
For the first phase of internationalization, the expectation from the platform was that the internal teams should be able to market upGrad’s courses independently in different countries, i.e. the content displayed in each country could be different and it should be possible to maintain it separately. In addition to this, visitors should be auto-redirected to their country-specific URLs when they try opening any page on the upgrad.com website. This is a general behavior across many websites where content is distinguished using certain codes that are appended to the base URL.
An additional requirement that might not seem obvious is maintaining this appended code throughout the visitor session. If someone lands on upgrad.com/us, then all the subsequent routing should be handled keeping the initial location code in context. E.g. if the viewer clicks on a hyperlink or a button that causes routing to /data-science-pgd-iiitb, then the resultant URL should be upgrad.com/us/data-science-pgd-iiitb.
So in all, we had three problems to solve:
- Enable saving location-specific pages in DB and allow respective editing
- Handle client-side routing
- Auto-redirect to these specific pages according to user location
Updating CMS to handle international pages
We have an in-house content management system (CMS), named Apollo, to manage content on upgrad.com. ‘Pages’ can be described as documents in a database mapped to a unique slug. Previously, it was only possible to have single-level slugs which directly mapped to the URLs on upgrad.com. E.g. data-science-pgd-iiitb mapped to upgrad.com/data-science-pgd-iiitb. To allow pages to have a two-level URL like upgrad.com/us/data-science-pgd-iiitb, we introduced a new property in the page document called i18n. This is an object which holds the value for locale which is a unique two-digit code assigned to each country. Locales are good textual identifiers for countries and can be directly mapped to URLs.
While creating pages, editors could specify which country page they wished to create and the value would get attached to the page document once created. We broke the previously used slug identifier for the page into two parts viz. name and i18n.locale.
// Old
{
"slug": "data-science-pgd-iiitb",
...
}// New
{
"slug": "us/data-science-pgd-iiitb",
"name": "data-science-pgd-iiitb",
"i18n": {
"locale": "us"
}
}
The name property would contain the more logical value of the program for which the page belonged to. This could be used to later group all pages falling under the same name. As mentioned in the above snippet, the slug property now included the complete URL that the document would signify, i.e. if requested for the United States page for /data-science-pgd-iiitb, I can expect that I’d get data from the above document. Yet, the page data loading logic isn’t so straight forward, I’ll talk about it ahead.
You might question here why we are maintaining duplicate information in name, slug, and i18n.locale. The answer would be simple — to simplify various fetching scenarios. During this project, we wanted to keep breaking changes as minimum as possible and provide fallback mechanisms wherever possible. The CMS, which we call Apollo internally, was previously only being used by the web clients. But after the introduction of upGrad’s mobile applications, it was extended to Android and iOS mobile clients. Hence, keeping CMS API endpoints and response schema unchanged was of utmost importance.
Fallback Mechanisms
To be able to reach as many locations as possible and simultaneously to keep a low data footprint, international marketing pages followed a region-wise fallback mechanism. If thought in a straight-forward manner, creating a page for each country per program would mean an m * n rise in our data. This would also mean that content maintainers would have to update several pages in order to make changes across all country pages of a specific program. This wasn’t a happy scenario.
To solve this problem the idea was to maintain a minimum number of international pages and serve relevant content from these handful pages to all locations worldwide. This meant that we would have to maintain only a few pages and could update which page had to be served for a particular location on the fly.
Generalizing this, we introduced a configuration object in our APIs. This was a JSON structure which held region wise grouping of locations. The logic was simple, the values at the leaf nodes of this tree denoted individual countries/locations which could potentially have a page associated with them. If they did not, the page fetching API would go up one level and try to fetch a page linked with that locale recursively.
For e.g. if the client requested a page for locale=ae, we would check if ae page existed, if not then check if asia page existed, again if not then return the global page for the requested entity.
// Configuration
{
"global": {
"asia": {
"sg": {},
"ae": {}
},
"europe": {
"gb": {}
}
}
}
Introducing this allowed us to associate countries to specific regions so that we could serve the region data in every country which fell under it. To avoid making this configuration huge, we ensured on our level that a global page variant existed for every page. This ensured that if a page was requested for a country that did not exist in our configuration, it would always fall back to the global page.
The page endpoint now accepted a locale parameter through which the clients could specify which country page was required. This primarily covered most of the compatibility issues, mostly with gradually updating mobile clients, as the endpoints remained the same, and even if no locale query parameter was sent, the API assumed that it was in (India) by default, which was the case before internationalization.
Handling client-side routing
We use Nuxt in the universal mode to serve our marketing pages. Fortunately, Nuxt (and the Vue community in general) has a set of official modules for popular use cases, including internationalization. Thanks to the nuxt/i18n module, a lot of our work was simplified. This provided auto-generation of static routes on startup, client-side routing and also provided various ways to configure them.
At a lot of places, we were redirecting users to URLs in or outside the app. With i18n, this had to be pushed through a transformer function that attached the current locale value to the new URL. nuxt/i18n exposes a function called localePath for this. We found this to be a good opportunity to clean up our existing redirection logic and bring it to a single place by writing a wrapper function to redirect to URLs that internally optionally used localePath if the URL was an internal one.
When the app boots up, nuxt/i18n saves the data related to the current locale from the URL. E.g. upgrad.com/us/data-science-pgd-iiitb sets a value of locale us to the store, with other related information. Using this, the Nuxt app fires a call to the CMS API with the locale query parameter equal to the one set by the module. Post this, the API takes care of returning the exact matching data or applies some fallback mechanism to retrieve the nearest matching one. Once the data is returned, the Nuxt app renders according to the layouts specified in the data.
Auto-redirection to country-specific URLs
This was the most aesthetic feature of the entire project. No matter how much DB and API changes were made behind the scenes, one would look at the URL magically changing to their location code and be amazed. But we as developers knew how much effort went into making this possible along with ensuring that the rest of the platform worked fine as before.
Our existing marketing website is served via CloudFront to achieve low latency by enabling edge-caching. We decided to leverage this setup to add an extra step of routing to the location URLs. Just to restate the problem at hand:
When a viewer opens upgrad.com/data-science-pgd-iiitb from, say, the United States, the request should be redirected to upgrad.com/us/data-science-pgd-iiitb.
This was a perfect use case for on-demand executing functions — AWS Lambda. AWS Lambda has a CloudFront variant called Lambda@Edge, which creates copies of a function and deploys them across the CloudFront distribution network. This can then be configured to execute on certain triggers.
As mentioned in the diagram above, we placed the Lambda function on the origin request trigger. This event fires when CloudFront requests content from the origin server — the function sits between CloudFront and the origin. It is Lambda’s duty to create a redirection or pass on the URL to the origin server. To detect user location, CloudFront’s country headers came in handy. In addition to this, we also used our internal location resolving API powered by MaxMind’s IP database.
To access CloudFront’s country headers, they are required to be added to the whitelisted headers in the behavior configuration. You can find it by the name CloudFront-Viewer-Country .
The Lambda function, implemented in Node.js, roughly looks as follows:
exports.handler = async (event, context, callback) => {
let request = event.Records[0].cf.request;
const uri = request.uri;
const querystring = request.querystring;
const countryCode = request.headers['cloudfront-viewer-country'][0].value; // Skip some URLs which you might not want to redirect
if(shouldSkipRedirection(uri)) return callback(null, request);
// Resolve location from headers or IP
let { locale } = await resolveLocation(countryCode, request.clientIp);
// Create a 302 redirect
let response = {
status: 302,
statusDescription: 'Found',
headers: {
location: [{
key: 'Location',
value: '/' + locale + uri + (querystring ? `?${querystring}` : '')
}]
}
}
return callback(null, response);
}
Once this Lambda was published over to the edge locations, origin requests started to flow through the lambda function and users viewing the website from locations worldwide saw different content. This was the last step (first in the user journey) in the entire flow and if we retract one point at a time, all things fall into place.
When a user opens the upgrad.com website from the US, the lambda detects the location and assigns locale value of us, and redirects to /us before hitting the origin server — the Nuxt app. The nuxt/i18n module takes over and sets the locale value as us in the Nuxt context which leads to an API being fired with the respective query parameter attached. The API server tries looking for the US page in the database and returns the nearest matching document to the client. On receiving this data, the client successfully renders the page and returns it back to CloudFront. It then returns this result back to the viewer and also caches it against the original request URL as well as the redirected URL. And that’s all folks!
Future Scope
This was the first phase of internationalization across the marketing website and has enabled marketing our courses in multiple countries now, and we are able to drive content independently everywhere. The next steps would include adding local language support and allowing manually changing regions. In addition to this, we think there could be improvements in the Lambda approach. Currently, the logic of skipping certain URLs that do not fall under this criteria is a bit scattered. We are thinking of ways to make it more centralized so that it can be configured via a single source of truth. Let us know in the comments if you have implemented something similar to this in your applications, would love to have different perspectives.
This project wouldn’t have been possible without the entire team that made it a success. Kudos to Maitrey, Jitesh, Shahir, Abhishek, Harshita, Omkar, Akshay, Vishal, Yuvaraj & team, and all others who contributed. Do visit upGrad.com to check out our programs that are completely online! If you wish to work with our ever-enthusiastic team, check out our careers page. We are always on the lookout for ambitious, talented folks!
