Node.JS Turned This E-Commerce Giant Around: A Developer’s Memoir

Node.JS Development Technology

Some of our developers, who worked for over 10 years in Java, have been bleeding Java bytecodes and breathing abstract interfaces. In 2011, most of them found themselves learning Node.JS. After 8 years, they remember it was instant love.

Probably this love for Node.JS turned us into a Node.JS development company – an idea which faces a fierce opposition from our other development teams.

Let’s fast rewind to Pierre Omidyar’s San Jose living room in September 1995. It was exactly then and there the AuctionWeb was founded. It was meant to be an online platform that permitted efficient peer-to-peer trading in an auction format. This online site provided a space for people to buy, sell, and trade any number of items from almost any category.

In September 1997, AuctionWeb was renamed eBay.

By the beginning of 2019, eBay has:
• 180 million active buyers.
• 25 million sellers.
• 1.2 billion total number of listings.
• $10.7 billion as revenue (2018).

Trivia: In 2000, Justin Timberlake’s half eaten French toast was sold for $1,025.

At eBay, engineers have an approach of re-inventing how to build web front end in a way that scales to hundreds of teams and handle monstrous traffic. Over the past few years, their applications have been migrating to a Node.JS stack using Marko as the view layer. A lot is still changing at eBay where they ensure to tackle any amount of traffic load and still deliver top-notch customer service.

Coming back to our Node.JS developers, they now act as if Node has taken over the programming world and are non-stop talking about its advantages. Give it a beak, guys!

It’s true that it’s tough to escape the gravitational pull of JavaScript and its omnipresence in the wild wild web. But with all its huge libraries and frameworks, it never did deploy any competitive back-end platform. Then Node.JS happened, and it smashed all that jinx offering viable alternatives to those limitations of JavaScript.

eBay & Node.JS Development

eBay mostly runs on a Java-based tech stack. Java and JVM constitute its entire workflow. eBay required an enormous amount of stability and ability to handle that scale of traffic, so only a proven technology could be the logical choice. However, the engineering team has been open to new technologies, and they have been mulling over Node.JS for some time.

Let’s see how eBay partnered with Node.JS to develop its first Node.JS application.

Scale

A group of eBay engineers have been trying to find ways to bring an eBay Hackathon-winning project to production. They realized that Java will not be enough to meet the project requirements, so started exploring possibilities of other frameworks, specifically Node.JS.

The primary requirements of the project were:
1. Make the application as real time as possible, i.e., maintain live connections with the server.
2. Manage a huge number of eBay-specific services that display information on the page.

The team had concerns about scalability on Java, and so they went for building a mid-range application from scratch. Node.JS seemed like a good fit.

Influencing Perceptions

eBay was rooted in Java and so it was almost impossible to convince folks to use JavaScript on the back end. There were a lot of questions arising about type safety, error handling, scalability, etc. Moreover, JavaScript being JavaScript (the most misread language) has the quality of attracting debates. To deal with all these questions and apprehensions, the team opened an internal wiki page and invited engineers to express their concerns, doubts and questions about Node.JS development.

In a couple of days, the team had an encyclopedic list of queries and questions to work on. Not quite contrary to the expectations, the most common questions centered around the reliability of the stack and efficiency of Node.JS in handling functionalities which has already been implemented in Java. The team went through all the questions painstakingly and answered each one of them with real-world examples. At times, the questions were eye-openers to the team and pushed them to think in various perspectives.

By the end of the exercise, the core value of Node.JS was recognized by people, and in fact, some of the arguments against Node.JS did become part of those which leverage the functionality of the framework.

Once the team put an end to all speculations and apprehensions, it was show time!

Building from Scratch

They started from tabula rasa. The guiding principle was to build a bare minimum boilerplate Node.JS server that scales. They added four node modules as dependencies – express, cluster, request and async. They decided on MongoDB for data persistence since it is easy to use and also to leverage its existing infrastructure at eBay. With the basic setup being done, they got the server up and running in the developer boxes. The server accepted requests, managed a few APIs and survived some data.

The team did some end-to-end testing, configuring their front end servers to point to the Node.JS server, and all seemed to work fine.

They spent a lot of time designing data model and schema, identifying the best Node modules for the tasks and started implementing each end point. Then for the next few weeks they were like ostriches in a desert – burying their heads down doing nothing but coding.

Deploy

When the application gained some stability, it was needed to move it to a staging environment. Next step is to deploy the Node.JS stack. Their deployment objectives were simple – build once and deploy everywhere. They were just hoping Node.JS to work as easy and seamless as possible.

They could leverage their existing cloud-based deployment system. They wrote a shell script and ran it through Hudson CI job. Hudson is activated when the code is checked into the master branch. This job builds the Node.JS bundle and pushes it to the deployment cloud. Finally it chooses an environment, QA, staging or pre-production, and activates the application on the machines.

This let the Node.JS web service run in various stable environments. eBay engineers were amazed at the speed and simplicity by which the whole deployment setup worked.

They said, Node.JS is happiness!

Monitor

eBay already has a brilliant monitoring dashboard built on top of the log data. This system could generate reports and real-time alerts if anything goes wrong. The team hooked the Node.JS stack to the centralized logging system and had a similar monitoring system set up. They developed a logger module and implemented three different logging APIs: code level logging, machine level logging and load balancer level logging.

At the machine level logging, which occurs at the cluster module level, they extended the NPM Cluster Module to create an eBay-specific version.

The log data formats were created in such a way that they matched with the Java-based logs. This way people didn’t feel much of a difference using it because it generated the same dashboards and reports.

The asynchronous nature of Node.JS event loop presented some challenges for logging, like crossing of the logging transactions. But in no time the engineers came up with both short-term and long-term solutions for that.

The Day of Jubilation

The Hackathon project had gone live. It was the first eBay application to run its back end completely in Node.JS. The feedback on performance after an internal launch was extremely encouraging.

eBay’s development team had developed a full-fledged front end stack on Node.JS to leverage most of their implementations.

Today, the technology stack of eBay, for application and data, contains ES6, Apache Tomcat, Cassandra, Hadoop, Oracle, Marko and more along with Node.JS and JavaScript. eBay’s engineers have come a long way to build a robust, dynamic front end and back end systems to handle the most busiest traffic on the site.

Bring it on Christmas and New Year!!

Inspiration for a Node.JS Development Company

Node.JS developers at Spericorn have always been thrilled by the fact that they are able to run the server-side code and the front-end code without the need to learn a completely different language. As a Node.JS development company, we have numerous projects running on Node.JS which allows handling of thousands of concurrent connections with a single server with the least threads concurrency.

A huge part of our web development pivots around JavaScript frameworks, especially Node.JS for backend. Being a Node.JS development company, we see this as an opportunity to exploit all possibilities that Node.JS brings to the table.

If you are also equally excited by the opportunities Node.JS presents or have a compelling idea to change your business around, let’s have a talk.