Migrating data, technology and people to Google Cloud

Editor’s note: Bukalapak, an ecommerce company based in Jakarta, is one of Indonesia’s largest businesses. As their platform grew to serve over 100 million customers and 12 million merchants, they needed a solution that would reliably and securely scale to handle millions of transactions a day. Here, they discuss their migration to Google Cloud and the value added from its managed services.Similar to many other enterprises, Bukalapak’s ecommerce platform did not originate in the cloud. It was initially built leveraging on-premises technologies that worked quite well at the beginning. However, as our business grew—processing over 2 million transactions per day and supporting 100 million customers—it became challenging to keep up with the necessary scale and availability needs. It wasn’t uncommon to see traffic spikes following promotional events, which were frequent. Our infrastructure and overall architecture, however, just wasn’t designed to handle this scale of demand. It was clear we needed a new way to support the success of the business, a way that would allow us to scale to meet fast-growing demand, while providing the best experience to our customers, all without overburdening our team. This led us to implement significant architectural changes, and consider a migration to the cloud.Choosing Google CloudGiven that this migration would be a large and complex endeavor, we wanted a partner in this journey, not just a vendor. We started by evaluating the product and services portfolio of potential providers, along with their ability to innovate and solve cutting-edge problems. With our very limited experience in the cloud, it was critical to have an experienced professional services team that could effectively guide and support us throughout the migration journey. We also evaluated the overall cost and the availability of data centers in Indonesia that would allow us to comply with government requirements for financial products. Finally, we needed to plan for how we would attract and retain talent, so we looked at the degree of adoption across the providers across Southeast Asia, and specifically Indonesia. After careful consideration across these areas, Google Cloud was the right choice for us.Embarking on the cloud migrationOur on-premises deployment included over 160 relational and NoSQL databases, We also maintained a Kubernetes cluster of over 1,000 nodes and over 30,000 cores, running 550 production microservices and one large monolith application. To address the large amount of technical debt our platform had, we decided against a lift-and-shift approach. Instead, we spent a good deal of time refactoring our services, particularly our monolith application (a.k.a., the mothership), and partitioning our databases. Enhancing our monitoring and alerting, deployment tooling, and testing frameworks were critical to improve the quality of our software, development and release processes, and performance and incident management. We also invested heavily in automation, moving away from manual testing to integration testing, API testing and front-end testing. Adopting the toolings and best practices of DevOps, MLOps and ChatOps increased our engineering velocity and improved the quality of our products and services. For a team that had very limited cloud experience, it was clear early on that this was not just a technology migration. It involved a cultural migration as well, and we wanted to ensure our team could perform the migration while gaining the skill set and experience needed to maintain and develop cloud-based applications. We started by training a smaller team, which took on the task of migrating our first services. Incrementally, we worked on expanding the training, and looping in more and more engineers in the migration efforts. As more engineering teams got involved, we paired them with one of the engineers who joined the migration early on and acted as a coach. This approach allowed us to transfer knowledge and roll out best practices, incrementally but surely, across the entire organization. We took a multi-step approach for the migration. We started by focusing on the cloud foundation work, introducing automation and new technologies like Ansible and Terraform. We also invested heavily in establishing a strong security foundation, onboarding WAF and Anti-DDoS, domain threat detection, network scanning, and image hardening tools, to name a few. From there, we started to migrate the smaller, simpler services and worked our way up to the more complex. That helped the team gain experience over time while managing risk appropriately. In the end, we successfully completed the migration in just 18 months, with very minimal downtime.  Managed services for greater peace of mindOur team selected Cloud SQL early on as the fully managed service for most of our MySQL and PostgreSQL databases. We appreciated how easy Cloud SQL made it to manage and maintain our databases. With just a few simple API calls, we could quickly set up a new instance or read replica. Auto-failovers and auto-increasing disk size ensured we could run reliably without a heavy operational burden. In addition to Cloud SQL, we’ve now been able to integrate across the other Google Cloud data services, including BigQuery, Data Studio, Pub/Sub, and Dataflow. These services have been instrumental in helping us process, store, and gain insights from a massive amount of data. That in turn allowed us to better understand our customers and consistently find new opportunities to make improvements on their behalf.Google Cloud’s managed services give a greater peace of mind. Our team spends less time on maintenance and operations. Instead, we have more time and resources to focus on building products and solving problems related to our core business. Our engineering velocity has increased, and our team has access to Google’s cutting-edge technology, enabling us to solve problems more efficiently. In addition, our platform now has higher uptime, and can scale with ease to keep up with unpredictable and growing demand. We also were able to improve the overall security of our platform and now have a standardized security model that can easily be implemented for new applications. The larger impact has been on what our lean infrastructure team is now able to accomplish. Migrating to Google Cloud gave us the strategic and competitive advantages we were looking for. Both throughout the migration and now that we’re running in production, Google Cloud has been a great partner to us. The Google Cloud team put a lot of effort into understanding what we needed to be successful, and advocating for our needs, often connecting us to product teams or experts from others in the organization. Their desire to go the extra mile on behalf of their customers made our experience positive and ultimately made our cloud migration successful.Learn more about Bukalapak and how you can migrate to Google Cloud managed databases.Related ArticleTo run or not to run a database on Kubernetes: What to considerIt can be a challenge to run a database in a distributed container environment like Kubernetes. Try these tips and best practices.Read Article
Quelle: Google Cloud Platform