To the Cloud
The company decided to migrate the main services to the Google Cloud. That required rewrite the services from Apache Spark to Apache Beam, and use Google Dataflow to run the services. During the migration, we need to ensure that the services were still running and that the data was consistent across the two platforms.