How BigQuery helped Theta Labs and NASA bring science and hope to streaming

Editor’s note: We’re hearing today from Theta Labs, a leading decentralized video streaming platform that is powered by users and decentralized on a new blockchain. With their peer-to-peer bandwidth sharing distributed ledger technology, Theta Labs has been able to revolutionize the livestream experience. By adopting Google Cloud, Theta Labs has been able to scalably meet a growing active user base on their blockchain platform which in turn, has helped them expand their strategic partnership with NASA, including hosting the latest SpaceX rocket launch.When we established Theta Labs back in 2016, the goal was to set up a streaming video service with an emphasis on rendering popular PC video games like League of Legends, CS:GO and Dota2 into immersive 360° virtual reality experiences. And yet, thanks to our unique approaches to streaming, video rendering, and patented blockchain video technology, we’ve grown into something so much bigger that we’ve even caught the attention of NASA. All of this was possible thanks to Google Cloud and their databases and analytics products such as BigQuery, Dataflow, Pub/Sub and Firestore. Reaching the heights of video streamingBack when we first launched Sliver.tv—now Theta.tv—we decided to differentiate ourselves from the competition by creating unique live streaming video experiences, especially for streamers and viewers in regions with little or no access to high-speed internet. Our blockchain-based peer-to-peer video delivery technology lets users share their bandwidth with others, letting our streamers reach audiences they never could before.It was this ability to reach more unique and remote viewers and to give larger audiences the opportunity to discover new things that caught NASA’s attention. NASA saw the potential in our service to spread interest in science and technology to an audience of mostly younger viewers. They gave us the privilege of becoming one of only four or five video services with direct access to NASA’s source video feed, and we recently collaborated to premiere NASA’s August Women’s Equality Day  broadcast.The biggest highlight of this partnership so far was the opportunity to livestream the SpaceX launch. In a year where we all needed a bit more hope, being able to bring the live launch of a space shuttle to a wider audience was an amazing experience, inspiring so many to reach for the stars.Video of the stars starts with the cloudTo facilitate an event as large as a space launch with so many viewers takes a powerful infrastructure. To do all of this with our unique peer-to-peer blockchain system that rewards viewers and streamers for sharing bandwidth, we needed Google Cloud’s reliable, scalable, and stable infrastructure. With the strength of Google Cloud and their help on creating auto-scaling DevOps solutions, we were able to reach more viewers than ever without hitting the VM caps that previously caused issues with latency and customer experience. Previously, we’d faced challenges like infrastructure scaling limitations to meet application demands, high costs, and too much of our time wasted on managing and maintaining solutions.Google Cloud offers us better scalability, so we’re no longer capped by the number of active streamers we could have on our platform. Google Cloud gave us:Performance, and flexibility of implementationBreadth of capabilities and supportAbility to ingest streaming data for real-time insightsRelationship & communication with the Google account teamExpansive feature optionsPrice point compared to offered features/servicesOur partnership with Google Cloud has also let us reach viewers in regions that normally would have trouble accessing streaming video. Edge computing allows most of the computation work to be done near the source, improving response times and bandwidth usage—a perfect synergy leveraging Google and Theta Network’s core strengths. And with Google Cloud’s over 1600 nodes, we are able to get closer to our users than ever before.Running analytics on our skyrocketing dataBeyond the video streaming, Google Cloud’s enterprise data warehouse BigQuery gave us the capacity to do the typically difficult—if not impossible—task of sorting real-time data from the blockchain system. We have built a real-time pipeline for the viewership data using Dataflow, Pub/Sub and BigQuery. A Dataflow job continuously pulls the data from a Pub/Sub topic and ingests into BigQuery. We’ve seen Pub/Sub quickly ingest roughly 12,000 to 14,000 blocks of data containing 60,000 to 200,000 transactions daily into BigQuery for real-time analysis. We also used Pub/Sub and Dataflow to create the listener/subscriber for the topic our ETL pipeline publishes, then ingest that into BigQuery tables. By running fast queries in BigQuery, we were able to uncover findings such as: How many people watched and shared a certain video stream in the past hour;How many donations were made to a streamer in total;Which livestream has the highest donation to viewer ratio;What was the most impactful moment during a livestream.Prior to BigQuery, finding this information required writing customized scripts to analyze the blockchain raw data and the analysis used to take hours or even days of engineering time. Now we can gain such insights in a little as a few seconds, and effectively in real time. Now we can gather information to let streamers, advertisers, and partners know when more viewers were online and engaged. This means that NASA and other content creators could better find and reach their audiences. Results that let us grow and scale to the moon and beyondGoogle Cloud helps us better to forecast how many concurrent users we need to support during livestream events and predict multi-variable reputation scores for our network of thousands of edge and guardian nodes to identify and address bad actors and under-performing nodes. Today, our BigQuery environment has 45GB of data, which contains almost 7.5 million blocks and 57 million transactions and counting. We migrated to Google Cloud in less than six months, and saw the return on investment almost immediately. We’re able to bring top-notch connectivity, scalability and security capabilities for our branded content partners like NASA, enterprise validator partners including Google and community members that run Theta edge and guardian nodes, and we’re reducing costs over time.All of this is just the beginning of the ways we’re looking to spread more entertainment, science, and hope during these dark times. And thanks to Google Cloud’s strength and scalability, we’ll be able to keep growing, reaching even more audiences and partners.Learn more about Theta Labs here.
Quelle: Google Cloud Platform