Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics. With HDInsight, you get managed clusters for various Apache big data technologies, such as Spark, MapReduce, Kafka, Hive, HBase, Storm and ML Services backed by a 99.9% SLA. In addition, you can take advantage of HDInsight’s rich ISV application ecosystem to tailor the solution for your specific scenario.
HDInsight covers a wide variety of big data technologies, and we have received many requests for a detailed guide. Whether you want to just get started with HDInsight, or become a Big Data expert, this post has you covered with all the latest resources.
Latest content
The HDInsight team has been working hard releasing new features, including the launch of HDInsight 4.0. We make major product announcements on the Azure HDInsight and Big Data blogs. Here is a selection of the most recent updates:
Launch of HDInsight 4.0 at Microsoft Ignite 2018 (Session Video)
Azure HDInsight brings next generation Apache Hadoop 3.0 and enterprise security to the cloud
Deep dive into Azure HDInsight 4.0
HDInsight Enterprise Security Package now generally available
Exciting new capabilities on Azure HDInsight
6-part best practice guide for on premises Hadoop to cloud migration
Azure Toolkit for IntelliJ – Spark Interactive Console
Secure incoming traffic to HDInsight clusters in a virtual network with private endpoint
Apache Spark jobs gain up to 9x speed up with HDInsight IO Cache
Bring Your Own Keys for Apache Kafka on HDInsight
New Azure HDInsight management SDK now in public preview
HDInsight Developer Guide
The HDInsight Developer Guide covers both basic as well as advanced scenarios for developers, data scientists, or data engineers getting started or learning more with Azure HDInsight. This step-by-step guide starts with a basic overview and use-cases, followed by best practices on how to configure clusters, plan capacity, and develop applications for different workloads such as Hive, Spark, HBase and others. Finally, the guide concludes with advanced use-cases and scenarios along with samples.
HDInsight training resources
In addition to the guide, we would also like to highlight the other resources available to learn or know more about HDInsight. Please see below for the different learning resources available for HDInsight including self-paced training, documentation, videos, and more.
Self-paced online trainings
Self-paced online training on edX, an online learning destination, offers high-quality courses from the world’s best universities and institutions to learners everywhere. These self-paced training courses are available for free as part of Microsoft Professional Program for Big Data, or you can add a verified certificate for a fee. These courses have been updated and below are the three specific courses on HDInsight.
Processing Big Data in Azure HDInsight: This course teaches you how to use the Hadoop technologies in Microsoft Azure HDInsight to build batch processing solutions that cleanse and reshape data for analysis.
Implementing Real Time Analytics in Azure HDInsight: In this course, you’ll learn how to implement low-latency and streaming big data solutions using Hadoop technologies like HBase, Storm, and Spark on Microsoft Azure HDInsight.
Implementing Predictive Analytics in Azure HDInsight: In this course, learn how to implement predictive analytics solutions for big data using Apache Spark in Microsoft Azure HDInsight.
Also see self-paced online training on Microsoft Virtual Academy, which provides free online training by world-class experts to help you build your technical skills and advance your career. Ready to continue your big data deep dive? Below are the in-depth course to explore Hadoop and Spark on HDInsight, which are a key part of the analytics portion of MVA Data Series.
Hadoop on HDInsight
Spark on HDInsight
Self-serve documentation
HDInsight Documentation: This is the landing page for HDInsight documentation that is useful to any developer, data scientist, or big data administrator. This documentation includes everything from getting started to specific scenarios and use-cases with HDInsight. You can download the complete documentation using the “Download as PDF” option available on bottom left side of the page, or search for specific topics on the top left search box.
HDInsight Troubleshooting Guide: We are constantly updating the troubleshooting guide so that you can easily debug or troubleshoot issues.
Instructor led training
Whether you’re looking to enhance your proficiency in specific technologies like Azure Machine Learning Studio or in overall architecture of Big Data and Analytics, we’ve likely got a course that can get you on your way. The instructor-led and self-paced video courses span from short webinars, to multi-day workshops, to longer-term deep dives on demand. Check back frequently because new offerings are regularly added by Microsoft and our training partners.
Videos
HDInsight videos: Apart from the above resources, you can also search for specific topics from getting started to advanced topics on Channel 9 or YouTube.
The following videos are great to learn about the scope and features in HDInsight.
Deep Dive on Apache Spark Performance Tuning on HDInsight: Part 1, Part 2, Part 3, and Part 4
New Spark UI extensions for better job performance analysis
Optimizing HBase Performance in HDInsight
Introduction to Apache Kafka on Azure HDInsight
Fine-grained security with Apache Ranger on HDInsight Kafka
Bring your own keys on Apache Kafka with Azure HDInsight
HDInsight: Fast Interactive Queries with Hive on LLAP
Introducing ML Services 9.3 in Azure HDInsight
Compliance Standards on HDInsight
Big Data Partner Program
How to use Machine Learning on Azure Government with HDInsight
StreamSets on Azure HDInsight
2017-18 conference recordings
Ignite 2018
Gaining deeper insights from big data using open source analytics on Azure HDInsight
Five essential new enhancements in Azure HDInsight
DataWorks Summit 2018
Building a Modern Data Warehouse on Microsoft Azure with Azure HDInsight and Azure Databricks
Zero ETL analytics with LLAP in Azure HDInsight
//build
Ingestion in data pipelines with Managed Kafka Clusters in Azure HDInsight
ISV Showcase: End-to-end Machine Learning using H2O on Azure
Interactive ad-hoc analysis at petabyte scale with HDInsight Interactive Query
Real-time data streams with Apache Kafka and Spark
Connect()
Breakpoint debugging of Spark jobs in Azure HDInsight
Hands on labs
Data science lab: This lab specifically focuses on the Spark ML component of Spark and highlights its value proposition in the Apache Spark Big Data processing framework.
Hive lab: This lab focuses on how customers can leverage HDInsight Hive to analyze big data stored in Azure Blob Storage.
Get Microsoft certified on HDInsight
Perform Data Engineering on Microsoft Azure HDInsight
Designing and Implementing Big Data Analytics Solutions
Other Resources
Training to build expertise in Azure
We hope that you will find the developer guide and all the other resources helpful. If you have any feedback or questions, feel free to send us an email at AskHDInsight@microsoft.com. We’d love to hear from you. You can also stay up-to-date on the latest Azure HDInsight news and features by following us on Twitter #HDInsight and @AzureHDInsight.
Quelle: Azure
Published by