Dank der Avanced Matrix Extension (AMX) hat Sapphire Rapids die achtfache AI-Leistung, auch Krypto und Netzwerk laufen signifikant flotter. Von Marc Sauter (Xeon, Prozessor)
Quelle: Golem
This post is a guest post from 451 Research’s Jean Atelsek, an analyst working across the Cloud Transformation team and Digital Economics Unit of 451 Research, on moving to hybrid architectures.
Quelle: CloudForms
Rossi Residencial is one of Brazil’s largest construction companies and real estate developers, with more than 115,000 clients in residential and commercial properties. The company had successfully relied on a managed deployment of SAP for its financial operations since 1999, first on premises and later with a private cloud provider. But in 2017, Rossi began a strategic financial restructuring focused on improving its operating efficiency. To support its vision for the business, Rossi needed the flexibility to add or subtract SAP resources based not just on growth, but also on demand, which can vary considerably from year to year. The company decided that the best way to accomplish this would be to migrate its SAP environment to the public cloud.The original cloud provider Rossi chose had become extremely costly as the U.S. dollar rose against the Brazilian Real. What’s more, the Rossi team was less than satisfied with the provider’s service. So the search began for a cloud provider that could offer high availability and scalability, as well as an implementation partner familiar with the company’s previous cloud provider.After researching its options, the company chose Google Cloud as its provider and Sky.One as its implementation partner. “Out of the cloud options we researched, Google Cloud offered us the best financial conditions and a solution that truly catered to us,” says Eduardo Araújo, Rossi Residencial’s IT manager. “And out of the many partners we contacted, Sky.One offered the best work planning and service.”A delightfully uneventful move to Google CloudThe contract with the original cloud provider was nearing its end, which compressed the timeframe for the migration. Given the time constraints, Sky.One and Rossi determined that instead of migrating every app in Google Cloud from scratch, the best approach would be to mirror its architecture completely by migrating all of its virtual machines from the company’s four SAP environment servers directly to Google Cloud. To complete the operation, Sky.One used Google Migrate for Compute Engine, a tool designed to facilitate and streamline this type of migration while reducing risk.Working closely with the Rossi team, Sky.One mapped the source structure in detail to prepare the destination with the integrations and accesses it would need to support the SAP environment, which included ECC with four VM instances for DEV, QAS, PRD, and Legacy. The process took just a month to complete with zero impact on operations — something Rossi had never achieved over three previous cloud migrations. Given that history, the ease of integration between Google Cloud and SAP was a pleasant surprise. “We were worried about potential incompatibilities, and we didn’t know if we would be able to work like before,” Araújo says. “Sky.One spared no effort to make the cloud migration safe and without impacting our operation. Today, we can work even better than before.” Adds Ricardo Nunes, Sky.One’s Solution Expert, “Google Cloud’s solutions support all SAP migration steps, but after migrating, we noticed that daily operations had become even more tightly integrated.” The company’s SAP environment now runs in Compute Engine, a service from Google Cloud for creating and running VMs. Rossi’s SAP data is stored in Google Cloud Storage. As a result, the company has a great deal more flexibility to maintain availability and performance even when resizing VMs. “If I need to open a new branch or break ground on a project, the entire system core is already in the cloud, and I don’t have to worry about local infrastructure,” Araújo explains. “Being in Google Cloud gives us the flexibility to increase or decrease our resources according to our needs. We also don’t worry about the local data center and its updating, generating a very important medium- to long-term saving for us.”Rossi also gained another, even more dramatic savings: Because Google Cloud bills in local currency at a fixed exchange rate with dollars, the company has experienced a 50% cost reduction versus its previous provider.New plans for even more operational improvementsWith the migration to Google Cloud complete, Rossi’s IT team has more time to focus on business needs instead of operational issues. Araújo says the company is now able to consider taking advantage of additional Google Cloud tools — including data analytics and AI — to make the most of its extensive data and improve operations and customer service even further. By moving its SAP environment to Google Cloud, Rossi has not only been able to spend less for better service. It has also gained a flexible, scalable platform upon which to build the efficiencies and customer experiences of its future.Learn more about Rossi Residential’s SAP migration to Google Cloud and hear from other SAP customers about their experiences.Related ArticleCasa dos Ventos advances sustainability mission with SAP S/4HANA on Google CloudThanks to its new cloud infrastructure and scalable services with Google Cloud, Casa dos Ventos was also able to process 20 years of data…Read Article
Quelle: Google Cloud Platform
Data growth is a massive challenge for all organizations, but just as important is ensuring access to the data doesn’t become a bottleneck. For high performance computing (HPC) and ML/AI applications, reducing time to insight is key, and so finding the right storage solution that can support low latency, high bandwidth data access at an affordable price is critical. Today, we are excited to announce that Google Cloud, working with its partners NAG and DDN, demonstrated the highest performing Lustre file system on the IO500 ranking of the fastest HPC storage systems. About the IO500 listIO500 is an HPC storage benchmark that seeks to capture the full picture of an HPC storage system by calculating a score based upon a wide range of storage characteristics. The IO500 website captures all of the technical details of each submission and allows users to understand the strengths and weaknesses of each storage system. For example, it is easy to see if a specific storage system excels at bandwidth, metadata operations, small files, or all of the above. This gives IT organizations access to realistic performance expectations and to help administrators select their next HPC storage system.At Google Cloud, we appreciate the openness of the IO500, since all configurations they evaluate are readily available to all organizations. Even though most users don’t need to deploy at extreme scale, by looking through the details of the Google Cloud submissions, users can see the potential of what is possible and feel confident that we can meet their needs over the long haul. Google Cloud first participated in 2019 in an earlier collaboration with DDN, and at that time the goal was to demonstrate the capabilities of a Lustre system using Persistent Disk Standard (HDD) and Persistent Disk SSD. Lustre on Persistent Disk is a great choice for a long term persistent storage system where data must be stored safely.Since 2019, Google Cloud has released numerous new HPC capabilities such as 100 Gbps networking, larger and faster Local SSD volumes, and pre-tuned HPC VM Images. Working with our partners at NAG, who provided Cloud HPC integration and benchmarking expertise, along with DDN’s storage specialists, we decided to resubmit to the IO500 this year to demonstrate how these new capabilities can help users deploy an extreme scale scratch storage system. When using scratch storage, the goal is to go fast for the entire runtime of an application, but initial data must first be copied into the system and the final results stored persistently elsewhere. For example, all data can start in Cloud Storage, be transferred into the Lustre storage system for the run of a job, and then the results can be copied back to Cloud Storage.We’re proud to report our latest submission ranked 8th and is currently the highest ranked Lustre storage system on the list—quite a feat considering Lustre is one of the most widely deployed HPC file systems in the world. Our submission deployed a 1.8PB Lustre file system on 200 N2 VM instances with Local-SSD for the storage servers (Lustre OSSs), 50 N2 VM instances with Local-SSD for the metadata servers (Lustre MDTs), and 1,000 C2 VM instances using the HPC Image for the compute nodes (Lustre clients). The storage servers utilized 75Gbps networking (to match the read bandwidth of the SSD).All of the performance details are on the IO500 website, but some key results include:On read bandwidth, this is a 12x improvement over our submission in 2019 using Persistent Disk, which really demonstrates the potential for deploying scratch file systems in the cloud. Further, Lustre on Google Cloud is one of only 3 systems demonstrating greater than 1TB/s, indicating that even high bandwidth (and not just IOPs) remains challenging for many HPC storage deployments.Each of our three IO500 submissions demonstrates that while it is possible to build a single monolithic extremely-fast Lustre file system on Google Cloud, it can be more cost effective to tailor the compute/networking/storage deployment aspects to the needs of your application. For example, different levels of price/performance can be achieved by leveraging the range of Persistent Disk options for long lasting persistent deployments and optimizing the ratio of Local-SSD capacity to the number of vCPUs for relatively short lived deployments that really need to go fast.Real-world applicationsFull automation of Lustre on Google Cloud enables HPC workloads such as data-intensive ML training using Compute Engine A2 instances, highly available and high bandwidth SAS Grid analytics, and other HPC applications that need either high-bandwidth access to large files or low latency access to millions to billions of small files.HPC in the cloud is growing faster than the HPC industry overall, which makes sense when you think about how easy it is to spin up very large compute and storage clusters in the cloud in mere minutes. Typically on-premises HPC deployments take many months to plan: procure hardware and software, install and configure the infrastructure, and optimize the application. Google has demonstrated the highest performing Lustre deployment that took a few minutes to deploy with a few keystrokes. Whether for born-in-the-cloud HPC applications, full migration of HPC applications in the cloud, hybrid and burst deployments, or simply for POC evaluations to improve on-prem supercomputers, the elasticity, pay per use, and the lower associated maintenance cost of HPC in the cloud has many of benefits.When deploying both new and existing high performance applications to the cloud, there are a number of decisions to consider, including rightsizing VMs, deployment and scheduling tools, monitoring frameworks and much more. One of the most significant decisions is the storage architecture, as there are many great options in Google Cloud. When high performance storage is done right, it’s magical, flowing data seamlessly to compute nodes at an astounding speed. But when HPC storage is done wrong, it can limit time to insight (and even grind all progress to a halt), cause many management headaches, and be unnecessarily expensive.The use of parallel file systems such as Lustre in Google Cloud fills a critical need for many HPC applications that balances many of the benefits of both Cloud Storage and cloud-based NAS solutions such as Filestore. Cloud Storage effortlessly scales beyond petabytes of data at very high bandwidth and at a very low cost per byte, but requires the use of Cloud Storage APIs and/or libraries, incurs extra operation charges, and has higher latency relative to on-prem storage systems. NAS filers like Filestore include robust enterprise security and data management features, have very low latency, no per-operation charges, all while enabling use of NFS/SMB that allows applications to be seamlessly deployed from laptops and on-premises to the cloud. But users must be mindful of the lack of parallel I/O (which can constrain maximum performance), relatively low capacity limits (currently up to 100TB per volume), and the relatively high cost as compared to Cloud Storage.DDN EXAScaler on Google Cloud, an enterprise version of Lustre by the Lustre open-source maintainers at DDN, delivers a balance of both Cloud Storage and NAS filer storage. EXAScaler enables full POSIX-based file APIs and low-latency access, but also scales to petabytes. As shown by our performance results on the IO500, using DDN EXAScaler on Google Cloud can scale to TB/s of bandwidth and millions of metadata operations per second. Because the features are balanced, the price also ends up being balanced and typically falls somewhere between the cost of using Cloud Storage and the cost of using NAS (although this is highly dependent on the type of storage).We would like to thank the Lustre experts at DDN and NAG for their incredibly valuable insights in tuning and tailoring Google Cloud and Lustre for the IO500. An easy way to get started using DDN’s EXAScaler on Google Cloud is by using the Marketplace console, or for more advanced users, get more control by using Terraform deployment scripts. There’s also an easy-to-follow demo, and if you continue to need some guidance, HPC experts here at Google as well as at NAG and DDN are here to help.Related ArticleRead Article
Quelle: Google Cloud Platform
Enterprise application teams need to manage more than just containers in their software supply chain. That’s why we created Artifact Registry, a fully-managed service with support for both container images and non-container artifacts.Artifact Registry improves and extends upon the existing capabilities of Container Registry, such as customer-managed encryption keys, VPC-SC support, Pub/Sub notifications, and more, providing a foundation for major upgrades in security, scalability and control. While Container Registry is still available and will continue to be supported as a Google Enterprise API, going forward new features will only be available in Artifact Registry, and Container Registry will only receive critical security fixes.Below, we’ll highlight the key improvements Artifact Registry provides over Container Registry, as well as the steps to start using it today.A unified control plane for container, OS and language repositoriesArtifact Registry includes more than just container images: as a developer, you can store multiple artifact formats, including OS packages for Debian and RPM, as well as language packages for popular languages like Python, Java, and Node. In addition, you can manage them all from a single, unified interface. A more granular permission model with Cloud IAMArtifact Registry comes with fine-grained access control via Cloud IAM. Unlike Container Registry, this allows you to control access on a per-repository basis, rather than all images stored in a project. This enables you to scope permissions as granularly as possible, for example to specific regions or environments as necessary.Repositories in the region of your choiceArtifact Registry supports the creation of regional repositories, which allows you to put your artifacts and data directly in the location that they’ll be used, allowing for higher availability and speed. In Container Registry, you’re limited to “multi-regions”: for example, the closest multi-region for Australia is Asia. However, with Artifact Registry’s regional support, you can create a repository directly in the Sydney data center.A pricing model that respects your regionWhile Artifact Registry’s pricing is still based on a combination of network egress and storage usage, support for regional repositories means that you can choose in what region to host your container repositories. Although per unit storage costs are higher for Artifact Registry, optimizing the locations of your repositories to be hosted in the same region where they are used can result in cost savings, because any network traffic within the same region is not considered egress and is thus free.Part of a secure supply chainArtifact Registry was designed from the ground up to integrate into our suite of secure supply chain products. This means that it can optionally use Container Analysis to scan your container images for vulnerabilities as they’re uploaded to Artifact Registry, and works directly with Binary Authorization to secure your deployments.We’re here to help you migrateIf you already use Container Registry, you can take advantage of all the current and upcoming features of container image storage with Artifact Registry by migrating to it. To help, we’ve prepared the following guides:Transitioning from Container Registry provides an overview of how to use Artifact Registry instead of Container Registry in a backwards-compatible wayCopying images from Container Registry guide you to move container images from an existing repository to an Artifact Registry repositoryIf you’re currently hosting your container images with a third party, you can begin using Artifact Registry directly, by following the instructions in our guide, Migrating containers from a third-party registry, which shows you how to avoid rate limits on image pulls or third-party outages which can disrupt your builds and deployments.And if you’re just getting started storing container images, you can begin using Artifact Registry as your image repository right away. To learn how, check out Artifact Registry quickstart for Docker, a guide to using Artifact Registry as a single location for managing private packages and Docker container images.Join our community Our Artifact Registry communities are also great resources to help answer your questions and for guidance on best practices: Ask questions on Stack Overflow using the google-artifact-registry tagVisit the Google Cloud Slack community and ask a question in the #artifact-registry channel. If you haven’t already joined the Slack community, use this form to sign up.Related ArticleNode, Python and Java repositories now available in Artifact RegistryExpanded language support lets you store Java, Node and Python artifacts in Artifact Registry, for a more secure software supply chain.Read Article
Quelle: Google Cloud Platform
So far in this series, we’ve been focused on generic concepts and console-based workflows. However, when you’re working with huge amounts of data or surfacing information to lots of different stakeholders, leveraging BigQuery programmatically becomes essential. In today’s post, we’re going to take a tour of BigQuery’s API landscape – so you can better understand what each API does and what types of workflows you can automate with it.Leveraging Google Cloud APIsYou can access all the Google Cloud APIs from server applications with our client libraries in many popular programming languages, from mobile apps via the Firebase SDKs, or by using third-party clients. You can also access Cloud APIs with theGoogle Cloud SDK tools or Google Cloud Console. If you are new to Cloud APIs, see Getting Started on how to use Cloud APIs. All Cloud APIs provide a simple JSON HTTP interface that you can call directly or via Google API Client Libraries,Most Cloud APIs also provide a gRPC interface you can call via Google Cloud Client Libraries, which provide better performance and usability. For more information about our client libraries, checkout Client Libraries Explained.BigQuery v2 APIThe BigQuery v2 API is where you can interact with the “core” of BigQuery. This API gives you the ability to manage data warehousing resources like datasets, tables (including both external tables and views), and routines (functions and procedures). You can also leverage BigQuery’s machine learningcapabilities, and create or poll jobs for querying, loading, copying or extracting data. Programmatically getting query resultsOne common way to leverage this API is to programmatically get answers to business questions by running BigQuery queries and then doing something with the results. One example that quickly came to mind was automatically filling in a Google Slide template. This can be especially useful if you’re preparing slides for something like a quarterly business review – where each team may need a slide that shows their sales performance for the last quarter. Many times an analyst is forced to manually run queries and copy-paste the results into the slide deck. However, with the BigQuery API, Google Slides APIand a Google Apps Script we can automate this entire process!If you’ve never used Google Apps scripts before, you can use them to quickly build serverless functions that run inside of Google Drive. Google Apps Scripts already have the Google Workspace and Cloud Libraries available, so you simply need to add the Slides and BigQuery service into your script.In your script you can do something like loop through each team’s name and use it as a parameter to run a parameterized query. Finally, you can use that to replace a template in that team’s slide within the deck. Check out someexample code here, and look out for a future post on more details on the entire process!Loading in new dataAside from querying existing data available in BigQuery, you can also use the API to create and run a load job to add new data into BigQuery tables. This is a common scenario when building batch loading pipelines. One example might be if you’re transforming and bringing data into BigQuery from a transactional database each night. If you remember from our post on tables in BigQuery, you can actuallyrun an external query against a Cloud SQL database. This means that we can simply send a query job, through BigQuery’s API, to grab new data from the Cloud SQL table. Below, we’re using the magics command from the google.cloud.bigquery Python library to save the results into a pandas dataframe.Next, we may need to transform the results. For example, we can use the Google Maps GeoCoding API to get the latitude and longitude coordinates for each customer in our data. Finally, we can create a load job to add the data, along with the coordinates, into our existing native BigQuery table.You can access this code in our sample Jupyter Notebook. However, if you were using this in production you may want to leverage something like a Google Cloud Function.Reservations APIWhile the “core” of BigQuery is handled through the BigQuery v2 API, there are other APIs to manage tangential aspects of BigQuery. The Reservations API, for example, allows you to programmatically leverage workload management resources like capacity commitments, reservations and assignments as we discussed in a previous post. Workload managementLet’s imagine that we have an important dashboard loading at 8am on the first Monday of each month. You’ve decided that you want to leverageflex slotsto ensure that there are enough workers to make the dashboard load super fast for your CEO. So, you decide to write a program that purchases a flex slot commitment, creates a new reservation for loading the dashboard and then assigns the project where the BI tool will run the dashboard to the new reservation. Check out the full sample code here!Storage APIAnother relevant API for working with BigQuery is theStorage API. The Storage API allows you to use BigQuery like a Data Warehouse and a Data Lake. It’s real-time so that you don’t have to wait for your data, it’s fast so that you don’t need to reduce or sample your data, and it’s efficient so that you should only read the data you want. It’s broken down into two components.The Read Client exposes a data-stream suitable for reading large volumes of data. It also provides features for parallelizing reads, performing partial projections, filtering data, and offering precise control over snapshot time.The Write Client (preview) is the successor to the streaming mechanism found in the BigQuery v2 API. It supports more advanced write patterns such as exactly one semantics. More on this soon!The Storage API was used to build a series of Hadoop connectors so that you can run your Spark workloads directly on your data in BigQuery. You can also build your own connectors using the Storage API!Connections APIThe BigQuery Connections API is used to create a connection to external storage systems, like Cloud SQL. This enables BigQuery users to issue live, federated, queries against other systems. It also supports BigQuery Omni to define multi-cloud data sources and structures.Programmatically Managing Federation ConnectionsLet’s imagine that you are embedding analytics for your customers. Your web application is structured such that each customer has a single-tenant Cloud SQL instance that houses their data. To perform analytics on top of this information, you may want to create connections to each Cloud SQL database. Instead of manually setting up each connection, one option could be using the Connections API to programmatically create new connections during the customer onboarding process.Data Transfer Service APIBigQuery data transfer serviceallows you to automate work to ingest data from known data sources, with standard scheduling. DTS offers support for migration workloads, such as specialized integrations that help with synchronizing changes from a legacy, on premise warehouse. Using the API you can do things likecheck credentials, trigger manual runs and get responses from prior transfers.Data QnA APIThe last API I’ll mention is one of my favorites—Data QnA which is currently in preview. Are business users at your organization always pinging you to query data on their behalf? Well, with the QnA API you can convert natural language text inquiries into SQL – meaning you can build a super powerful chatbot that fulfills those query requests, or even give your business users access to connected sheets so they can ask analytics questions directly in a spreadsheet. Check out this post to learn more about how customers are using this API today!Wrapping upHopefully this post gave you a good idea of how working with the BigQuery APIs can open up new doors for automating data-fueled workflows. Keep in mind that there are lots of more relevant APIs when working with BigQuery, like some of the platforms we discussedlast week in our post on Data Governance- including IAM for managing access policies, DLP for managing sensitive data, andData Catalog for tracking and searching metadata. Next week is our last post for this season of ourBigQuery Spotlight YouTube series, and we’ll be walking through monitoring use cases. Remember to follow me on LinkedIn and Twitter!Related ArticleBigQuery Admin reference guide: Data governanceLearn how to ensure your data is discoverable, secure and usable inside of BigQuery.Read Article
Quelle: Google Cloud Platform
Ein bisschen 5G-Signal geht angeblich auch unter die Erde. Telefónica dementiert einen Ausbau im Untergrund. (5G, Telekom)
Quelle: Golem
Die Gesundheitsfunktionen von Smartwatches führen bei manchen Patienten zu regelrechter Angst um die Gesundheit – mit belastenden Folgen. (Smartwatch, Medizin)
Quelle: Golem
Was am 19. August 2021 neben den großen Meldungen sonst noch passiert ist, in aller Kürze. (Kurznews, Microsoft)
Quelle: Golem
Bei Katastrophen könnten laut ARD SMS-Warnungen beziehungsweise Cell Broadcast helfen. Besser sei 5G Broadcast. (ARD, Technologie)
Quelle: Golem
