Amazon FSx for Lustre bietet neue Optionen und höhere Geschwindigkeiten für das Arbeiten mit S3-Daten

Amazon FSx for Lustre, ein Hochleistungsdateisystem für Machine Learning, High Performance Computing, Analyse und andere Datenverarbeitungs-Workloads, macht die Verarbeitung von Daten auf Amazon S3 durch die neuen Optionen zum Zurückschreiben der Ergebnisse nach S3 und der bis um das zweifache höheren Datenübertragungsgeschwindigkeiten zwischen FSx for Lustre und S3 noch einfacher.
Quelle: aws.amazon.com

Amazon CloudFront kündigt sechs neue Edge-Standorte in Nordamerika und Frankreich an.

Details: Amazon CloudFront kündigt sechs neue Edge-Standorte, die unser Netzwerk in ihren jeweiligen Regionen um zusätzliche Kapazitäten erweitern. Fünf der neuen Edge-Standorte befinden sich in Nordamerika: Atlanta (2), Chicago, Dallas und Houston. Diese neue Kapazität erhöht die Anfragebearbeitungskapazität von CloudFront in diesen Bereichen um durchschnittlich bis zu 50%. Der sechste neue Edge-Standort befindet sich in Paris, Frankreich. Wie immer verbessert jeder neue CloudFront Edge-Standort die Lieferfähigkeit und Leistung Ihrer Webanwendungen für Ihre Kunden. Eine vollständige Liste der weltweiten Standorte von CloudFront finden Sie auf der Webseite CloudFront – Funktionen.
Quelle: aws.amazon.com

Colfax amplifies the power of its ESAB product portfolio with IoT

If you’ve welded two pieces of metal together, chances are you’ve used equipment from ESAB, a business unit of Colfax Corporation. Started in Sweden in 1904, ESAB was acquired by Colfax in 2012, and offers a broad portfolio of welding, cutting, and gas management equipment to customers in virtually every industry across the globe including agriculture, building construction, energy, light and heavy manufacturing, transportation, and even medical and hospitals. Wherever things are made, ESAB is there.

With the evolution of the Internet of Things (IoT), Colfax saw an opportunity to transform its businesses. What was unique about Colfax’s IoT initiative – named Data Driven Advantage (DDA) –  was their vision of enabling customers to leverage the extensive ESAB portfolio.

Leveraging synergies of a broad portfolio

Many of ESAB’s customers manufacture highly configurable products like tractors, mining equipment, wind towers, and agricultural feed tanks. Imagine the complex process steps involved. First, hundreds of metal pieces are cut and placed on shelves. Then the parts move through the factory. At each step, there is labor required to set up a machine, fuse or cut parts, check for quality, and prepare for the next job. Workers grind metal, weld parts, and refill filler metal. Welds are documented extensively, often manually, for traceability in critical applications like shipbuilding where the paperwork can literally fill a shipping container.

“If customers can leverage the broad ESAB portfolio of welding equipment, power supplies, filler metals, welding tips, mechanized cutters – even helmets, gloves, and protective gear – in a connected way, they can gain incredible insights across their entire manufacturing processes,” explained Ryan Cahalane, Vice President of Digital Growth, Colfax.

Challenges on the IoT journey

Similarly to other companies, Colfax ran into challenges on its IoT journey, including educating executives, dealing with legacy products, ensuring their IoT solution integrated with existing distribution channels and sales teams, and a lack of coordination and inconsistent technology choices across business units.

After years in pilot purgatory, the company accelerated their digital growth success when they refocused on their core differentiation, gaining efficiencies from having a common technology platform across businesses, and leveraging ESAB’s deep application expertise and wide portfolio to unlock value for their customers. As part of this refocus, the company exited the business of developing a custom IoT platform. They selected PTC Thingworx for Azure and the Microsoft Azure IoT platform. “Deploying ThingWorx for Azure provides businesses like Colfax with a much more holistic offering around digital transformation,” said Ron Salvador, Senior Director at PTC.

The re-platforming started in December 2018, and by the HMI 2018 event, ESAB had a field-ready prototype of WeldCloud™ based on Thingworx and Azure that they began testing with customers. By FabTech in the fall of 2018, they had expanded the portfolio to include cutting applications with CutCloud™ and began rolling out the common Thingworx/Azure reference architecture to other business units’ products like orbital welding and gas monitoring.

Customers gain immediate benefits

Customers have seen immediate benefits. “With insights from WeldCloud™, one of our customers realized that they were using one station’s equipment less than 10 percent of the available time, yet it bottlenecked the overall process,” said Ludvig Enlund, GM and head of ESAB’s DDA initiative. “Plus, they were staffing it with expensive labor where the work didn’t require that level of expertise, meanwhile taking away from other more critical processes where higher skill was required. With this awareness, they rebalanced their processes, adjusted labor, and helped gain significant throughput on a line where every product made can be sold.”

With ESAB Digital Solutions, customers now have data to understand how processes, labor, and material contribute to the cost of each part. “In a job shop environment, customers are able to improve their quoting process, increase profit, and win even more business,” said Enlund. Not only do ESAB Digital Solutions like WeldCloud™ and CutCloud™ make otherwise impossible productivity improvements possible, they also give everyone involved in the operation the chance to elevate their performance with data. Some examples include:

•    Operations managers can identify new productivity drivers
•    Quality engineers can trace a defective weld and determine the correct repairs and how to prevent future issues
•    Service technicians can be proactively alerted if equipment has an anomaly or breaks down, allowing for preventative and even predictive maintenance
•    Welding engineers can use data to more quickly set up new test welding processes for new applications

Additionally, ESAB’s approach to digital transformation parallels their collaborative approach to customer preferences, where their broad portfolio provides value even where competitive equipment may be preferred. “Welding may be a science, but its practitioners consider good welding an art and become very attached to the tools they know,” said Enlund. “With the Universal Connector interface and Weld Quality (HKS), WeldCloud™ can easily work with other brands, and deliver most of the same value.”

Partnering to deliver customer value

As ESAB moves forward, it plans to continue focusing on partnerships that deliver customer value, having seen success from close relationships with Microsoft and PTC. “We are taking a page out of the Microsoft playbook,” said Cahalane. “The world is a different place. It’s moving fast. Competitors can become collaborators, and collaboration is key.” The company will continue to expand its Data Driven Advantage initiative across its large portfolio and is now piloting the same common technology platform in its own operations. They expect that the power of standard tools, consistent data models, and modern analytics will increase their own productivity, improve quality, and potentially enable new insights by closing the loop between products in the field and the operations that make them.
Quelle: Azure

Simplify Always On availability group deployments on Azure VM with SQL VM CLI

Always On availability groups (AG) provide high availability and disaster recovery capabilities to your SQL Server database, whether on-premises, in the cloud, or a combination of both. Manually deploying an availability group for SQL Server on Azure Virtual Machines (VM) is a complex process that requires understanding of Azure’s infrastructure, but new enhancements have greatly simplified the process.

We recently published a new method to automate Always On AG deployments on Azure VM with SQL Virtual Machine Resource Provider via Azure quickstart templates. Today, we are proud to share that we have further simplified this automation with Azure SQL VM CLI, the management API for SQL VM resource provider.

Deploying an Always on AG configuration for SQL Server on Azure VM is now possible with following simple steps.

Define Windows Failover Cluster metadata

az sql vm group manages the metadata about the Windows Failover Cluster service that will host the Always on AG. Cluster metadata includes the Active Directory (AD) domain, cluster accounts, and the storage account to be used as cloud witness and SQL Server version. Use az sql vm group create to define the Windows Failover Cluster metadata so that when the first VM is added, the cluster will be created as defined. An example command is provided below.

az sql vm group create -n <cluster name> -l <region ex:eastus> -g <resource group name> –image-offer <SQL2016-WS2016 or SQL2017-WS2016> –image-sku Enterprise –domain-fqdn <FQDN ex: domain.com> –operator-acc <domain account ex: testop@domain.com> –bootstrap-acc <domain account ex:bootacc@domain.com> –service-acc <service account ex:testservice@domain.com> –sa-key ‘<PublicKey>’ –storage-account ‘<ex:https://cloudwitness.blob.core.windows.net/>’

We only support AD domain joined Windows Failover Cluster definition. The FQDN is a must have property and all AG replicas should already be joined to the AD domain before they are added to the cluster.

You can use any existing storage account as a Cloud Witness in the cluster, or you can create a new storage account. An example an Azure CLI command to create the storage account is below:

az storage account create -n <name> -g <resource group name> -l <region ex:eastus> –sku Standard_LRS –kind StorageV2 –access-tier Hot –https-only true

Add SQL VMs to the Cluster – Adding the first VM will create the cluster

az sql vm add-to-group manages adding AG replicas to the Windows Failover Cluster defined above. The cluster is created when the first VM is added to the group. Installing of cluster role on the VM and creating the cluster with the given name is automated by this command. Following add-to-group calls will add next replica to the Cluster.

az sql vm add-to-group -n <VM Name> -g <Resource Group Name> –sqlvm-group <cluster name> -b <bootstrap account password> -p <operator account password> -s <service account password>

You can deploy a new SQL VM instance from Enterprise SQL Server 2016 or 2017 images on Azure Marketplace to use as AG replicas. If you deploy SQL VM from the Azure portal, then it will have the SQL IaaS extension installed and registered with SQL VM RP by default. If you deploy with Azure Power Shell, CLI, or from a non-SQL Server image, you will need to manually follow these steps:

Install the SQL IaaS extension on the virtual machine.
Create a SqlVirtualMachine resource associated with the VM with az sql vm create. An example of this is below:

az sql vm create -n <VM Name> -g <Resource Group Name> -l <region ex:eastus>

You can add an existing SQL VM to the cluster as AG replicas if these prerequisites are followed.

Create an Availability Group through SSMS

Once all SQL VMs are added to the cluster, you can log in to one of them and setup the availability group through SSMS new Availability Group Wizard. At this point, creating the availability group is very simple as all replicas are already added to the cluster.

Create an Availability Group Listener

The last step in the Always On AG configuration is creating an AG Listener to enable automated connection routing after a failover. You can create an AG Listener with the az sql vm ag-listener create command, as shown below.

az sql vm group ag-listener create -n <listener name> -g <resource group name> –ag-name <availability group name> –group-name <cluster name> –ip-address <ag listener IP address> –load-balancer {lbname} –probe-port <Load Balancer probe port, default 59999> –subnet {subnet resource id} –sqlvms <names of SQL VM’s hosting AG replicas ex: sqlvm1 sqlvm2>

AG Listener requires an Internal Load Balancer (ILB) on Azure VMs. If your SQL VMs are in the same availability set, then you can use a Basic ILB, otherwise you need to use a Standard ILB. You can create the ILB via Azure CLI as shown in the example below.

az network lb create –name <ILB name> -g <resource group name> –sku Standard –vnet-name <VNet Name> –subnet <subnet name>

That is all to deploy SQL Server on Azure Virtual Machines with Always On AG Configuration. Start taking advantage of these expanded capabilities enabled by SQL VM resource provider and Azure SQL VM CLI today. If you have a question or would like to make a suggestion, you can contact us through UserVoice. We look forward to hearing from you!
Quelle: Azure

Azure Marketplace new offers – Volume 31

We continue to expand the Azure Marketplace ecosystem. From January 1 to January 15, 2019, 67 new offers successfully met the onboarding criteria and went live. See details of the new offers below:

Virtual machines

Akumo Software: Akumo Software's platform extends datacenter environments between virtualized or cloud-based infrastructure. It provides a consistent and simple way to cost-effectively manage an on-demand datacenter.

BlogEngine.NET on Windows Server 2016: BlogEngine.NET is a lightweight, simple, user-friendly blog engine that can be an excellent alternative to WordPress. Easy to modify and extend, it is specifically designed for .NET developers.

BlogEngine.NET on Windows Server 2019: BlogEngine.NET is a lightweight, simple, user-friendly blog engine that can be an excellent alternative to WordPress. Easy to modify and extend, it is specifically designed for .NET developers.

Conductor4SQL Central Server: This virtual machine comes with all the components required for using Conductor4SQL, including Windows Server 2016, Microsoft SQL Server 2017, Microsoft Power BI Desktop, and Conductor4SQL.

Dell EMC NetWorker Virtual Edition 18.2: Dell EMC NetWorker software provides fast, efficient backup and recovery for enterprise applications and databases.

F5 BIG-IP Cloud Edition: This edition is comprised of per-app VEs and BIG-IQ centralized management. The former provides intelligent traffic management and web application firewall security, while the latter delivers deployment automation, management, and visibility.

GnuCash on Windows Server 2016: GnuCash helps you to track your bank accounts, income, stocks, expenditures, and more. Users have the freedom to run, copy, distribute, study, change, and improve the software. It also works on mobile operating systems.

GnuCash on Windows Server 2019: GnuCash helps you to track your bank accounts, income, stocks, expenditures, and more. Users have the freedom to run, copy, distribute, study, change, and improve the software. It also works on mobile operating systems.

IIS on Windows Server 2019: Key features of IIS on Windows Server 2019 include wildcard host headers, IIS administration PowerShell cmdlets, and improved coalescing of connections to deliver an uninterrupted and properly encrypted browsing experience.

Incorta Free Trial: With Incorta’s Direct Data Mapping engine, you get real-time aggregation of large, complex business data without needing a data warehouse.

InterSystems IRIS Community Edition: InterSystems IRIS is a complete data platform that provides developers the freedom to choose the language and data model best suited to rapidly develop their applications.

InterSystems IRIS Express Edition: InterSystems IRIS is a complete data platform that provides developers the freedom to choose the language and data model best suited to rapidly develop their applications. See additional pricing options for this edition.

Neo4j Enterprise VM Version 3.5: Neo4j's graph database platform helps organizations make sense of their data by revealing how people, processes, locations, and systems are interrelated. This approach powers apps tackling AI, fraud detection, master data, and recommendations.

NetScaler MA Service Agent 13.0: The NetScaler MA Service agent software works as an intermediary between the NetScaler Management and Analytics Service and the NetScaler instances within Microsoft Azure.

Nginx on Windows Server 2016: Features of Nginx on Windows Server 2016 include reverse proxy with caching, IPv6, load balancing, FastCGI support with caching, WebSockets, TLS/SSL with SNI, and the handling of static files, index files, and automatic indexing.

OpenCart on Windows Server 2016: Written in PHP, OpenCart is a free, open-source e-commerce platform available under the GNU General Public License, which allows end users to modify the software.

OpenCart on Windows Server 2019: Written in PHP, OpenCart is a free, open-source e-commerce platform available under the GNU General Public License, which allows end users to modify the software.

Puppet Enterprise: Puppet Enterprise lets you automate the entire lifecycle of your Azure infrastructure simply and securely, from initial provisioning through application deployment.

Pyramid 2018 – Windows Server: Pyramid 2018 lets business users do high-end analytics and data science on any browser or device without needing IT help. It's the next generation of self-service analytics with governance.

SQL 2017 Enterprise Edition w/ ER/Builder: With the ER/Builder data modeler for SQL 2017 on Windows Server 2016, you can manage an unlimited number of tables. You can also create an index, triggers, keys, stored procedures, views, generators, and domains.

SQL Server 2017 Standard Edition w/ ER/Builder: With the ER/Builder data modeler for SQL 2017 on Windows Server 2016, you can manage an unlimited number of tables. You can also create an index, triggers, keys, stored procedures, views, generators, and domains.

SQL Server 2017 Web Edition w/ ER/Builder: With the ER/Builder data modeler for SQL 2017 on Windows Server 2016, you can manage an unlimited number of tables. You can also create an index, triggers, keys, stored procedures, views, generators, and domains.

Strokk Webservices Demo: Wherever a password is used in a web form or an internal application, that piece of knowledge-based authentication can be hardened almost transparently with a behavioral biometrics second factor called keystroke dynamics.

Untangle NG Firewall: Use NG Firewall to connect remote locations and ensure safety, reliability, and performance while providing protection for your data, applications, and users.

Varnish Enterprise 6: Varnish Enterprise (VE), previously known as Varnish Plus, is our commercial/enterprise version of the popular open-source HTTP engine/reverse HTTP proxy Varnish Cache (VC).

VyOS 1.2 LTS: VyOS is a Linux-based open-source network operating system for routers and firewalls.

Windows Virtual Desktop: With Windows Virtual Desktop, Microsoft Office and Windows can be deployed and scaled on Azure in a few moments, including compliance and built-in security.

WordPress on Windows Server 2016: Quickly deploy WordPress on Windows 2016 with built-in MySql and phpMyAdmin. Host as many websites or applications as you need.

Xeams on CentOS: Get this secure and powerful mail server with a strong junk-filtering engine on CentOS. Xeams Community Edition is available as a free software supporting multiple platforms and all mail servers with smart-host functionality.

Xeams on Ubuntu: Get this secure and powerful mail server with a strong junk-filtering engine on Ubuntu. Xeams Community Edition is available as a free software supporting multiple platforms and all mail servers with smart-host functionality.

Xeams on Windows Server 2016: Get this secure and powerful mail server with a strong junk-filtering engine on Windows Server 2016. Xeams Community Edition is available as a free software supporting multiple platforms and all mail servers with smart-host functionality.

Xeams on Windows Server 2019: Get this secure and powerful mail server with a strong junk-filtering engine on Windows Server 2019. Xeams Community Edition is available as a free software supporting multiple platforms and all mail servers with smart-host functionality.

Web applications

Aggregion Blockchain Node: Aggregion operates a blockchain ecosystem enabling major copyright holders to fully control their global end-to-end distribution networks and licensing of digital content. Microsoft Azure products enhance the Aggregion blockchain platform.

Archive One: Archive One is a document management system designed to help document administrators classify, store, secure, search for, and retrieve essential company records. Make compliance and audits easy with Archive One.

Drupal with Azure Database for MariaDB: This solution uses a virtual machine for the application front end and the Azure Database for MariaDB service for the application data. Drupal is an open-source content management system used to create websites and apps.

Lavelle Networks ScaleAon SD-WAN: Lavelle Networks ScaleAon SD-WAN is a hybrid WAN. ScaleAon SD-WAN accelerates cloud adoption for enterprises by seamlessly extending the wide area network (WAN) across physical and virtual resources.

Lightning Network for Azure: This distribution provides a virtual machine instance that runs Bitcoin (btcd), Litecoin (ltcd), and Lightning Network (lnd or c-lightning). You can also run BTCPayServer as a sample application for the node network.

MariaDB Galera Cluster: MariaDB Galera is a multi-master database cluster solution for synchronous replication and high availability. This solution uses multiple virtual machines to replicate your data in a configurable number of nodes.

Mediant VE Session Border Controller (SBC): Enable Microsoft Teams Direct Routing or connect SIP trunks to Skype for Business Server. AudioCodes’ Mediant session border controllers make deployment easier and help users set up multi-SBC network interfaces.

Spanning Backup For Office 365: Spanning Backup for Office 365 provides automatic daily backup and recovery for Office 365 mail, calendars, OneDrive, and SharePoint.

Surge Identity (SaaS): Surge Identity is a cloud-based identity solution that enables secure sign-in using trusted identity and social providers, and it secures app-to-app communication using the latest industry security standards.

Tidal Migrations -Premium Insights for Source Code: Tidal Migrations provides your team with a simple, fast, and cost-effective cloud migration management solution. This add-on empowers your team with actionable insights on the apps you plan to refactor or re-platform to Azure.

TimeXtender Discovery Hub: This virtual machine runs Windows Server 2016 and the TimeXtender Discovery Hub. The Discovery Hub application server for Azure allows customers to build, deploy, and manage an enterprise-grade analytical architecture.

Vnomic Management for SAP Workloads: Select your SAP HANA workload requirements without worrying about underlying technical details. Vnomic will automatically compute and provision a complete and validated SAP HANA workload and deliver it on Azure in minutes.

Consulting services

2008 Windows/SQL End of Support Workshop – 2 days: End of support is looming for Windows 2008 and SQL Server 2008, and Piksel Group's workshop is here to help you understand your options and create an action plan.

2-Hour Azure Migration Briefing: This briefing by Flat Rock Technology will provide high-level information on what it takes to migrate to the cloud and to Microsoft Azure in particular.

Airlines ChatBot: 3 week implementation: This is a conversational AI implementation for airlines over Amadeus/Sabre supporting multiple channels and covering flight booking, status, disruption notification, check-in, boarding passes, FAQs, and more.

Azure Analytics 5-Day Readiness Assessment: Pythian Kick Analytics-as-a-Service puts the power of data analytics in the hands of your business users and solves the data silo problem. This five-day assessment is for customers in Canada.

Azure Analytics 5-Day Readiness Assessment (UK): Pythian Kick Analytics-as-a-Service puts the power of data analytics in the hands of your business users and solves the data silo problem. This five-day assessment is for U.K. customers.

Azure Analytics 5-Day Readiness Assessment (USA): Pythian Kick Analytics-as-a-Service puts the power of data analytics in the hands of your business users and solves the data silo problem. This five-day assessment is for U.S. customers.

Azure Design Assessment: 4-Day Assessment: CDW will review your Microsoft Azure environment to verify configuration and provide recommendations according to best practices.

Azure Governance Workshop: 5-Day Workshop: CDW will provide an in-depth look at the people, processes, and technology currently in place and document a governance plan that enables IT professionals to effectively support business needs.

Azure Jumpstart: 3-Day Implementation: CDW will help your organization choose and implement virtual networking technology. Learn the best approaches to deploying virtual machines, including the associated cloud services and storage accounts.

Azure migration & transformation two-day workshop: In this Azure migration and transformation workshop, Piksel Group will identify candidate cloud services and the benefits of moving to Azure.

Azure migration and transformation briefing (3h): Select from a range of Azure migration and transformation consultancy, implementation, and managed services, starting with a cloud briefing and initial cloud readiness assessment.

Azure Migration Assessment: 2-Day Assessment: CDW will work with you to deploy an assessment tool in your environment, ensure the tool is configured properly, run the tool, and help review and interpret the results.

Azure transformation five-day proof of concept: This Azure migration and transformation briefing, assessment, planning, and proof-of-concept activity by Piksel Group will identify and validate candidate services and benefits.

Cloud Operations & Monitoring: 3-day Assessment: Objektkultur Software will plot a strategy for your migration to the cloud and will support you in your change management process, enabling a conversion that integrates into your system landscape.

Connecting with ExpressRoute: 2 Day Implementation: Your organization will first need to engage with a WAN provider that supports ExpressRoute connectivity. CDW will configure the virtual network and gateway and will assist in establishing an ExpressRoute connection.

Cyber Security PEN Testing: 4 Week Assessment: Networks come under attack every day, and these attacks can disrupt business, create chaos, and cause reputational damage. A penetration test helps you understand how threat actors might penetrate your network.

Envisioning AI for IoT data: 2-day Workshop: This offer by TheDataTeam is for an AI envisioning workshop conducted at the client's site for discovering use cases that are of immense business value and solvable using Azure and TheDataTeam's Intellegion platform.

EOS Migration Pilot: 8-Wk Implementation: This is a pilot migration of legacy Windows Server workloads to Azure using both discovery and containerization tooling.

Leadership Development Solution: The Leadership Development Solution helps K-12 organizations make informed decisions. It provides strategic education services along with an Azure data warehouse and visualizations to improve the leadership placement process.

Migrate to Azure at 20% of Consumption 8-wk Impl: Migrate your workloads and apps to Azure at just 20 percent of your Azure consumption for the first year. This package by NetEnrich combines our tools expertise with our knowledge of datacenter and app migrations.

SAP on Azure / QAS Migration Service: 2-Week POC: This proof of concept involves free Azure consumption credits and a migration service offered by MSR IT Services covering SAP QAS/Sandbox landscapes.

SpotLITE Discovery for Azure MSP: 2 WKS Assessment: Green House Data’s SpotLITE discovery process is designed to determine the overall health of your IT systems and outline a plan to improve operational performance and include Azure.

SSO Using ADFS and ADConnect: 3-Day Implementation: Integrate your on-premises directories with Azure Active Directory to provide a common identity for access to both cloud and on-premises resources, simplifying things for your end users.

Quelle: Azure

Performance best practices for using Azure Database for PostgreSQL – Connection Pooling

This blog is a continuation of a series of blog posts to share best practices for improving performance and scale when using Azure Database for PostgreSQL service. In this post, we will focus on the benefits of using connection pooling and share our recommendations to improve connection resiliency, performance, and scalability of applications running on Azure Database for PostgreSQL. If you have not read the previous performance best practice blogs in the series, we would highly recommend reading the following blog posts to learn, understand, and adopt the recommended best practices for using Azure Database for PostgreSQL service.

Performance best practices for using Azure Database for PostgreSQL
Performance updates and tuning best practices for using Azure Database for PostgreSQL
Performance troubleshooting best practices using Azure Database for PostgreSQL features

In PostgreSQL, establishing a connection is an expensive operation. This is attributed to the fact that each new connection to the PostgreSQL requires forking of the OS process and a new memory allocation for the connection. As a result, transactional applications frequently opening and closing the connections at the end of transactions can experience higher connection latency, resulting in lower database throughput (transactions per second) and overall higher application latency. It is therefore recommended to leverage connection pooling when designing applications using Azure Database for PostgreSQL. This significantly reduces connection latency by reusing existing connections and enables higher database throughput (transactions per second) on the server. With connection pooling, a fixed set of connections are established at the startup time and maintained. This also helps reduce the memory fragmentation on the server that is caused by the dynamic new connections established on the database server.

The connection pooling can be configured on the application side if the app framework or database driver supports it. If that is not supported, the other recommended option is to leverage a proxy connection pooler service like PgBouncer or Pgpool running outside the application and connecting to the database server. Both PgBouncer and Pgpool are developed by the community and can be used with Azure Database for PostgreSQL. As we continue on, we will focus our conversation on PgBouncer in the context of real user experiences.

PgBouncer is a lightweight connection pooler that can be installed on the virtual machine (VM) running the application. The application connects to the PgBouncer proxy service running locally on the VM while PgBouncer service in-turn connects to the Azure Database for PostgreSQL service using the credentials and configuration settings specified in the pgbouncer.ini file. The maximum number of connections and default pool size can be defined in the configuration settings in pgbouncer.ini.

If your application is containerized and running on Azure Kubernetes Service (AKS), you can run PgBouncer as a sidecar proxy. As part of our commitment to provide native integration of best in class OSS databases with Azure’s industry leading ecosystem, we have published a PgBouncer sidecar proxy image in Microsoft container registry. PgBouncer sidecar runs with the application container in the same pod in AKS and provides connection pooling proxy to Azure Database for PostgreSQL. If the application container fails over or restarts, the sidecar container will follow thereby providing high availability with connection resiliency and predictable performance. Visit the docker hub page to learn more on how to access and use this image. For best practices around development with Azure Kubernetes Services, we would recommend to follow the documentation, “Connecting Azure Kubernetes Service and Azure Database for PostgreSQL.”

To give some estimates of the performance improvement when using PgBouncer for connection pooling with Azure Database for PostgreSQL, we ran a simple performance benchmark test with pgbench. pgbench provides a configuration setting to create new connection for every transaction so we leveraged that to measure the impact of connection latency on throughput of the application. The following are the observations with A/B testing comparing throughput with standard pgbench benchmark testing with and without PgBouncer. In the test, we ran pgbench with scale factor of 5 against Azure Database for PostgreSQL running on general purpose tier with 2 vCores (GP_Gen5_2). The only variable during the tests was PgBouncer. With PgBouncer, the throughput improved 4x times as shown below while connection latency was reduced by 40 percent.

PgBouncer, with its built-in retry logic can further ensure connection resiliency, high availability, and transparent application failover during the planned (scheduled/scale-up/scale-down) or unplanned failover of the database server. The retry logic is found to be very useful for OSS applications like CKAN or Apache Airflow using SQLAlchemy. Without the use of PgBouncer, the database failover events require the application service to be restarted for connections to be re-established following a connection failure. In this scenario, it is also important to set connection timeout sufficiently higher than the retry interval to allow retry attempts to proceed before timing out.

To summarize, as new connections are an expensive operation with PostgreSQL, especially for applications which opens new connections frequently, we highly recommend using connection pooling while running applications against Azure Database for PostgreSQL. If the application is not designed to leverage connection pooling out of the box you can leverage PgBouncer as a connection pooling proxy. The benefits of running application with PgBouncer proxy are:

Improved throughput and performance
No connection leaks by defining the maximum number of connections to the database server
Improved connection resiliency against restarts
Reduced memory fragmentation

We hope that you are taking advantage of Azure Database for PostgreSQL. Please continue to provide feedback on the features and functionality that you want to see next. If you need any help or have questions, please check out the “Azure Database for PostgreSQL Documentation.” You can also reach out to us by emailing the Ask Azure DB for PostgreSQL alias, and be sure to follow us on Twitter @AzureDBPostgres and #postgresql for the latest news and announcements.

Acknowledgements

Special thanks to Diana Putnam, Rachel Agyemang, Sudhakar Sannakkayala, Sunil Agrawal, Sunil Kamath, Bhavin Gandhi, Anitah Cantele, and Prabhat Tripathi for their contributions to this posting.
Quelle: Azure