Scale your Python service with Managed Disks

The Azure Python SDK now supports Azure Managed Disks!

 

Azure Managed Disks and 1000 VMs in a Scale Set are now generally available. Azure Managed Disks provide a simplified disk management, enhanced scalability, and better security. It takes away the notion of storage account for disks, enabling developers to scale without worrying about the limitations associated with storage accounts. This post provides a quick introduction and reference to consuming key service features from Python.

 

From a developer perspective, the Managed Disks experience in Azure CLI is idomatic to the CLI experience in other cross-platform tools. You can use the Azure Python SDK and the azure-mgmt-compute package 0.33.0 to administer Managed Disks. You can create a compute client using this tutorial. The complete API documentation is available on ReadTheDocs.

Standalone Managed Disks

Prior to Managed Disks, developers needed to maintain images for their VMs in multiple storage accounts to avoid the risk of running out of disk space. It is easy to see how this can complicate the architecture, and the dev-ops, for a service that requires a large number of VMs quickly, and has to be available across multiple regions. With Managed Disks, you do not need to worry about replicating images into new storage accounts. You can have a single image per region, and the service will make sure they are available for up to 10,000 VMs under a single subscription.

 

You can create new disks from various starting points with a lines of Python code. Here are a few specific examples:

Create an empty Managed Disk
Create a Managed Disk from Blob Storage
Create a Managed Disk from our own Image

 

Here’s a quick preview for creating an empty Managed disk in Python with a few lines of code:

from azure.mgmt.compute.models import DiskCreateOption

 

        async_creation = compute_client.disks.create_or_update(

            &;my_resource_group&039;,

            &039;my_disk_name&039;,

            {

                &039;location&039;: &039;westus&039;,

                &039;disk_size_gb&039;: 20,

                &039;creation_data&039;: {

                    &039;create_option&039;: DiskCreateOption.empty

                }

            }

        )

        disk_resource = async_creation.result()

Virtual Machine with Managed Disks

Now that you know the basics of creating managed disks, but how do you configure your service to create VMs from images stored on a Managed Disk? The service affords flexibility to create VMs from various types of Managed Disks. You can create a VM with an implicit Managed Disk for a specific disk image. Creation is simplified with implicit creation of managed disks without specifying all the disk details. You do not have to worry about creating and managing Storage Accounts.

 

A Managed Disk is also implicitly created when provisioning a VM from an OS image on the Azure Marketplace. Here’s an example for a Ubuntu VM. Notice how the storage account parameter is optional in the VM definition.

 

storage_profile = azure.mgmt.compute.models.StorageProfile(

                image_reference = azure.mgmt.compute.models.ImageReference(

                    publisher=&039;Canonical&039;,

                    offer=&039;UbuntuServer&039;,

                    sku=&039;16.04.0-LTS&039;,

                    version=&039;latest&039;

                )

            )

 

You can easily attach a previously provisioned Managed Disk as shown here. See a complete example on how to create a VM in Python (including network), and how check the full VM tutorial in Python.

Virtual Machine Scale Sets with Managed Disks

For very large scale services, Azure recommends using Virtual Machine Scale Sets (VMSS). VMSS allows developers to create a pool of VMs with identical configuration. The service allows “true autoscale” – developers do not need to pre-provision VMs. Prior to Managed Disks, the developers needed to consider the design carefully to ensure efficient Disk IO, ideally using a single storage account for up to 20 VMs. This limitation required developers to create and manage additional storage accounts to support a larger scale. With Managed Disk, you don’t have to manage any storage account at all. If you are used to the VMSS Python SDK, your storage_profile can now be exactly the same as the one used in VM creation. This feature also simplifies programming – you no longer have to manage any storage account at all.

 

The official guide to transitioning from user managed storage to Managed Disks is available in this article. Quick samples are also available for preview.

Get productive with the Azure CLI

If the CLI is your management tool of choice, there are several handy commands available for various scenarios. For example, here’s how you can create a stand alone Managed Disk from the Azure CLI with a single command:

 

az disk create -n myDisk -g myResourceGroup –size-gb 20

 

Check out Aaron Roney’s blog post to learn more CLI commands for programming Managed Disks.

Other operations

There are numerous other quick management operations you might need to get started with Managed Disks. See sample code for the following operations:

Resizing a managed disk from the Azure CLI

Updating the Storage Account type of the Managed Disks

Creating an image from Blob Storage

Creating a snapshot of a Managed Disk that is currently attached to a Virtual Machine

In summary

Managed Disks can tighten your workflow, simplify your service architecture, and offer you greater peace of mind in running a highly scalable Python cloud service. It also offers better reliability for Availability Sets by ensuring that the disks of VMs in an Availability Set are sufficiently isolated from each other to avoid single points of failure, and offers better security via granular role based access to resources. You can use the Azure CLI to create and manage your Managed Disks. Hopefully this blog post serves as a quick reference as you try Managed Disks on your own. For more information about the service, head over the Azure documentation. For feedback on the Python SDK, please send an email to azurepysdk@microsoft.com.
Quelle: Azure

After A Big Bank Bought Simple, Things Got Complicated For Customers

Scott Beale / Flickr / Via Flickr: laughingsquid

Customers of Simple — the 21st century bank with no branches or checkbooks, just a website, sleek interface, and debit cards — signed up because they wanted banking to be just that: simple. But after European banking behemoth BBVA Compass acquired Simple for $117 million three years ago, customers say the process of transitioning their accounts to the new bank has been rife with headaches and complication. And for Simple customers who were not US citizens, the transition wasn’t initially possible at all.

To integrate with BBVA, Simple had to rebuild its banking system, which took two years, according to a company spokesperson. When that process was complete, Simple began the process of transferring individual accounts from Bancorp, which previously backed Simple, to BBVA. This transition requires customers to provide multiple pieces of identification, transfer funds, and reconfigure features like direct deposit or autopay.

“It&;s opening a new bank account, which is why we fully understood that there would be frustration,” said Simple spokesperson Amy Dunn. “It’s a terrible experience no matter what. We tried to make it as painless as possible.”

Despite Simple warning customers of the pending transition back in April 2016, things didn’t go smoothly for everyone.

For example, in Google Play reviews and on Twitter, US residents here on green cards or work visas complained that they were unable to transition to new Simple accounts. A spokesperson for Simple said the problem arose because BBVA “didn&039;t have the existing infrastructure to open accounts for permanent residents who aren&039;t citizens.”

BuzzFeed News first inquired about this issue on Friday; since then, Simple — which recently publicly reiterated its commitment to immigrants’ rights — says it’s resolved the problem. “We’ll be able to approve new applicants with a green card or Visa (so long as they fit all other qualifications), and our team will be following up with customers who were impacted in the coming weeks,” Dunn wrote in an email. “If they&039;ll have us, we&039;d like to welcome them back.”

Dunn couldn’t say how many immigrants have returned to Simple since the fix was made, but the overall number of non-citizens using Simple is very small. However, plenty of people had other problems.

Neil Robertson of Denver opened a Simple account after a bad experience at Wells Fargo. Because he’s a musician who’s often paid in cash, Simple’s lack of branch locations can be annoying for him. But not as annoying as when his Simple debit card recently got declined at a bar despite him having plenty of money in his account.

“[Simple was] pretty exciting, but they were acquired and things seemed to stagnate.”

“It wasn&039;t just my card getting declined, which is always a weird feeling, but when I pulled up the app, I had to expose what&039;s in my account to show it to [the bartender],” Robertson said. “And every time you keep swiping that card, my account is saying it&039;s going through — $22, $22, $22.” Robertson didn’t ultimately lose any money, though he says he’s still waiting for “several hundred dollars” to show up in the new account.

Robertson transitioned onto the new BBVA-backed Simple platform in late January, but as of yet, his old account hasn’t closed; for him and other customers, this awkward transition period has caused confusion — it’s unclear which account actually holds their money. Robertson says he’ll keep his Simple account until he can find a better option, but not everyone else is as patient.

One woman said she quit Simple after suddenly being unable to use her debit card while travelling abroad. Another woman said she was temporarily cut off from her funds after her account was closed without sufficient warning as part of the transition process.

And other customers are having problems with automatic bill payments still being withdrawn from their old accounts.

Gordon Emmerson’s power company charged him $24 for an unexpected overdraft after his money had already been moved to his new Simple account. After some back and forth, Simple agreed to cover the fee. But Emmerson is no longer a customer. “That whole episode caused me to lose faith in the company and worry about the safety of my money, so I’m in the process of transitioning to a local credit union now,” he said. “I wouldn’t recommend Simple anymore, that’s for sure.”

Simple says it “can’t speak to specific customers’ accounts or issues for security reasons” but described many of these complaints as one-off issues. A spokesperson pointed to tweets from people who praised its handling of the transition, and said the majority of feedback has been positive. And, indeed, it seems like a lot of customers are happy with the service; even some who initially had a problem with the transition will be sticking with Simple.

“Simple was founded on the belief that a banking company can be on the same side as our customers,” said Dunn in an email. “We believe in our mission to help people change their relationship with money, and to help people feel confident and empowered instead of ashamed, confused, and taken advantage of.”

But the challenge for Simple, which is owned by a big bank, will be retaining the customers who came to it to get away from big banks.

“This isn’t the first time Simple has faced this problem — some of its customers are also frustrated by the fact that BBVA Compass is one of more than a dozen banks invested in the Dakota Access Pipeline. Though Simple says it “neither financially nor philosophically” supports the pipeline, the issue serves as another example of how a big bank isn’t what Simple customers signed up for.

For Aaron Shekey, the real sign of trouble with Simple came after he transitioned accounts, when he (and thousands of other Simple users) received a marketing email from BBVA about a Valentine’s Day promotion. Simple had told their customers that BBVA would never contact them.

“[Simple was] pretty exciting, but they were acquired and things seemed to stagnate,” said Shekey, who is a programmer and startup founder. “I think they&039;re the best of the online banks, but the email from BBVA felt like a breach of privacy and trust.”

Simple sent an email to customers apologizing for the mistake, and said in an email statement, “We had put agreements in place up-front to prevent this sort of cross-marketing from happening, and we’ve all been working to make sure that’s the case moving forward.” But another Simple customer, Ben Lopatin, is still worried. “Now I’m not sure what other data BBVA has access to and ‘isn’t supposed to use’,” he said via DM.

“They aren&039;t going to be &039;simple&039; to use anymore. If anything, BBVA is probably going to ruin this.”

Simple always expected it would lose a few customers in the transition process, but so far, with 87% of accounts moved over to the BBVA platform, it says it hasn’t lost a significant amount of business. The company says it’s grown 68% percent in the last year, and has around the same number of accounts as a bank with 1,010 branches with just a fraction of the corporate overhead.

Still, BBVA has lost money on its investment in Simple every year since the acquisition, with the total write-down in value since 2014 totaling $89.5 million, according to the Houston Chronicle.

In a statement to BuzzFeed News, BBVA said valuations for tech companies are “very volatile.” “As with most investments, return is delivered in the medium to long term,” the statement continues, “and BBVA Compass remains committed to Simple and its continued success and ongoing growth.”

Also still committed to Simple is Sean Ferguson, a customer who plans to keep his account despite what he calls “completely irresponsible” behavior because it still makes it easier for him to manage his money.

“Part of why I signed up for Simple is that it was 100% online and ridiculously easy to use. Painless transfers. Easy debit card management. Effortless deposits,” he said via DM. “So yeah, I&039;m still with them and they fixed my problem, but I definitely feel like they aren&039;t going to be &039;simple&039; to use anymore. If anything, BBVA is probably going to ruin this.”

Quelle: <a href="After A Big Bank Bought Simple, Things Got Complicated For Customers“>BuzzFeed

RDO Ocata released

The community is pleased to announce the general availability of the RDO build for OpenStack Ocata for RPM-based distributions, Linux 7 and Red Hat Enterprise Linux.
RDO is suitable for building private, public, and hybrid clouds. Ocata is the 15th release from the OpenStack project, which is the work of more than 2500 contributors from around the world (source).

The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Linux and is a member of the CentOS Cloud Infrastructure SIG.
The Cloud Infrastructure SIG focuses on delivering a great user experience for CentOS Linux users looking to build and maintain their own on-premise, public or hybrid clouds.

All work on RDO, and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first.

Interesting things in the Ocata release include:

Significant Improvements to Tempest and Tempest plugin packaging in RDO

The OpenStack-Ansible project now supports deployment on top of CentOS with the help of RDO-packaged dependencies

For cloud operators, RDO now provides packages for some new OpenStack Services:

Tacker: an ETSI MANO NFV Orchestrator and VNF Manager
Congress: an open policy framework for the cloud
Vitrage: the OpenStack RCA (Root Cause Analysis) Service
Kolla: The Kolla project provides tooling to build production-ready container images for deploying OpenStack clouds

Some other notable additions:

novajoin: a dynamic vendordata plugin for the OpenStack nova metadata service to manage automatic host instantiation in an IPA server
ironic-ui: a new Horizon plugin to view and manage baremetal servers
python-virtualbmc VirtualBMC is a proxy that translates IPMI commands to libvirt calls. This allows projects such as OpenStack Ironic to test IPMI drivers using VMs.
python-muranoclient: a client for the Application Catalog service.
python-monascaclient: a client for the Monasca monitoring-as-a-service solution.
Shaker: the distributed data-plane testing tool built for OpenStack
Multi-architecture support: aarch64 builds are now provided through an experimental repository – enable the RDO ‘testing’ repositories to get started

From a networking perspective, we have added some new Neutron plugins that can help Cloud users and operators to address new use cases and scenarios:

networking-bagpipe: a mechanism driver for Neutron ML2 plugin using BGP E-VPNs/IP VPNs as a backend
networking-bgpvpn: an API and framework to interconnect BGP/MPLS VPNs to Openstack Neutron networks
networking-fujitsu: FUJITSU ML2 plugins/drivers for OpenStack Neutron
networking-l2gw: APIs and implementations to support L2 Gateways in Neutron
networking-sfc: APIs and implementations to support Service Function Chaining in Neutron

From the Packstack side, we have several improvements:

We have added support to install Panko and Magnum
Puppet 4 is now supported, and we have updated our manifests to cover the latest changes in the supported projects

Getting Started

There are three ways to get started with RDO.

To spin up a proof of concept cloud, quickly, and on limited hardware, try the All-In-One Quickstart. You can run RDO on a single node to get a feel for how it works.
For a production deployment of RDO, use the Quickstart and you’ll be running a production cloud in short order.
Finally, if you want to try out OpenStack, but don’t have the time or hardware to run it yourself, visit TryStack, where you can use a free public OpenStack instance, running RDO packages, to experiment with the OpenStack management interface and API, launch instances, configure networks, and generally familiarize yourself with OpenStack. (TryStack is not, at this time, running Ocata, although it is running RDO.)

Getting Help

The RDO Project participates in a Q&A service at ask.openstack.org, for more developer-oriented content we recommend joining the rdo-list mailing list. Remember to post a brief introduction about yourself and your RDO story. You can also find extensive documentation on the RDO docs site.

The rdo channel on Freenode IRC is also an excellent place to find help and give help.

We also welcome comments and requests on the CentOS mailing lists and the CentOS and TripleO IRC channels (centos, centos-devel, and tripleo on irc.freenode.net), however we have a more focused audience in the RDO venues.

Getting Involved

To get involved in the OpenStack RPM packaging effort, see the RDO community pages and the CentOS Cloud SIG page. See also the RDO packaging documentation.

Join us in rdo on the Freenode IRC network, and follow us at @RDOCommunity on Twitter. If you prefer Facebook, we’re there too, and also Google+.
Quelle: RDO

Microsoft Networking Academy with the Azure Black Belt Team – Winter 2017!

Welcome to our new networking webinar series! We’ve changed the name of our bi-weekly talk to better reflect what we want to convey to you, our customers and partners! And this comes with a new name: Microsoft Networking Academy !

The Microsoft Network Academy session is taking place every other Friday this winter and spring. It is open to all customers and partners to learn more about Azure Networking, including ExpressRoute and Virtual Networking, and how to plan and design their connectivity to the Microsoft Cloud.

As a team, we’ve decided to create two formats for our Microsoft Networking Academy series: the introductory sessions, and deep dive sessions, generally presented in alternance.

In both formats, there will be an open Q&A session at the end where customers can ask the experts. Content and partner speakers will vary for each session, but the general agenda is as follows:

Introductory sessions

Azure Networking fundamentals (10 minutes)
Deep dive topic of the week (15-20 minutes)
Partner spotlight of the week (15-20 minutes)
Q&A

Deep dive sessions

Introduction (5 minutes)
Deep dive topic of the week (35-45 minutes)
Q&A (10 minutes)

We will post the agenda in advance on this blog, and to our interested viewers, you can join a distribution list by sending an email to gbb-anf@microsoft.com with the subject-line “Join Microsoft Networking Academy List”. We will email you a reminder and the agenda in advance for the upcoming sessions.

We kicked off this new Academy Series on Friday, February 17th, 2017.

Join the Skype Meeting and make sure you don’t miss out on future sessions by adding this the series to your Outlook calendar. You can also download ICS (same agenda entry for both introductory and deep dive sessions).

Here are a few links for convenience:

Session recordings for Microsoft Networking Academy will be posted on Channel 9
Previous sessions are already posted on Channel 9’s Azure Networking Fridays channel
Check list presented in our sessions
Fall 2016 season’s sessions and early winter 2017 sessions

February 3rd’s – Introductory session call recap:

Azure Networking Fundamentals
Technical overview of Azure Load Balancing with our new team member, Bryan Woodworth!
Partner Spotlight – Riverbed’s SD-WAN solution accelerates the adoption of Azure via unique automation algorithms, cloud-centric management workflows and an industry-leading focus on app performance. Come see how this really happens!
Links to the deck and to the video recording on Channel 9

February 17th’s – Deep Dive session call recap:

Quick introduction and announcement
Deep dive on the 3 ExpressRoute peerings with Eddie Villalba
Ask the Experts Q&A!
Links to the deck and to the video recording on Channel 9

March 3rd’s – Introductory session call agenda:

Azure Networking Fundamentals
Partner Spotlight – Citrix’s SD-WAN solution
The sessions will be recorded and posted on Channel 9!

Quelle: Azure

Universal access: An early prototype for graphical VNC console in Ironic

The post Universal access: An early prototype for graphical VNC console in Ironic appeared first on Mirantis | Pure Play Open Cloud.
Ideally, users should be able to have a universal experience when it comes to accessing their nova instances, but when integrated as part of OpenStack deployment, nova instances deployed on Ironic baremetal nodes have certain limitations compared to standard virtual machines created by nova. In particular, it is not currently possible to access the graphical VNC console of these instances via the Horizon Dashboard.
To fix this problem, the ironic community has started to work on introducing a framework for graphical console access for baremetal nodes. Because each hardware vendor implements a different way of providing graphical console access, the framework is planned to be quite generic, leaving details of the actual graphical console configuration and enablement to a proposed GraphicalConsole interface of an ironic driver.
One interesting type of hardware to consider in this regard is Dell servers supporting iDRACv7 or newer (PDF). The iDRAC firmware on such servers supports native access to the server’s graphical console over the OpenVNC-compatible protocol directly, without the need for proprietary VNC proxies or clients. An administrator who has appropriate access to the iDRAC configuration can enable this built-in VNC server and set the password, connection timeout and SSL encryption options.
In order to test the VNC capabilities of such hardware, I have implemented a prototype of a graphical console interface for the DRAC driver. It uses the WS-MAN HTTP API (as do the rest of the DRAC-specific driver interfaces) to toggle the VNC server feature on and off and set its properties. I have also created a prototype of get_vnc_console method for the Ironic virt-driver in Nova. As a result, I was able to get access to the graphical console in the Horizon Dashboard for the nova instance deployed on top of a Dell R630 server managed by Ironic.

Of course, no prototype is complete without any bugs/problems discovered during testing. Here is what I’ve been hitting my head against and hacking around while making this to work:

This prototype was done prior to the generic graphical console framework implementation done in ironic. Thus the prototype implementation is, for now, overriding the existing serial console interface in an Ironic driver that was specifically created for this purpose. That means that currently it is not possible to have both a serial console and graphical console.
Conveniently though, the proposed base GraphicalConsole interface will have the same API as the current Console (SerialConsole in the future) interface. This means that once the generic framework for the graphical console interfaces is implemented in Ironic, this prototype can be plugged in as graphical console interface basically as-is.
The interface implementation is using the low-level WS-MAN Python client calls for now, because support for managing the iDracCardService is yet lacking from python-dracclient. The work to enable this functionality is already ongoing in the community, though.
The ironic virt-driver changes are rather specific for this particular case are meant exclusively to let me quickly test this functionality. After the generic graphical console is implemented in Ironic and the required complementary functionality is available in python-ironicclient, this will change.
The OpenVNC implementation in iDRAC does not seem to be complete, as noVNC can not properly connect to it. The result is an apparently connected console with no graphical output (issue#). Resolving this problem involves disabling a single passed encoding parameter in the noVNC code. For now, I have had to patch noVNC, but I have not yet determined the implications of these changes on access to a standard VM graphical console.
In order for noVNC to connect, you must set the password on the VNC server, because noVNC cannot accept an empty password in its password prompt, and setting the password for the iDRAC VNC server to None/empty string still results in the VNC server requesting a password on connection. I am not sure if this should be considered a bug in the iDRAC VNC server or in noVNC.
I have not tested yet how the iDRAC VNC server works with noVNC when SSL is enabled in iDRAC VNC Server.
The iDRAC VNC server is limited to a single VNC session at a time, so it is not really suitable for a multi-user setup. On the other hand, this still might suffice for undercloud-like use cases such as TripleO.
Note that in the current prototype, all nodes running the nova-novncproxy service (or the single one specified as “vncserver_proxyclient_address” in the config for nova-compute with the Ironic virt-driver) must effectively have access to the BMC network, as the built-in iDRAC VNC server is serving from its own BMC IP address. Take care to set up such proxying securely in a clustered nova deployment.

As you can see, there&;s still a ways to go before this functionality is available in a production capacity. Nevertheless, this seems like an interesting and promising development in the hardware market. I consider it as yet another small step on the way forward to close the gap between baremetal and virtual servers in OpenStack, and to enable a unified user experience for the compute service.
The post Universal access: An early prototype for graphical VNC console in Ironic appeared first on Mirantis | Pure Play Open Cloud.
Quelle: Mirantis

Announcing Azure Network Watcher – Network Performance Monitoring and Diagnostics Service for Azure

Have you ever felt the need to diagnose a critical problem and you needed access to packet data from a virtual machine? What if you could capture the packet data from a virtual machine in just a few clicks? How about the ability to log flow data for Network Security Groups, visualize and interpret the information with a tooling platform of your choice?

With Azure Network Watcher, you can now access a plethora of logging and diagnostic capabilities that empower you with insights to understand your network performance and health. These capabilities are accessible via Portal, Power Shell, CLI, Rest API and SDK.

What does Network Watcher enable for you?

Topology

You can now view the network topology of your deployments with just a few clicks. For example, the figure below represents the network topology of a simple web application deployed on Azure. With Network Watcher, you can now visualize the complete network topology of your application.

Sample topology view of a web application

IP flow verify

A common diagnostic need is to check whether a flow is allowed or denied to or from a virtual machine. Using “IP flow verify” you can now validate if a flow (combination of source IP, destination IP, source port, destination port and protocol) is allowed or denied. You will also be provided with the specific Network Security Group and security rule allowing or denying the flow in question.

Validate IP flow from the Portal

Next hop

Typical issues with network connectivity is misconfiguration of user defined routes. Next hop provides the ability to get the next hop type and IP address based on a specified virtual machine, allowing you to investigate any route being black-holed and conditions caused by incorrect configuration.

Get next hop from the Portal

Security Group view

Auditing your network security is vital for detecting network vulnerabilities and ensuring compliance with your IT security and regulatory governance model.

With Security Group view, you can retrieve the configured Network Security Group and security rules, as well as the effective security rules. With the list of rules applied, you can determine the ports that are open and assess network vulnerability.

In addition, your IT security and compliance governance can define prescriptive security rules that can now be programmatically audited using this feature.

As an example, PCI DSS compliance emphasizes the need to store logs and review logs that perform security functions such as firewalls. The primary intent for this is to identify anomalies and suspicious activity. With a combination of flow logs, Security Group view and Azure Automation, periodic and frequent audit can be done in a programmatic manner. You can detect and alert on suspicious and anomalous activity.

Network Security Group view for a virtual machine from the Portal

Packet capture

Capturing and accessing packet data enables you to address various needs from diagnosing a connectivity issue to network security and compliance. With Network Watcher, you can trigger packet capture on virtual machines. Applying advanced rule matching options, you can capture packets that have a specific source IP, destination IP, source port or destination port, or a byte offset from the start of the packet – even a combination of all the above. This feature is supported on both Windows and Linux virtual machines.

Configuring packet capture from the Portal

Network Subscription limits

You can now view the usage of network resources against the limits in your subscription.

View limits for network resources in your subscription in a region

NSG flow logs

Flow data is a critical component for diagnosing and validating your Network Security Group configurations. You can now enable logging of NSG flow data that is allowed or denied per Network Security Group setting to help meet these needs. The NSG flow information includes timestamp, source IP, destination IP, source port, destination port and protocol, the Network Security Group and the security rule. This data can be ingested and visualized by Microsoft tools such as Power BI, as well as security information and event management tools provided by 3rd party partners and open source tools.

Configuring NSG flow logs from the Portal

A sample Power BI dashboard with the ingested flow log

Diagnostic logs

You can now configure diagnostic logs for all the network resources in a resource group from a single pane.

Configuring Diagnostic logs for network resources in a resource group

Virtual Network Gateway Connectivity Troubleshooting

A Virtual Network Gateway provides connectivity between your on-premises site and Azure VNets. Network Watcher will enable you to troubleshoot issues due to connectivity. A comprehensive suite of built-in tests are executed to isolate over fifteen different fault conditions and the results are logged in a customer specified storage. The log contains information such as connection status, bytes sent/received, IKE errors and WFP logs.

Integration with Azure Services

Using the native capabilities offered by Network Watcher, you can build powerful end to end network monitoring scenarios using Azure services like Azure Automation, Azure Functions and Azure Log Analytics.

Proactive monitoring of VPN connection using Azure Automation and Network Watcher

Partners and ecosystem integration

We have partnered with the following 3rd party tool providers to integrate their products with Network Watcher and provide you with a holistic experience in monitoring your network in Azure.

Splunk have built an operational intelligence platform by turning data generated from Network Watcher into valuable insights.

Observable Networks have integrated the packet capture capability of Network Watcher with their ONA platform (Observable Network Appliance) to detect security issues in your virtual machine.

Bryan Doerr, CEO of Observable Networks said, “We’re excited that the results of our continuous and close collaboration with Microsoft are now reaching our mutual customers. Digital transformation and the fast-growing transition to cloud platforms, like Azure, are creating demand for new cloud native security services."

Sumo Logic provides a machine data analytics platform that can ingest flow data for Network Security Groups to help you understand network vulnerabilities.

Kalyan Ramanathan, VP of Product Marketing at Sumo Logic said, “The cloud is changing the IT landscape. New business models, rapidly changing innovation and operations are driving a new set of needs. We are pleased to be teaming with Microsoft to further enhance the cloud experience for our mutual customers. Sumo Logic Machine Data analytics solution provides real-time operational insights into today’s modern applications with deep Microsoft Azure Integration, to help customers address the volume, variety and velocity of cloud generated data.”

Open source tools

Your network monitoring needs can be augmented by open source tools such as Capanalytics, Suricata and ELK (Elastic Search, Logstash and Kibana). We hope you will be able to leverage and build on the sample integration scenarios for visualizing packet capture, network intrusion detection and visualizing flow logs.

A sample dashboard highlighting network intrusion – integrating Network Watcher, Suricata and ELK

Network Watcher availability

Azure Network Watcher is available now in preview in the following regions – US West Central, US North and US West. We are in the process of rolling out Network Watcher the rest of Azure regions around the world.

How much does it cost?

We understand the current capabilities in Network Watcher are critical to a variety of your needs from diagnostics to security and compliance. These capabilities will be available free with your subscription. Standard storage costs are applicable in certain cases.

Your requirements and requests for an integrated solution and tooling is at the center of building this advanced network monitoring capability in Azure. Your feedback from using Network Watcher is vital to help steer the product development and eco system growth.

Enjoy the preview!
Quelle: Azure

AWS Lambda Adds Enhanced Visibility into Stream-based Processing Operations

You can now monitor and track the time that records spend in Amazon Kinesis and Amazon DynamoDB streams before being processed by your AWS Lambda functions. Using the new IteratorAge metric, you can easily detect delays in stream processing and create alarms that help you efficiently monitor your stream-based operation’s health and performance. The IteratorAge metric is now available as a default metric which is free of charge. It is viewable in the Lambda console’s monitoring tab. 
Quelle: aws.amazon.com

AWS Lambda Adds Enhanced Visibility into Stream-based Processing Operations

You can now monitor and track the time that records spend in Amazon Kinesis and Amazon DynamoDB streams before being processed by your AWS Lambda functions. Using the new IteratorAge metric, you can easily detect delays in stream processing and create alarms that help you efficiently monitor your stream-based operation’s health and performance. The IteratorAge metric is now available as a default metric which is free of charge. It is viewable in the Lambda console’s monitoring tab. 
Quelle: aws.amazon.com