Introducing Change Feed support in Azure DocumentDB

 We’re excited to announce the availability of Change Feed support in Azure DocumentDB! With Change Feed support, DocumentDB provides a sorted list of documents within a DocumentDB collection in the order in which they were modified. This feed can be used to listen for modifications to data within the collection and perform actions such as:

Trigger a call to an API when a document is inserted or modified
Perform real-time (stream) processing on updates
Synchronize data with a cache, search engine, or data warehouse

DocumentDB&;s Change Feed is enabled by default for all accounts, and does not incur any additional costs on your account. You can use your provisioned throughput in your write region or any read region to read from the change feed, just like any other operation from DocumentDB.

In this blog, we look at the new Change Feed support, and how you can build responsive, scalable and robust applications using Azure DocumentDB.

Change Feed support in Azure DocumentDB

Azure DocumentDB is a fast and flexible NoSQL database service that is used for storing high-volume transactional and operational data with predictable single-digit millisecond latency for reads and writes. This makes it well-suited for IoT, gaming, retail, and operational logging applications. These applications often need to track changes made to DocumentDB data and perform various actions like update materialized views, perform real-time analytics, or trigger notifications based on these changes. Change Feed support allows you to build efficient and scalable solutions for these patterns.

Many modern application architectures, especially in IoT and retail, process streaming data in real-time to produce analytic computations. These application architectures (“lambda pipelines”) have traditionally relied on a write-optimized storage solution for rapid ingestion, and a separate read-optimized database for real-time query. With support for Change Feed, DocumentDB can be utilized as a single system for both ingestion and query, allowing you to build simpler and more cost effective lambda pipelines. For more details, read the paper on DocumentDB TCO.

 

Stream processing: Stream-based processing offers a “speedy” alternative to querying entire datasets to identify what has changed. For example, a game built on DocumentDB can use Change Feed to implement real-time leaderboards based on scores from completed games. You can use DocumentDB to receive and store event data from devices, sensors, infrastructure, and applications, and process these events in real-time with Azure Stream Analytics, Apache Storm, or Apache Spark using Change Feed support.

Triggers/event computing: You can now perform additional actions like calling an API when a document is inserted or modified. For example, within web and mobile apps, you can track events such as changes to your customer&039;s profile, preferences, or location to trigger certain actions like sending push notifications to their devices using Azure Functions or App Services.

Data Synchronization: If you need to keep data stored in DocumentDB in sync with a cache, search index, or a data lake, then Change Feed provides a robust API for building your data pipeline. Change feed allows you to replicate updates as they happen on the database, recover and resume syncing when workers fail, and distribute processing across multiple workers for scalability.

 

Working with the Change Feed API

Change Feed is available as part of REST API 2016-07-11 and SDK versions 1.11.0 and above. See Change Feed API for how to get started with code.

 

 

The change feed has the following properties:

Changes are persistent in DocumentDB and can be processed asynchronously.
Changes to documents within a collection are available immediately in the change feed.
Each change to a document appears only once in the change feed. Only the most recent change for a given document is included in the change log. Intermediate changes may not be available.
The change feed is sorted by order of modification within each partition key value. There is no guaranteed order across partition-key values.
Changes can be synchronized from any point-in-time, that is, there is no fixed data retention period for which changes are available.
Changes are available in chunks of partition key ranges. This capability allows changes from large collections to be processed in parallel by multiple consumers/servers.
Applications can request for multiple Change Feeds simultaneously on the same collection.

Next Steps

In this blog post, we looked the new Change Feed support in Azure DocumentDB.

Learn more about Change Feed support in Azure DocumentDB
Upgrade to .NET SDK 1.11.0 with Change Feed support
Create a new DocumentDB account from the Azure Portal or download the DocumentDB Emulator
Stay up-to-date on the latest DocumentDB news and features by following us on Twitter @DocumentDB or reach out to us on the developer forums on Stack Overflow

Quelle: Azure

Elon Musk And Uber CEO Travis Kalanick To Advise Trump, Apple CEO To Meet Privately

Toru Hanai / Reuters

President-elect Donald Trump named Elon Musk, the head of SpaceX and Tesla, and Uber CEO Travis Kalanick to his growing panel of industry chieftains Wednesday, a group convened to provide regular economic guidance to the next president. PepsiCo&;s CEO Indra Nooyi has been added as well.

Billed as the President’s Strategic and Policy Forum, the panel was announced earlier this month, comprised of 16 business leaders, including the heads of Walmart, Disney, and General Motors. Among the initial batch of CEOs, IBM was the sole technology company represented, making the addition of Musk and Kalanick, all the more important in Trump&039;s efforts to channel and recruit the expertise of Silicon Valley.

“America has the most innovative and vibrant companies in the world, and the pioneering CEOs joining this Forum today are at the top of their fields,” Trump said, when the forum was first unveiled. “My Administration is going to work together with the private sector to improve the business climate and make it attractive for firms to create new jobs across the United States from Silicon Valley to the heartland.”

Tesla has not yet responded to a request for comment.

Kalanick told BuzzFeed News in a statement: “I look forward to engaging with our incoming president and this group on issues that affect our riders, drivers and the 450+ cities where we operate.”

Trump is slated to meet with Musk and a who&039;s who of tech industry titans Wednesday afternoon, with job creation expected at the top of the agenda. Among the guests planning to attend: Tim Cook of Apple, Jeff Bezos of Amazon, Sheryl Sandberg of Facebook, and Satya Nadella of Microsoft.

Cook and Musk are also expected to meet with Trump privately. Apple has not responded to a request for comment.

Quelle: <a href="Elon Musk And Uber CEO Travis Kalanick To Advise Trump, Apple CEO To Meet Privately“>BuzzFeed

Facebook Tests "M Suggestions," Laying Groundwork For More AI In Messenger

Facebook’s “M” virtual assistant hasn’t been rolled out to all that many people, but its interactions with a limited user base have helped train Facebook’s artificial intelligence systems, and now the masses may benefit.

M suggestions in action

Today, Facebook is starting a small test in the US for “M suggestions,” a new feature that will suggest certain actions based on the context in conversations within the Messenger app.

The test feature will suggest things like sending a location after someone asks a question like “where are you?,” or it will offer a small selection of stickers you may want to send in response to a message. “M” will show up in these conversations with its own chat avatar.

“Think of this as a version of M that can actually help suggest the right capabilities at the right time,” Messenger head David Marcus told BuzzFeed News.

The virtual assistant version of M has been in tests for over a year and still has no set date for a bigger rollout, but Marcus said Facebook wants to bring it to the public. “M suggestions,” if rolled out more broadly, could help push the schedule up a bit, getting a completely automated version of M into the hands of Messenger&;s over 1 billion users, albeit in a much more lightweight capacity than the assistant version.

“Hopefully with this side of it, we’ll have a path to opening it up to everyone fairly quickly,” Marcus said.

There’s still a good deal of technology that goes into creating these basic suggestions. Facebook essentially needs to understand what you’re saying in messages to make sure its suggestions aren’t annoying, and its technology also adjusts its suggestions based on how you interact with M. “It seems completely pedestrian, but it’s actually very hard to do,” Marcus said, referencing Messenger’s sticker suggestions.

Other suggestions Messenger already makes, such as event reminders and ride hailing, will be rolled under the “M suggestions” umbrella as well.

Facebook M in action

Facebook&039;s M suggestions can be seen as a defensive move in some ways. Google Assistant, a virtual assistant inside Google’s Allo messaging app, can also be called into conversations, and Allo suggests responses based on the conversation’s context with “Smart Replies.” Asked for his thoughts on Google Assistant, Marcus declined to address it directly.

The virtual assistant competition between Google and Facebook could get interesting very quickly, especially since assistants fit in both companies’ territory. A great assistant could be a competitive advantage for a messaging app, and if it&039;s developed outside of Facebook, it could be a threat since the company owns both Messenger and WhatsApp, two massive messaging apps. On the other hand, virtual assistants can help people look for something online, the pillar Google’s business is built on. For now though, neither company appears to be under imminent threat from the others’ virtual assistant efforts.

M suggestions will roll out to a very small group, but Marcus said Facebook hopes to gradually expand it in the first half of next year.

Quelle: <a href="Facebook Tests "M Suggestions," Laying Groundwork For More AI In Messenger“>BuzzFeed

Azure IoT Hub message routing dramatically simplifies IoT solution development

IoT solutions can be complex, and we’re always working on ways to simplify them.

As we work with customers on real-world, enterprise-grade IoT solutions built on Azure IoT, one pattern we’ve noticed is how businesses route inbound messages to different data processing systems.

Imagine millions of devices sending billions of messages to Azure IoT Hub. Some of those messages need to be processed immediately, like an alarm indicating a serious problem. Some messages are analyzed for anomalies. Some messages are sent to long term storage. In these cases, customers have to build routing logic to decide where to send each message:

While the routing logic is straightforward conceptually, it’s actually really complex when you consider all of the details you have to handle when you build a dispatching system: handling transient faults, dealing with lost messages, high reliability, and scaling out the routing logic.

To make all this easier, we’ve made a great new feature to IoT Hub generally available: message routing. This allows customers to setup automatic routing to different systems via Azure messaging services and routing logic in IoT Hub itself, and we take care of all of the difficult implementation architecture for you:

You can configure your IoT hub to route messages to your backend processing services via Service Bus queues, topics, and Event Hubs as custom endpoints for routing rules. Queuing and streaming services like Service Bus queues and Event Hubs are used in many if not all messaging applications. You can easily set up message routing in the Azure portal. Both endpoints and routes can be accessed from the left-hand info pane in your IoT Hub:

You can add routing endpoints from the Endpoints blade:

You can configure routes on your IoT Hub by specifying the data stream (device telemetry), the condition to match, and the endpoint to which matching messages are written.

Message routing conditions use the same query language as device twin queries and device jobs. IoT Hub evaluates the condition on the properties of the messages being sent to IoT Hub and uses the result to determine where to route messages. If messages don’t match any of your routes, the messages are written to the built-in messages/events endpoint just like they are today.

We have also enhanced our metrics and operations monitoring logs to make it easy for youto tell when an endpoint is misbehaving or whether a route was incorrectly configured. You can learn about the full set of metrics that IoT Hub provides in our documentation including how each metric is used.

Azure IoT is committed to offering our customers high availability message ingestion that is secure and easy to use. Message routing takes telemetry processing to the next step by offering customers a code-free way to dispatch messages based on message properties. Learn more about today&;s enhancements to Azure IoT Hub messaging by reading the developer guide. We firmly believe in customer feedback, so please continue to submit your suggestions through the Azure IoT User Voice forum or join the Azure IoT Advisors Yammer group.
Quelle: Azure

Amazon's Prime Video Is Now Live In The Company's Largest Market Outside The US

Brian Ach / Getty Images

Amazon has launched its video streaming service Prime Video in India, the company&;s largest geographical market outside the United States.

The service is bundled with a Prime subscription in India, which is the cheapest in the world at $15 a year, but has been available for an introductory price of $7 since the company launched it in the country in July.

Amazon Prime Video competes directly with Netflix, which expanded simultaneously to 130 countries, including India, in January.

Prime Video sharply undercuts Netflix on price in India, where the cheapest Netflix subscription costs Rs. 500 a month ($7.4). It also competes with more than a dozen local streaming services which offer a mix of free ad-supported content, as well as paid options.

Launching Prime Video in India is a strategic milestone for Amazon. Thanks to a rapid smartphone penetration and falling data prices, the country is an important market for most US tech companies.

At an event in Washington in June, Amazon CEO Jeff Bezos said that the company would invest up to $5 billion in India. Amazon&039;s aggressive expansion in the country has worried local rivals like Flipkart, India&039;s largest e-commerce startup, who recently appealed for protectionist policies from the Indian government to stave off cash-rich rivals like Amazon.

Unlike Netflix, which launched in India with no Indian content at all, Amazon Prime Video already has dozens of launch titles in Hindi, Bengali, Marathi, Tamil, and Telugu. Amazon has also partnered with several major film production studios in India to make dozens of Bollywood titles available on its service, and has included local language subtitles for English-language shows like Mad Men, Seinfeld, Person of Interest, Mad Men, and more. Also available are US-produced Amazon Originals series like Transparent and Bosch.

Quelle: <a href="Amazon&039;s Prime Video Is Now Live In The Company&039;s Largest Market Outside The US“>BuzzFeed

Facebook Spokesperson Calls Muslim Registry "Straw Man"

Lluis Gene / AFP / Getty Images

A spokesperson for Facebook, accidentally responding to a BuzzFeed News reporter via email, called the notion of a Muslim registry a “straw man.” Seemingly thinking he was addressing a colleague, he suggested that the best course of action was to not respond to the reporter&;s inquiry.

Earlier today BuzzFeed News emailed Facebook to ask whether the social networking giant would make a commitment to limit data collection that could be used for ethnic or religious targeting, including a pledge not to build a registry of Muslims, if asked to do so by the government. A Facebook public relations representative intended to forward our request, along with a message about how to respond, within Facebook, but accidentally sent the email to BuzzFeed News instead and in doing so, provided inadvertent insight into how the company plays the optics game.

BuzzFeed News

Facebook had not responded to previous requests for clarification on whether or not it would participate in building a Muslim registry. BuzzFeed News asked again today in light of the fact that Facebook COO Sheryl Sandberg will attend a tech summit at Trump Tower tomorrow, and because a group of 60 or so engineers and employees for major tech companies just signed a pledge not to comply with practices that could be used to target people or build databases based on their religious beliefs.

In the email that was accidentally sent to BuzzFeed News, the PR rep calls this “attacking a straw man.”

Happy to talk to her off record about why this is attacking a straw man. Also I heard back from her that she may or may not write an additional piece depending on what response she gets from companies. So sounds like not making any stmt on record is the way to go.

The representative subsequently called, and asked that the email be considered off the record. This preference for off the record spin over on the record comments is fairly typical of large tech companies. Facebook ultimately declined to comment.

However, the possibility that President Elect Donald Trump will try to create a Muslim registry is not a straw man. Trump has repeatedly been given a chance to clarify whether he supports the idea. He has been asked about it by everyone from Fox News to George Stephanopoulos. On more than one occasion he has chosen not to rule out the possibility of a Muslim registry. The question of such a registry arose based on Trump&039;s comments about Muslims and his intention to build a database of Syrian refugees.

Since the election, Kansas Secretary of State Kris Kobach, an advisor to Trump&039;s transition team, told Reuters that Trump&039;s immigration policy group is considering reinstating the National Security Entry-Exit Registration System (NSEERS), a suspended federal program used to keep a database of immigrants from countries that have a Muslim majority from 2001 to 2012.

Facebook, of course, already asks for and retains sensitive information about the race, religion, and location of its users and allows advertisers to target narrow segments of people based on that personal information. Government officials here and abroad already use the social network to track activists and dissidents.

Quelle: <a href="Facebook Spokesperson Calls Muslim Registry "Straw Man"“>BuzzFeed

How Malik Obama Became A Twitter "Shitlord" And Alt-Right Darling

Perhaps the weirdest subplot in this year&;s unprecedentedly weird presidential election was the bromance between Donald Trump and Malik Obama, President Obama&039;s half-brother. In July, Malik, a 58-year-old with dual US and Kenyan citizenship, announced that he would support Trump&039;s presidential bid. Shortly afterward, Trump welcomed his support on Twitter. By October, the Trump campaign invited Obama to the final presidential debate, where he snapped a photo alongside Kellyanne Conway.

Malik Obama&039;s reasons for supporting Trump remain somewhat unclear and may stem from hard feelings he has towards his half-brother. But whatever the reason, that support was vehement, particularly on Twitter, where he was so adamantly pro-Trump that his account was often accused of being a parody.

But shortly before the election, Obama was verified by Twitter. And since that verification, something weird has happened: Malik Obama has gone full chanterculture, shitlord, troll, adopting much of the language and similar tactics to those used by the alt-right.

To wit: Obama frequently calls his opponents cucks. Obama discusses the Swahili meaning of the “Harambe.” Obama rails against “fake news” like CNN and the Huffington Post. Obama goes back and forth with Bill Mitchell. Obama assures his followers that he has not been “spirit cooked” (Pizzagate for “satanic rite.”) Obama shares Pepe memes of himself bearing the name “Memelike Obameme.” Etc.

It&039;s a lot to take in: Barack Obama&039;s half brother, who was the best man at the president&039;s wedding, is now a proud -loving shitlord. How could a Kenyan immigrant who only a few years away from social security, so quickly develop such a mastery of the current alt-right cultural memefield of tropes and jokes?

The only thing that&039;s sure about the answer is that it involves Chuck Johnson in some way. According to both Obama and Johnson — the notorious journalist and troll who was controversially kicked off Twitter for asking his followers for money so he could “take out” civil rights activist DeRay Mckesson — it&039;s all very simple: the latter taught the former how to use Twitter really, really well. On Saturday, Obama tweeted

And responding to an email from BuzzFeed News asking how he had mastered Twitter so quickly, Obama wrote “It&039;s a wonderful medium for expressing oneself. Chuck Johnson got me started and my followers have taught me a lot. But most of it is just plain knack&;”

Reached via email, Johnson concurred. According to him, the two connected in the spring in Washington, DC (Obama lived for years in Maryland), where Johnson began teaching Obama his method.

“I teach people neurolinguistic programming,” Johnson wrote. “I have them speak while they tweet so that it appears more conversational and I make sure that they tap into people&039;s emotions and not just their intellect. When Malik Obama told me he supported Donald Trump I helped him throughout the process.”

But today, Wesearchr, Johnson&039;s crowd-sourced investigations company, retweeted a tweet suggesting that Johnson had groomed Malik Obama as part of a master plan to get back at Twitter.

Both Obama and Johnson denied that Johnson writes the president&039;s half-brother&039;s tweets.

“Malik Obama is a grown man who does his own tweets,” Johnson wrote. “I do my own stunts.”

There&039;s no question that Johnson and Obama know each other — they were photographed together for Johnson&039;s other venture, a news site called GotNews.com.

Asked for details into how Johnson taught him Twitter, Malik Obama asked for a “token payment” of $1000. (Obama has a history of asking journalists for money; last year he asked a POLITICO reporter for $10,000 to talk.)

“You want everything for free?” he asked, when refused payment.

Quelle: <a href="How Malik Obama Became A Twitter "Shitlord" And Alt-Right Darling“>BuzzFeed

Scaling Up "Project Springfield" using Azure

This guest post is from the Project Springfield team in Microsoft’s Artificial Intelligence and Research group. Project Springfield delivers pioneering artificial intelligence for finding security issues as a cloud service. Learn how the team used Azure to meet and exceed scaling challenges on a tight timeline.

The Project Springfield engineering team, led by William Blum, had built the first release of “Project Springfield,” which helped customers find “million-dollar” security bugs by combining pioneering “whitebox fuzzing” technology from Microsoft Research with the elasticity of the cloud. Customers could upload their software to Project Springfield, which created a fuzzing lab in Azure. Each fuzzing lab tested with a portfolio of methods, looked for crashes from the test cases, and then picked the highest value issues to report. The power of Azure enabled this compute intensive process to scale up and scale down as customers’ demands changed, while simultaneously collecting data from every run to improve the service. To do this, Project Springfield had to dynamically create large numbers of virtual machine and network resources and manage them on behalf of the customer.

We had built the initial product on Azure, using the classic Azure management interface to dynamically provision virtual machines and networking resources. Now it was time to prepare for a new wave of customers – which meant scaling up the service by orders of magnitude. Scaling with the classic Azure would be a challenge. For example, each fuzzing lab used up a different cloud service on Project Springfield, yet there was limit of just 200 cloud services per subscription. That meant if the customers, in aggregate, ever needed to test more than 200 pieces of software at a time, Project Springfield would need to partition fuzzing labs across subscriptions even if each subscription otherwise had enough virtual machines available to serve customers. There had to be a better way.

We found that better way by re-architecting the service with the Azure management interface Azure Resource Manager as well as Service Fabric, Microsoft&;s micro-service-based application platform. With Azure Resource Manager, virtual machines, virtual networks, and load balancers are all treated as different resources. These resources can be combined in an Azure Resource Manager template, which is a JSON object defining what resources we needed and how they fit together. All the resources Project Springfield needs for a security testing lab are specified by a single template. When a customer needs a new lab, Azure can read the template and then dynamically create all the resources needed from the template. With Service Fabric, we could easily port our backend worker roles to micro services and dynamically scale up and scale down backend resources based on customer needs. The payoff was that instead of being locked into a single inflexible bundle, we could dynamically reshape the way resources were deployed.

Re-architecting the service around the new deployment concepts introduced by ARM required some work. The work paid off as we found that Azure’s infrastructure-as-a-service capabilities gave us better control and finer granularity over the configuration of our network and compute resources. Once adjusted to a new way of thinking, we could see how to make Project Springfield even more efficient and deliver value. For example, we realized that by using Azure Network Security Groups, we could enable each customer to set different IP address restrictions on who could access their Project Springfield resources – a key feature for enterprise users.

Even better, betting on Azure made us “future proof.” As Azure launched new features, such as support for Red Hat Linux, Windows Server Containers and more, we could see how they would let Project Springfield meet customer needs. With Azure Resource Manager, these are now just different kinds of resources in a resource manager template. That gave the team a single consistent way for managing fuzzing labs and laid the foundation for eventually offering different types of fuzzing labs for different customer needs.

By using the new capabilities of Azure, such as Azure Resource Manager, we achieved our scale goals in four months. That meant we could bring on customers and partners for trials of Project Springfield as fast as we could call them, without worrying that we would run out of capacity. What’s more, building on Azure set the team up for success as new capabilities came online. At Microsoft Ignite 2016, OSIsoft & Deschutes Brewery, EY, and Leviathan Security Group stood on stage and told the world about the value they saw in Project Springfield. Within a week over a thousand people signed up for trials! That’s a win by any standard.

Related content

Security testing in the cloud with F# and Project Springfield

Azure Resource Manager overview

Project Springfield: a cloud service built entirely in F#

More about Project Springfield

The Project Springfield team
Quelle: Azure

Google Spins Out Self-Driving Car Unit Into A New Company Called Waymo

Waymo

Google is spinning out its self-driving car program into a new company called Waymo, its chief executive John Krafcik announced Tuesday. Waymo will live under the umbrella of Alphabet, Google’s parent company.

“We’ll continue to have access to the resources and infrastructure that Alphabet provides,” Krafcik told reporters on Tuesday.

Krafcik said that Google conducted its first fully driverless rides in every day traffic in Austin last year, using a car with no steering wheels and no pedals. “We’ve taken over 10,000 trips with Googlers and guests in cities where we’re currently driving,” he observed.

Waymo

Google&;s spin-off of Waymo comes as the company, which began developing self-driving technology in 2009, moves to scale its technology. Competitors – namely, Uber – have already begun pilot programs to put passengers in self-driving cars in Pittsburgh. Those cars, however, still include steering wheels and pedals, as well as human drivers and co-pilots who steer cars to the road before turning on autonomous mode.

During today&039;s Waymo launch event, engineers said the company continues to calibrate its autonomous cars, improving their navigational abilities as well as their rides.

“We’re putting a lot of effort into making our cars more comfortable and having them be smoother,” Dmitri Dolgov, a principal engineer at Waymo, told reporters. “We’re continuing to build up map technology, and take our cars to new and different places.”

Quelle: <a href="Google Spins Out Self-Driving Car Unit Into A New Company Called Waymo“>BuzzFeed