Thought Microsoft's licence plans were Kafkaesque? How about a Kafka extension for Azure Functions?
(2020/04/28)
- Reference: 1588084209
- News link: https://www.theregister.co.uk/2020/04/28/azure_functions_kafka_extension/
- Source link:
Microsoft has slung a Kafka Extension for Azure Functions out to preview as well as sneaking in Java 8 support for Linux hosting.
The [1]public preview of the Kafka extension, currently only available on the Premium Plan for Azure Functions, adds the ability to have functions trigger in response to messages in Kafka Topics, or write to Kafka Topics. Both are handy for dealing with the stream processing platform.
The Kafka communication is based on the Confluent.Kafka library. It is also possible to use Kafka with Azure Functions containers in Kubernetes alongside [2]KEDA .
Azure Functions might be a little late to the game on the Kafka front. While Microsoft's extension is in public preview, the Kafka Connect AWS Lambda sink connector [3]is already available on the Confluent platform and will cheerfully pull records from Apache Kafka topics, convert them to JSON and run an AWS Lambda function. The output can then be spat back at another Kafka topic.
Apache Kafka itself is an open-source project aimed at handling real-time data feeds, and hooks up to external systems via Kafka Connect. While pushers of the tech, such as Confluent, have attempted to persuade those more familiar with traditional RDBMS systems that stream processing isn't that scary with the likes of [4]ksqlDB , it does remain best suited for tasks such as monitoring, messaging and activity tracking as well as stream processing.
The Kafka extension came as Linux hosting for Azure Functions applications built with [5]Java 8 also entered preview . The preview means that Java container workloads can be slung at Azure Functions and Kubernetes with KEDA, if that's your thing.
The Kafka extension is being developed in the open, and Microsoft added that it would "continue to innovate to bring in more Cloud Native and Open Source event sources."
You lucky, lucky people. ®
Sponsored: [6]How to Build Your Digital Experience Portfolio
[1] https://techcommunity.microsoft.com/t5/azure-functions/announcing-public-preview-of-the-azure-functions-kafka-extension/ba-p/1342348
[2] https://keda.sh/scalers/apache-kafka/
[3] https://docs.confluent.io/current/connect/kafka-connect-aws-lambda/index.html
[4] https://www.theregister.co.uk/2019/11/26/ksqldb/
[5] https://azure.microsoft.com/en-us/updates/announcing-java-feature-support-azure-functions/
[6] https://go.theregister.co.uk/tl/1936/-8578/how-to-build-your-digital-experience-portfolio?td=wptl1936
The [1]public preview of the Kafka extension, currently only available on the Premium Plan for Azure Functions, adds the ability to have functions trigger in response to messages in Kafka Topics, or write to Kafka Topics. Both are handy for dealing with the stream processing platform.
The Kafka communication is based on the Confluent.Kafka library. It is also possible to use Kafka with Azure Functions containers in Kubernetes alongside [2]KEDA .
Azure Functions might be a little late to the game on the Kafka front. While Microsoft's extension is in public preview, the Kafka Connect AWS Lambda sink connector [3]is already available on the Confluent platform and will cheerfully pull records from Apache Kafka topics, convert them to JSON and run an AWS Lambda function. The output can then be spat back at another Kafka topic.
Apache Kafka itself is an open-source project aimed at handling real-time data feeds, and hooks up to external systems via Kafka Connect. While pushers of the tech, such as Confluent, have attempted to persuade those more familiar with traditional RDBMS systems that stream processing isn't that scary with the likes of [4]ksqlDB , it does remain best suited for tasks such as monitoring, messaging and activity tracking as well as stream processing.
The Kafka extension came as Linux hosting for Azure Functions applications built with [5]Java 8 also entered preview . The preview means that Java container workloads can be slung at Azure Functions and Kubernetes with KEDA, if that's your thing.
The Kafka extension is being developed in the open, and Microsoft added that it would "continue to innovate to bring in more Cloud Native and Open Source event sources."
You lucky, lucky people. ®
Sponsored: [6]How to Build Your Digital Experience Portfolio
[1] https://techcommunity.microsoft.com/t5/azure-functions/announcing-public-preview-of-the-azure-functions-kafka-extension/ba-p/1342348
[2] https://keda.sh/scalers/apache-kafka/
[3] https://docs.confluent.io/current/connect/kafka-connect-aws-lambda/index.html
[4] https://www.theregister.co.uk/2019/11/26/ksqldb/
[5] https://azure.microsoft.com/en-us/updates/announcing-java-feature-support-azure-functions/
[6] https://go.theregister.co.uk/tl/1936/-8578/how-to-build-your-digital-experience-portfolio?td=wptl1936
Re: Kafka extension
jake
"Nothing works as expected." ... Check.
"Nobody can find out why." ... Check.
Not an omen, no. It's nothing more than truth in advertising, which is the real shocker, considering where it's coming from.
Kafka extension
It's worthy of note that in Franz Kafka's universe nothing worked as expected (as in The Castle) and nobody could find out why (as in The Trial). Extending Kafka to the real world in any useful way would therefore seem almost impossible. I hope the name's not an omen.