Wednesday, January 01, 2020

Data Platform Tips 35 - Azure Data Explorer vs. Azure Stream Analytics


Azure Data Explorer
Azure Data Explorer is a secure Analytics service that allow users to do real time analysis on large volumes of data streaming from multiple applications, IoT devices etc. You can ingest data from about 80 data sources, Egress data using Kusto Query Language (KQL), invoke control flow actions like Lookup and GetMetaData against ADLS Gen 2.

Azure Data Explorer Tools - https://docs.microsoft.com/en-us/azure/kusto/tools/

KQL cheat sheet - https://github.com/marcusbakker/KQL/blob/master/kql_cheat_sheet.pdf

Azure Stream Analytics
Azure Stream Analytics is a real time analytics and complex event processing engine to process large volumes of streaming data from multiple applications simultaneously. The inputs supported for Azure Stream Analytics are IoT Hub, Event Hub and Blob Storage.

So exactly what's the real difference between them?

Azure Stream Analytics is mainly used for processing data as part of the data flowing to provide insights without the need to store it. If you want to persist the data then the output from Azure Stream Analytics needs to be stored to a data store like Azure Data Lake Storage, Blob Storage, Azure SQL Database, Azure Synapse Analytics etc.

Azure Data Explorer on the other hand is a database and it can ingest, store and query large volumes of data in tables. It has the ability to hold the data for few days and you can either drop the tables or delete data from those tables based on setting retention policies. It supports only bulk delete operations and not record by record deletions.




No comments:

Post a Comment