Data Platform Tips 32 - Azure Data Lake Storage Gen2 and Azure Stream Analytics

In this post, we will be looking at how to store streaming data from IoT Hub to Azure Data Lake Storage Gen2 using Azure Stream Analytics.

Save streaming data to Azure Data Lake Storage Gen2 using Azure Portal

a) Logon to the Azure Portal.

b) First provision an Azure IoT Hub to ingest real time sensor data.

c) Now your IoT Hub would have been successfully provisioned.

d) Add a new device named "Device1" and note down the connectionstring for Device1.

e) Click on the "eye" icon to reveal the Primary Connection string of Device1 and save it for future use.

f) Next, let us provision an Azure Stream Analytics Job.

g) You can now see the Azure Stream Analytics Job provisioned.

h) Add the IoTHub as the input for the Azure Stream Analytics Job.

i) Add the Azure Data Lake Storage Gen2 as output for the Azure Stream Analytics Job.

j) Now add the query to the Azure Stream Analytics Job.

k) Let us simulate some data to for the IoT Hub to ingest using https://azure-samples.github.io/raspberry-pi-web-simulator/

l) Replace the connectionstring with the Device1 connectionstring noted in step d) and click on Run to simulate the messages.

m) Start the Azure Stream Analytics Job.

n) Now you can check the Azure Data Lake Storage Gen2 using Azure Storage Explorer to see the newly created JSON file based on the streaming data.