Friday, January 03, 2020

Data Platform Tips 37 - Ingesting data into Azure Data Explorer

a) Logon to the Azure Portal.

b) Provision Azure Data Explorer Service as outlined in the post "Azure Data Explorer Service".

c) Create a new Event Hub named "EventHubDemo13".

d) Create a new Event Hub named "StreamingDataDemo".

e) Create a table named "TestTable" in the Azure Data Explorer Database.

.create table TestTable (TimeStamp: datetime, Name: string, Metric: int, Source:string)

f) Map the columns from the incoming json data to the columns in the TestTable.

.create table TestTable ingestion json mapping 'TestMapping' '[{"column":"TimeStamp","path":"$.timeStamp","datatype":"datetime"},{"column":"Name","path":"$.name","datatype":"string"},{"column":"Metric","path":"$.metric","datatype":"int"},{"column":"Source","path":"$.source","datatype":"string"}]'

g) Connect the Event Hub from Azure Data Explorer to get the ingested data.

h) Create a new Data Connection.

i) Make a note of the Connection string-primary key of the Event Hub.

j) Download the sample application to simulate messages to the Event Hub and replace the connectionstring with the information copied in step i)

k) Run the Application.

l) Check the message content on Azure Data Explorer by typing "TestTable" in Query window of Azure Data Explorer and click Run.

No comments:

Post a Comment