Google BigQuery – Analytics Data Warehouse

Google handles Big Data every second of every day to provide services like Search, YouTube, Gmail and Google Docs. Google created a Query Service named “Dremel” which was used just internally within Google.
Dremel is a query service that allows you to run SQL-like queries against very, very large data sets and get accurate results in mere seconds. You just need a basic knowledge of SQL to query extremely large datasets in an ad hoc manner.
BigQuery is the public implementation of Dremel. BigQuery provides the core set of features available in Dremel to third party developers. It does so via a REST API, a command line interface, a Web UI, access control and more, while maintaining the unprecedented query performance of Dremel.
BigQuery can scan billions of rows in a highly performant manner for ad hoc query analysis. It does achieve high performance through Columnar Storage and Tree Architecture. BigQuery Client Libraries - https://cloud.google.com/bigquery/client-libraries
Currently Microsoft is planning to provide Google BigQuery connector for Power BI. In the interim, you can import data from Google BigQuery using an ODBC driver, which is fully supported for Import scenarios in Power BI Desktop, and Personal/Enterprise Gateway for Refresh purposes.

BigQuery vs MapReduce

MapReduce is a distributed computing technology that allows to implement custom “mapper” and “reducer” functions programmatically and run batch processes with them on hundreds or thousands of servers concurrently. MapReduce is designed as a batch processing framework, so it’s not suitable for ad hoc and trial-and-error data analysis.
BigQuery is designed to handle structured data using SQL.MapReduce is a better choice when you want to process unstructured data programmatically. The mappers and reducers can take any kind of data and apply complex logic to it.
Use BigQuery

• Finding particular records with specified conditions. For example, to find request logs with specified account ID.
• Quick aggregation of statistics with dynamically-changing conditions. For example, getting a summary of request traffic volume from the previous night for a web application and draw a graph from it.
• Trial-and-error data analysis. For example, identifying the cause of trouble and aggregating values by various conditions, including by hour, day and etc...

Use MapReduce

• Executing a complex data mining on Big Data which requires multiple iterations and paths of data processing with programmed algorithms.
• Executing large join operations across huge datasets.
• Exporting large amount of data after processing.

Technorati Tags: Google,BigQuery,MapReduce,Power BI Desktop,Power BI

Power BI Desktop – Google Analytics Integration

Download Power BI Desktop or from the Power BI Portal. Power BI provides out of the box integration with Google Analytics through the Google Analytics Core Reporting API. Google Analytics Core Reporting API change log to track any changes released by Google.
I tried to get some analytics on my blog through Power BI Desktop – Google Analytics Integration.
Note: The Google Analytics content pack and the connector in Power BI Desktop rely on the Google Analytics Core Reporting API. As such, features and availability may vary over time.
a) Open Power BI Desktop – Free tool provided by Microsoft to install on desktops and immediately start pulling data from disparate sources and start building Visualisations over that.
b) Click on “Get Data” from the tool bar and select “Online Services” and choose “Google Analytics”.

c) You will then displayed with the message advising that Power BI connects to a third party service which in this case is the Google Analytics Core Reporting API. You can check the Don’t warn me again for this connector checkbox and click “Continue”

d) If you haven’t already connected your Google Analytics account with Power BI Desktop then you will be provided with the below screen to connect to your Google Analytics account.




e) Once provided the credentials and access to Power BI you can then connect to your Google Analytics account from Power BI Desktop.’

f) Select the Google Analytics account and start choosing your dimensions and measures you would like to analyse and Power BI Desktop will import them for you.


g) Now you can start building your visualisations on your Google Analytics data using Power BI Desktop similar to the below one.

Technorati Tags: Power BI,Google Analytics,Google Analytics Connector,Power BI Desktop

Startup Weekend Wellington – Nov 25-27 2016

Startup Weekends are 54-hour events where developers, designers, marketers, product managers and startup enthusiasts come together to share ideas, form teams, build products, and launch startups!
Startup Weekends are weekend-long, hands-on experiences where entrepreneurs and aspiring entrepreneurs can find out if startup ideas are viable.  On average, half of Startup Weekend’s attendees have technical or design backgrounds, the other half have business backgrounds.
Beginning with open mic pitches on Friday, attendees bring their best ideas and inspire others to join their team. Over Saturday and Sunday teams focus on customer development, validating their ideas, practicing LEAN Startup Methodologies and building a minimal viable product. On Sunday evening teams demo their prototypes and receive valuable feedback from a panel of experts
Buy your tickets here

SSRS - Parameter Date format in Google Chrome is mm/dd/yyyy instead of dd/mm/yyyy

• To fix the issue, you need to change the accept-languages property in the preferences file for Google Chrome. To do that, type “about:version” in the address path and copy the profile path value.
• Close down the Chrome Browser.
• Open the preferences file from the profile path copied from step 1 and look for the “accept-languages” property
• Change the property from “en-US”,en to “en-NZ”,en and the parameter dates in SSRS will now display in dd/mm/yyyy format in Google Chrome.

Reference: https://productforums.google.com/forum/#!msg/chrome/psPDoulWfzc/7CrsYYO576wJ

Technorati Tags: Google,Chrome,SSRS

Azure Data Catalog – Enterprise Data Assets

Azure Data Catalog is an enterprise-wide metadata catalog that stores, describes, indexes, and shows how to access any registered data asset. The focus of Data Catalog is to bridge the gap between IT and business. Crowdsourced annotations let users who are knowledgeable about the data assets registered in the Catalog to enrich the system at any time. This helps others understand the data more readily, including its intended purpose and how it’s being used within the business.
Azure Data Catalog extracts the following information from the data sources:

• Asset Name
• Asset Type
• Asset Description
• Attribute/Column Names
• Attribute/Column Data Types
• Attribute/Column Description

The Data Catalog REST API is a REST-based API that provides programmatic access to Data Catalog resources to register, annotate and search data assets programmatically. The service gives capabilities that enable any user, from analysts to data scientists to developers, to register, discover, understand, and consume data assets.

Technorati Tags: Azure Data Catalog,Data Assets