Use the API Server and Elasticsearch ADO.NET Provider in Microsoft Power BI

The CData API Server enables your organization to create Power BI reports based on the current Elasticsearch data (plus data from 200+ other ADO.NET Providers). The API Server is a lightweight Web application that runs on your server and, when paired with the ADO.NET Provider for Elasticsearch, provides secure OData services of Elasticsearch data to authorized users. The OData standard enables real-time access to the live data, and support for OData is integrated into Power BI. This article details how to create data visualizations based on Elasticsearch OData services in Power BI.

Getting Started

---

Set Up the API Server

Follow the steps below to begin producing secure Elasticsearch OData services:

Deploy

The API Server runs on your own server. On Windows, you can deploy using the stand-alone server or IIS. On a Java servlet container, drop in the API Server WAR file. See the help documentation for more information and how-tos.

The API Server is also easy to deploy on Microsoft Azure, Amazon EC2, and Heroku.

Connect to Elasticsearch

After you deploy the API Server and the ADO.NET Provider for Elasticsearch, provide authentication values and other connection properties needed to connect to Elasticsearch by clicking Settings -> Connection and adding a new connection in the API Server administration console.

Set the Server and Port connection properties to connect. To authenticate, set the User and Password properties, PKI (public key infrastructure) properties, or both. To use PKI, set the SSLClientCert, SSLClientCertType, SSLClientCertSubject, and SSLClientCertPassword properties.

The data provider uses X-Pack Security for TLS/SSL and authentication. To connect over TLS/SSL, prefix the Server value with 'https://'. Note: TLS/SSL and client authentication must be enabled on X-Pack to use PKI.

Once the data provider is connected, X-Pack will then perform user authentication and grant role permissions based on the realms you have configured.

When you configure the connection, you may also want to set the Max Rows connection property. This will limit the number of rows returned, which is especially helpful for improving performance when designing reports and visualizations.

You can then choose the Elasticsearch entities you want to allow the API Server access to by clicking Settings -> Resources.

Authorize API Server Users

After determining the OData services you want to produce, authorize users by clicking Settings -> Users. The API Server uses authtoken-based authentication and supports the major authentication schemes. Access can also be restricted based on IP address; by default, only connections to the local machine are allowed. You can authenticate as well as encrypt connections with SSL.

Connect to Elasticsearch

Follow the steps below to connect to Elasticsearch data from Power BI.

1. Open Power BI Desktop and click Get Data -> OData Feed. To start Power BI Desktop from PowerBI.com, click the download button and then click Power BI Desktop.

2. Enter the URL to the OData endpoint of the API Server. For example:

   http://MyServer:8032/api.rsc

3. Enter authentication for the API Server. To configure Basic authentication, select Basic and enter the username and authtoken for a user of the OData API of the API Server.

   The API Server also supports Windows authentication using ASP.NET. See the help documentation for more information.

4. In the Navigator, select tables to load. For example, Orders.

   

Create Data Visualizations

After pulling the data into Power BI, you can create data visualizations in the Report view. Follow the steps below to create a pie chart:

1. Select the pie chart icon in the Visualizations pane.
2. Select a dimension in the Fields pane: for example, OrderName.
3. Select a measure in the Freight in the Fields pane: for example, Freight.

You can change sort options by clicking the ellipsis (...) button for the chart. Options to select the sort column and change the sort order are displayed.

You can use both highlighting and filtering to focus on data. Filtering removes unfocused data from visualizations; highlighting dims unfocused data. You can highlight fields by clicking them:

You can apply filters at the page level, at the report level, or to a single visualization by dragging fields onto the Filters pane. To filter on the field's value, select one of the values that are displayed in the Filters pane.

Click Refresh to synchronize your report with any changes to the data.

Upload Elasticsearch Data Reports to Power BI

You can now upload and share reports with other Power BI users in your organization. To upload a dashboard or report, log into PowerBI.com, click Get Data in the main menu and then click Files. Navigate to a Power BI Desktop file or Excel workbook. You can then select the report in the Reports section.

Refresh on Schedule and on Demand

You can configure Power BI to automatically refresh your uploaded report. You can also refresh the dataset on demand in Power BI. Follow the steps below to schedule refreshes through the API Server:

1. Log into Power BI.
2. In the Dataset section, right-click the Elasticsearch Dataset and click Schedule Refresh.
3. If you are hosting the API Server on a public-facing server like Azure, you can connect directly. Otherwise, if you are connecting to a feed on your machine, you will need to expand the Gateway Connection node and select a gateway, for example, the Microsoft Power BI Personal Gateway.
4. In the settings for your dataset, expand the Data Source Credentials node and click Edit Credentials.
5. Expand the Schedule Refresh section, select Yes in the Keep Your Data Up to Date menu, and specify the refresh interval.

You can now share real-time Elasticsearch reports through Power BI.