Seamlessly Bring Elasticsearch Data into Retool Using CData Connect Cloud


Integrate Elasticsearch data into Retool with CData Connect Cloud to build smarter apps and streamline workflows with real-time data access.

Retool enables the rapid creation of custom internal tools using a low-code or no-code approach. By integrating Elasticsearch into Retool with CData Connect Cloud, you can access live data from Elasticsearch in real time, enabling you to build more efficient apps and streamline workflows.

This guide walks you through the simple steps to connect Elasticsearch to Retool via CData Connect Cloud using OpenAPI or the Virtual SQL Server API. You can securely connect to your Elasticsearch data, ensuring seamless integration and real-time updates, empowering you to create smarter apps and enhance operational efficiency. Let's get started!

About Elasticsearch Data Integration

Accessing and integrating live data from Elasticsearch has never been easier with CData. Customers rely on CData connectivity to:

  • Access both the SQL endpoints and REST endpoints, optimizing connectivity and offering more options when it comes to reading and writing Elasticsearch data.
  • Connect to virtually every Elasticsearch instance starting with v2.2 and Open Source Elasticsearch subscriptions.
  • Always receive a relevance score for the query results without explicitly requiring the SCORE() function, simplifying access from 3rd party tools and easily seeing how the query results rank in text relevance.
  • Search through multiple indices, relying on Elasticsearch to manage and process the query and results instead of the client machine.

Users frequently integrate Elasticsearch data with analytics tools such as Crystal Reports, Power BI, and Excel, and leverage our tools to enable a single, federated access layer to all of their data sources, including Elasticsearch.

For more information on CData's Elasticsearch solutions, check out our Knowledge Base article: CData Elasticsearch Driver Features & Differentiators.


Getting Started


Overview

Here is an overview of the steps:

  1. Configure: (Optional) Set up your Elasticsearch data source in CData Connect Cloud, add a Workspace to Virtual Datasets, add assets to your Workspace, and generate a Personal Access Token (PAT).
  2. Connect: Link Retool to CData Connect Cloud using OpenAPI or the Virtual SQL Server API.
  3. Explore: Use your connected data to build smarter apps and explore advanced integrations.

Prerequisites

Ensure you have the following before starting:

  1. An active Elasticsearch account and credentials.
  2. A CData Connect Cloud account. Log in or sign up for a free trial here.
  3. A Retool account. Sign up here.

STEP 1: Configure the Elasticsearch Connectivity for Retool (in CData Connect Cloud)

Add a Elasticsearch connection in CData Connect Cloud to integrate it with Retool.

Note: This step is optional and can be skipped if you have already connected to the Elasticsearch data source.

  • Log into Connect Cloud, click Connections and click Add Connection
  • Select "Elasticsearch" from the Add Connection panel
  • Enter the necessary authentication properties to connect to Elasticsearch.

    Set the Server and Port connection properties to connect. To authenticate, set the User and Password properties, PKI (public key infrastructure) properties, or both. To use PKI, set the SSLClientCert, SSLClientCertType, SSLClientCertSubject, and SSLClientCertPassword properties.

    The data provider uses X-Pack Security for TLS/SSL and authentication. To connect over TLS/SSL, prefix the Server value with 'https://'. Note: TLS/SSL and client authentication must be enabled on X-Pack to use PKI.

    Once the data provider is connected, X-Pack will then perform user authentication and grant role permissions based on the realms you have configured.

  • Click Create & Test
  • Navigate to the Permissions tab in the Add Elasticsearch Connection page and update the User-based permissions.

    If the connection fails, refer to error messages for guidance in the documentation. Some data sources require direct sign-in, which can be completed via the Sign In button if prompted.

    Configure the Virtual Datasets and Workspace, and Generate a Personal Access Token (PAT).

    Once you've connected Elasticsearch into the CData Connect Cloud, configure virtual Datasets and Workspaces, and then generate a personal access token (PAT) to enable seamless integration and data management across various connected data sources.

    Virtual Datasets in CData Connect Cloud simplify access to diverse data sources, supporting tables, views, and derived views with full CRUD capabilities. Workspaces organize datasets, offer SQL Server, connectivity endpoints, and include assets like folders for efficient management. Learn more about Virtual Datasets here

    • Create a Workspace:
      • Click Virtual Datasets () on the left, then Add on the top right.
      • Enter a Workspace Name and click Confirm to create the workspace.
    • Create a Virtual Dataset or Folder:
      • In the workspace, click Add on the top right, select from the menu (Folder, Tables and Views, or Derived Views), and provide the necessary details (e.g., folder name, connection, tables/views).
      • Click Confirm to add the asset.
      • For any asset, the tabs show:
        • Columns: Metadata table.
        • Preview: Data item preview.
        • SQL: Generated SQL query, including the configurable Alias.

    Personal Access Tokens (PATs) in CData Connect Cloud allow connections via REST API, OData, or Virtual SQL Server.

    • To create a new PAT:
      • Go to Settings (top right) > Access Tokens.
      • Click Create PAT, enter a name, then click Create.
      • Copy the PAT using the copy icon. NOTE: The PAT can't be viewed again after closing the dialog, so make sure to copy it before clicking OK.
      • Click OK to close the dialog, and save the copied PAT for use in proceeding steps.
    • To regenerate a PAT, click the regeneration icon () in the PAT list and follow the prompts.

    STEP 2: Connect Retool to Elasticsearch in CData Connect Cloud

    Now that you've configured your Elasticsearch connection in CData Connect Cloud, it's time to integrate it with Retool.

    Retool can be connected to Elasticsearch in CData Connect Cloud via either OpenAPI or the Virtual SQL Server API. Choose the method that best suits your workflow:

    Connect via OpenAPI

    To connect using the OpenAPI specification, follow these steps:

    1. On CData Connect Cloud, from the left-hand side, go to Virtual Database > select a Workspace.
    2. Click the View Endpoints, and under the OpenAPI, copy the OpenAPI URL for the workspace. Keep the default options. See OpenAPI section for more information.
    3. Login to Retool and then go to the Resources tab, click Create new > Resource.
    4. In the Select a resource type search bar, and search for OpenAPI.
    5. Click OpenAPI. The Configure OpenAPI screen appears. Enter the following information to configure the OpenAPI connector:
      • Name: The Connection Name to identify the connector on Retool.
      • Specification URL: The OpenAPI URL you copied from the Workspace in CData Connect Cloud. Paste the URL here.
      • Forward headers when fetching OpenAPI spec: Check this box.
      • Under the URL parameters, keep the key-value pair unchanged.
      • In the Headers section, write the key as Authorization and in the value, write Basic.
      • For the Basic connection value, Retool requires a base64-encoded version of your CData Connect Cloud username:PAT combination (where PAT is the token you generated earlier). Use a tool like this to encode the value. For example, test@cdata.com:12345 encodes to dGVzdEBjZGF0YS5TIzNDU=. Copy this encoded value and paste it into the value field after Basic with a space.
      • Username: Enter your CData Connect Cloud username. This is displayed in the top-right corner of the CData Connect Cloud interface. For example, test@cdata.com.
      • Password: Enter the PAT you generated on the Settings page.
    6. Click Test connection to test your connection configuration. Once you have a successful connection, click Create resource.
    7. Create the app in Retool in which to integrate your data.
    8. Click Create resource query.
    9. Select the new resource you created, and view the endpoints that are available.
    10. You can now build your app with your live Elasticsearch data.

    Connect via Virtual SQL Server API

    To connect using the Virtual SQL Server API, follow these steps:

    1. Login to Retool, then go to Resources. Click Connect New at the top-right and select Resources.
    2. Enter Microsoft SQL in the search bar, click the resulting option, and enter the following details:
      • Name: A name for the connection to CData Connect Cloud.
      • Host: tds.cdata.com.
      • Port: 1433.
      • Database Name: Enter the Connection Name of the CData Connect Cloud data source you want to connect to (for example, Salesforce1).
      • Database Username: Enter your CData Connect Cloud username. This is displayed in the top-right corner of the CData Connect Cloud interface. For example, test@cdata.com.
      • Database Password: Enter the PAT you generated on the Settings page.
    3. Click Test Connection. If the connection fails, you might need to whitelist CData Connect Cloud IPs for Retool and other data sources. Check out the documentation here.
    4. If it succeeds, click Create Resource.
    5. Your connection is now set up, and you can build your app with live Elasticsearch data.


    STEP 3: Build Retool Apps with Your Live Elasticsearch Data

    Now that you have integrated Elasticsearch with Retool using CData Connect Cloud, it's time to utilize your live data to its full potential. Let's build a simple app:

    • Step 1: Set Up the Table (on home screen)
      • Create a New App in Retool.
      • Select your Elasticsearch connection.
      • Add a Table component and bind it to Elasticsearch's data.
    • Step 2: Add Filters (on top-right)
      • Add a Dropdown to filter by Opportunity Stage.
      • Add a Text Input for searching Account Name.
    • Step 3: Display Key Metrics (below Table)
      • Add a Text component to show Total Amount of filtered opportunities.
      • Use a query to sum the Amount for filtered data.
    • Step 4: Add Visuals (top-right or side panel)
      • Add a Bar Chart to visualize opportunities by Stage.
      • Add a Line Chart to show Expected Revenue over time.
    • Step 5: Test and Deploy (top-right)
      • Test filters, search, and charts.
      • Deploy the app and share the link for access.

    With these steps, you've successfully created a dynamic app to manage and visualize your Salesforce opportunities. Enjoy exploring your live data and leveraging it for better insights!


    Try CData Connect Cloud Free for 30 Days

    Experience real-time Elasticsearch data access and enhanced app development with CData Connect Cloud in Retool. Start your free trial today!

    • Ready to get started?

    Learn more about CData Connect Cloud or sign up for free trial access:

    Free Trial

    In this article

    Related articles