CData Virtuality 4.9 Release Notes: Enhanced Security Platform, Improved Clustering, Increased Connectivity, and More

This release is packed with improvements, most importantly, on the Server's side, support for third-party security vaults by replacing the legacy PicketBox solution with Elytron, centralizing security management, lowering development and maintenance costs, and enhancing regulatory compliance. Please note that this means that the CLI Exporter does not export LDAP settings anymore and make a backup manually.

BigQuery can now be used not only as a data source but also as analytical storage. We've also added several possibilities for BigQuery as a data source: disabled creation of physical indexes for BigQuery data sources, implemented the ability to use MERGE statement for rewriting UPSERT queries, and added the possibility to use a folder within a storage bucket.

Starting with this release, CData Virtuality supports eight new CData drivers: Kintone, Google Sheets, Zuora, Sage Intacct, Paylocity, ServiceNow, and Workday.

The Server now has the option to control handling of multiple underscores in column names and use wildcards for the jobs, procedures, functions, tables and views permissions. We've also improved the rules for assigning and removing permissions: previously, a user with "E" permission for the SYSADMIN.setPermissions procedure could grant or revoke any permissions on any object. Now, in addition to needing "E" permission for SYSADMIN.setPermissions, a user must meet one of the following criteria to modify permissions on an object:

• Be an admin user
• Be the owner of the object (procedure or job)
• Have "A" (Alter) permission on the object, in which case the user can only assign or remove permissions they themselves hold for that object.

As for bug fixes, most importantly, we've fixed two bugs affecting the Cloud Agent: one causing the creation of MySQL data source to fail and another where some agent-specific properties were not present in CLI templates and "SYSADMIN.ConnectorProperties" table. We've also resolved a bug with OData V4 where filtering on GUID column would fail, a bug preventing role assignment from being applied to existing sessions, and a bug cauding the PREFER_DWH option not working when used in subqueries. Now all is well.

We've also disabled the Python language permission for the admin-role meaning that from now on, to enable python scripting, you'll need to explicitly add python L permission to the custom python role.

As for the Studio, improvements include extending the BigQuery data source wizards to support "folderInBucket" translator property, as well as bucket name, bucket prefix, and credentials. Another improvement is the possibility to use Google Analytics API v4 by default in the query builder wizard.

On the connectors side, we've updated the Facebook connector to API v21 and added column geo_target_country to "Performance_Campaign" report for the Google Ads API connector. We've also worked on the Amazon Selling Partner connector for which we've added READY_TO_SHIP status to Shipments Report and fixed two bugs: one causing Report_FlatFileSettlement to fail with "ERROR: value too long for type character varying(50)" and another resulting in loading Orders and its subtables failing with HTTP response 400. Other bug fixes are one each for Facebook (where the performance reports had no "end_date" parameter), Zendesk (where the getSatisfactionRatings and getUsersIncremental procedures would fail), and Airtable (where the InstallExamples procedure throwed error). Now all is well.

Here are all issues in this release:

Server

• DVCORE-8620 (Improvement): BigQuery: disable creation of physical indexes for BigQuery data sources
• DVCORE-8418 (Improvement): BigQuery: use MERGE statement for rewriting UPSERT queries
• DVCORE-6736 (Improvement): BigQuery: add a possibility to use a folder within a storage bucket
• DVCORE-8613 (Improvement): Integrate Kintone CData JDBC driver
• DVCORE-8609 (Improvement): Integrate Google Sheets CData JDBC driver
• DVCORE-8604 (Improvement): Integrate Zuora CData JDBC driver
• DVCORE-8601 (Improvement): Integrate Jira CData JDBC driver
• DVCORE-8600 (Improvement): Integrate Sage Intacct CData JDBC Driver
• DVCORE-8597 (Improvement): Integrate Paylocity CData JDBC driver
• DVCORE-8593 (Improvement): Integrate ServiceNow CData JDBC driver
• DVCORE-8592 (Improvement): Integrate Workday CData JDBC driver
• DVCORE-8588 (Improvement): Disable the Python language permission for the admin-role
• DVCORE-8572 (Improvement): Add an option to control handling of multiple underscores in column names
• DVCORE-8520 (Improvement): Make it possible to connect Grafana to CData Virtuality
• DVCORE-8432 (Improvement): Add wildcard for the jobs, procedures, functions, tables and views permissions
• DVCORE-8318 (Improvement): Improve rules for assigning and removing permissions
• DVCORE-7711 (Improvement): Migrate authentication and authorization mechanisms from PicketBox to Elytron
• DVCORE-8622 (Bug Fix): Google Analytics Data: listProperties procedure fails when run for a long time
• DVCORE-8102 (Bug Fix): Google Analytics Data: some new dimensions and metrics are missing
• DVCORE-8612 (Bug Fix): Data source records are not removed from the configuration file when the data source is replaced by the SYSADMIN.createOrReplaceDatasource procedure
• DVCORE-8611 (Bug Fix): Dataverse JDBC driver version used by CData Virtuality produces wrong results
• DVCORE-8608 (Bug Fix): Cloud Agent: creation of MySQL data source fails
• DVCORE-8472 (Bug Fix): Cloud Agent: some agent-specific properties are not present in CLI templates and "SYSADMIN.ConnectorProperties" table
• DVCORE-8605 (Bug Fix): Google Analytics Data: data source creation fails with "Unable to get public no-arg constructor" error
• DVCORE-8603 (Bug Fix): OData V4: filtering on GUID column fails
• DVCORE-8602 (Bug Fix): Gathering metadata via CData driver results in NullPointerException
• DVCORE-8584 (Bug Fix): Snowflake: history update job throws "Stored procedure execution error: Scoped transaction started in stored procedure is incomplete and it was rolled back" when no upload mode or INTERNAL_STAGE mode is used
• DVCORE-8576 (Bug Fix): Web service connector: POST requests with NTLM authentication fail
• DVCORE-8571 (Bug Fix): Exasol: upsert and history update jobs fail when columnNameType is set to UPPERCASE or PRESERVE for mixed case columns
• DVCORE-8566 (Bug Fix): BigQuery: it is impossible to insert into a table with columns having unknown nullable status
• DVCORE-6599 (Bug Fix): BigQuery: region mismatch between the dataset and the temporary bucket causes an error upon inserting
• DVCORE-8538 (Bug Fix): Names of system functions in "SYS.Functions" and "SYS.FunctionParams" tables are in lowercase
• DVCORE-8537 (Bug Fix): Some functions are not shown in "SYS.Functions" and "SYS.FunctionParams" tables
• DVCORE-8495 (Bug Fix): Oracle ADWC: wallet is not encoded in base64 in connection properties
• DVCORE-8130 (Bug Fix): Oracle ADWC: SYSADMIN.createOrReplaceDatasource procedure does not use the "wallet" connection property
• DVCORE-8478 (Bug Fix): Role assignment is not applied to existing sessions
• DVCORE-8315 (Bug Fix): ClickHouse: INSERT INTO fails with NullPointerException when NULL is inserted into NOT NULL column
• DVCORE-8300 (Bug Fix): ClickHouse: some functions like LPAD, LEFT, TIMESTAMPADD and others are not pushed down to the source
• DVCORE-314 (Bug Fix): PREFER_DWH option does not work when used in subqueries

Studio

• DVCORE-8632 (Improvement): Extend Google BigQuery data source wizards to support "folderInBucket" translator property
• DVCORE-6409 (Improvement): Extend Google BigQuery wizard to support bucket name, bucket prefix, and credentials
• DVCORE-7365 (Improvement): Use Google Analytics API v4 by default in the query builder wizard
• DVCORE-7335 (Improvement): Hide server connection information in the window header
• DVCORE-8637 (Bug Fix): SQL Editor autocompletion suggests either data sources or virtual schemas, but not both
• DVCORE-8214 (Bug Fix): The color scheme of the error dialog remains light when the dark theme is enabled throughout Studio on Linux

Connectors

• SQL-967 (Improvement): Facebook: update connector to API v21
• SQL-957 (Improvement): Amazon Selling Partner: add READY_TO_SHIP status to Shipments Report
• SQL-954 (Improvement): Google Ads API: add column geo_target_country to "Performance_Campaign" report
• SQL-979 (Bug Fix): Facebook: the performance reports have no "end_date" parameter
• SQL-975 (Bug Fix): Amazon Selling Partner: Report_FlatFileSettlement fails with "ERROR: value too long for type character varying(50)"
• SQL-958 (Bug Fix): Amazon SP: loading Orders and its subtables fails with HTTP response 400
• SQL-963 (Bug Fix): Zendesk: getSatisfactionRatings and getUsersIncremental procedures fail
• SQL-944 (Bug Fix): Airtable: InstallExamples procedure throws error