Skip to main content

TDM 2.9 Release Notes

· 4 min read
Jake Cave

2.9 Major Release Items​

What's New​

Base DCS - Redshift Support​

  • Added support for Redshift in Base DCS.

File Masking​

  • Conditional masking has been added for all file types.
  • Optimizations made by using chunking when masking delimited files.

Data Compliance Insight​

  • View your data operation execution reports across different data sources and types.

Data Connection Insight​

  • Centralized view to monitor your data source status and execute different data operations from a single view.

REST API​

  • Ecosystem API endpoints for DCS jobs.

Data Bricks​

  • New data widgets in custom dashboards.

Changed​

Data Source Management​

  • The "Link with System Components" option has been removed and replaced with labels.

PII Splatter Board​

  • The redesigned view allows users to search for PII compliance across different data sources.

Execution Queue​

  • View the current status of all data operation executions.

Fixed​

Global​

  • Extended global timeout for database type Couchbase.

Database Masking​

  • Fixed issue where Oracle tables might not be compressed correctly.
  • Altered Oracle queries used in masking to allow for larger DDL queries to be saved.
  • Resolved bug where masking an empty column did not display the correct error.

File Profiling​

  • Log updates for delimited file type now work correctly in all scenarios.

File Masking​

  • Fixed issue with delimited files where quotes were not retained if the value contained a comma (e.g., "hello, world").
  • Resolved bug where an incorrect engine was used to read delimited files.
  • Fixed issue where result count could be incorrect for delimited and fixed-width file types.
  • Added better handling for delimited file masking when dealing with bad inputs.

Decommissioned​

  • Data Compliance
  • Compliance Dashboard
  • Profile Dashboard
  • Validate Dashboard
  • Masking Dashboard

2.9.1​

Release Date: 1st October 2022

Changed​

File Masking - Delimited​

  • Enhanced progress updates for delimited file masking. The process now incorporates folders and the size of a file to report an accurate percentage of completion.

File Masking​

  • Improved chunking algorithm to strengthen performance. Masking is now based on distinct values found: if greater than 1 million, masking is based on distinct values; otherwise, it is based on row count.

AWS Redshift​

  • Reworked logic involving vacuums to maximize performance on masking.

Fixed​

Database Masking​

  • Performance improvements for databases with large counts of distinct values. The reworked algorithm constitutes improvements for the following database types:

    • MSSQL
    • MySQL
    • MariaDB
    • PostgreSQL
    • AWS Redshift
    • Teradata

  • Resolved a rare bug when masking full names where many-to-one masking returned an index error.

File Masking - XML​

  • Improvement in the file gathering process when conducting a folder mask.
  • Revised algorithmic changes to update recursion when failing on edge cases.

File Masking - Delimited​

  • Resolved issues related to breaking due to unique characters, such as certain Chinese characters.
  • Resolved a TFN masking bug on RedHat 8.1 systems.

Masking - Oracle​

  • Improvements to masking diagnosis to resolve merging conflicts with special characters.
  • Enhanced trigger enabling/disabling when multiple correlated triggers are attached to one table.

Profiling - AWS Redshift​

  • Now only retaining tables of the required "type" and excluding types such as views.

DCS Base​

  • Fixed temporary table logging issues when failing in rare cases.

2.9.2​

Release Date: 1st February 2023

Changed​

File Masking - Delimited​

  • Enhanced performance when masking multiple large files, along with optimized memory usage.
Database Masking - Oracle​
  • Will now utilize a chunking feature to break data inserts into blocks of data to increase performance and reduce transaction log.

File Masking​

  • Mismatch in row count between input and output file will now be logged. Since row counts can differ due to bad lines, empty lines, etc.

Fixed​

Database Masking - Redshift​

  • Fixed issue where string values were being skipped if they included quotes.

File Masking - Delimited​

  • Fixed issue where empty space values were being masked.
  • Resolved issue where values starting with escape characters or empty space were being skipped.
  • Fixed reporting structure to accurately convey when a column is masked multiple times in one scan.

Profile Scanner - Oracle​

  • Fixed wildcard functionality to incorporate rules to abide by include/exclude table functionality.

Decommissioned​

  • Nothing has been decommissioned.