Skip to main content

TDM 3.6 Release Notes

· 5 min read
Jake Cave

3.6 Major Release Items​

What's New​

File Connection - Azure & S3 Support​

  • Users can now connect to and process files stored on both Amazon S3 and Azure.
  • Allows for profiling, masking, and validation directly from these cloud storage providers.

Profile Config - Multi-threading with Configurable Thread Count​

  • Multi-threading is now available in the profile configuration screen.
  • Users can set a thread count for faster profiling, provided the database supports it. This will significantly speed up the profiling process for large datasets.

Compliance Report Generator - Multiple Data Connections in One Report​

  • The TDM Report Generator has been enhanced to support multiple data connections within a single report.
  • Provides more comprehensive insights and flexibility in reporting.

Conditional Masking 'Like' Query​

  • New functionality allows users to apply conditional masking based on SQL 'LIKE' queries.
  • Enables the exclusion or inclusion of specific rows based on text patterns.

Group Feature for Data Connections​

  • Data connections can now be linked together into groups.
  • Allows users to perform group profiling, masking, validation, and reporting. This enables more efficient workflows for large-scale data management.
  • Enable the TDM Queue Manager to make use of this new feature!

Data Connection Bulk Upload/Update​

  • New feature allows users to upload or update multiple data connections in bulk.
  • Streamlines the management of large data environments.

New TDM API Endpoint - Get Logs and Reports​

  • A new API endpoint is available to retrieve the JSON version of logs and reports.
  • Allows users to programmatically access profile report data.

Changes​

Improved Performance for Masking Large Files​

  • Optimized chunk size for file masking, particularly for configurations exceeding 10,000 records.
  • Results in better performance and scalability for large file masking operations.

File Validation - Unsupported Pattern Handling​

  • A new validation mechanism compares the difference between the original and masked files when the pattern of the column is unsupported.
  • Ensures data integrity during masking.

Fixed Issues​

DOB Masking - ORACLE​

  • Addressed issues related to Date of Birth (DOB) masking.
  • Ensured more consistent and accurate masking results across various configurations.

Data Generator - Name Field Order​

  • Resolved an issue where the Data Generator was incorrectly ordering 'Full Name' with first and last names.

Masking Performance for Large Datasets​

  • Improved performance when masking datasets with over 40 million rows.
  • Reduced processing times and improved system scalability.

JSON Masking Performance​

  • Resolved an issue causing slower-than-expected performance and inconsistent results during the masking of JSON files.

3.6.1​

Release Date: 15th October 2024

Fixed​

DB Profiling:​

Sybase​
  • Fixed issue where profiling would hang if the connection pool is exceeded.
Teradata​
  • Addressed masking issue by recollecting sessions after profiling to prevent session limit from exceeding.
Oracle​
  • Resolved issue concerning materialised view logs being included in profiling.

Dataview​

  • Resolved Dataview not showing the output of stored procedures for Sybase.

DataCatalogue​

  • Resolved issue concerning views being included in DataCatalogue for Oracle.

MongoDB Connection Timeout:​

  • Implemented more connection retries if it fails to connect to the database.

Bulk Reports:​

  • Ensured values in bulk reports are enclosed in double quotes.
  • Fixed an issue with conditional masking parameters not showing up in the report.
  • Enhanced the displayed error message when generating bulk reports for failed profile scans to provide clearer feedback.

3.6.2​

Release Date: 30th October 2024

Fixed​

Teradata:​

  • Enhanced the accuracy of conditional validation.

Oracle:​

  • Resolved an issue where DOB masking for 20th century dates would fail with incompatible nls_date_format.

Sybase:​

  • Enhanced performance for forensic scanner.

Couchbase:​

  • Updated compatibility with later Couchbase versions.

DB/File:​

  • Added conditional masking parameters to validation reports.

File:​

  • Fixed an issue with wildcards not working as intended for Azure Blob.

3.6.3​

Release Date: 15th November, 2024

Fixed​

DB - General​

  • Refined authentication handling, so switching types now properly defaults username and password fields to empty strings.
  • Removed fabrication config creation for unsupported types.

MSSQL Masking​

  • Resolved an issue that could interrupt fixed value masking, ensuring more reliable data processing.

Oracle Masking​

  • Added better handling for temporary table and column deletion on exceptions during masking.

Sybase Profiling​

  • Updated Profile report to ensure row count displays accurately.

File - General​

  • Validate scans now support validation based on data length specifications from masking config.
  • Added tailored error messages for unsupported wildcard usage.

Fixed Width Masking​

  • Improved parsing of fixed values during masking, now based on data connection-defined column widths.

Validation Report​

  • Resolved an issue where Custom Function names were being displayed as an ID in Validation report.

API - MongoDB​

  • Resolved an issue with remote MongoDB functionality, allowing for seamless model backup and restore.