3.5 Major Release Items
What's New
File Validation
- Files can now be validated based on their masking configurations.
- We recommend using the following workflow for the best results:
profiling → masking → validation
File Connection - FTP
- Collect files using FTP an connection
- Users can Profile, Mask, Validate files using FTP login details
- Connecting to FTP is as simple as connecting to a remote server. Ensure the server type is "FTP".
Profiling - Deep Scan
- This feature conducts a thorough scan of each column individually rather than scanning the entire table at once. Particularly effective for tables containing a significant amount of null data. Note: This might increase profiling time by 5 times!
New Front-End Changes
Please see the DCS user guide for more information
Optimised Chunking
- The chunking process is expedited by determining the batch size based on the higher value between the total row count and distinct row count.
- This can still be overridden by user input
Changes and Enhancements
Database Masking - PostgreSQL, DB2, MariaDB, MSSQL
- Skip columns with constraints such as:
Primary key
Unique key
Foreign key
- By default fixed value masking will now not use chunking. To enable chunking use the parameters field in the masking config
- Attached additional information to the dcs server check api endpoint to give users the:
- TDM Version
- Database start-up time
- Current time on TDM server
Access the endpoint via
http://[dcs_server]/dcs/api/dcs_server_check
- Report now includes the time taken to mask each column
Custom Functions - Test Function
- Can now test look-up and parameter based functions.
Fixed
Masking - MySQL
- Resolved performance issues by optimising the insertion of data
File Masking - JSON
- Fixed an issue where minimum and maximum overlap input was not validated
DB Validation
- Skip columns with no data with a log message, rather than producing an error
- Now reducing the lookup of pattern telnum to 1000, so it can be validated accurately
3.5.1
Release Date: 7th June 2024
Fixed
PostgreSQL Check Connection
- Resolved an edge case where conflicting packages could cause the connection to fail
File Masking
- Delimited
- Fixed an issue where data length was not trimmed correctly if the length was specified
- Resolved a complication where user defined chunk size would not be updated
Pattern Finder
- Solved an issue where an empty data set would raise errors
DB Masking
- MSSQL
- Fixed an error where clustered indexes were not being disabled
3.5.2
Release Date: 21st June 2024
Fixed
File Profiling
- Fixed an issue where files containing only invisible characters and multiple empty lines were not recognised as empty files.
Conditional Masking
- Resolved an issue where the new user interface changes could disrupt the order of conditional masking, leading to a row count error.
Profiling Report
- Fixed an issue where the profiling report URL was not updated upon failure.
3.5.3
Release Date: 8th July 2024
Fixed
MongoDB Profiling
- Resolved an issue where profiling would fail for users lacking permissions to certain tables.
Large File Masking
- Resolved a high memory utilisation issue, optimising overall performance and stability.
Large File Profiling
- Optimised the profiling process for large files, enhancing speed and efficiency.
Oracle Masking
- Fixed a bug where the unpredictable behavior of using rownum could potentially skip rows.
Teradata Insert
- Corrected an issue where failed inserts in Teradata could enable debug mode and return the partial failed SQL query.
Teradata Masking
- Fixed a problem in Teradata where empty data could cause complications in the generation of masked data.