4 posts tagged with "decommissioned"

3.6 Major Release Items​

What's New​

File Connection - Azure & S3 Support​

• Users can now connect to and process files stored on both Amazon S3 and Azure.
• Allows for profiling, masking, and validation directly from these cloud storage providers.

Profile Config - Multi-threading with Configurable Thread Count​

• Multi-threading is now available in the profile configuration screen.
• Users can set a thread count for faster profiling, provided the database supports it. This will significantly speed up the profiling process for large datasets.

Compliance Report Generator - Multiple Data Connections in One Report​

• The TDM Report Generator has been enhanced to support multiple data connections within a single report.
• Provides more comprehensive insights and flexibility in reporting.

Conditional Masking 'Like' Query​

• New functionality allows users to apply conditional masking based on SQL 'LIKE' queries.
• Enables the exclusion or inclusion of specific rows based on text patterns.

Group Feature for Data Connections​

• Data connections can now be linked together into groups.
• Allows users to perform group profiling, masking, validation, and reporting. This enables more efficient workflows for large-scale data management.
• Enable the TDM Queue Manager to make use of this new feature!

Data Connection Bulk Upload/Update​

• New feature allows users to upload or update multiple data connections in bulk.
• Streamlines the management of large data environments.

New TDM API Endpoint - Get Logs and Reports​

• A new API endpoint is available to retrieve the JSON version of logs and reports.
• Allows users to programmatically access profile report data.

---

Changes​

Improved Performance for Masking Large Files​

• Optimized chunk size for file masking, particularly for configurations exceeding 10,000 records.
• Results in better performance and scalability for large file masking operations.

File Validation - Unsupported Pattern Handling​

• A new validation mechanism compares the difference between the original and masked files when the pattern of the column is unsupported.
• Ensures data integrity during masking.

---

Fixed Issues​

DOB Masking - ORACLE​

• Addressed issues related to Date of Birth (DOB) masking.
• Ensured more consistent and accurate masking results across various configurations.

Data Generator - Name Field Order​

• Resolved an issue where the Data Generator was incorrectly ordering 'Full Name' with first and last names.

Masking Performance for Large Datasets​

• Improved performance when masking datasets with over 40 million rows.
• Reduced processing times and improved system scalability.

JSON Masking Performance​

• Resolved an issue causing slower-than-expected performance and inconsistent results during the masking of JSON files.

---

---

3.6.1​

Release Date: 15th October 2024

Fixed​

DB Profiling:​

Sybase​

• Fixed issue where profiling would hang if the connection pool is exceeded.

Teradata​

• Addressed masking issue by recollecting sessions after profiling to prevent session limit from exceeding.

Oracle​

• Resolved issue concerning materialised view logs being included in profiling.

Dataview​

• Resolved Dataview not showing the output of stored procedures for Sybase.

DataCatalogue​

• Resolved issue concerning views being included in DataCatalogue for Oracle.

MongoDB Connection Timeout:​

• Implemented more connection retries if it fails to connect to the database.

Bulk Reports:​

• Ensured values in bulk reports are enclosed in double quotes.
• Fixed an issue with conditional masking parameters not showing up in the report.
• Enhanced the displayed error message when generating bulk reports for failed profile scans to provide clearer feedback.

---

---

3.6.2​

Release Date: 30th October 2024

Fixed​

Teradata:​

• Enhanced the accuracy of conditional validation.

Oracle:​

• Resolved an issue where DOB masking for 20th century dates would fail with incompatible nls_date_format.

Sybase:​

• Enhanced performance for forensic scanner.

Couchbase:​

• Updated compatibility with later Couchbase versions.

DB/File:​

• Added conditional masking parameters to validation reports.

File:​

• Fixed an issue with wildcards not working as intended for Azure Blob.

---

---

3.6.3​

Release Date: 15th November, 2024

Fixed​

DB - General​

• Refined authentication handling, so switching types now properly defaults username and password fields to empty strings.
• Removed fabrication config creation for unsupported types.

MSSQL Masking​

• Resolved an issue that could interrupt fixed value masking, ensuring more reliable data processing.

Oracle Masking​

• Added better handling for temporary table and column deletion on exceptions during masking.

Sybase Profiling​

• Updated Profile report to ensure row count displays accurately.

File - General​

• Validate scans now support validation based on data length specifications from masking config.
• Added tailored error messages for unsupported wildcard usage.

Fixed Width Masking​

• Improved parsing of fixed values during masking, now based on data connection-defined column widths.

Validation Report​

• Resolved an issue where Custom Function names were being displayed as an ID in Validation report.

API - MongoDB​

• Resolved an issue with remote MongoDB functionality, allowing for seamless model backup and restore.

3.5 Major Release Items​

What's New​

File Validation​

• Files can now be validated based on their masking configurations.
  • We recommend using the following workflow for the best results: profiling → masking → validation

File Connection - FTP​

• Collect files using FTP an connection
• Users can Profile, Mask, Validate files using FTP login details
• Connecting to FTP is as simple as connecting to a remote server. Ensure the server type is "FTP".

Profiling - Deep Scan​

• This feature conducts a thorough scan of each column individually rather than scanning the entire table at once. Particularly effective for tables containing a significant amount of null data. Note: This might increase profiling time by 5 times!

New Front-End Changes​

• Enhanced user interface:

  • User interface has been rebuilt to ease the configuration process
  • All parameters for masking are now separate grouped fields
  • UDS has extended configuration fields

• Changes to the execution screen

  • This enables easier access for not only viewing but executing scans on a datasource

• Config changes

  • Revamped config for more control over scan logic

• Conditional Masking

  • Separated fields to ease the process of creating and modifying conditional mask statements.

Please see the DCS user guide for more information

Optimised Chunking​

• The chunking process is expedited by determining the batch size based on the higher value between the total row count and distinct row count.
  • This can still be overridden by user input

---

Changes and Enhancements​

Database Masking - PostgreSQL, DB2, MariaDB, MSSQL​

• Skip columns with constraints such as: Primary key Unique key Foreign key
• By default fixed value masking will now not use chunking. To enable chunking use the parameters field in the masking config

TDM API - Additional Information​

• Attached additional information to the dcs server check api endpoint to give users the:
  • TDM Version
  • Database start-up time
  • Current time on TDM server Access the endpoint via http://[dcs_server]/dcs/api/dcs_server_check

Masking Report - Additional Information​

• Report now includes the time taken to mask each column

Custom Functions - Test Function​

• Can now test look-up and parameter based functions.

---

Fixed​

Masking - MySQL​

• Resolved performance issues by optimising the insertion of data

File Masking - JSON​

• Fixed an issue where minimum and maximum overlap input was not validated

DB Validation​

• Skip columns with no data with a log message, rather than producing an error
• Now reducing the lookup of pattern telnum to 1000, so it can be validated accurately

---

---

3.5.1​

Release Date: 7th June 2024

Fixed​

PostgreSQL Check Connection​

• Resolved an edge case where conflicting packages could cause the connection to fail

File Masking​

• Delimited
  • Fixed an issue where data length was not trimmed correctly if the length was specified
  • Resolved a complication where user defined chunk size would not be updated

Pattern Finder​

• Solved an issue where an empty data set would raise errors

DB Masking​

• MSSQL
  • Fixed an error where clustered indexes were not being disabled

---

---

3.5.2​

Release Date: 21st June 2024

Fixed​

File Profiling​

• Fixed an issue where files containing only invisible characters and multiple empty lines were not recognised as empty files.

Conditional Masking​

• Resolved an issue where the new user interface changes could disrupt the order of conditional masking, leading to a row count error.

Profiling Report​

• Fixed an issue where the profiling report URL was not updated upon failure.

---

---

3.5.3​

Release Date: 8th July 2024

Fixed​

MongoDB Profiling​

• Resolved an issue where profiling would fail for users lacking permissions to certain tables.

Large File Masking​

• Resolved a high memory utilisation issue, optimising overall performance and stability.

Large File Profiling​

• Optimised the profiling process for large files, enhancing speed and efficiency.

Oracle Masking​

• Fixed a bug where the unpredictable behavior of using rownum could potentially skip rows.

Teradata Insert​

• Corrected an issue where failed inserts in Teradata could enable debug mode and return the partial failed SQL query.

Teradata Masking​

• Fixed a problem in Teradata where empty data could cause complications in the generation of masked data.

3.4 Major Release Items​

What's New​

SSL Support​

• Users can now connect to databases using custom SSL certificates.
• For MSSQL, MySQL, MariaDB, PostgreSQL

Regex Database Masking​

• Inclusion of an integrated function to manage regex generation.
• A user can place their required regex in the parameter field to create fabricated regex data.

Cross Table Conditional Masking​

• Masking based on cross table relationships. This will join two columns to mask the correct PII. This can be enabled using the parameters field

  Example:

  col|=|value||chunk (optional)|external_table|table_ref|cond_table_ref

Changes and Enhancements​

Database Masking - MySQL​

• Skip columns with constraints such as: Primary key Unique key Foreign key
• By default fixed value masking will now not use chunking. To enable chunking use the parameters field in the masking config

TDM Report - Template Changes​

• Enhancements for better visualisation of the database life cycle.
• Additional graphs and diagrams to view the database life cycle.

Fixed​

Profile MSSQL​

• Resolved an issue where tables where being skipped due to exceeding the holding byte size variable.

DB Masking​

• Oracle Resolved a logs issue where it stated triggers could not be disabled when in fact there was no triggers. Fixed an issue where table or view could not be found due oracle version incompatibility.

• MySQL Resolved an issue where the conditional masking configuration for the 'Address' column in the 'where' clause was ambiguous.

• Custom Functions Fixed an issue where fullname custom function could not be run with other custom functions.

DB Validation​

• Fixed an issue where trailing space was not trimmed before validating the last 4 digits against the lookup value.

Fabrication​

• Resolved an error where column generation failed when the data length was undefined.

Data Source​

• Fixed an error where special characters where being filtered out in the schema name.

Decommissioned​

• Removal of pattern Encrypt One to One

3.4.1​

Release Date: 1st March 2024

Fixed​

Profiling​

• MSSQL Fixed an issue where columns with empty information were not being skipped
• MYSQL Resolved a profile issue in the latest version of MySQL by adding additional support to exclude table type 'views' in databases

DB Masking​

• Oracle Fixed an issue where detailed error logs were not being displayed when using the oracle function 'dbms_parallel'
• MSSQL Resolved an issue where date was out of range due to the way small-datetime behaves It requires data to be the year 1900 and later.

Fabrication​

• MSSQL Resolved a syntax error when creating tables.

TDM Report Generator​

• Corrected logic for folders and output file generation

DataLibrary​

• Others_HashLast4Digits Fixed an issue where masked data generation failed when the source data's last 4 digits were alphabetical values
• Drivers_LicenseManyToOne Corrected an issue wherein pattern name failed to correspond with function name

3.4.2​

Release Date: 18th March 2024

Fixed​

Validation​

• Oracle Resolved an issue where a case sensitive search was being used for schemas

DB Masking​

• Firstname Many to One Fixed an edge case where encoded source values were not being adjusted before masking.
• Parameter Checks Resolved an edge case where conditional masking was attempted due to empty string conditions.
• Oracle Fixed an issue, so constraint types, listed below, are ignored: Primary Key Unique Key Foreign Key
• MySQL Performance improvements by no longer using order methods.

File Masking​

• JSON Resolved an error when using fixed value masking.

What's New​

Information Template​

This feature allows you to design input forms to capture additional information as part of user requests.

DevOps Manager​

• GIT Integration- New Feature allowing you execute scripts directly from your GIT repository.
• Folder Management - New Feature allowing you to create or manage folders and files within your scripts in file explorer.

Webhook DevOps Integration​

This new feature allows you to call DevOps Script on a webhook call.

Project PI Planning​

This feature allows projects to plan and track their activities for the release train.

Package Deployment Tracker​

This new data brick allows you to keep track of all the package deployment status across your different environments.

Release Milestone Tracker​

This new report will allow release manager to track how projects are progressing with their release activities.

Changes & Enhancements​

Connectivity View​

System view has been enhanced to use different symbols for systems, instances and components.

Enhanced Reports​

These reports have been revamped to support new fresh look.

• Configure Environments (Environment Management >> Management)
• Environment Activity (Environment Management >> Management)
• PI Activity (Release Train Management >> Reports)
• Manage Release Trains (Release Train Management >> Management)
• Manage PI Plans (Release Train Management >> Management)
• Project Activity (Project Management >> Reports)
• Project Demand (Project Management >> Reports)
• Manage Projects (Project Management >> Management)
• Booking Activity (Booking Management >> Reports)
• Booking Queue (Booking Management >> Reports)
• Manage Bookings (Booking Management >> Management)
• Event Activity (Event Management >> Reports)
• Manage Events (Event Management >> Management)
• Service Activity (Service Management >> Reports)
• Manage Service Requests (Service Management >> Management)

Booking Gantt Reports​

Additional Filter to view data by System Instance Assigned To

User Groups​

Ability to send emails to user group email address

Bulk Upload​

Added feature to automatically remove unsupported characters from CSV

Deployed Version Dashboard​

• Ability to see multi instance details in the same environment on hover.
• Added ability to view data by"Environment" & "Environment Type"

Environment Connectivity (System Instance View)​

This view was showing decommissioned objects which has been fixed now.

Decommissioned​

Release Train Management >> Management​

• PI Editor

Project Management >> Management​

• Project Editor