Skip to main content

TDM Report Generator

Introduction​

The TDM Overview Report generator is a critical tool designed for optimising the management and analysis of test data in software development. It automatically generates detailed reports in PDF format, utilising Ecosystem and DCS API, to provide a comprehensive overview of the test data lifecycle. This tool is essential for database administrators, data engineers, and software testers, offering insights into each stage of test data management, including pre-scan details, forensic scan metrics, profiling, masking, and validation.

Key functionalities include:

  • Pre-scan: Offers initial database information such as type and name.
  • Forensic Scan: Details database size, user count, and schema specifics.
  • Profiling: Analyses profiling duration, configuration, and sensitive data identification.
  • Masking: Summarises data masking activities and outcomes for data security.
  • Validation: Reports on the final validation of the test data management process.

A detailed explanation on the key functionalities of the report are further discussed below in the Structure section.

Purpose​

The purpose of this report generator is multifaceted:

  1. Detailed Analysis: It provides in-depth insights into various stages of test data management, including pre-scan details, forensic scan metrics, profiling, masking, and validation.

  2. Enhanced Decision-Making: By offering detailed data on database type, size, user count, and schema specifics, it facilitates informed decision-making and efficient management strategies.

  3. Data Security and Compliance: The tool emphasises data security through its masking and validation reports, ensuring compliance with data privacy standards.

  4. Operational Efficiency: The generator's ability to quickly produce comprehensive reports aids in timely issue identification and resolution, thereby enhancing operational efficiency and data integrity.

This tool is not just a reporting mechanism, but a comprehensive solution that supports the continuous improvement and integrity of test data management, pivotal in today's data-driven software development landscape.

Structure​

The Test Data Management Report is meticulously organised to provide a thorough understanding of the test data lifecycle. Each section is crafted to offer both detailed information and clarity.

Pre-scan​

This section introduces the database, specifying the type (such as relational or NoSQL) and the database name. This foundational information sets the stage for the subsequent analysis.

Pre Scan Example

Forensic Scan​

Here, the report delves into key database metrics. It covers the database size, quantified in gigabytes or terabytes, and the user count. The section also includes a time prediction for running scans and the date of the last forensic scan. An essential part of this section is the detailed schema breakdown, offering a count and comprehensive details of each schema within the database.

Forensic Scan Example
Datatype Distribution

Profiling​

The profiling section is divided into two parts. The first part lists the dates and duration of the profiling process, along with a summary of the configuration settings used. The second part presents the profiling results, including the total number of Personally Identifiable Information (PII) found based on metadata and content scans, along with a detailed categorisation and count of the PII identified.

Profile Scan Example
PII Breakdown

Masking​

In the masking section, the report first outlines the date range and duration of the masking process, followed by a summary of the configuration settings. The results of the masking process are then detailed, indicating the overall status and specifics, such as the number of columns successfully masked and any failures.

Masking Scan Example
Mask Function Distribution

Validation​

Similar to the masking section, the validation segment starts with information on the dates and duration of the validation process and a summary of the configuration settings. The section concludes with the validation results, providing insights into the overall status of the process and the specifics of the columns validated, including successes and failures.

Validation Scan Example

This structured approach in the report ensures that every critical aspect of test data management is comprehensively covered, providing valuable insights for efficient data management and compliance.

Configurability​

Notably, the report generator offers flexibility in its configuration. Users can tailor the report to include or exclude specific sections based on their requirements. This feature ensures that the report is not only informative but also relevant to the specific context in which it is required.

Compatibility​

Currently, the report generator is specifically designed to work with database connections. This focus ensures that the tool provides in-depth and accurate insights for database-related test data management activities. It is important to note that the current version of the report generator does NOT support file scans.

Usage​

Firstly, navigate to Data Management > Administration > Data Source Management. Then select a data source (note: make sure it is a database source), and click on the download report button.

Download Report

Then, a popup window will appear which can be used to select the sections to include in the report.

Report Gen Section Selection

After selecting the desired sections, press OK. This should start the report generation process. Once the report compilation is complete, it will be ready for download.

Report Gen Complete