Profiling
Overview of Profiling
The first step of the process is to Profile your Data Source. Profiling, scans the data source and determines, with a degree of certainty, the type of data in the column. During the process a Data Masking configuration file and a Data Validation (Audit) configuration file is generated as per the TDM solution.
Profile Configurations
The rulesets and patterns scanned as part of the profiling process are defined in Profile Configurations.
For DCS version 3.5:
Navigate to Data Management Hub > Manage Configurations > Profile Configurations. Here you can Add, Edit, Clone or Delete a Profile Configuration.
For DCS versions 3.4 or earlier:
Navigate to Profiling > Profiling Configurations here you can Add, Edit, Clone or Delete a Profile Configuration.
Adding a Profile Configuration
Click the Add Profile Configuration button. This will present a new Profile Configuration form for adding the following information.
Note: You can add a default sample by clicking the "Load Sample" button.
For DCS version 3.5:
For DCS versions 3.4 or earlier:
On the Profile Configuration form, you can edit the following configuration information.
| Type | Description |
|---|---|
| Configuration Name | The profile configuration name. |
| Default Tolerance | The default quantity of matches for it to be classified as a pattern. |
| Depth | The number of maximum records to search. (Note: The higher this is set the slower the profile will be). |
| Description | The profile configuration description or long name. |
| Include/Exclude Table | The tables to exclude from profiling or alternatively only include for profiling. |
| Notification | List of users who will be notified. |
| Status | Set the profile to Active for use or Draft/Inactive to hide it from the Profile Execution area. |
| Deep Scan | An advanced feature which ignores null values when identifying PII data. This is advantageous in an empty table that may contain PII. (DCS3.5) |
Add/Edit Pattern(s) section: -
| Type | Description |
|---|---|
| Category | The pattern category. (This filters the results visible in the Pattern drop list). |
| Identifier | Pattern Name used in the profile config. |
| Tolerance | Overrides the default tolerance with a specific tolerance. |
| PII Level | Sets the pattern as Primary PII, Secondary PII, or Other. |
| Pattern Type | The pattern type as defined in the Data Library Administration area. |
| Pattern | Pattern which has to be used. |
Once complete, scroll to the bottom of the main window and click Save to Save the Profile or Close to Cancel out without saving.
Editing a Profile Configuration
Click the Edit Profile Connection button on the Profile Configuration you want to edit. This will present the Profile Configuration form for modification.
Once complete, scroll to the bottom of the main window and click Save to Save the configuration or Close to Cancel out without saving.
Cloning a Profile Configuration
Click the Clone Profile Connection button on the Profile Configuration you want to duplicate. This will present the Profile Configuration form with a duplication of all data for saving as a new Profile Configuration.
Once complete, scroll to the bottom of the main window and click Save to Save the configuration or Close to Cancel out without saving.
Deleting a Profile Configuration
Click the Delete Profile Connection button on the Profile Configuration you want to remove. This will present a confirmation window confirming that you want to delete the Profile Configuration or not.
Comparing Profile Configurations
Tick on the check boxes in front of the two configurations which you want to compare and then press the Config Comparison button to view the comparison report. The text highlighted in red marks the differences in the two configurations.
Execute Profiling
For DCS version 3.5:
To execute a new profile scan, navigate to Data Management Hub > Execution Console.
Here, you can select your data source then press View. Scroll down to the Execution Details section, select Profile in the first dropdown menu, and your chosen validation/mask configuration in the second dropdown menu. The Execute button can be used to then run a profile scan or View Results to simply view the results from the previously ran profile jobs.
For DCS versions 3.4 or earlier:
To Execute a new profiling request, Navigate to Profiling > Execute Profiling
Here you can select a data source connection and Profile Configuration for execution. Press Run button to run the selected profile configuration. On Run button press, the list down will be refreshed with past executions and will also show the status of current execution request.
Validation Log
In the Details column, the Log button can be used to view logs for the execution. execution.
Note: The Log window refreshes every 15 seconds if the profile is currently executing.
A currently executing validation scan can also be cancelled by clicking the Cancel button.
Validation Report
Successfully completed profiles will generate a Profile Report which can be viewed by clicking the Report button.
This can also be searched on or exported out to Excel or CSV.
