MongoDB Staging Environment

Introduction

Efficient performance of the MongoDB server is crucial for the optimal functioning of the enov8 Test Data Management (TDM) Tool. This document outlines the standard configurations required for a MongoDB staging server and provides a comprehensive set of troubleshooting steps for Database Administrators (DBAs) to ensure that the database aligns with these standards and operates at peak efficiency.

Disclaimer

This documentation applies to all MongoDB versions currently supported by MongoDB, Inc., specifically for use in staging environments. The configurations and queries provided are recommendations based on typical staging setups and may not be suitable for all environments. It is essential to evaluate these recommendations in the context of your specific staging environment and requirements. Consult with your organization's database administrators and MongoDB documentation for guidance tailored to your specific needs.

NoSQL Differences

MongoDB is a document-oriented NoSQL database. Key differences from relational databases relevant to TDM operations: there are no schemas enforced at the database level (fields are discovered at runtime via aggregation), there are no SQL triggers (MongoDB does not support them), and transactions are not required for TDM masking operations since document updates are atomic at the collection level.

1. Standard Database Server Configuration

1.1 Hardware Requirements

The hardware requirements should scale based on the size of the database and the volume of data being masked. Below is a comprehensive guideline for CPU, memory, disk, and network:

CPU

Cores: Start with 4-6 cores, add 2 cores per 1 TB of data.
Example:
- For a 1 TB database: 4 cores.
- For a 5 TB database: 12 cores.

Memory (RAM)

Base Requirement: Start with 16 GB, add 16 GB per 1 TB of data.
WiredTiger Cache: MongoDB's WiredTiger storage engine uses a cache sized at 50% of (RAM − 1 GB) by default. For masking operations this cache should be at least 4 GB.
Example:
- For a 1 TB database: 16 GB of RAM (7.5 GB WiredTiger cache).
- For a 5 TB database: 80 GB of RAM (39.5 GB WiredTiger cache).

Disk

Type: Use SSDs for optimal performance, especially for I/O-intensive masking operations and index creation.
Capacity: Provision at least 2× the total data size to accommodate temporary masking collections, index builds, and WiredTiger journal files.
IOPS: Minimum of 10,000 IOPS for SSDs; 300 IOPS for HDDs.
Example:
- For a 1 TB database: 2 TB of SSD storage.

Network

Bandwidth: At least 1 Gbps network interface card (NIC).
Latency: Low-latency connections between the enov8 TDM application server and the MongoDB host. High latency amplifies the cost of the many round-trips made during bulk document updates.

1.2 System-Level Database Optimizations

Optimizing for Bulk Document Masking

Disable Journaling (Standalone Only):
- Purpose: For standalone staging servers that do not require crash recovery, disabling journaling reduces I/O during masking. Do not disable journaling on replica set members.
- Implementation (in mongod.conf):
```
storage:
  journal:
    enabled: false
```
- Note: Restart mongod after changing this setting.
Set Write Concern to Unacknowledged for Staging:
- Purpose: The TDM tool connects with retryWrites=true&w=majority by default. For a standalone staging server where data loss on crash is acceptable, reducing write concern improves throughput.
- Note: This is a connection-level option. The TDM connection string can be adjusted if your staging policy allows it.
Increase WiredTiger Cache:
- Purpose: A larger cache reduces disk I/O during collection scans and bulk updates used by masking operations.
- Implementation (in mongod.conf):
```
storage:
  wiredTiger:
    engineConfig:
      cacheSizeGB: 8
```
- Sizing guideline: Set to 60–70% of available RAM, leaving headroom for the OS page cache and sort memory.
Disable the Profiler for Staging:
- Purpose: The MongoDB profiler (slowms logging) adds overhead during high-volume masking operations. Disable it on the staging instance.
- Implementation:
```
db.setProfilingLevel(0);
```

Operating System Configuration

Transparent Huge Pages (THP): Disable THP on Linux. MongoDB explicitly recommends against it.

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

ulimit settings (Linux):

Limit Recommended Setting
Open files (nofile) 65536
Max processes (nproc) 64000
Virtual memory (as) unlimited

Limit	Recommended Setting
Open files (`nofile`)	65536
Max processes (`nproc`)	64000
Virtual memory (`as`)	unlimited

1.3 Connection Configuration (TDM UI)

The TDM connection form uses the following fields. Configure them to match your staging MongoDB instance.

Required fields

Field	Description
Server	Hostname or IP of the MongoDB server (or comma-separated list for replica sets).
Database	Database name used for masking.
Connection Format	Standard (default) — direct host:port; DNS Seedlist — for MongoDB Atlas or replica sets that use SRV records (`mongodb+srv://`).

Optional connection fields

Field	Default	Description
Port	27017	TCP port.
Authentication Database Source	admin	Database where user credentials are stored (`authSource`).
Replica Set Name	—	Set this when the staging deployment is a replica set. The TDM tool appends `replicaSet=<name>` to the connection. For staging, a single-node replica set is often preferred over a standalone instance so oplog-based behaviour and read/write semantics match production.
Username / Password	—	Used for DEFAULT, SCRAM-SHA-1, SCRAM-SHA-256, MONGODB-CR, PLAIN, and GSSAPI. Not used for MONGODB-X509.

Authentication Mechanism

Choose the mechanism that matches your MongoDB deployment. The form shows or hides fields based on this selection.

Value	When to use	Extra fields shown in UI
DEFAULT	Let the server negotiate (typical for username/password).	—
SCRAM-SHA-256	Recommended for password-based auth.	—
SCRAM-SHA-1	Legacy password auth.	—
MONGODB-CR	Legacy; avoid for new deployments.	—
PLAIN	LDAP.	—
GSSAPI	Kerberos.	Service Name (default `mongodb`), Realm, Keytab Path, Kinit Path
MONGODB-X509	TLS client certificate.	Certificate Authentication File Path, Certificate Key File Path

Connection Format is only available when Authentication Mechanism is one of DEFAULT, SCRAM-SHA-1, SCRAM-SHA-256, MONGODB-CR, or PLAIN (not GSSAPI or MONGODB-X509).

Index Management (staging requirements)

Regardless of how the connection is configured in the UI, the staging database user must allow the following:

Temporary Indexes: The TDM tool creates a temporary index (e8_idx_{fieldName}) on the match field before merge-based masking and drops it afterwards. The user needs createIndex and dropIndex privileges.
Index Definition Storage: The tool uses a collection named e8_index_def in the target database to store index definitions when disabling and re-enabling indexes around masking. Do not modify or drop this collection during a job. It is created automatically.
Background builds: Temporary indexes are built with background=True; on current MongoDB versions index builds are non-blocking by default.

1.4 Network Configuration

Port: The TDM connection form uses Port default 27017. Ensure the TDM application server can reach the MongoDB host on the port you configure. MongoDB listens on this port unless changed in mongod.conf.

mongod.conf (self-hosted):

net:
  port: 27017
  bindIp: 0.0.0.0   # Or restrict to specific IPs
  tls:
    mode: disabled   # Set to 'requireTLS' if using MONGODB-X509 auth

MongoDB Atlas: Use Connection Format: DNS Seedlist in the TDM connection and whitelist the TDM application server IP in Atlas Network Access.
Firewall: Open the configured port from the TDM application server to the MongoDB host. For replica sets, open the port between members as well.

1.5 Authentication

The enov8 TDM tool supports the following MongoDB authentication mechanisms. In the TDM connection form, choose the matching option from the Authentication Mechanism dropdown; the form then shows the relevant fields (e.g. Username/Password, or for GSSAPI the Keytab Path, or for MONGODB-X509 the certificate paths).

SCRAM-SHA-256 (Recommended)

What it is: The default and most secure password-based authentication mechanism. Uses the Salted Challenge Response Authentication Mechanism with SHA-256.
When to use: The standard choice for all new MongoDB deployments. Preferred over SCRAM-SHA-1 for all staging environments.
TDM configuration: authMechanism=SCRAM-SHA-256, authSource=<auth_database>

Connection string:

mongodb://user:pass@host:27017/dbname?authMechanism=SCRAM-SHA-256&authSource=admin

Setup:

db.createUser({
  user: "tdm_user",
  pwd: "strong_password",
  roles: [{ role: "readWrite", db: "staging_db" },
          { role: "dbAdmin",   db: "staging_db" },
          { role: "clusterMonitor", db: "admin" }]
})

SCRAM-SHA-1

What it is: An older SCRAM variant using SHA-1. Still supported but less secure than SCRAM-SHA-256.
When to use: When connecting to legacy MongoDB 3.x instances that do not support SCRAM-SHA-256, or when compatibility with older drivers is required.
TDM configuration: authMechanism=SCRAM-SHA-1, authSource=<auth_database>

Connection string:

mongodb://user:pass@host:27017/dbname?authMechanism=SCRAM-SHA-1&authSource=admin

DEFAULT

What it is: Lets the MongoDB server negotiate the best available mechanism. The server will use SCRAM-SHA-256 if supported, falling back to SCRAM-SHA-1.
When to use: When the MongoDB version or configuration is uncertain, or when you want the driver to automatically negotiate.
TDM configuration: authMechanism=DEFAULT, authSource=<auth_database>

PLAIN (LDAP)

What it is: SASL PLAIN authentication. Sends the username and password in cleartext (must be used over TLS). Enables MongoDB to delegate authentication to an external LDAP server.
When to use: Environments that centralise identity management with LDAP (e.g. Active Directory) and require MongoDB to authenticate against it.
TDM configuration: authMechanism=PLAIN, authSource=$external

Connection string:

mongodb://user:pass@host:27017/dbname?authMechanism=PLAIN&authSource=%24external

Prerequisites:
- MongoDB Enterprise required for LDAP authentication.
- TLS must be enabled (net.tls.mode: requireTLS) to prevent password exposure.
- mongod.conf LDAP configuration:
```
security:
  ldap:
    servers: "ldap.company.com"
    transportSecurity: tls
    authz:
      queryTemplate: "ou=users,dc=company,dc=com"
```

GSSAPI (Kerberos)

What it is: Kerberos-based authentication via GSSAPI. The user principal is obtained from the Kerberos ticket cache rather than embedded in the connection string. The TDM tool passes the username as a separate parameter (not in the URI) to preserve the Kerberos principal format (e.g. user@REALM.COM).
When to use: Enterprise environments that use Kerberos for centralized authentication (common in large financial and government organizations).
TDM configuration: authMechanism=GSSAPI, authSource=$external, optionally Service Name=<service_name> if the MongoDB service uses a non-default Kerberos service name (default: mongodb)

Connection string (internal):

mongodb://host:27017/dbname?authMechanism=GSSAPI&authSource=%24external&authMechanismProperties=SERVICE_NAME:mongodb

Prerequisites:

MongoDB Enterprise required.
The TDM application server must have a valid Kerberos ticket (kinit tdm_user@REALM.COM) before starting a masking job.

The MongoDB server must be configured with a Kerberos keytab:

security:
  authorization: enabled
setParameter:
  authenticationMechanisms: GSSAPI

The TDM service account Kerberos principal must be mapped to a MongoDB user:

db.getSiblingDB("$external").createUser({
  user: "tdm_user@REALM.COM",
  roles: [{ role: "readWrite", db: "staging_db" },
          { role: "dbAdmin",   db: "staging_db" },
          { role: "clusterMonitor", db: "admin" }]
})

MONGODB-X509 (TLS Certificate)

What it is: Authenticates using a client TLS certificate instead of a username/password. The subject Distinguished Name (DN) of the client certificate is used as the MongoDB username. Requires both a client certificate/key file and a CA certificate.
When to use: High-security environments where password-based authentication is prohibited, or for service-to-service authentication where certificate rotation is managed centrally.
TDM configuration: authMechanism=MONGODB-X509, cert_key_path=<path to .pem>, ca_path=<path to CA .pem>. Optionally, set username to the certificate subject DN if required.
Connection string (internal):
```
mongodb://host:27017/?authMechanism=MONGODB-X509
```
The TDM tool passes tlsCertificateKeyFile and tlsCAFile as direct MongoClient parameters.

Prerequisites:

TLS must be enabled on the MongoDB server:

net:
  tls:
    mode: requireTLS
    certificateKeyFile: /etc/ssl/mongodb.pem
    CAFile: /etc/ssl/ca.pem

The TDM service account must be created in $external using the certificate subject DN:

db.getSiblingDB("$external").createUser({
  user: "CN=tdm_service,OU=TDM,O=Company,C=AU",
  roles: [{ role: "readWrite", db: "staging_db" },
          { role: "dbAdmin",   db: "staging_db" },
          { role: "clusterMonitor", db: "admin" }]
})

The client certificate and CA certificate must be accessible on the TDM application server at the configured paths.

MONGODB-CR (Deprecated)

What it is: The original MongoDB challenge-response authentication mechanism. Removed in recent MongoDB versions.
When to use: Only for legacy environments that have not yet migrated to SCRAM. Do not use for new deployments.
TDM configuration: authMechanism=MONGODB-CR, authSource=<auth_database>

1.6 Security and Patch Management

Patching:
- Apply MongoDB patches via the standard package manager (apt, yum) or MongoDB Atlas automatic patch management.
- Maintain MongoDB at the latest patch release within your major version to receive security fixes.
Enable Authentication:
- Ensure security.authorization: enabled is set in mongod.conf. Without this, any connecting client has full access.
```
security:
  authorization: enabled
```
TLS in Production:
- For any staging environment that mirrors production data, enable TLS (net.tls.mode: requireTLS) regardless of the auth mechanism to protect data in transit.

2. Troubleshooting Steps for DBAs

When experiencing performance issues with the TDM masking tool on MongoDB, the following steps can help identify and resolve common problems.

2.1 Verify Hardware and OS Configuration

Check CPU and Memory

Purpose: Ensure hardware meets specifications and THP is disabled.

Commands:

lscpu
free -h
cat /sys/kernel/mm/transparent_hugepage/enabled

Verification: Confirm CPU and RAM meet the requirements in section 1.1. The THP output should show [never].

Disk I/O Performance

Purpose: Ensure disk I/O supports bulk document updates and index builds.
Commands:
```
iostat -x 1 5
```
Verification: The %util column should remain below 80%. High utilisation during masking indicates I/O bottlenecks affecting WiredTiger write performance.

2.2 Validate MongoDB Configuration

Check WiredTiger Cache Size

Purpose: Confirm sufficient cache is allocated for masking operations.

Commands:

db.serverStatus().wiredTiger.cache["maximum bytes configured"]

Expected Output: A value of at least 4 GB (4,294,967,296 bytes).
Verification: If the cache size is too small, increase storage.wiredTiger.engineConfig.cacheSizeGB in mongod.conf.

Confirm Authentication Mechanism

Purpose: Verify the MongoDB server supports the auth mechanism configured in the TDM tool.

Commands:

db.adminCommand({ getParameter: 1, authenticationMechanisms: 1 })

Expected Output:

{ "authenticationMechanisms": ["SCRAM-SHA-1", "SCRAM-SHA-256"], "ok": 1 }

Verification: The mechanism configured in the TDM tool (SCRAM-SHA-256, GSSAPI, MONGODB-X509, etc.) must appear in this list. If it does not, add it to setParameter.authenticationMechanisms in mongod.conf.

Verify TLS Configuration (for MONGODB-X509 and PLAIN)

Purpose: Confirm TLS is enabled and the server certificate is valid.

Commands:

openssl s_client -connect <mongodb-host>:27017

Verification: The certificate chain should be valid and the subject should match the configured hostname. For MONGODB-X509 auth, confirm the client certificate subject DN matches the MongoDB user created in $external.

2.3 Assess Database Performance

Check Server Status

Purpose: The TDM tool calls db.command("serverStatus") at connection time to retrieve the MongoDB version. Use it to assess overall server health during masking.

Commands:

db.serverStatus().connections
db.serverStatus().opcounters

Expected Output:

{ "current": 5, "available": 51195, "totalCreated": 12 }

Verification: High current connection counts indicate connection accumulation. The TDM tool opens one connection per masking job; if the count grows unexpectedly, check for leaked connections.

Check Collection Document Counts

Purpose: The TDM tool uses db.command("count", collectionName) to retrieve document counts per collection. Verify counts are accurate.

Commands:

db.runCommand({ count: "my_collection" })
// Or for faster estimates on large collections:
db.my_collection.estimatedDocumentCount()

Verification: For accurate counts after an unclean shutdown, run db.my_collection.countDocuments({}). The TDM tool uses count, which returns fast estimates from collection metadata.

Check Database Size

Purpose: The TDM tool calls db.command('dbstats')['fsTotalSize'] for database size reporting during forensic scan.
Commands:
```
db.stats()
```

Expected Output:

{ "db": "staging_db", "dataSize": 1073741824, "storageSize": 536870912, "fsTotalSize": 107374182400 }

Verification: storageSize reflects actual on-disk usage after WiredTiger compression. fsTotalSize is the total filesystem size, not the database size — this is what the TDM tool reports as the database size.

2.4 Check Session and Query Performance

Identify Currently Running Operations

Purpose: Find active masking operations and confirm they are progressing.

Commands:

db.currentOp({ "active": true, "op": { "$in": ["update", "command"] } })

Verification: Look for long-running update or aggregate operations from the TDM service account. The secs_running field indicates how long an operation has been active.

Monitor Slow Operations

Purpose: Identify masking operations that are slower than expected.

Commands (temporarily enable profiling at level 1):

db.setProfilingLevel(1, { slowms: 1000 });  // Log ops slower than 1 second

// Query the profiler
db.system.profile.find().sort({ ts: -1 }).limit(10).pretty();

// Disable after investigation
db.setProfilingLevel(0);

Verification: Review slow update and aggregate operations. Missing indexes on match fields are the most common cause of slow bulk updates during masking.

Review Execution Plans

Purpose: Determine if collection scans are causing slow masking updates.

Commands:

db.my_collection.explain("executionStats").find({ field_name: "value" })

Verification: Look for COLLSCAN in the winning plan. A COLLSCAN on a large collection during merge operations indicates the temporary index (e8_idx_{fieldName}) was not created successfully, or the wrong field is being used as the match key.

2.5 Evaluate Storage and Space Usage

Check Collection Storage Usage

Purpose: Ensure the staging database has sufficient space for temporary masking collections and index builds.

Commands:

db.stats({ scale: 1024 * 1024 })  // Output in MB

Expected Output:

{
  "db": "staging_db",
  "collections": 12,
  "dataSize": 1024,
  "storageSize": 512,
  "indexSize": 128,
  "freeStorageSize": 64
}

Verification: Total used space (dataSize + indexSize) should not exceed 80% of available disk. Budget additional space for temporary masking collections (approximately 1× the size of each masked collection) and the e8_index_def collection.

Check the e8_index_def Collection

Purpose: The TDM tool stores index definitions in e8_index_def during index disable/enable cycles. If a masking job failed mid-run, orphaned entries in this collection can prevent indexes from being re-created correctly.
Commands:
```
db.e8_index_def.find().pretty()
```
Verification: After a failed masking job, check for orphaned entries in e8_index_def. If the collection contains stale entries from a previous failed run, they should be cleared before re-running:
```
db.e8_index_def.drop()
```

2.6 Review Indexes

List Indexes on a Collection

Purpose: The TDM tool calls collection.list_indexes() to enumerate indexes. Confirm that indexes on frequently masked collections are healthy.
Commands:
```
db.my_collection.getIndexes()
```

Expected Output:

[
  { "name": "_id_", "key": { "_id": 1 } },
  { "name": "field1_1", "key": { "field1": 1 }, "unique": true }
]

Verification: Ensure the _id index is always present. Check for any indexes in an invalid state, which can cause masking merge operations to fail.

Rebuild Invalid Indexes

Purpose: Invalid or corrupt indexes cause bulk_write operations to fail during masking.
Commands:
```
db.my_collection.reIndex()
```
Verification: Run db.my_collection.validate() after reindexing to confirm the collection is healthy.

2.7 Network Diagnostics

Test Connectivity on Port 27017

Purpose: Confirm the enov8 TDM application server can reach the MongoDB instance.

Commands (from the TDM application server):

# Windows
Test-NetConnection -ComputerName <mongodb-host> -Port 27017
# Linux
nc -zv <mongodb-host> 27017

Expected Output (Linux):

Connection to <mongodb-host> 27017 port [tcp] succeeded!

Test the ismaster Command

Purpose: The TDM tool calls client.admin.command('ismaster') immediately after connecting to verify the connection is live. Test this manually to confirm authentication and connectivity.

Commands:

mongosh "mongodb://tdm_user:password@host:27017/staging_db?authMechanism=SCRAM-SHA-256&authSource=admin" \
  --eval "db.adminCommand({ ismaster: 1 })"

Expected Output:
```
{ "ismaster": true, "ok": 1 }
```

2.8 Verify Security and Patch Levels

Check MongoDB Version

Purpose: The TDM tool calls db.command("serverStatus")['version'] to retrieve the version at runtime.

Commands:

db.version()
// Or:
db.adminCommand({ serverStatus: 1 }).version

Verification: Confirm the instance is running a version currently supported by MongoDB, Inc.

Check Applied Patches

Purpose: Confirm the MongoDB instance is on the latest patch release.

Commands (Linux):

mongod --version
apt list --installed 2>/dev/null | grep mongodb   # Debian/Ubuntu
yum list installed | grep mongodb                 # RHEL/CentOS

2.9 Application Interaction

Confirm Driver Installation

Purpose: The TDM tool uses the pymongo Python library. Confirm it is installed and at a compatible version.
Verification:
```
pip show pymongo
```
Expected Output: A current release of pymongo compatible with your MongoDB server version.

Test the TDM Service Account Connection

Purpose: Verify that the TDM service account can connect using the configured auth mechanism before running a masking job.

SCRAM-SHA-256:

mongosh "mongodb://tdm_user:password@host:27017/staging_db?authMechanism=SCRAM-SHA-256&authSource=admin"

GSSAPI (Kerberos) — verify ticket first:

klist   # Confirm a valid Kerberos ticket is present
mongosh "mongodb://host:27017/staging_db?authMechanism=GSSAPI&authSource=%24external" \
  --username "tdm_user@REALM.COM"

MONGODB-X509:

mongosh "mongodb://host:27017/?authMechanism=MONGODB-X509" \
  --tls --tlsCertificateKeyFile /path/to/client.pem --tlsCAFile /path/to/ca.pem

2.10 System Resource Monitoring

Monitor CPU and Memory Usage

Purpose: Detect resource bottlenecks during bulk document updates.
Commands:
```
top
```
Verification: Monitor mongod process CPU and memory usage. High CPU during masking is expected; sustained 100% CPU suggests the WiredTiger cache is too small and the engine is doing excessive disk I/O.

Check Swap Usage

Purpose: Ensure MongoDB is not swapping, which degrades WiredTiger cache performance significantly.
Commands:
```
swapon -s
free -h
```
Verification: Confirm minimal swap usage. If swap is active during masking, increase available RAM or reduce the masking batch size.

3. Database Permissions

MongoDB uses Role-Based Access Control (RBAC). Unlike relational databases, permissions are granted by assigning roles to users rather than by granting individual privileges on tables. The following roles are required for the enov8 TDM service account.

Authentication Database

The TDM service account should be created in the admin database (using authSource=admin) for SCRAM-based auth, or in $external for GSSAPI and MONGODB-X509 auth.

Required Roles

Role	Database	Privileges Granted	Used By
readWrite	Target staging database	`find`, `insert`, `update`, `delete`, `createIndex`, `dropIndex`, `listIndexes`, `listCollections`, `createCollection`, `dropCollection`	All
dbAdmin	Target staging database	`dbStats`, `collStats`, `validate`, `createIndex`, `dropIndex`, `dropCollection`, `compact`	All, Masking
clusterMonitor	`admin` database	`serverStatus`, `connPoolStats`, `top` — required for the `serverStatus` command used to retrieve the MongoDB version	All

Creating the TDM Service Account

SCRAM-SHA-256 / SCRAM-SHA-1 / DEFAULT:

use admin
db.createUser({
  user: "tdm_user",
  pwd: "strong_password",
  roles: [
    { role: "readWrite",      db: "staging_db" },
    { role: "dbAdmin",        db: "staging_db" },
    { role: "clusterMonitor", db: "admin"       }
  ]
})

GSSAPI (Kerberos) — user created in $external:

use $external
db.createUser({
  user: "tdm_user@REALM.COM",
  roles: [
    { role: "readWrite",      db: "staging_db" },
    { role: "dbAdmin",        db: "staging_db" },
    { role: "clusterMonitor", db: "admin"       }
  ]
})

MONGODB-X509 — user created in $external with certificate DN:

use $external
db.createUser({
  user: "CN=tdm_service,OU=TDM,O=Company,C=AU",
  roles: [
    { role: "readWrite",      db: "staging_db" },
    { role: "dbAdmin",        db: "staging_db" },
    { role: "clusterMonitor", db: "admin"       }
  ]
})

Command-Level Permissions

The following MongoDB commands are invoked directly by the TDM tool. These are covered by the roles above but are listed explicitly for auditing purposes:

Command	Called By	Required Privilege	Used By
`ismaster`	Connection validation	Any authenticated user	All
`count`	`getNumRows`, `getNumRowsPerTable`	`find` (via `readWrite`)	Profile, Forensic Scan
`dbstats`	`getDBSize`	`dbStats` (via `dbAdmin`)	Forensic Scan
`usersInfo`	`getNumUsers`	`viewUser` (via `dbAdmin`)	Forensic Scan
`serverStatus`	`getDBVersion`	`serverStatus` (via `clusterMonitor`)	All
`listCollections`	`getTables`, `checkTableExistence`	`listCollections` (via `readWrite`)	All
`aggregate`	`getColumns`, `getColumnType`, `validate_column`	`find` (via `readWrite`)	All
`find`	`getColumnValues`, `mergeTable`, `getTableData`	`find` (via `readWrite`)	All, Masking
`update` / `bulk_write`	`updateColumnFixedValue`, `updateColumnByAnotherColumn`, `mergeTable`	`update` (via `readWrite`)	Masking
`insert` / `insert_many`	`insertDataTempTable`, `save_index_def`	`insert` (via `readWrite`)	Masking
`drop` (collection)	`deleteTable`	`dropCollection` (via `dbAdmin`)	Masking
`createIndex`	`create_tmp_index`, `recover_index`	`createIndex` (via `readWrite`)	Masking
`dropIndex`	`drop_index`	`dropIndex` (via `readWrite`)	Masking
`listIndexes`	`get_indexes`, `get_index_def`	`listIndexes` (via `readWrite`)	All, Masking

MongoDB Staging Environment

Introduction​

1. Standard Database Server Configuration​

1.1 Hardware Requirements​

CPU​

Memory (RAM)​

Disk​

Network​

1.2 System-Level Database Optimizations​

Optimizing for Bulk Document Masking​

Operating System Configuration​

1.3 Connection Configuration (TDM UI)​

Required fields​

Optional connection fields​

Authentication Mechanism​

Index Management (staging requirements)​

1.4 Network Configuration​

1.5 Authentication​

SCRAM-SHA-256 (Recommended)​

SCRAM-SHA-1​

DEFAULT​

PLAIN (LDAP)​

GSSAPI (Kerberos)​

MONGODB-X509 (TLS Certificate)​

MONGODB-CR (Deprecated)​

1.6 Security and Patch Management​

2. Troubleshooting Steps for DBAs​

2.1 Verify Hardware and OS Configuration​

Check CPU and Memory​

Disk I/O Performance​

2.2 Validate MongoDB Configuration​

Check WiredTiger Cache Size​

Confirm Authentication Mechanism​

Verify TLS Configuration (for MONGODB-X509 and PLAIN)​

2.3 Assess Database Performance​

Check Server Status​

Check Collection Document Counts​

Check Database Size​

2.4 Check Session and Query Performance​

Identify Currently Running Operations​

Monitor Slow Operations​

Review Execution Plans​

2.5 Evaluate Storage and Space Usage​

Check Collection Storage Usage​

Check the e8_index_def Collection​

2.6 Review Indexes​

List Indexes on a Collection​

Rebuild Invalid Indexes​

2.7 Network Diagnostics​

Test Connectivity on Port 27017​

Test the ismaster Command​

2.8 Verify Security and Patch Levels​

Check MongoDB Version​

Check Applied Patches​

2.9 Application Interaction​

Confirm Driver Installation​

Test the TDM Service Account Connection​

2.10 System Resource Monitoring​

Monitor CPU and Memory Usage​

Check Swap Usage​

3. Database Permissions​

Required Roles​

Creating the TDM Service Account​

Command-Level Permissions​

Introduction

1. Standard Database Server Configuration

1.1 Hardware Requirements

CPU

Memory (RAM)

Disk

Network

1.2 System-Level Database Optimizations

Optimizing for Bulk Document Masking

Operating System Configuration

1.3 Connection Configuration (TDM UI)

Required fields

Optional connection fields

Authentication Mechanism

Index Management (staging requirements)

1.4 Network Configuration

1.5 Authentication

SCRAM-SHA-256 (Recommended)

SCRAM-SHA-1

DEFAULT

PLAIN (LDAP)

GSSAPI (Kerberos)

MONGODB-X509 (TLS Certificate)

MONGODB-CR (Deprecated)

1.6 Security and Patch Management

2. Troubleshooting Steps for DBAs

2.1 Verify Hardware and OS Configuration

Check CPU and Memory

Disk I/O Performance

2.2 Validate MongoDB Configuration

Check WiredTiger Cache Size

Confirm Authentication Mechanism

Verify TLS Configuration (for MONGODB-X509 and PLAIN)

2.3 Assess Database Performance

Check Server Status

Check Collection Document Counts

Check Database Size

2.4 Check Session and Query Performance

Identify Currently Running Operations

Monitor Slow Operations

Review Execution Plans

2.5 Evaluate Storage and Space Usage

Check Collection Storage Usage

Check the e8_index_def Collection

2.6 Review Indexes

List Indexes on a Collection

Rebuild Invalid Indexes

2.7 Network Diagnostics

Test Connectivity on Port 27017

Test the ismaster Command

2.8 Verify Security and Patch Levels

Check MongoDB Version

Check Applied Patches

2.9 Application Interaction

Confirm Driver Installation

Test the TDM Service Account Connection

2.10 System Resource Monitoring

Monitor CPU and Memory Usage

Check Swap Usage

3. Database Permissions

Required Roles

Creating the TDM Service Account

Command-Level Permissions