4 posts tagged with "vme"

1.10.0 Major Release Items​

What's New​

Choose What Each Onboarding Workflow Does​

New Onboarding shows a workflow as the pipeline it runs: Ingest, Snapshot, Send, Clone and Mount. Tick only the parts you need. A workflow that skips Ingest starts from a source already on the appliance, and one that skips Snapshot starts from an existing snapshot, so refreshing a test copy from last night's snapshot is a Clone + Mount workflow of a few steps. Combinations that cannot run are refused before the workflow is created.

See Onboard Application for the full workflow.

One Volume per Database, Mounted as a Folder​

Each SQL Server database is onboarded onto a single volume, and every volume mounts as a folder under the host's vME folder: <vME folder>\sources\<id> for a source and <vME folder>\clones\<id> for a clone. A host can hold as many volumes as it needs, and a folder path tells you at a glance which dataset a volume belongs to. You choose the vME folder when you install the Windows connector, and you can change it later in the connector's tray settings.

Snapshots and Rollbacks Coordinated with the Host​

vME stops and starts the host's databases for you around a snapshot or a rollback, so a volume a SQL Server host has mounted is never changed under that host. When you take a snapshot of a mounted source, or Create Bookmark or Rollback To Bookmark on a mounted clone, the host's agent:

1. takes the databases on that volume offline and takes its disk offline,
2. waits while vME takes the snapshot or rolls back,
3. brings the disk back at the same folder and sets the databases online again.

Each step is shown as the host reports it.

The onboarding wizard's Snapshot step does the same. Tick Don't stop/start the host databases only when you have already taken them offline yourself. If no agent can reach the host, vME refuses and lists the steps to do it by hand; once the databases are offline, Proceed without the host finishes the job.

See iSCSI Connector for details.

See Where Every Dataset Is Attached​

The Sources, Snapshots and Clones lists show what is attached where and who is using it:

• Attached to names the host that has the volume mounted, whether it is connected, and the folder it is mounted at.
• Used by links to the unfinished workflows that depend on a dataset.
• Sources and clones show when they were created.
• Tick several rows and Delete selected. vME deletes clones first, then snapshots, then sources, and explains in plain words anything it will not delete, for example a source that still has snapshots or one an unfinished workflow is using.

Rename iSCSI Sources and Clones​

Sources and clones exported over iSCSI can be renamed from the web console, the API or the CLI. The export keeps its target name, so a host that has the volume mounted stays connected while you rename it. A dataset an unfinished workflow is using is not renamed, and the message names the workflow.

Follow Every Workflow from One List​

The Recent Workflows list shows each workflow's Technology, with filters for state, date, snapshot, host and source, and grouping. Open a workflow's Stages to see each step and how long it took. Deleting a workflow warns you when it mounted a clone on a host and removes only what that workflow created. A cancelled workflow opens read-only, so you can still see what it did.

Agent Manager: Host Details, Editing and Host Maintenance​

The Agent Manager lists your agents and shows each host's details as its agent reports them: Windows version, CPU and memory, domain, IP addresses, iSCSI initiator name, vME folder, SQL Server instances and versions, and the databases on the host. Edit Agent sets an agent's label, environment and tags.

Host maintenance shows every iSCSI target and portal a host still holds, marked in use, idle (the dataset is on this appliance but the host is not using it) or orphaned (no dataset on this appliance answers for it any more). Select the ones you no longer need and Forget selected: the host removes those entries and touches nothing else.

See Agent Manager for more.

Register Your vME Appliances Once​

The vME Appliances tab of the Agent Manager keeps the appliances this vME sends data to. Add an appliance, Test the connection and Establish trust once. The Send step of a workflow then offers the registered appliances by name, with their pools.

Set Up a New Appliance from Its Console​

A new vME appliance opens a setup menu on its console the first time it starts: configure IPv4 networking (DHCP or a static address with gateway and DNS), set the hostname and the NTP server, and test connectivity. When you finish, it shows a summary with the address to continue at, and the console keeps showing that address. The browser Setup Wizard then shows the network settings already in effect, with Edit these settings if you want to change them.

See the Setup Wizard (First Boot) Guide.

Improvements​

• iSCSI exports need no client address. Creating an iSCSI source or clone asks only for its size; the export's settings show its target name. NFS exports keep their client address and user mapping.
• Safer replication. A send with Overwrite if it already exists is refused when the destination has clones, and the refusal names them. Nothing is destroyed.
• The connector keeps to its own disks. Source preparation and clone mount bring online and format only the disks behind the job's own iSCSI targets, wait for a slow disk to appear, and stop with the targets they expected and the disks they found when nothing matches. Windows automount is restored whenever a clone mount ends.
• Staging databases detach cleanly. Other connections to a staging database are closed before it is detached or restored over, so an open session does not stop an ingest or a teardown.

API Changes & Upgrading​

Update the Windows connector (MSI) to 1.4.0 on your SQL Server hosts. Folder mounts, host-coordinated snapshots and rollbacks, host details and Host maintenance all need it. The installer asks for the vME folder (VME_ROOT for a silent install). vME checks what each connector reports it can do, so a host still on an older connector keeps working for what it supports, and a feature it lacks is refused with a message rather than attempted.

Workflows created before the upgrade keep their drive letters. They are shown under Advanced on the mount step.

Changes to check in a custom REST integration:

• POST /snapshot/create and POST /snapshot/rollback on a mounted iSCSI dataset can answer 202 Accepted with a task_id while the host is coordinated. Poll GET /snapshot/task/<task_id>. A refusal answers 409 with the manual steps; pass "skip_host": true only once you have taken the databases offline yourself.
• iSCSI network_info takes no client_address, anonuid or anongid. NFS keeps all three.
• Workflows take parts, start_from and config_mode, and return parts and live_steps. Each SQL Server database uses one volume.
• POST /source/rename and POST /clone/rename accept iSCSI datasets. A dataset an unfinished workflow is using answers 409 with used_by.
• A request without a username or api-key answers with a JSON error rather than a server error page.

New endpoints:

• The appliance registry: /peer and /peer/<id> with test, trust, pools and agents.
• DELETE /object/destroy deletes up to 100 sources, snapshots and clones in one call and reports each one.
• PATCH /agent with "action": "update" sets an agent's label, environment and tags. Agent listings return each agent's tags, capabilities and host details.
• Source and clone listings return guid, attached_to and used_by.

The vME API reference documents every route, and the vME Postman collection includes an Onboarding (API) folder that runs the one-volume, folder-mount flow.

CLI: vme snapshot create and vme snapshot rollback take --skip_host and --wait; without --wait they print the task id. New vme peer and vme object destroy commands match the API. See the CLI guide.

1.9.0 Major Release Items​

What's New​

First-Time Setup Wizard​

New appliances now boot into a guided Setup Wizard that walks you through every first-run step in the browser: hostname, network (DHCP or static IP with DNS), timezone, TLS certificate, licence activation, admin password, authentication method, and the initial storage pool. There is no more manual bootstrapping over SSH.

See the Setup Wizard guide for the full step-by-step walkthrough, including DHCP vs static IP networking and installing your own TLS certificate.

End-to-End iSCSI Agent Orchestration​

vME can now drive the SQL Server iSCSI workflow remotely and automatically. A lightweight Windows agent (installed from an MSI, run as a background service) enrols each SQL Server host with vME and then picks up and runs onboarding, migration, and clone mount/unmount jobs on demand - with no one logging in to run PowerShell.

Agents are managed centrally from the new Agent Manager in the web console: create one-time enrollment tokens, watch each agent's live status, and rotate keys - all over an outbound-only connection (port 443), so no inbound ports need to be opened on the SQL host.

See the iSCSI Connector - Automated (Windows Agent) guide for setup and usage.

Application Onboarding Wizard (Beta)​

A new Onboard Application wizard (Beta) guides you through cloning a production database onto another vME appliance end to end - pairing the two appliances, discovering the source host, and driving the iSCSI agent through prep, snapshot, clone, and mount. This is an early Beta; we'd love your feedback.

See Onboard Application (Beta) for the full workflow.

Container Connection Details​

vME now surfaces each database container's connection details - host, port mappings, mount paths, database type, and environment variables (such as POSTGRES_USER / POSTGRES_PASSWORD) - in the web UI, so you can connect to a running database without hunting for its settings. The container list API (GET /docker/container/list) returns the port mappings, mounts, database type, and environment variables as well.

See Getting container connection info for details.

Consistent Snapshots​

When you snapshot a source or clone that has running database containers, vME now stops those containers first, takes the snapshot, and restarts them - so the captured data is consistent rather than mid-write. The UI shows a container-aware confirmation listing exactly what will be stopped.

See Snapshots for more info.

Rename Sources and Clones in the UI​

You can now rename sources and clones directly from the web console, instead of destroying and recreating them. (Renaming is not supported for iSCSI sources and clones.)

Unix CLI Install​

The vME CLI can now be installed on Unix, in addition to Windows, so you can run it from the command line on either platform. The CLI is also up to date with the 1.9.0 API. See the CLI guide to download and install.

Improvements​

• Clearer LDAP errors - a failed LDAP login now shows the actual error returned by the directory, instead of a generic message.
• Licence errors surfaced - operations blocked by an unlicensed or expired appliance now show a clear licence error.
• Longer sessions - the web session timeout has been extended to 24 hours.

Fixed Issues​

iSCSI connector

• Snapshots now flush the LUN write cache before the snapshot is taken, so clones are no longer occasionally created RAW/empty.
• Skip-backup no longer crashes when pointed at a single backup file.

Platform

• Login now signs in correctly on the button press after a session timeout.
• Fixed a UI refresh glitch when switching between menu items.
• Fixed cross-appliance API calls failing due to certificate handling.
• Resolved intermittent stale database-connection errors.

API Changes & Upgrading​

The 1.9.0 REST API is documented in the vME API reference. A few endpoints changed shape in this release - if you maintain a custom REST integration, review these before upgrading. (The vME web UI and CLI ship updated with the release and need no changes.)

Grab the updated vME Postman collection to get all the new and changed endpoints ready to run - it also includes an Onboarding (API) folder that replicates the full Onboard Application flow as ordered, chained requests.

Breaking changes

• POST /user/modify now requires your current_password, and a password change is sent as a structured object:

  - { "username": "jane", "password": "NewPass456!" }
  + {
  +   "username": "jane",
  +   "current_password": "OldPass123!",
  +   "password": { "current": "OldPass123!", "new": "NewPass456!", "confirm": "NewPass456!" }
  + }
  
• DELETE /user/delete now requires your password and a typed confirmation:

  - { "username": "ubuntu123" }
  + { "username": "ubuntu123", "password": "MyPassword123!", "confirm_text": "DELETE ubuntu123" }
  
• POST /db/receive is now asynchronous - it returns 202 Accepted with a task_id. Poll GET /tasks/<task_id> for completion instead of expecting a synchronous result.

New and improved

• New endpoints: POST /db/send/cancel, POST /user/changepass, POST /db/network/create, DELETE /db/network/delete, and GET /tasks/<task_id>.
• iSCSI sources & clones over the API: POST /source/create and POST /clone/create accept a network_info block (type: iscsi, with auto_grow / vsize), and the iSCSI target is provisioned automatically as part of creation.
• System-to-system db send no longer needs an SSH/PEM key - vME establishes trust with the receiver automatically using its admin api-key.

1.9.1​

Version 1.9.1 is a focused update to the Onboard Application workflow and the iSCSI connector, based on feedback from teams using them day to day.

What's New​

Choose Your Own Database Name Prefix​

Databases ingested onto iSCSI storage were always named with a fixed ISCSI_ prefix. The ingest step now has a Target database prefix field (under Advanced options) so you can pick a naming scheme that suits your environment. The default remains ISCSI_, already-ingested databases are detected using your chosen prefix, and teardown detaches using the same prefix the workflow ingested with.

Tear Down the Source Host​

Once an application has been onboarded and cloned to the target appliance, you can clean up the source SQL host with one click. Tear down source detaches the staging databases, disconnects the workflow's iSCSI sessions, and frees the drive letters on the source host - without touching the source dataset on the appliance, so you can keep cloning from it. Only the workflow's own sessions and databases are affected.

Recognisable Snapshot Names​

Snapshots created in the wizard now default to a name derived from your source name (for example SalesDB-20260729103000) instead of the generic prepared-<timestamp>. The name stays fully editable, and invalid characters are rejected with a clear message.

Automatic Drive Letter Assignment​

The ingest and clone-mount steps now pick free drive letters automatically instead of defaulting to fixed letters that may already be in use. The pickers prefer the host's recommended letters when free, and never assign the same letter to data and log. Selecting an in-use letter manually still asks for confirmation before reformatting.

Improvements​

• P: and S: are now assignable as data/log drive letters across the wizard and the Windows connector.
• Teardown is kinder to shared hosts - a full iSCSI cleanup no longer restarts the Windows iSCSI service, so other iSCSI workloads on the same host are not interrupted, and mount-point cleanup only touches the drive letters the workflow owns.

Fixed Issues​

Onboard Application​

• Workflow steps completed directly via the REST API (for example with the Postman collection) are now recorded on the workflow and shown on the wizard's stepper, which refreshes live - previously the stepper only updated for actions taken inside the wizard itself.

iSCSI Connector​

• Tearing down a clone could leave the iSCSI session reconnecting automatically: persistent target registrations are now parsed and removed correctly, so a torn-down target stays disconnected.
• Sessions are now disconnected reliably even when the target is no longer in the discovered-targets list.
• Disks are taken offline before their session is disconnected, so volumes dismount cleanly during teardown.
• The teardown's database-detach step can be skipped cleanly when there is nothing to detach, instead of stalling the run.

API Changes & Upgrading​

Update the Windows connector to use the new features. The custom database prefix and the teardown fixes live in the Windows iSCSI connector - update the connector (MSI) on your SQL Server hosts as part of this upgrade. Until a host's connector is updated, ingests on that host keep the default ISCSI_ prefix regardless of what is set in the wizard.

Behaviour change for direct connector use: in the teardown script, the disconnect selection now takes Windows disk numbers (as shown in the script's disk listing) instead of session list positions. ALL and MATCH selections are unchanged, and wizard-driven teardowns are unaffected.

API additions (all optional, backwards compatible):

• The ingest and teardown jobs accept a dbPrefix value (migrate.dbPrefix / unmount.dbPrefix).
• The snapshot, clone, source, and send APIs accept optional workflow tracking fields (workflow_id, wizard_step, role) so API-driven onboarding steps appear on the wizard's stepper.
• The vME Postman collection's Onboarding (API) folder has been updated to use these fields.

1.9.2​

Version 1.9.2 continues the Onboard Application and iSCSI connector work.

What's New​

Choose Which Databases To Restore From Your Backups​

The ingest step can now probe backup paths: every database found is listed with its backup set (stripes, differential, backup time) and you tick the ones to ingest. Sets that cannot be restored are marked with the reason, and a name that already exists on the host is flagged as a replacement. A probe reads backup headers only, so it can run before the iSCSI drives are prepared.

Ingest From Full Plus Differential Backups​

A striped full plus its newest differential now restores as one backup set, and stripes may span several backup paths. An incomplete set stops the run naming the files it found.

Choose the Source SQL Instance​

An optional SQL instance field on the ingest step pins the source instance on hosts running more than one. Teardown targets the instance the ingest used.

Database State in Discovery​

Host discovery reports each database's state, and a database that cannot be read cannot be selected as a source.

Improvements​

• Ingest jobs are no longer limited to one hour, and restore progress appears in the job log as it runs.
• Drive letter suggestions are simply the first two free letters.
• Backup paths are entered one per line; a trailing comma or semicolon is ignored.
• A step whose agent stops reporting shows in amber as stopped reporting instead of running forever. An agent that comes back still reports the real outcome.

Fixed Issues​

Onboard Application​

• Deleting a workflow with cleanup now detaches the staging databases on the source SQL host before destroying the source dataset, instead of leaving them unreadable on the instance.
• A delete that reports itself as still in progress can simply be run again and completes the cleanup.
• A clone attached with a custom database prefix no longer gains an extra ISCSI_ on top; clones attach under the name the ingest created.
• The setup wizard rejects invalid hostnames, and the NTP field accepts IPv4-mapped IPv6 addresses.

iSCSI Connector​

• A workflow could prepare, format or reclaim drive letters from another workflow's disks when one source name was a prefix of another. Workflows now match their own disks exactly and stop rather than touch a disk they cannot confirm as their own.
• Clones no longer attach with a doubled name such as ISCSI_Sales_Sales. Existing doubled names can be renamed with ALTER DATABASE ... MODIFY NAME, a metadata-only change.
• The attach honours the configured database prefix instead of always applying ISCSI_.

API Changes & Upgrading​

Update the Windows connector (MSI) on your SQL Server hosts. Everything above except the wizard changes lives in the connector. In particular, ingesting from existing backup files requires the updated connector: on an older one the probe reports a configuration error and the ingest cannot start. Other ingest modes keep their previous behaviour until a host is updated.

If you used a custom database prefix in 1.9.1: clones mounted before this upgrade may carry a doubled name the teardown will not match. Detach them on the host or rename them with ALTER DATABASE ... MODIFY NAME first.

API additions (all optional, backwards compatible):

• migrate.probeOnly reports the backup sets in migrate.backupPaths (as backupSets in the job result) without restoring; drive letters are not required. Pass the reported files back as backupPaths to restore a subset.
• migrate.sqlInstance pins the source SQL instance.
• attach.dbPrefix applies the wizard's prefix to an API-driven attach.
• job_timeout_seconds overrides a job's default timeout (24 hours for an ingest, one hour otherwise).
• Host discovery includes each database's state.
• Agent jobs can report a non-terminal timed_out state when the host's agent stops reporting; a returning agent can still move the job to completed or failed.
• The vME Postman collection carries a version so it can be matched to a release.

Behaviour note for direct connector use: the clone attach derives each database's name from its primary data file rather than the file name.

1.9.3​

Version 1.9.3 adds wrong-instance protection for ingest and rounds out driving the Onboard Application through the API.

What's New​

Wrong-Instance Protection for Ingest​

On hosts running more than one SQL instance, database ids repeat across instances. The wizard now records which instance your databases were selected on and blocks a selection that spans instances, and the connector stops an ingest before anything is backed up or restored when the job resolves to a different instance than the selection was made on. Runs without an explicit database selection are unaffected.

Improvements​

• Portal addresses on the discovery, ingest, and mount steps are validated when you submit: a mistyped address is rejected immediately with a message instead of queuing a job that fails later on the host.
• A failed agent job now fails its workflow: the failed step is named on the final page, and Mark complete asks for explicit confirmation while a step has failed.

Fixed Issues​

Onboard Application​

• Workflow steps completed through the API (for example with the vME Postman collection) now record the same details the wizard records, so the wizard renders API-driven workflows correctly.
• Resume source provisioning and Re-apply iSCSI settings now succeed on a partially or fully provisioned source.
• Two sends started at the same time no longer conflict during first-time transfer setup.

API Changes & Upgrading​

API additions and behaviour (backwards compatible):

• /db/network/create on a dataset that already has its network settings returns success with "Network already configured" instead of an error.
• migrate.selectionInstance on an ingest job names the instance a database selection was made against; the job aborts before restoring when it resolves to a different instance. Omit it to keep the previous behaviour.
• receiver.ssh_address on /db/send sets the data-transfer host when it differs from the target's API address (for example a private address between appliances).

Update the Windows connector (MSI) to 1.3.3 on your SQL Server hosts to get the wrong-instance protection; other changes live on the appliance.

Release Date: 6th March 2026

Key Features​

• User & Role Management: Added full administrative user management capabilities within vME. Administrators can now add, modify and delete users directly from the platform.
• LDAP Integration: vME now supports integration with Active Directory via LDAP, allowing centralised authentication and improved enterprise security alignment.
• ZFS Pool Disk Discovery: When creating a ZFS storage pool, vME now automatically retrieves and displays available disks and partition information, simplifying storage configuration and reducing manual effort.
• iSCSI Volume Autogrow: iSCSI storage volumes now automatically grow as needed. Manual size configuration has been removed, providing a simpler and more reliable provisioning experience.
• CHAP Authentication for MSSQL iSCSI: Added CHAP authentication support when mounting MSSQL databases over iSCSI for enhanced security.

Enhancements​

• Operating System Disk Protection: System disks are now clearly identified and protected from accidental pool creation, reducing risk during storage configuration.
• Improved Logging: Logging has been enhanced to provide clearer, more structured messages to assist with troubleshooting and operational visibility.
• CLI Alignment: The CLI has been updated to align with the latest API capabilities, ensuring consistent behaviour across interfaces.
• Container Image Pull Refresh: Added automatic refresh during container image pulls to provide better visibility into progress.
• Dark Mode Improvements: Improved text consistency and readability in Dark Mode.
• Frontend Stability Improvements: Addressed UI refresh behaviour and improved overall frontend responsiveness.
• Documentation Updates: Updated API, CLI and frontend documentation to reflect new capabilities and screenshots.

Bug Fixes​

• Disk Validation: Resolved an issue where invalid physical device paths could be passed during disk operations.
• Pool Disk Display: Fixed an issue where newly created iSCSI disks were incorrectly appearing as available disks for pool creation.
• Security Library Update: Addressed frontend security warnings related to third party libraries.

Release Date: 16th January 2025

Key Features​

• Network File Share: NFS support added for sources and clones. This enables vME to be used with a wide range of database technologies and hosting methods. This is an alternative to container based database images on the vME appliance itself.
• Virtual Mainframe: IBM ZD&T support added for Virtual z/OS mainframes to leverage vME's agility and space saving benefits to provision tiny mainframes for Development, Test and Training.
• File Explorer: Added a new frontend file explorer to view and manage files within a source or clone.

Enhancements​

• Form Redesign: Redesigned the forms to be more user friendly and consistent with the rest of the platform.
• Operating System: Upgraded the base Operating System to Debian 12 / Ubuntu 22.04 LTS.
• Log Messages: Improved Logging and Messaging.

Bug Fixes​

• Image Pull: Fixed a rare scenario where pulling a docker image could get stuck “In Progress” status.
• Clone Destroy: Clone Destroy now checks if a child bookmark exists and will not destroy the clone unless recursive flag is set.