File level restore error: Error while scanning disk

[carloum70-lab]

Are you using XOA or XO from the sources?

XO from the sources

Which release channel?

None

Provide your commit number

781e8

Describe the bug

File level restore fails with "Error while scanning disk" the moment I select a disk.

For testing I created a new virtual machine based on Debian13 and I also using LVM. (Problem only occurs when using LVM) The first time I backup the virtual machine I can do a file level restore without any problems. I am using Delta backups.
According XO the second backup is successful, but the xo-server logging says:

Apr 28 14:25:19 dacsvm-xoa xo-server[3398089]: 2026-04-28T12:25:19.265Z xo:main INFO - Console proxy (admin - ::1) Apr 28 14:25:23 dacsvm-xoa xo-server[3400547]: 2026-04-28T12:25:23.101Z xo:backups:worker INFO starting backup Apr 28 14:25:25 dacsvm-xoa xo-server[3400547]: 2026-04-28T12:25:25.943Z xo:xapi:xapi-disks INFO export through vhd Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: 2026-04-28T12:25:29.230Z xo:xapi WARN retry { Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: attemptNumber: 0, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: delay: 5000, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: error: XapiError: VDI_IN_USE(OpaqueRef:868c9405-3886-2a52-e073-35bd7730ca8a, destroy) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at XapiError.wrap (file:///opt/xo/xo-builds/xen-orchestra-202604281224/packages/xen-api/_XapiError.mjs:16:12) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at default (file:///opt/xo/xo-builds/xen-orchestra-202604281224/packages/xen-api/_getTaskResult.mjs:13:29) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202604281224/packages/xen-api/index.mjs:1078:24) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at file:///opt/xo/xo-builds/xen-orchestra-202604281224/packages/xen-api/index.mjs:1112:14 Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at Array.forEach (<anonymous>) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202604281224/packages/xen-api/index.mjs:1102:12) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202604281224/packages/xen-api/index.mjs:1275:14) Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: at process.processTicksAndRejections (node:internal/process/task_queues:104:5) { Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: code: 'VDI_IN_USE', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: params: [ 'OpaqueRef:868c9405-3886-2a52-e073-35bd7730ca8a', 'destroy' ], Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: call: undefined, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: url: undefined, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: task: task { Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: uuid: 'e7762044-ced6-5551-0926-b53d0f354e8d', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: name_label: 'Async.VDI.destroy', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: name_description: '', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: allowed_operations: [], Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: current_operations: {}, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: created: '20260428T12:25:27Z', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: finished: '20260428T12:25:27Z', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: status: 'failure', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: resident_on: 'OpaqueRef:24259a39-9c18-f096-e8ac-ed47fcffa473', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: progress: 1, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: type: '<none/>', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: result: '', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: error_info: [Array], Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: other_config: {}, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: subtask_of: 'OpaqueRef:NULL', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: subtasks: [], Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: backtrace: '(((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 5189))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/helpers.ml)(line 1706))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 5178))((process xapi)(filename ocaml/xapi/rbac.ml)(line 188))((process xapi)(filename ocaml/xapi/rbac.ml)(line 197))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 77)))' Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: } Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: }, Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: fn: 'destroy', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: arguments: [ 'OpaqueRef:868c9405-3886-2a52-e073-35bd7730ca8a' ], Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: pool: { Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: uuid: 'a93d2904-5934-5e69-5c1e-b17e93d8150f', Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: name_label: 'dacspool01' Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: } Apr 28 14:25:29 dacsvm-xoa xo-server[3400547]: } Apr 28 14:25:32 dacsvm-xoa xo-server[3400547]: 2026-04-28T12:25:32.793Z xo:xapi:vdi INFO OpaqueRef:781520b2-4a0a-2e38-07df-d89a663be61d was already destroyed { Apr 28 14:25:32 dacsvm-xoa xo-server[3400547]: vdiRef: 'OpaqueRef:868c9405-3886-2a52-e073-35bd7730ca8a', Apr 28 14:25:32 dacsvm-xoa xo-server[3400547]: vbdRef: 'OpaqueRef:781520b2-4a0a-2e38-07df-d89a663be61d' Apr 28 14:25:32 dacsvm-xoa xo-server[3400547]: } Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: 2026-04-28T12:25:38.261Z xo:backups:worker INFO backup has ended Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: 2026-04-28T12:25:38.415Z xo:backups:worker INFO process will exit { Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: duration: 15314369, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: exitCode: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: resourceUsage: { Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: userCPUTime: 495054, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: systemCPUTime: 64895, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: maxRSS: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: sharedMemorySize: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: unsharedDataSize: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: unsharedStackSize: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: minorPageFault: 28025, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: majorPageFault: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: swappedOut: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: fsRead: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: fsWrite: 45264, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: ipcSent: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: ipcReceived: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: signalsCount: 0, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: voluntaryContextSwitches: 3066, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: involuntaryContextSwitches: 217 Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: }, Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: summary: { duration: '15s', cpuUsage: '4%', memoryUsage: '0 B' } Apr 28 14:25:38 dacsvm-xoa xo-server[3400547]: }

During the second restore I see the following logging of the xo-server:

Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: 2026-04-28T12:26:44.554Z xo:api WARN admin | backupNg.listPartitions(...) [5s] =!> Error: Command failed: vgchange -an carlo-test3-vg Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: File descriptor 23 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 3398089: node Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: File descriptor 24 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 3398089: node Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: File descriptor 26 (/var/lib/xo-server/data/leveldb/MANIFEST-000412) leaked on vgchange invocation. Parent PID 3398089: node Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: File descriptor 31 (/dev/fuse) leaked on vgchange invocation. Parent PID 3398089: node Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: File descriptor 47 (/dev/fuse) leaked on vgchange invocation. Parent PID 3398089: node Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: File descriptor 49 (/var/lib/xo-server/data/leveldb/000416.log) leaked on vgchange invocation. Parent PID 3398089: node Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: WARNING: Not using device /dev/loop2 for PV H4a1eG-AH95-SnSs-1eUk-9Peb-pTEY-bwkaTn. Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: WARNING: PV H4a1eG-AH95-SnSs-1eUk-9Peb-pTEY-bwkaTn prefers device /dev/loop0 because device is used by LV. Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: Logical volume carlo-test3-vg/root in use. Apr 28 14:26:44 dacsvm-xoa xo-server[3398089]: Can't deactivate volume group "carlo-test3-vg" with 1 open logical volume(s)

Error message

Error while scanning disk

To reproduce

1. Go to 'Backup'
2. Click on 'File level restore'
3. Click on restore button of the VM
4. Select yoour backup
5. Select your desk
6. error occurs

Expected behavior

After selecting the disk, you can choose the filesystem and the files/directories you want to restore.

Screenshots

Node

v24.14.1

Hypervisor

XCP-NG 8.3

Additional context

See also https://xcp-ng.org/forum/topic/12085/error-while-scanning-disk/

[fbeauchamp]

can you try the file level restore with this branch ? #9776

[carloum70-lab]

can you try the file level restore with this branch ? #9776

How can I do this ? Should I change the BRANCH variable to "pull/9776/head" in xo-install.sh ?