Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
vm11.local vps False False centos 9.stream x86_64 /archive/kyr-2026-03-09_23:33:15-rgw:verify-squid-none-default-vps/691
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
dead 691 2026-03-09 23:33:24 2026-03-10 00:52:40 2026-03-10 00:58:09 0:05:29 vps clyso-debian-13 centos 9.stream rgw:verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/valgrind} 2
fail 684 2026-03-09 23:33:21 2026-03-10 00:40:51 2026-03-10 00:54:07 0:13:16 0:09:56 0:03:20 vps clyso-debian-13 centos 9.stream rgw:verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/lockdep} 2
Failure Reason:

Command failed on vm09 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/kshtsk/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 569c3e99c9b32a51b4eaf08731c728f4513ed589'

fail 678 2026-03-09 23:33:19 2026-03-09 23:40:25 2026-03-10 00:39:47 0:59:22 0:22:15 0:37:07 vps clyso-debian-13 ubuntu 22.04 rgw:verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/lockdep} 2
Failure Reason:

Command failed on vm09 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/kshtsk/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 569c3e99c9b32a51b4eaf08731c728f4513ed589'

pass 672 2026-03-09 23:33:17 2026-03-09 23:33:17 2026-03-10 00:14:24 0:41:07 0:38:44 0:02:23 vps clyso-debian-13 ubuntu 22.04 rgw:verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/lockdep} 2
pass 668 2026-03-09 11:24:30 2026-03-09 21:50:14 2026-03-09 22:00:59 0:10:45 0:09:50 0:00:55 vps clyso-debian-13 ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_cephadm} 1
fail 661 2026-03-09 11:24:26 2026-03-09 21:13:00 2026-03-09 21:20:10 0:07:10 0:04:18 0:02:52 vps clyso-debian-13 centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"grep: /var/log/ceph/79ecf65a-1bfd-11f1-82dd-e5af7d3bb33e/ceph.log: No such file or directory" in cluster log

pass 648 2026-03-09 11:24:20 2026-03-09 20:33:32 2026-03-09 20:42:16 0:08:44 0:07:43 0:01:01 vps clyso-debian-13 centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli} 1
fail 644 2026-03-09 11:24:18 2026-03-09 20:24:08 2026-03-09 20:33:17 0:09:09 0:07:45 0:01:24 vps clyso-debian-13 centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2026-03-09T20:30:19.156049+0000 mon.vm02 (mon.0) 495 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.all-available-devices (CEPHADM_APPLY_SPEC_FAIL)" in cluster log

fail 639 2026-03-09 11:24:16 2026-03-09 20:09:33 2026-03-09 20:22:49 0:13:16 0:10:14 0:03:02 vps clyso-debian-13 centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2026-03-09T20:18:08.135317+0000 mon.vm10 (mon.0) 494 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.all-available-devices (CEPHADM_APPLY_SPEC_FAIL)" in cluster log

pass 634 2026-03-09 11:24:14 2026-03-09 19:57:22 2026-03-09 20:08:38 0:11:16 0:08:29 0:02:47 vps clyso-debian-13 centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 625 2026-03-09 11:24:10 2026-03-09 19:31:21 2026-03-09 19:55:58 0:24:37 0:19:11 0:05:26 vps clyso-debian-13 centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_host_drain} 3
pass 620 2026-03-09 11:24:07 2026-03-09 19:19:46 2026-03-09 19:34:03 0:14:17 0:08:44 0:05:33 vps clyso-debian-13 ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
dead 586 2026-03-09 11:23:52 2026-03-09 17:19:26 2026-03-09 19:24:51 2:05:25 vps clyso-debian-13 centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

hit max job timeout

pass 563 2026-03-09 11:23:41 2026-03-09 16:44:55 2026-03-09 17:20:19 0:35:24 0:32:56 0:02:28 vps clyso-debian-13 ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
dead 560 2026-03-09 11:23:40 2026-03-09 16:41:40 2026-03-09 16:44:43 0:03:03 0:00:47 0:02:16 vps clyso-debian-13 centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
Failure Reason:

vm11.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []vm10.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []

dead 557 2026-03-09 11:23:39 2026-03-09 16:38:21 2026-03-09 16:41:24 0:03:03 0:01:13 0:01:50 vps clyso-debian-13 centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

vm07.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []vm11.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []

dead 548 2026-03-09 11:23:34 2026-03-09 16:26:07 2026-03-09 16:29:47 0:03:40 0:00:46 0:02:54 vps clyso-debian-13 centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

vm11.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []vm02.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []vm10.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []

dead 545 2026-03-09 11:23:33 2026-03-09 16:22:55 2026-03-09 16:26:10 0:03:15 0:00:42 0:02:33 vps clyso-debian-13 centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
Failure Reason:

vm02.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []vm11.local: _ansible_no_log: false changed: false invocation: module_args: allow_downgrade: false allowerasing: false autoremove: false best: null bugfix: false cacheonly: false conf_file: null disable_excludes: null disable_gpg_check: false disable_plugin: [] disablerepo: [] download_dir: null download_only: false enable_plugin: [] enablerepo: [] exclude: [] install_repoquery: true install_weak_deps: true installroot: / list: null lock_timeout: 30 name: - ceph - ceph-base - ceph-selinux - ceph-common - ceph-debuginfo - ceph-release - libcephfs1 - ceph-radosgw - python-ceph - python-rados - python-rbd - python-cephfs - librbd1 - librados2 - mod_fastcgi nobest: null releasever: null security: false skip_broken: false sslverify: true state: absent update_cache: false update_only: false use_backend: auto validate_certs: true msg: 'Failed to download metadata for repo ''baseos'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried' rc: 1 results: []

pass 543 2026-03-09 11:23:32 2026-03-09 16:04:50 2026-03-09 16:21:46 0:16:56 0:13:21 0:03:35 vps clyso-debian-13 centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli_mon} 5
fail 540 2026-03-09 11:23:31 2026-03-09 15:56:16 2026-03-09 16:04:51 0:08:35 0:02:38 0:05:57 vps clyso-debian-13 ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

Command failed on vm07 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype"