1scancel(1) Slurm Commands scancel(1)
2
6 scancel - Used to signal jobs or job steps that are under the control
7 of Slurm.
8
11 scancel [OPTIONS...] [job_id[_array_id][.step_id]] [job_id[_ar‐
12 ray_id][.step_id]...]
13
16 scancel is used to signal or cancel jobs, job arrays or job steps. An
17 arbitrary number of jobs or job steps may be signaled using job speci‐
18 fication filters or a space separated list of specific job and/or job
19 step IDs. If the job ID of a job array is specified with an array ID
20 value then only that job array element will be cancelled. If the job
21 ID of a job array is specified without an array ID value then all job
22 array elements will be cancelled. While a heterogeneous job is in a
23 PENDING state, only the entire job can be cancelled rather than its in‐
24 dividual components. A request to cancel an individual component of a
25 heterogeneous job while in a PENDING state will return an error. After
26 the job has begun execution, an individual component can be cancelled.
27 A job or job step can only be signaled by the owner of that job or user
28 root. If an attempt is made by an unauthorized user to signal a job or
29 job step, an error message will be printed and the job will not be sig‐
30 naled.
31
34 -A, --account=account
35 Restrict the scancel operation to jobs under this charge ac‐
36 count.
37 -b, --batch
39 By default, signals other than SIGKILL are not sent to the batch
40 step (the shell script). With this option scancel signals only
41 the batch step, but not any other steps. This is useful when
42 the shell script has to trap the signal and take some applica‐
43 tion defined action. Most shells cannot handle signals while a
44 command is running (i.e. is a child process of the batch step),
45 so the shell needs to wait until the command ends to then handle
46 the signal. Children of the batch step are not signaled with
47 this option. If this is desired, use -f, --full instead. NOTE:
48 If used with -f, --full, this option ignored. NOTE: This option
49 is not applicable if step_id is specified. NOTE: The shell it‐
50 self may exit upon receipt of many signals. You may avoid this
51 by explicitly trap signals within the shell script (e.g. "trap
52 <arg> <signals>"). See the shell documentation for details.
53 -M, --clusters=<string>
55 Cluster to issue commands to. Implies --ctld. Note that the
56 SlurmDBD must be up for this option to work properly.
57 --ctld Send the job signal request to the slurmctld daemon rather than
59 directly to the slurmd daemons. This increases overhead, but of‐
60 fers better fault tolerance. This is the default behavior on
61 architectures using front end nodes (e.g. Cray ALPS computers)
62 or when the --clusters option is used.
63 -f, --full
65 By default, signals other than SIGKILL are not sent to the batch
66 step (the shell script). With this option scancel also signals
67 the batch script and its children processes. Most shells cannot
68 handle signals while a command is running (i.e. is a child
69 process of the batch step), so the shell needs to wait until the
70 command ends to then handle the signal. Unlike -b, --batch,
71 children of the batch step are also signaled with this option.
72 NOTE: srun steps are also children of the batch step, so steps
73 are also signaled with this option.
74 --help Print a help message describing all scancel options.
76 -H, --hurry
78 Do not stage out any burst buffer data.
79 -i, --interactive
81 Interactive mode. Confirm each job_id.step_id before performing
82 the cancel operation.
83 -n, --jobname=job_name, --name=job_name
85 Restrict the scancel operation to jobs with this job name.
86 --me Restrict the scancel operation to jobs owned by the current
88 user.
89 -w, --nodelist=host1,host2,...
92 Cancel any jobs using any of the given hosts. The list may be
93 specified as a comma-separated list of hosts, a range of hosts
94 (host[1-5,7,...] for example), or a filename. The host list will
95 be assumed to be a filename only if it contains a "/" character.
96 -p, --partition=partition_name
98 Restrict the scancel operation to jobs in this partition.
99 -q, --qos=qos
101 Restrict the scancel operation to jobs with this quality of ser‐
102 vice.
103 -Q, --quiet
105 Do not report an error if the specified job is already com‐
106 pleted. This option is incompatible with the --verbose option.
107 -R, --reservation=reservation_name
109 Restrict the scancel operation to jobs with this reservation
110 name.
111 --sibling=cluster_name
113 Remove an active sibling job from a federated job.
114 -s, --signal=signal_name
116 The name or number of the signal to send. If this option is not
117 used the specified job or step will be terminated. Note. If this
118 option is used the signal is sent directly to the slurmd where
119 the job is running bypassing the slurmctld thus the job state
120 will not change even if the signal is delivered to it. Use the
121 scontrol command if you want the job state change be known to
122 slurmctld.
123 -t, --state=job_state_name
125 Restrict the scancel operation to jobs in this state.
126 job_state_name may have a value of either "PENDING", "RUNNING"
127 or "SUSPENDED".
128 --usage
130 Print a brief help message listing the scancel options.
131 -u, --user=user_name
133 Restrict the scancel operation to jobs owned by the given user.
134 -v, --verbose
136 Print additional logging. Multiple v's increase logging detail.
137 This option is incompatible with the --quiet option.
138 -V, --version
140 Print the version number of the scancel command.
141 --wckey=wckey
143 Restrict the scancel operation to jobs using this workload char‐
144 acterization key.
145
147 job_id The Slurm job ID to be signaled.
148 step_id
150 The step ID of the job step to be signaled. If not specified,
151 the operation is performed at the level of a job.
152 If neither --batch nor --signal are used, the entire job will be
154 terminated.
155 When --batch is used, the batch shell processes will be sig‐
157 naled. The child processes of the shell will not be signaled by
158 Slurm, but the shell may forward the signal.
159 When --batch is not used but --signal is used, then all job
161 steps will be signaled, but the batch script itself will not be
162 signaled.
163
165 Executing scancel sends a remote procedure call to slurmctld. If enough
166 calls from scancel or other Slurm client commands that send remote pro‐
167 cedure calls to the slurmctld daemon come in at once, it can result in
168 a degradation of performance of the slurmctld daemon, possibly result‐
169 ing in a denial of service.
170 Do not run scancel or other Slurm client commands that send remote pro‐
172 cedure calls to slurmctld from loops in shell scripts or other pro‐
173 grams. Ensure that programs limit calls to scancel to the minimum nec‐
174 essary for the information you are trying to gather.
175
178 Some scancel options may be set via environment variables. These envi‐
179 ronment variables, along with their corresponding options, are listed
180 below. (Note: Command line options will always override these set‐
181 tings.)
182 SCANCEL_ACCOUNT -A, --account=account
185 SCANCEL_BATCH -b, --batch
187 SCANCEL_CTLD --ctld
189 SCANCEL_FULL -f, --full
191 SCANCEL_HURRY -H, --hurry
193 SCANCEL_INTERACTIVE -i, --interactive
195 SCANCEL_NAME -n, --name=job_name
197 SCANCEL_PARTITION -p, --partition=partition_name
199 SCANCEL_QOS -q, --qos=qos
201 SCANCEL_STATE -t, --state=job_state_name
203 SCANCEL_USER -u, --user=user_name
205 SCANCEL_VERBOSE -v, --verbose
207 SCANCEL_WCKEY --wckey=wckey
209 SLURM_CONF The location of the Slurm configuration file.
211 SLURM_CLUSTERS -M, --clusters
213 SLURM_DEBUG_FLAGS Specify debug flags for scancel to use. See De‐
215 bugFlags in the slurm.conf(5) man page for a full
216 list of flags. The environment variable takes
217 precedence over the setting in the slurm.conf.
218
220 If multiple filters are supplied (e.g. --partition and --name) only the
221 jobs satisfying all of the filtering options will be signaled.
222 Cancelling a job step will not result in the job being terminated. The
224 job must be cancelled to release a resource allocation.
225 To cancel a job, invoke scancel without --signal option. This will
227 send first a SIGCONT to all steps to eventually wake them up followed
228 by a SIGTERM, then wait the KillWait duration defined in the slurm.conf
229 file and finally if they have not terminated send a SIGKILL. This
230 gives time for the running job/step(s) to clean up.
231 If a signal value of "KILL" is sent to an entire job, this will cancel
233 the active job steps but not cancel the job itself.
234 On Cray systems, all signals except SIGCHLD, SIGCONT, SIGSTOP, SIGTSTP,
236 SIGTTIN, SIGTTOU, SIGURG, or SIGWINCH cause the ALPS reservation to be
237 released. The job however will not be terminated except in the case of
238 SIGKILL and may then be used for post processing.
239
242 When using SlurmDBD, users who have an AdminLevel defined (Operator or
243 Admin) and users who are account coordinators are given the authority
244 to invoke scancel on other users jobs.
245
248 Send SIGTERM to steps 1 and 3 of job 1234:
249 $ scancel --signal=TERM 1234.1 1234.3
251 Cancel job 1234 along with all of its steps:
254 $ scancel 1234
256 Send SIGKILL to all steps of job 1235, but do not cancel the job it‐
259 self:
260 $ scancel --signal=KILL 1235
262 Send SIGUSR1 to the batch shell processes of job 1236:
265 $ scancel --signal=USR1 --batch 1236
267 Cancel all pending jobs belonging to user "bob" in partition "debug":
270 $ scancel --state=PENDING --user=bob --partition=debug
272 Cancel only array ID 4 of job array 1237
275 $ scancel 1237_4
277
280 Copyright (C) 2002-2007 The Regents of the University of California.
281 Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
282 Copyright (C) 2008-2011 Lawrence Livermore National Security.
283 Copyright (C) 2010-2022 SchedMD LLC.
284 This file is part of Slurm, a resource management program. For de‐
286 tails, see <https://slurm.schedmd.com/>.
287 Slurm is free software; you can redistribute it and/or modify it under
289 the terms of the GNU General Public License as published by the Free
290 Software Foundation; either version 2 of the License, or (at your op‐
291 tion) any later version.
292 Slurm is distributed in the hope that it will be useful, but WITHOUT
294 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
295 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
296 for more details.
297
300 slurm_kill_job (3), slurm_kill_job_step (3)
301August 2022 Slurm Commands scancel(1)