Fix RuntimeError: CUDA Error: Invalid Device Ordinal

Quick answer: CUDA invalid device ordinal means the process selected a logical GPU index that is not visible or does not exist. Check torch.cuda.device_count(), inspect CUDA_VISIBLE_DEVICES, remember that visible devices are renumbered from zero, and only then choose a torch.device.

Python Pool infographic troubleshooting CUDA invalid device ordinal with visible GPUs, device count, and zero-based IDs — A CUDA device ordinal is an index in the GPUs visible to the process; inspect visibility and use a valid zero-based index before changing model code.

The RuntimeError: CUDA error: invalid device ordinal message means your Python process asked CUDA for a GPU number that is not visible to that process. The most common case is simple: the code requests cuda:1 or cuda:2, but only one GPU is available, so the only valid CUDA ordinal is 0.

This is not a Python syntax problem and it is rarely fixed by reinstalling your whole environment. Treat it like a device selection bug. First count the GPUs PyTorch can see, then choose an ordinal inside that range, and finally check whether CUDA_VISIBLE_DEVICES, a notebook runtime, Docker, or a distributed launcher is hiding or remapping devices. The same zero-based indexing idea appears in regular Python sequences; if you need a refresher, see this guide to Python list index out of range.

Contents

Why the Error Happens

CUDA device ordinals are zero-based. If PyTorch reports two visible GPUs, valid device strings are cuda:0 and cuda:1. Requesting cuda:2 will fail because the third visible GPU does not exist. The word visible matters: a machine can have several physical GPUs while your Python process only sees one of them because the runtime or environment variable filters the list.

import torch

print("CUDA available:", torch.cuda.is_available())
print("Visible GPU count:", torch.cuda.device_count())

for index in range(torch.cuda.device_count()):
    print(index, torch.cuda.get_device_name(index))

Run that check in the same shell, notebook, container, or job where the error happens. Checking GPU count in a different terminal can give a misleading result if the failing job has a different environment.

Validate the Requested GPU ID

If your code accepts a command-line argument such as --gpu 1, validate it before creating the device. This produces a clearer error than letting the training loop fail after the model has already started loading. It also makes configuration mistakes easier to catch in CI, notebooks, and scheduled jobs.

import torch

requested_device = 1
gpu_count = torch.cuda.device_count()

if requested_device >= gpu_count:
    raise ValueError(f"GPU {requested_device} is not available; only {gpu_count} visible")

device = torch.device(f"cuda:{requested_device}")
print(device)

If the count is zero, do not request any CUDA ordinal. Either install and expose a supported GPU runtime or fall back to CPU mode until CUDA is available.

Python Pool infographic showing host, visible GPU list, device index, CUDA request, and invalid ordinal error — The requested device index must exist in the GPUs visible to the current process.

Use a Safe Default Device

For scripts that can run on laptops, CI runners, cloud notebooks, and GPU servers, choose the device at startup and reuse it everywhere. Avoid scattering hard-coded cuda:1 strings through the codebase because each one becomes another place that can drift from the actual machine.

import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Running on {device}")

This pattern selects the default CUDA device when one is available and CPU otherwise. It is also easier to test because the rest of the code only receives a single device object.

Check CUDA_VISIBLE_DEVICES

CUDA_VISIBLE_DEVICES can hide GPUs and remap ordinal numbers. For example, if the environment exposes only physical GPU 3, PyTorch may still call it cuda:0 inside the process. That is expected behavior. After filtering, always use the visible ordinal, not the physical GPU number printed by a cluster dashboard.

import os

os.environ["CUDA_VISIBLE_DEVICES"] = "0"

import torch

print(torch.cuda.device_count())
print(torch.device("cuda:0"))

Set this variable before importing PyTorch in the process. In Docker, cloud notebooks, and schedulers such as Slurm, the platform may set it for you. When debugging, print it near startup so you know whether the runtime is changing your visible device list. PyTorch’s CUDA semantics documentation explains this behavior in more detail.

Python Pool infographic mapping CUDA_VISIBLE_DEVICES, remapped indices, application request, and selected GPU — CUDA_VISIBLE_DEVICES can hide and remap GPUs, so application indices may differ from physical labels.

Move the Model and Tensors to the Same Device

After the ordinal is valid, use the same device for the model and tensors. Mixing CPU tensors with CUDA tensors causes a different error, but it often appears after developers have already changed device selection while chasing the invalid ordinal problem.

import torch

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model = torch.nn.Linear(4, 2).to(device)
batch = torch.randn(8, 4, device=device)
output = model(batch)

print(output.shape)

Centralizing the device value keeps the training code predictable. It also makes it easier to swap between CPU, one GPU, and multiple GPU jobs without editing every tensor allocation.

Distributed and Notebook Runtimes

Distributed launchers usually pass a local rank for each worker. That local rank must be smaller than the visible GPU count for that worker. If a job launches four workers but only exposes two GPUs, two workers will request invalid ordinals. Validate this at process startup before initializing the full training stack.

import os
import torch

local_rank = int(os.environ.get("LOCAL_RANK", "0"))
gpu_count = torch.cuda.device_count()

if torch.cuda.is_available() and local_rank < gpu_count:
    torch.cuda.set_device(local_rank)
    device = torch.device(f"cuda:{local_rank}")
else:
    device = torch.device("cpu")

print(device)

In notebooks, restart the kernel after changing CUDA environment variables or switching runtime types. A long-running kernel can keep old CUDA state in memory, so restarting is often faster than repeatedly editing code that is already correct. If your script builds a list of available devices, check that the list is not empty before selecting from it; this Python empty list check pattern applies cleanly here.

Quick Fix Checklist

Print torch.cuda.device_count() in the failing process.
Use only ordinals from 0 to device_count() - 1.
Check whether CUDA_VISIBLE_DEVICES is hiding or remapping GPUs.
Use a single device variable for the model and tensors.
For distributed jobs, make sure each worker local rank is valid.
Update the NVIDIA driver only when the GPU is not detected or the CUDA stack is incompatible.

When the GPU is not detected at all, check the driver and CUDA installation. The NVIDIA driver download page is the right starting point for driver updates, while the PyTorch CUDA API reference documents the device-count and device-selection functions used above. For Python code that imports optional GPU libraries only when present, this guide to conditional imports in Python can help keep CPU and GPU environments working from the same script.

Python Pool infographic showing torch.cuda availability, device count, index check, and model placement — Check CUDA availability and device count before constructing a device or moving a model.

Check CUDA Availability And Count

A machine may have physical GPUs that are hidden from the current process by a scheduler, container, driver, or CUDA_VISIBLE_DEVICES. PyTorch reports the devices visible to that process, so validate the count before constructing cuda:1 or another ordinal.

import torch

if torch.cuda.is_available():
    count = torch.cuda.device_count()
    print("visible GPUs:", count)
    for index in range(count):
        print(index, torch.cuda.get_device_name(index))
else:
    print("CUDA is unavailable")

Use Zero-Based Logical IDs

If device_count() returns one, the only valid logical CUDA index is zero. A request for cuda:1 is invalid even if the host has another physical adapter that is hidden. Device ordinals are indexes in the process-visible list, not guaranteed physical labels.

import torch

if torch.cuda.is_available() and torch.cuda.device_count() > 0:
    device = torch.device("cuda:0")
else:
    device = torch.device("cpu")

print(device)

Python Pool infographic testing driver, runtime, container mapping, multiprocessing, and validation — Check driver and runtime compatibility, container GPU mapping, process visibility, index bounds, and startup logs.

Understand CUDA_VISIBLE_DEVICES

CUDA_VISIBLE_DEVICES filters and renumbers devices for child processes. If it is set to 2, the physical GPU 2 can appear as logical cuda:0. Log the environment variable and the PyTorch count from inside the failing process instead of relying on a host-level GPU listing.

import os
import torch

print("CUDA_VISIBLE_DEVICES:", os.environ.get("CUDA_VISIBLE_DEVICES"))
print("visible count:", torch.cuda.device_count())

Make Selection Defensive

For configurable training or inference, validate a requested ordinal and fail with the visible count, or fall back to CPU when that is part of the application policy. Avoid silently changing a requested GPU in a distributed job, because two processes may then use different devices than the scheduler assigned.

import torch

def choose_device(requested=0):
    if not torch.cuda.is_available():
        return torch.device("cpu")
    count = torch.cuda.device_count()
    if not 0 <= requested < count:
        raise ValueError(f"requested GPU {requested}, visible GPUs: {count}")
    return torch.device(f"cuda:{requested}")

print(choose_device(0))

PyTorch’s official device_count() reference reports visible devices, and the CUDA semantics documentation describes device selection. Check the installed PyTorch and driver versions when availability itself is false.

For related accelerator workflows, compare tensor-to-NumPy conversion, deep-learning environments, and framework installation errors after verifying the runtime itself.

Frequently Asked Questions

What causes CUDA error invalid device ordinal?

The program selected a GPU index that is not available to the current process, often because the machine has fewer GPUs or CUDA_VISIBLE_DEVICES remapped them.