CUDA out of memory. Tried to allocate 128.00 MiB. GPU 0 has a total capacty of 21.99 GiB of which 61.00 MiB is free. Process 32515 has 492.00 MiB memory in use. Process 32508 has 492.00 MiB memory in use. Process 32435 has 492.00 MiB memory in use. Process 6074 has 3.92 GiB memory in use. Including non-PyTorch memory, this process has 4.07 GiB memory in use. Process 19323 has 492.00 MiB memory in use. Process 20932 has 988.00 MiB memory in use. Process 20931 has 988.00 MiB memory in use. Process 26828 has 2.96 GiB memory in use. Process 30522 has 1.60 GiB memory in use. Process 582 has 1.57 GiB memory in use. Process 14972 has 3.90 GiB memory in use. Of the allocated memory 3.41 GiB is allocated by PyTorch, and 356.33 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF