4、TORCH.CUDA

港控/mmm° 2022-10-09 00:33 145阅读 0赞

这个包增加了对 CUDA 张量类型的支持，它实现了与 CPU 张量相同的功能，但它们利用 GPU 进行计算。

它被延迟初始化，因此您可以随时导入它，并使用 is\_available() 来确定您的系统是否支持 CUDA。CUDA 语义有更多关于使用 CUDA 的细节。

<table> 
 <tbody> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.StreamContext.html#torch.cuda.StreamContext" rel="nofollow"><code>StreamContext</code></a> </td> 
 <td> Context-manager that selects a given stream. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.can_device_access_peer.html#torch.cuda.can_device_access_peer" rel="nofollow"><code>can_device_access_peer</code></a> </td> 
 <td> Checks if peer access between two devices is possible. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.current_blas_handle.html#torch.cuda.current_blas_handle" rel="nofollow"><code>current_blas_handle</code></a> </td> 
 <td> Returns cublasHandle_t pointer to current cuBLAS handle </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.current_device.html#torch.cuda.current_device" rel="nofollow"><code>current_device</code></a> </td> 
 <td> Returns the index of a currently selected device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.current_stream.html#torch.cuda.current_stream" rel="nofollow"><code>current_stream</code></a> </td> 
 <td> Returns the currently selected&nbsp;<a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.Stream.html#torch.cuda.Stream" rel="nofollow"><code>Stream</code></a>&nbsp;for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.default_stream.html#torch.cuda.default_stream" rel="nofollow"><code>default_stream</code></a> </td> 
 <td> Returns the default&nbsp;<a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.Stream.html#torch.cuda.Stream" rel="nofollow"><code>Stream</code></a>&nbsp;for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.device.html#torch.cuda.device" rel="nofollow"><code>device</code></a> </td> 
 <td> Context-manager that changes the selected device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.device_count.html#torch.cuda.device_count" rel="nofollow"><code>device_count</code></a> </td> 
 <td> Returns the number of GPUs available. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.device_of.html#torch.cuda.device_of" rel="nofollow"><code>device_of</code></a> </td> 
 <td> Context-manager that changes the current device to that of given object. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_arch_list.html#torch.cuda.get_arch_list" rel="nofollow"><code>get_arch_list</code></a> </td> 
 <td> Returns list CUDA architectures this library was compiled for. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_device_capability.html#torch.cuda.get_device_capability" rel="nofollow"><code>get_device_capability</code></a> </td> 
 <td> Gets the cuda capability of a device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_device_name.html#torch.cuda.get_device_name" rel="nofollow"><code>get_device_name</code></a> </td> 
 <td> Gets the name of a device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_device_properties.html#torch.cuda.get_device_properties" rel="nofollow"><code>get_device_properties</code></a> </td> 
 <td> Gets the properties of a device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_gencode_flags.html#torch.cuda.get_gencode_flags" rel="nofollow"><code>get_gencode_flags</code></a> </td> 
 <td> Returns NVCC gencode flags this library was compiled with. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.init.html#torch.cuda.init" rel="nofollow"><code>init</code></a> </td> 
 <td> Initialize PyTorch’s CUDA state. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.ipc_collect.html#torch.cuda.ipc_collect" rel="nofollow"><code>ipc_collect</code></a> </td> 
 <td> Force collects GPU memory after it has been released by CUDA IPC. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.is_available.html#torch.cuda.is_available" rel="nofollow"><code>is_available</code></a> </td> 
 <td> Returns a bool indicating if CUDA is currently available. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.is_initialized.html#torch.cuda.is_initialized" rel="nofollow"><code>is_initialized</code></a> </td> 
 <td> Returns whether PyTorch’s CUDA state has been initialized. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.set_device.html#torch.cuda.set_device" rel="nofollow"><code>set_device</code></a> </td> 
 <td> Sets the current device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.set_stream.html#torch.cuda.set_stream" rel="nofollow"><code>set_stream</code></a> </td> 
 <td> Sets the current stream.This is a wrapper API to set the stream. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.stream.html#torch.cuda.stream" rel="nofollow"><code>stream</code></a> </td> 
 <td> Wrapper around the Context-manager StreamContext that selects a given stream. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.synchronize.html#torch.cuda.synchronize" rel="nofollow"><code>synchronize</code></a> </td> 
 <td> Waits for all kernels in all streams on a CUDA device to complete. </td> 
 </tr> 
 </tbody> 
</table>

## Random Number Generator ##

<table> 
 <tbody> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_rng_state.html#torch.cuda.get_rng_state" rel="nofollow"><code>get_rng_state</code></a> </td> 
 <td> Returns the random number generator state of the specified GPU as a ByteTensor. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.get_rng_state_all.html#torch.cuda.get_rng_state_all" rel="nofollow"><code>get_rng_state_all</code></a> </td> 
 <td> Returns a list of ByteTensor representing the random number states of all devices. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.set_rng_state.html#torch.cuda.set_rng_state" rel="nofollow"><code>set_rng_state</code></a> </td> 
 <td> Sets the random number generator state of the specified GPU. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.set_rng_state_all.html#torch.cuda.set_rng_state_all" rel="nofollow"><code>set_rng_state_all</code></a> </td> 
 <td> Sets the random number generator state of all devices. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.manual_seed.html#torch.cuda.manual_seed" rel="nofollow"><code>manual_seed</code></a> </td> 
 <td> Sets the seed for generating random numbers for the current GPU. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.manual_seed_all.html#torch.cuda.manual_seed_all" rel="nofollow"><code>manual_seed_all</code></a> </td> 
 <td> Sets the seed for generating random numbers on all GPUs. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.seed.html#torch.cuda.seed" rel="nofollow"><code>seed</code></a> </td> 
 <td> Sets the seed for generating random numbers to a random number for the current GPU. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.seed_all.html#torch.cuda.seed_all" rel="nofollow"><code>seed_all</code></a> </td> 
 <td> Sets the seed for generating random numbers to a random number on all GPUs. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.initial_seed.html#torch.cuda.initial_seed" rel="nofollow"><code>initial_seed</code></a> </td> 
 <td> Returns the current random seed of the current GPU. </td> 
 </tr> 
 </tbody> 
</table>

Communication collectives

<table> 
 <tbody> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.comm.broadcast.html#torch.cuda.comm.broadcast" rel="nofollow"><code>comm.broadcast</code></a> </td> 
 <td> Broadcasts a tensor to specified GPU devices. </td> 
 </tr> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.comm.broadcast_coalesced.html#torch.cuda.comm.broadcast_coalesced" rel="nofollow"><code>comm.broadcast_coalesced</code></a> </td> 
 <td> Broadcasts a sequence tensors to the specified GPUs. </td> 
 </tr> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.comm.reduce_add.html#torch.cuda.comm.reduce_add" rel="nofollow"><code>comm.reduce_add</code></a> </td> 
 <td> Sums tensors from multiple GPUs. </td> 
 </tr> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.comm.scatter.html#torch.cuda.comm.scatter" rel="nofollow"><code>comm.scatter</code></a> </td> 
 <td> Scatters tensor across multiple GPUs. </td> 
 </tr> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.comm.gather.html#torch.cuda.comm.gather" rel="nofollow"><code>comm.gather</code></a> </td> 
 <td> Gathers tensors from multiple GPU devices. </td> 
 </tr> 
 </tbody> 
</table>

## Streams and events ##

<table> 
 <tbody> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.Stream.html#torch.cuda.Stream" rel="nofollow"><code>Stream</code></a> </td> 
 <td> Wrapper around a CUDA stream. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.Event.html#torch.cuda.Event" rel="nofollow"><code>Event</code></a> </td> 
 <td> Wrapper around a CUDA event. </td> 
 </tr> 
 </tbody> 
</table>

## Memory management ##

<table> 
 <tbody> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.empty_cache.html#torch.cuda.empty_cache" rel="nofollow"><code>empty_cache</code></a> </td> 
 <td> Releases all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in&nbsp;nvidia-smi. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.list_gpu_processes.html#torch.cuda.list_gpu_processes" rel="nofollow"><code>list_gpu_processes</code></a> </td> 
 <td> Returns a human-readable printout of the running processes and their GPU memory use for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_stats.html#torch.cuda.memory_stats" rel="nofollow"><code>memory_stats</code></a> </td> 
 <td> Returns a dictionary of CUDA memory allocator statistics for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_summary.html#torch.cuda.memory_summary" rel="nofollow"><code>memory_summary</code></a> </td> 
 <td> Returns a human-readable printout of the current memory allocator statistics for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_snapshot.html#torch.cuda.memory_snapshot" rel="nofollow"><code>memory_snapshot</code></a> </td> 
 <td> Returns a snapshot of the CUDA memory allocator state across all devices. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_allocated.html#torch.cuda.memory_allocated" rel="nofollow"><code>memory_allocated</code></a> </td> 
 <td> Returns the current GPU memory occupied by tensors in bytes for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.max_memory_allocated.html#torch.cuda.max_memory_allocated" rel="nofollow"><code>max_memory_allocated</code></a> </td> 
 <td> Returns the maximum GPU memory occupied by tensors in bytes for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.reset_max_memory_allocated.html#torch.cuda.reset_max_memory_allocated" rel="nofollow"><code>reset_max_memory_allocated</code></a> </td> 
 <td> Resets the starting point in tracking maximum GPU memory occupied by tensors for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_reserved.html#torch.cuda.memory_reserved" rel="nofollow"><code>memory_reserved</code></a> </td> 
 <td> Returns the current GPU memory managed by the caching allocator in bytes for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.max_memory_reserved.html#torch.cuda.max_memory_reserved" rel="nofollow"><code>max_memory_reserved</code></a> </td> 
 <td> Returns the maximum GPU memory managed by the caching allocator in bytes for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.set_per_process_memory_fraction.html#torch.cuda.set_per_process_memory_fraction" rel="nofollow"><code>set_per_process_memory_fraction</code></a> </td> 
 <td> Set memory fraction for a process. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_cached.html#torch.cuda.memory_cached" rel="nofollow"><code>memory_cached</code></a> </td> 
 <td> Deprecated; see&nbsp;<a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.memory_reserved.html#torch.cuda.memory_reserved" rel="nofollow"><code>memory_reserved()</code></a>. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.max_memory_cached.html#torch.cuda.max_memory_cached" rel="nofollow"><code>max_memory_cached</code></a> </td> 
 <td> Deprecated; see&nbsp;<a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.max_memory_reserved.html#torch.cuda.max_memory_reserved" rel="nofollow"><code>max_memory_reserved()</code></a>. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.reset_max_memory_cached.html#torch.cuda.reset_max_memory_cached" rel="nofollow"><code>reset_max_memory_cached</code></a> </td> 
 <td> Resets the starting point in tracking maximum GPU memory managed by the caching allocator for a given device. </td> 
 </tr> 
 <tr> 
 <td> &nbsp; <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.reset_peak_memory_stats.html#torch.cuda.reset_peak_memory_stats" rel="nofollow"><code>reset_peak_memory_stats</code></a> </td> 
 <td> Resets the “peak” stats tracked by the CUDA memory allocator. </td> 
 </tr> 
 </tbody> 
</table>

## NVIDIA Tools Extension (NVTX) ##

<table> 
 <tbody> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.nvtx.mark.html#torch.cuda.nvtx.mark" rel="nofollow"><code>nvtx.mark</code></a> </td> 
 <td> Describe an instantaneous event that occurred at some point. </td> 
 </tr> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.nvtx.range_push.html#torch.cuda.nvtx.range_push" rel="nofollow"><code>nvtx.range_push</code></a> </td> 
 <td> Pushes a range onto a stack of nested range span. </td> 
 </tr> 
 <tr> 
 <td> <a href="https://pytorch.org/docs/1.9.0/generated/torch.cuda.nvtx.range_pop.html#torch.cuda.nvtx.range_pop" rel="nofollow"><code>nvtx.range_pop</code></a> </td> 
 <td> Pops a range off of a stack of nested range spans. </td> 
 </tr> 
 </tbody> 
</table>