Vulkan 1.4.359 Expands Cooperative Matrix Capabilities
Way back in 2023 with Vulkan 1.3.255 the VK_KHR_cooperative_matrix extension was introduced for initial cooperative matrix support for Vulkan and the SPIR-V IR. VK_KHR_cooperative_matrix can be used for machine learning inference, graphics post-processing, physics simulations, and other tasks. Similar cooperative matrix support was since added to OpenCL.
With Vulkan 1.4.359 there is now VK_EXT_cooperative_matrix_maintenance1 with a collection of minor improvements to Vulkan's cooperative matrix support. VK_EXT_cooperative_matrix_maintenance1 brings additions to help accelerate features beyond just simple GEMM kernels and now supporting type/use conversions, reductions, per-element operations, and more. The new extension text explains:
"This extension adds several new features building on the cooperative matrix types added in VK_KHR_cooperative_matrix. The goal is to add and accelerate features beyond just simple GEMM kernels, including adding support for type/use conversions, reductions, per-element operations, and conversion of an element index to a matrix coordinate.
...
This extension also provides a new command to query the properties of cooperative matrices, which can be easily extended by other extensions in the future. The new query also includes inputs for the desired subgroup size and (when used together with extensions that support workgroup scope cooperative matrices, such as VK_NV_cooperative_matrix2) the number of invocations in the local workgroup, allowing implementations to expose properties that are optimal or compatible only with specific subgroup sizes or local workgroup sizes."
This cooperative matrix update was worked on by NVIDIA, Qualcomm, Intel, and Arm engineers.
Those wishing to learn more can do so via this commit of the Vulkan 1.4.359 changes.
3 Comments