Redefining the shading languages ecosystem with SPIR-V

古城微笑少年丶 2022-07-24 10:07 154阅读 0赞

SPIR-V，全称**Standard Portable Intermediate Representation (SPIR****)** 是一种用在GPU通用计算和图形学上的中间语言（[intermediate language][]，类似汇编）；由[Khronos][]开发, 最初是为[OpenCL][]准备的。 目前的版本是 SPIR-V，和下一代图形标准Vulkan差不多同时提出。前面版本的SPIR其实基于LLVM IR；而最新的SPIR-V则是重新定义了一套；当然还是和LLVM IR有些类似。

SPIR的目的是为Shader或OpenCL kernel，提供一个平台设备无关的类汇编语言的标准。这样应用程序发布时可以使用二进制的类汇编格式Shader Binary；而不是像以前OpenGL那样直接在线编译GLSL。以前的方式需要提供Shader的源代码，这不安全。Vulkan以后会大量的走类似D3D的方式，只提供一个Shader二进制格式。

At Siggraph 2014, the Khronos Group announced the development of [GLnext and a shader IL][]. They have a name now: [Vulkan][] and [SPIR V][].

First, let's clear out something: SPIR-V has nothing to do with SPIR. It's built from scratch with marketing thinking it was a good idea to name two products with different version values. Ohhh, I can't wait to battle against the confusions this idea will lead to. SPIR-V is not tied to LLVM, it's a fully specified and self-contained specification. It can represent both graphics code and compute code for any API, including Vulcan, OpenCL, OpenGL, OpenGL ES, WebGL, etc.

If we trust the industry is interested in solving actual developers' problems, it will eventually be used outside the Khronos Group, for Direct3D, Metal, consoles and beyond!

The Khronos Group has published some great documents for SPIR-V: [a white paper][], [a provisial portable core specification][], [an provisial extended instructions for graphics specification][], [an provisial extended instructions for compute specification][]. A work in progress [GLSL to SPIR-V compiler][] is also available.

SPIR-V in a nutshell:

*  Better portability
 *  Better runtime performance
 *  Source languages independent
 *  Fully specified
 *  Simple binary
 *  Extendable

#### A current shader cross-compilation pipeline ####

Engines will typically use a meta-HLSL or meta shading languages for their shader code. Historically, HLSL on desktop and consoles but GLSL on mobile and CAD/DCC/professional stuff because of markets differences and reality. An engine that wants to address multiple markets is quickly confronted to the requirement of cross-compiling shaders from HLSL to GLSL and/or GLSL to HLSL for example with game engines. Alternatively, in the off-line rendering ecosystem, we could also imagine the issue happenning between [Open Shading Language][] and[RenderMan][].

To match the reality of the ecosystem, we have to cross compile the shading languages but this is always [a painful process][] as illustrated in figure 1.

![Center][]

Figure 1: Shader compilation pipeline in Unity 5+

Is this crazy and insane? Yes. However, it addresses the reality of the market which is complex and fragmented with a lot of legacy hardware and software. If there are reasons to use an engine, this is one. This pipeline is particularly complex but still GLSL remains a second class shading language in the sense that this pipeline doesn't allow to run GLSL code on Direct3D for example.

#### Issues with HLSL11 IL and D3D11 compiler ####

In this pipeline HLSL and HLSL11 IL have a central place but HLSL11 IL is not specified and it's produced by a closed source Windows only compiler. Additionally, the D3D11 compiler should be a deprecated software because:

*  1 It was designed for legacy vec4 GPUs which implies wasting 50% of registers on vec2 and %25% on vec3. Considering that most GPUs hide memory access latencies by launching as many wavefronts as possible, GPUs can only launch new wavefronts when there is still space available in the register file. Thus, D3D11 compiler cost a great deal of performance by wasting registers.
 *  2 HLSL11 doesn't expose many hardware features ([gl\_DrawID][gl_DrawID], [pixel local storage][], [framebuffer fetch][]). How to express these features in HLSL when we want to use them in GLSL where they are available? Well, it's hard and at best possible through ugly hacks!
 *  3 D3D11 compiler performs destructive "optimizations" on the input shaders. Maybe, these "optimizations" where ok at some points in history but they prevent GPU vendors to properly optimize the shaders because source information gets lost. From where comes from these massive performance gains from hotfix drivers on new AAA released games? At least, part of it from shaders replacements. If HLSL11 IL input is A, use this totally different binary. This process is probably ok for hardware vendors that can afford it but this process won't scale beyond some flagship AAA games. Hence, the rest of us, the 99%, are paying the price of a poor compiler.

#### Easier and more robust cross compilation with SPIR-V ####

SPIR-V is a fully specified, cross APIs, binary intermediate language which is easy to read, to extend or to ignore unknown instructions without destructing the source in a tool chain.

![Center 1][]

Figure 2: A more desiable shader compilation pipeline to match the market reality

In figure 2 shows how SPIR-V could be used as a center piece of the compilation pipeline allowing multiple front-ends / languages to produce the SPIR-V IL. On platforms supporting SPIR-V, we could directly feed APIs with SPIR-V. The reality is that it will take a lot of time to transform the ecosystem and we need a solution for others platforms (eg: shipped mobiles which will never get new drivers). Hence, in a market real shader compilation pipeline, SPIR-V would need to be converted to HLSL9 IL, HLSL11 IL, GLSL, Metal, etc.

This is a lot of work but SPIR-V provides a simpler and more robust approach to cross compile to others ILs and languages. [HLSLcc][] is a great tool that demonstrated that cross compilation at IL level is a good direction. Another example is [IL2CPP][] used by Unity to cross compile C\# to C++. However, HLSL11 IL has many issues, as expressed previously. With SPIR-V, the source IL is fully specified and extendable making the translation from SPIR-V to HLSL11 IL easier (or even possible) than HLSL11 IL to SPIR-V for example.

I am glade to see that some frameworks are already [investigating about SPIR-V][]: [Jogamp][] is a [Java][]binding for multiple APIs including OpenGL and OpenCL. SPIR-V will allow the framework users to target independently OpenGL compute of OpenCL depending framework users needs for example.

![Center 2][]

Figure 3: Dreaming of a large market adoption, wish: that's the plan 5 years down this line

In figure 3, we show what would be the ultimate goal for SPIR-V: A massive adoption of the shading intermediate language so that we only have focus on innovations in the source language world. Great time to start a PHD on shading language!

In the current ecosystem, we have to generate N shaders per source because Direct3D 9, Direct3D 11, OpenGL and OpenGL ES accept different syntaxes. In many cases, only the syntax sugar is different but the functionality is exactly the same from API to API. The multiplication of generated shaders has a production cost in iteration time and latency until we get the results that SPIR-V could ultimately avoid. One SPIR-V => N platforms. Ideally, only hardware feature levels would condition the generation of multiple shader outputs.

#### Building bridges between ecosystems ####

Supporting multiple source languages as first class citizen is valuable for many reasons: The user case choice the shading language he like best but also to build bridges between ecosystem: off-line rendering and real-time rendering; Mobile and desktop; Simulation and rendering,; etc!

Let's take a common real-time use case: the case of skinning. For some games it might be valuable to do the skinning using a computer shader so that this work will be done on the GPU. However, some games might be GPU bound so to get better performance, it's probably a good idea to rely on the CPU. Let's imagine that [ISPC][] decides to support SPIR-V just like it supports [NVIDIA PTX][] for example. In such case an engine could create a single parallel friendly and optimized code and a game could choose either a GPU or CPU target for skinning depending on its needs.

Futhermore, SPIR-V allows to decouple the tools from the shipping software. There is no reason to ship a SPIR-V compiler such as ISPC or LLVM in a final game. It would increase export time and take a lot of space which is not desiable on mobile or WebGL platforms where memory is critical.

#### Better matching of the engine reality: custom shading languages! ####

If we consider engine shading languages, most are actually meta-HLSL using custom syntax to expose built-in shaders, engine specific functionality, fallback systems, making easier/possible cross compilation and platform targeting or offering a more natural way to expose shaders to technical shader artists. [Unity shading language][] is a good example of such meta-HLSL approach.

SPIR-V is offering us the opportunity to officialize these languages and even build the custom engine functionalities in their hearts through extended instruction sets. SPIR-V is built around a model where unrecognized blocks are just ignored and preserved, allowing interactions with SPIR-V tools unaware of these extensions.

Going crazier, we could imagine in the future storing OpenGL states in SPIR-V or describing rendering passes with off-line tools capable to filter redundant states changes. Many opportunities to run our imagination wild!

On the opposity side, we could imagine innovations to design a shading language friendly for technical artists, exposing [PBR][] parametrization for example or decoupling [surface shader, lighting model and light shader][surface shader_ lighting model and light shader] natively in the language.

#### NOT resolved by SPIR V: "Ohoho, on line shader compilation time is too long." ####

Some game developers would say that we need an IL for better online compilation performance. I think this is misunderstanding the cause of this issue. The slow part of the GLSL compilation isn't the translation from GLSL to IHVs IL (eg [NVIDIA PTX][NVIDIA PTX 1] or [AMD IL][]) but the optimizations path on the IHVs IL, particularly register allocation and scheduling transformation. These operations are fully dependent of GPU architectures hence should be performed by a targeted platform hardware vendor compiler.

Parsing GLSL / HLSL and performing non destruction optimizations have a cost and moving these off-line will give us some online performance gains. However, following the [80-20 rule][], we should expect only 20% performance gain. Compilation is a finer art than just parsing languages.

#### NOT fully resolved by SPIR V: "Ohoho, IHVs shader compilers are rubish." ####

A lot of issues with current GLSL compilers is that all of them support invalid syntaxes but hardware vendors don't want to fix them because it could break some applications. Removing the language syntax avoid potential errors but if tools generate invalid SPIR-V code, chances are that drivers will still do whatever it takes to garantee the IL works on their implementations. From an IHVs point of view, the worse is an application that doesn't work on their platforms by run fine else where.

By design, SPIR-V doesn't prevent IHVs to run invalid code so something else would need to be done to resolve this issue. Conformance tests? This said, I think that the simplicity of SPIR-V brings us to a much better state but the larger potential SPIR-V ecosystem than GLSL makes it required to ensure SPIR-V code quality.

The other issue is that some shader compilers are terrible at performance trivial optimizations hence we will probably need to have a shader pipeline path to do this job.

#### Better integration of the shading language in modern graphics APIs: ####

![Center 3][]

*Figure 4: Integration of SPIR-V in an explict graphics API such as Vulkan*

Figure 4 shows a potential integration of SPIR-V in Vulkan. Basically, with OpenGL and GLSL, all the cross compilation can be done off-line but the rest, including the actual shader compilations, needs to happen in the rendering thread. With Vulkan, we can decouple the tasks and guarantee that the rendering loop never compile shaders because of shader patching, play with shader cache or validate states. Additionally, we have a full control other the threaded shader compilations. With OpenGL, if we don't query the compilation results right after requesting a compilation, the drivers may thread and hide the compilation time but this behavior is not specified: Would the drivers actually thread the compilation? How many compilations can happen in parallel? Vulkan is an explict API, hence we have control for each step, when, where and how these steps should be performed.

#### Time for celebrations before hard work for an ecosystem revolution! ####

I am absolutely convinced that SPIR-V is a game changer for the industry just like WebGL before it. However, there is no magic. To become really successful, it will take a lot of efforts to ensure adoption by hardware and platform vendors but also to adapt SPIR-V to the reality of the market and build (open source!) cross compilation tools placing SPIR-V at their centers.

The head lines might be all about iPhone6 and Samsung S6 but the reality of the market is that we are seeing a race to the bottom with lot of old ES2 GPU IP shipping everyday. Furthermore, there is already a lot of devices already shipped that will never see the color of a drivers update during their entire life time. Finally, Vulkan might be great and all but realistically it will take years for the ecosystem to complete this transition. Meanwhile, there is a lot of OpenGL/ES, WebGL, Direct3D, console APIs out there. SPIR-V is solution for all of them!

I am looking forward the shading language revolution that SPIR-V will lead to, one step at a time!

Enjoy!

I cannot express how awesome this is. :-P[John W Kloetzli, Jr][John W Kloetzli_ Jr]

*  [SPIR-V: whitepaper][a white paper]
 *  [SPIR-V 0.99, provisiontal specification][a provisial portable core specification]
 *  [SPIR-V Extended Instructions for GLSL, provisiontal specification][an provisial extended instructions for graphics specification]
 *  [SPIR-V OpenCL 2.1 Extended Instruction Set, provisiontal specification][an provisial extended instructions for compute specification]
 *  [SPIR-V front page][SPIR V]
 *  [Vulkan front page][Vulkan]

[intermediate language]: https://en.wikipedia.org/wiki/Intermediate_language
[Khronos]: https://en.wikipedia.org/wiki/Khronos_Group
[OpenCL]: https://en.wikipedia.org/wiki/OpenCL
[GLnext and a shader IL]: https://www.khronos.org/assets/uploads/developers/library/2014-siggraph-bof/OpenGL-Ecosystem-BOF_Aug14.pdf
[Vulkan]: https://www.khronos.org/vulkan
[SPIR V]: https://www.khronos.org/spir
[a white paper]: https://www.khronos.org/registry/spir-v/papers/WhitePaper.pdf
[a provisial portable core specification]: https://www.khronos.org/registry/spir-v/specs/1.0/SPIRV.pdf
[an provisial extended instructions for graphics specification]: https://www.khronos.org/registry/spir-v/specs/1.0/GLSL.std.450.pdf
[an provisial extended instructions for compute specification]: https://www.khronos.org/registry/spir-v/specs/1.0/OpenCL.std.21.pdf
[GLSL to SPIR-V compiler]: https://www.khronos.org/opengles/sdk/tools/Reference-Compiler/
[Open Shading Language]: http://www.openshading.com/
[RenderMan]: https://renderman.pixar.com/resources/current/RenderMan/shadingLanguage.html
[a painful process]: http://aras-p.info/blog/2014/03/28/cross-platform-shaders-in-2014/
[Center]: /images/20220722/f41e7dcabe8b437e95638ca476b262f7.png
[gl_DrawID]: https://www.opengl.org/registry/specs/ARB/shader_draw_parameters.txt
[pixel local storage]: https://www.khronos.org/registry/gles/extensions/EXT/EXT_shader_pixel_local_storage.txt
[framebuffer fetch]: https://www.khronos.org/registry/gles/extensions/EXT/EXT_shader_framebuffer_fetch.txt
[Center 1]: /images/20220722/1488b1207700446289aa7315de3dba9c.png
[HLSLcc]: https://github.com/James-Jones/HLSLCrossCompiler
[IL2CPP]: http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-in-unity/
[investigating about SPIR-V]: https://jogamp.org/bugzilla/show_bug.cgi?id=1140#c2
[Jogamp]: https://jogamp.org/
[Java]: http://lib.csdn.net/base/17
[Center 2]: /images/20220722/b52f7f2c74b2421281a4918312b479ec.png
[ISPC]: https://github.com/ispc/ispc
[NVIDIA PTX]: https://github.com/ispc/ispc/blob/master/ptxtools/ptx.ll
[Unity shading language]: http://docs.unity3d.com/Manual/SL-Reference.html
[PBR]: http://www.frostbite.com/2014/11/moving-frostbite-to-pbr/
[surface shader_ lighting model and light shader]: http://aras-p.info/blog/2009/05/07/shaders-must-die-part-2/
[NVIDIA PTX 1]: http://docs.nvidia.com/cuda/parallel-thread-execution/#abstract
[AMD IL]: http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/AMD_Intermediate_Language_%28IL%29_Specification_v2.pdf
[80-20 rule]: http://en.wikipedia.org/wiki/Pareto_principle
[Center 3]: /images/20220722/54724977a7e2463dac513ca2c3aa1aad.png
[John W Kloetzli_ Jr]: https://twitter.com/JJcoolkl/status/572791033551036417