gotosl

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 15, 2025 License: BSD-3-Clause Imports: 31 Imported by: 0

README

Implementational details of Go to SL translation process

Overall, there are three main steps:

  1. Translate all the .go files in the current package, and all the files they //gosl:import, into corresponding .wgsl files, and put those in shaders/imports. All these files will be pasted into the generated primary kernel files, that go in shaders, and are saved to disk for reference. All the key kernel, system, variable info is extracted from the package .go file directives during this phase.

  2. Generate the main kernel .wgsl files, for each kernel function, which: a) declare the global buffer variables; b) include everything from imports; c) define the main function entry point. Each resulting file is pre-processed by naga to ensure it compiles, and to remove dead code not needed for this particular shader.

  3. Generate the gosl.go file in the package directory, which contains generated Go code for configuring the gpu compute systems according to the vars.

Go to SL translation

  1. files.go: Get a list of all the .go files in the current directory that have a //gosl: tag (ProjectFiles) and all the //gosl:import package files that those files import, recursively.

  2. extract.go: Extract the //gosl:start -> end regions from all the package and imported filees.

  3. Save all these files as new .go files in shaders/imports. We manually append a simple go "main" package header with basic gosl imports for each file, which allows the go compiler to process them properly. This is then removed in the next step.

  4. translate.go: Run TranslateDir on shaders/imports using the "golang.org/x/tools/go/packages" Load function, which gets ast and type information for all that code. Run the resulting ast.File for each file through the modified version of the Go stdlib src/go/printer code (printer.go, nodes.go, gobuild.go, comment.go), which prints out WGSL instead of Go code from the underlying ast representation of the Go files. This is what does the actual translation.

  5. sledits.go: Do various forms of post-processing text replacement cleanup on the generated WGSL files, in SLEdits function.

Struct types and read-write vs. read-only:

  1. In Go, all struct args are generally pointers, including method receivers.

  2. In WGSL, pointers are strongly constrained, and you cannot use a pointer to a struct field within another struct. In general, structs are best used for static parameters, rather than writable data.

  3. For a given method, it is not possible to know in advance based just on type data whether a given struct arg is read-only or read-write -- these are properties of the variables, not the types.

  4. Therefore, we make the strong simplifying assumption that all struct args to a method are read-only, and thus the GPU code sets them to non-pointers. Any code that passes non-read-only struct args to such methods will generate a WGSL compilation / validation error.

  5. There is one exception: a given method can be explicitly marked with //gosl:pointer-receiver to make the method receiver argument a pointer, and thus usable with read-write pointers.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var Replaces = []Replace{
	{[]byte("sltype.Uint32Vec2"), []byte("vec2<u32>")},
	{[]byte("sltype.Float32Vec2"), []byte("vec2<f32>")},
	{[]byte("float32"), []byte("f32")},
	{[]byte("float64"), []byte("f64")},
	{[]byte("uint32"), []byte("u32")},
	{[]byte("uint64"), []byte("su64")},
	{[]byte("int32"), []byte("i32")},
	{[]byte("math32.FastExp("), []byte("FastExp(")},

	{[]byte("math.Float32frombits("), []byte("bitcast<f32>(")},
	{[]byte("math.Float32bits("), []byte("bitcast<u32>(")},
	{[]byte("shaders."), []byte("")},
	{[]byte("slrand."), []byte("Rand")},
	{[]byte("RandUi32"), []byte("RandUint32")},
	{[]byte(".SetFromVector2("), []byte("=(")},
	{[]byte(".SetFrom2("), []byte("=(")},
	{[]byte(".IsTrue()"), []byte("==1")},
	{[]byte(".IsFalse()"), []byte("==0")},
	{[]byte(".SetBool(true)"), []byte("=1")},
	{[]byte(".SetBool(false)"), []byte("=0")},
	{[]byte(".SetBool("), []byte("=i32(")},
	{[]byte("slbool.Bool"), []byte("i32")},
	{[]byte("slbool.True"), []byte("1")},
	{[]byte("slbool.False"), []byte("0")},
	{[]byte("slbool.IsTrue("), []byte("(1 == ")},
	{[]byte("slbool.IsFalse("), []byte("(0 == ")},
	{[]byte("slbool.FromBool("), []byte("i32(")},
	{[]byte("bools.ToFloat32("), []byte("f32(")},
	{[]byte("bools.FromFloat32("), []byte("bool(")},
	{[]byte("num.FromBool[f32]("), []byte("f32(")},
	{[]byte("num.ToBool("), []byte("bool(")},
	{[]byte("sltype."), []byte("")},
}
View Source
var SLBools = []Replace{
	{[]byte(".IsTrue()"), []byte("==1")},
	{[]byte(".IsFalse()"), []byte("==0")},
	{[]byte(".SetBool(true)"), []byte("=1")},
	{[]byte(".SetBool(false)"), []byte("=0")},
	{[]byte(".SetBool("), []byte("=int32(")},
	{[]byte("slbool.Bool"), []byte("int32")},
	{[]byte("slbool.True"), []byte("1")},
	{[]byte("slbool.False"), []byte("0")},
	{[]byte("slbool.IsTrue("), []byte("(1 == ")},
	{[]byte("slbool.IsFalse("), []byte("(0 == ")},
	{[]byte("slbool.FromBool("), []byte("int32(")},
	{[]byte("bools.ToFloat32("), []byte("float32(")},
	{[]byte("bools.FromFloat32("), []byte("bool(")},
	{[]byte("num.FromBool[f32]("), []byte("float32(")},
	{[]byte("num.ToBool("), []byte("bool(")},
}

Functions

func CopyFile

func CopyFile(src, dst string) ([][]byte, error)

func Fprint

func Fprint(output io.Writer, pkg *packages.Package, node any) error

Fprint "pretty-prints" an AST node to output. It calls PrintConfig.Fprint with default settings. Note that gofmt uses tabs for indentation but spaces for alignment; use format.Node (package go/format) for output that matches gofmt.

func IsGoFile

func IsGoFile(f fs.DirEntry) bool

func IsWGSLFile

func IsWGSLFile(f fs.DirEntry) bool

func MathReplaceAll

func MathReplaceAll(mat, ln []byte) []byte

func MoveLines

func MoveLines(lines *[][]byte, to, st, ed int)

MoveLines moves the st,ed region to 'to' line

func ReadFileLines

func ReadFileLines(fn string) ([][]byte, error)

func RemoveGenFiles

func RemoveGenFiles(dir string)

RemoveGenFiles removes .go, .wgsl, .spv files in shader generated dir

func Run

func Run(cfg *Config) error

func SlBoolReplace

func SlBoolReplace(lines [][]byte)

SlBoolReplace replaces all the slbool methods with literal int32 expressions.

func SlEdits

func SlEdits(src []byte) (lines [][]byte, hasSlrand bool, hasSltype bool)

SlEdits performs post-generation edits for wgsl, replacing type names, slbool, function calls, etc. returns true if a slrand. or sltype. prefix was found, driveing copying of those files.

func SlEditsReplace

func SlEditsReplace(lines [][]byte) (bool, bool)

SlEditsReplace replaces Go with equivalent WGSL code returns true if has slrand. or sltype. to auto include that header file if so.

func SlRemoveComments

func SlRemoveComments(lines [][]byte) [][]byte

func WriteFileLines

func WriteFileLines(fn string, lines [][]byte) error

Types

type CommentedNode

type CommentedNode struct {
	Node     any // *ast.File, or ast.Expr, ast.Decl, ast.Spec, or ast.Stmt
	Comments []*ast.CommentGroup
}

A CommentedNode bundles an AST node and corresponding comments. It may be provided as argument to any of the Fprint functions.

type Config

type Config struct {

	// Output is the output directory for shader code,
	// relative to where gosl is invoked; must not be an empty string.
	Output string `flag:"out" default:"shaders"`

	// Exclude is a comma-separated list of names of functions to exclude from exporting to WGSL.
	Exclude string `default:"Update,Defaults"`

	// Keep keeps temporary converted versions of the source files, for debugging.
	Keep bool

	// Debug enables debugging messages while running.
	Debug bool
}

Config has the configuration info for the gosl system.

type File

type File struct {
	Name  string
	Lines [][]byte
}

File has contents of a file as lines of bytes.

type Function

type Function struct {
	Name    string
	Funcs   map[string]*Function
	Atomics map[string]*Var // variables that have atomic operations in this function
}

Function represents the call graph of functions

func NewFunction

func NewFunction(name string) *Function

type GetGlobalVar

type GetGlobalVar struct {
	// global variable
	Var *Var

	// name of temporary variable
	TmpVar string

	// index passed to the Get function
	IdxExpr ast.Expr

	// rw override
	ReadWrite bool
}

GetGlobalVar holds GetVar expression, to Set variable back when done.

type Group

type Group struct {
	Name string

	// comment docs about this group
	Doc string

	// Uniform indicates a uniform group; else default is Storage.
	Uniform bool

	Vars []*Var
}

Group represents one variable group.

type Kernel

type Kernel struct {
	Name string

	Args string

	// Filename is the name of the kernel shader file, e.g., shaders/Compute.wgsl
	Filename string

	// function code
	FuncCode string

	// Lines is full shader code
	Lines [][]byte

	// ReadWriteVars are variables marked as read-write for current kernel.
	ReadWriteVars map[string]bool
}

Kernel represents a kernel function, which is the basis for each wgsl generated code file.

type Mode

type Mode uint

A Mode value is a set of flags (or 0). They control printing.

const (
	RawFormat Mode = 1 << iota // do not use a tabwriter; if set, UseSpaces is ignored
	TabIndent                  // use tabs for indentation independent of UseSpaces
	UseSpaces                  // use spaces instead of tabs for alignment
	SourcePos                  // emit //line directives to preserve original source positions
)

type PrintConfig

type PrintConfig struct {
	Mode     Mode   // default: 0
	Tabwidth int    // default: 8
	Indent   int    // default: 0 (all code is indented at least by this much)
	GoToSL   *State // gosl:

	ExcludeFunctions map[string]bool
}

A PrintConfig node controls the output of Fprint.

func (*PrintConfig) Fprint

func (cfg *PrintConfig) Fprint(output io.Writer, pkg *packages.Package, node any) error

Fprint "pretty-prints" an AST node to output for a given configuration cfg. Position information is interpreted relative to the file set fset. The node type must be *ast.File, *CommentedNode, []ast.Decl, []ast.Stmt, or assignment-compatible to ast.Expr, ast.Decl, ast.Spec, or ast.Stmt.

type Replace

type Replace struct {
	From, To []byte
}

type State

type State struct {
	// Config options.
	Config *Config

	// path to shaders/imports directory.
	ImportsDir string

	// name of the package
	Package string

	// GoFiles are all the files with gosl content in current directory.
	GoFiles map[string]*File

	// GoVarsFiles are all the files with gosl:vars content in current directory.
	// These must be processed first!  they are moved from GoFiles to here.
	GoVarsFiles map[string]*File

	// GoImports has all the imported files.
	GoImports map[string]map[string]*File

	// ImportPackages has short package names, to remove from go code
	// so everything lives in same main package.
	ImportPackages map[string]bool

	// Systems has the kernels and variables for each system.
	// There is an initial "Default" system when system is not specified.
	Systems map[string]*System

	// GetFuncs is a map of GetVar, SetVar function names for global vars.
	GetFuncs map[string]*Var

	// SLImportFiles are all the extracted and translated WGSL files in shaders/imports,
	// which are copied into the generated shader kernel files.
	SLImportFiles []*File

	// generated Go GPU gosl.go file contents
	GPUFile File

	// ExcludeMap is the compiled map of functions to exclude in Go -> WGSL translation.
	ExcludeMap map[string]bool

	// GetVarStack is a stack per function definition of GetVar variables
	// that need to be set at the end.
	GetVarStack stack.Stack[map[string]*GetGlobalVar]

	// GetFuncGraph is true if getting the function graph (first pass).
	GetFuncGraph bool

	// CurKernel is the current Kernel for second pass processing.
	CurKernel *Kernel

	// KernelFuncs are the list of functions to include for current kernel.
	KernelFuncs map[string]*Function

	// FuncGraph is the call graph of functions, for dead code elimination
	FuncGraph map[string]*Function
}

State holds the current Go -> WGSL processing state.

func (*State) AllFuncs

func (st *State) AllFuncs(name string) map[string]*Function

AllFuncs returns aggregated list of all functions called be given function

func (*State) AppendGoHeader

func (st *State) AppendGoHeader(lines [][]byte) [][]byte

AppendGoHeader appends Go header

func (*State) AtomicVars

func (st *State) AtomicVars(funcs map[string]*Function) map[string]*Var

AtomicVars returns all the variables marked as atomic within the list of functions.

func (*State) CompileFile

func (st *State) CompileFile(fn string) error

func (*State) CopyPackageFile

func (st *State) CopyPackageFile(fnm, packagePath string) error

CopyPackageFile copies given file name from given package path into the current imports directory. e.g., "slrand.wgsl", "cogentcore.org/lab/gosl/slrand"

func (*State) ExtractFiles

func (st *State) ExtractFiles()

ExtractFiles processes all the package files and saves the corresponding .go files with simple go header.

func (*State) ExtractGosl

func (st *State) ExtractGosl(lines [][]byte) (outLines [][]byte, hasVars bool)

ExtractGosl gosl comment-directive tagged regions from given file.

func (*State) ExtractImports

func (st *State) ExtractImports()

ExtractImports processes all the imported files and saves the corresponding .go files with simple go header.

func (*State) ExtractWGSL

func (st *State) ExtractWGSL(lines [][]byte) [][]byte

ExtractWGSL extracts the WGSL code embedded within .Go files, which is commented out in the Go code -- remove comments.

func (*State) GenGPU

func (st *State) GenGPU()

GenGPU generates and writes the Go GPU helper code

func (*State) GenGPUSystemInit

func (st *State) GenGPUSystemInit(sy *System) string

GenGPUSystemInit generates GPU Init code for given system.

func (*State) GenGPUSystemOps

func (st *State) GenGPUSystemOps(sy *System) string

GenGPUSystemOps generates GPU helper functions for given system.

func (*State) GenKernelHeader

func (st *State) GenKernelHeader(sy *System, kn *Kernel, avars map[string]*Var) string

GenKernelHeader returns the novel generated WGSL kernel code for given kernel, which goes at the top of the resulting file.

func (*State) GenTensorFuncs

func (st *State) GenTensorFuncs(sy *System) string

GenTensorFuncs returns the generated WGSL code for indexing the tensors in given system.

func (*State) GetTempVar

func (st *State) GetTempVar(vrnm string) *GetGlobalVar

GetTempVar returns temp var for global variable of given name, if found.

func (*State) GlobalVar

func (st *State) GlobalVar(vrnm string) *Var

GlobalVar returns global variable of given name, if found.

func (*State) HasGoslTag

func (st *State) HasGoslTag(lines [][]byte) bool

HasGoslTag returns true if given file has a //gosl: tag

func (*State) ImportFiles

func (st *State) ImportFiles(lines [][]byte)

ImportFiles checks the given content for //gosl:import tags and imports the package if so.

func (*State) Init

func (st *State) Init(cfg *Config)

func (*State) PrintFuncGraph

func (st *State) PrintFuncGraph()

func (*State) ProjectFiles

func (st *State) ProjectFiles()

ProjectFiles gets the files in the current directory.

func (*State) RecycleFunc

func (st *State) RecycleFunc(name string) *Function

get or add a function of given name

func (*State) Run

func (st *State) Run() error

func (*State) System

func (st *State) System(sysname string) *System

System returns the given system by name, making if not made. if name is empty, "Default" is used.

func (*State) TranslateDir

func (st *State) TranslateDir(pf string) error

TranslateDir translate all .Go files in given directory to WGSL.

func (*State) VarIsReadWrite

func (st *State) VarIsReadWrite(vrnm string) bool

VarIsReadWrite returns true if var of name is set as read-write for current kernel.

func (*State) VarsAdded

func (st *State) VarsAdded()

VarsAdded is called when a set of vars has been added; update relevant maps etc.

type System

type System struct {
	Name string

	// Kernels are the kernels using this compute system.
	Kernels map[string]*Kernel

	// Groups are the variables for this compute system.
	Groups []*Group

	// NTensors is the number of tensor vars.
	NTensors int
}

System represents a ComputeSystem, and its kernels and variables.

func NewSystem

func NewSystem(name string) *System

type Var

type Var struct {
	Name string

	// comment docs about this var.
	Doc string

	// Type of variable: either []Type or F32, U32 for tensors
	Type string

	// ReadOnly indicates that this variable is never read back from GPU,
	// specified by the gosl:read-only property in the variable comments.
	// It is important to optimize GPU memory usage to indicate this.
	ReadOnly bool

	// ReadOrWrite indicates that this variable defaults to ReadOnly
	// but is also flagged as read-write in some cases. It is registered
	// as read_write in the gpu ComputeSystem, but processed as ReadOnly
	// by default except for kernels that declare it as read-write.
	ReadOrWrite bool

	// True if a tensor type
	Tensor bool

	// Number of dimensions
	TensorDims int

	// data kind of the tensor
	TensorKind reflect.Kind

	// index of tensor in list of tensor variables, for indexing.
	TensorIndex int
}

Var represents one global system buffer variable.

func (*Var) IndexFunc

func (vr *Var) IndexFunc() string

IndexFunc returns the tensor index function name

func (*Var) IndexStride

func (vr *Var) IndexStride(dim int) string

IndexStride returns the tensor stride variable reference

func (*Var) SLType

func (vr *Var) SLType() string

SLType returns the WGSL type string

func (*Var) SetTensorKind

func (vr *Var) SetTensorKind()

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL