FFN

Source: nn/FFN.luau

SwiGLU feed-forward block: FFN(x) = W3(silu(W1 x) .* (W2 x)).

Methods

new(name: string, C: number, hidden: number, rng: any)

forward(: any, x: any) -> any