Feedforward network (FFN)
AIFeedforward network (FFN): The position-wise sub-layer in a transformer block, two linear layers with an activation between, that transforms each token independently.
Feedforward network (FFN): The position-wise sub-layer in a transformer block, two linear layers with an activation between, that transforms each token independently.