Expert network
AIExpert network: One of several specialised sub-networks in a mixture-of-experts layer, of which only a few are activated for each token.
Expert network: One of several specialised sub-networks in a mixture-of-experts layer, of which only a few are activated for each token.