![MoEs comeback in GenAI with Mixtral](/content/images/size/w600/2024/02/DALL-E-2024-02-12-20.57.23---Imagine-a-raccoon--serving-as-the-mascot-for-a-concept-called--Mixture-of-Experts-.-This-raccoon--designed-in-a-charming--low-res-pixel-art-style--emb.webp)
MoEs
MoEs comeback in GenAI with Mixtral
Just yesterday, Mistral AI announced their new model Mixtral 8x7B. It is a high-quality Sparse Mixture of Expert models (SMoEs). Its weights are open-source and licensed under Apache 2.0 magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%https://t.co/uV4WVdtpwZ%3A6969%2Fannounce&