The trainer and the coed
Our strategy revolves round an idea known as information distillation, which makes use of a “trainer–pupil” mannequin coaching technique. We begin with a “trainer” — a big, highly effective, pre-trained generative mannequin that’s an professional at creating the specified visible impact however is much too sluggish for real-time use. The kind of trainer mannequin varies relying on the objective. Initially, we used a custom-trained StyleGAN2 mannequin, which was skilled on our curated dataset for real-time facial results. This mannequin could possibly be paired with instruments like StyleCLIP, which allowed it to govern facial options based mostly on textual content descriptions. This offered a powerful basis. As our mission superior, we transitioned to extra subtle generative fashions like Google DeepMind’s Imagen. This strategic shift considerably enhanced our capabilities, enabling higher-fidelity and extra numerous imagery, larger inventive management, and a broader vary of kinds for our on-device generative AI results.
The “pupil” is the mannequin that in the end runs on the person’s gadget. It must be small, quick, and environment friendly. We designed a pupil mannequin with a UNet-based structure, which is great for image-to-image duties. It makes use of a MobileNet spine as its encoder, a design identified for its efficiency on cell units, paired with a decoder that makes use of MobileNet blocks.
