Embed

---

Created: 23 Dec 2022, 02:16 PM | Modified: =dateformat(this.file.mtime,"dd MMM yyyy, hh:mm a") Tags: knowledge,

---

Embed images first

dp = embed(

data=dp, input_col=“image”, encoder=“clip”, device=0

)

In describe step:

• With candidate descriptions → embed again

text_dp = embed(

text_dp,

input_col=“output_phrase”,

encoder=“clip”,

device=0

)

Both embeds use CLIP encoder, which will use [pretrained model=“ViT-B/32”]{.mark} as per /domino/domino/_embed/clip.py to do both .encode_image and .encode_text based on the input_col="" type.

If it is mk.ImageColumn, will use .encode_image. If it is mk.PandasSeriesColumn, will use .encode_text

as per /domino/env/lib/python3.8/site-packages/clip/model.py

Will return the encoded data in dp with out_col name f”{encoder}({input_col})”, which will become “clip(image)” or “clip(output_phrase)”

https://domino-slice.readthedocs.io/en/latest/apidocs/embed.html#embed-reference