Import your AI Model into Lens Studio to enhance your Lens
SnapML lets you to integrate machine learning models (like the ones created with FieldDay) into your Lens Studio projects. Here's a step-by-step guide on how to export your trained model, and take your Snap Lens to the next level with AI.
Step 1
Exporting your model to Lens Studio
Once you're happy with the performance of your model in FieldDay, you can easily export it using the 'Share' button in FieldDay. Find the 'Lens Studio' option and tap 'Get Template via Email'. In your inbox you'll receive all relevant files, including:
.onnx
Open Neural Network Exchange File (the model)
.lsproj
Lens Studio Template
.js
A script with your labels
Step 2
Importing your model into Lens Studio
To import your model into Lens Studio follow these steps:
Open the Lens Studio template file that you generated,
or go to Lens Studio and open the FieldDay Classification Template.With the project open, drag the .onnx file and the Label script file into your project resources.
In the ML component setup pop-up, set the scale and bias input parameters to the following values for all FieldDay models:
Scale
0.0171248
0.0175070
0.0174292
Bias
-2.1179039
-2.0357143
-1.8044444
Step 3
Connect your ML Component
In your Objects panel, select your ML component. This will open the Inspector panel, where you can select your model. Uncheck "Stretch" and select "Screen Crop Texture" as the texture to input the camera feed to your model.
Step 4
Connect your Label Script
In your Objects panel, select the Labels script that lives under the Multi Class Classification object. In its Inspector panel, slide the Labels script you exported from FieldDay or click on the current Label script to replace it with the new one.
Step 5
Connect your Label Script
Adjust the 'Top K' value in the Inspector panel of the "Multi Class Classification" object to a number that is equal to the number of categories you have in your FieldDay model (including the 'Background' category if you have it enabled). The threshold defines the level of confidence your model needs to have to trigger a response. Illustrated below (0.70) is pretty high. Make sure to check the confidence of your model in FieldDay to adjust it. You can see the confidence in the analyse tab, with a double tap on the camera feed inside a fieldDay project.
