Troubleshooting and FAQ
Troubleshooting and FAQ#
How can I remove a facet from an item using a pipelet?
You can remove a facet within a pipelet by setting it to an empty list:
item["keywords"][facetName] = []
where facetName is the name of the facet you want to remove.
My pipelet is validated but upload fails.
The pipelet validate
throws no error but when uploading the pipelet, I see the following error:
2021-12-06 12:43:16,023 ERROR: Could not upload pipelet: b'{"error": "{\\"error\\": \\"Unknown pipelet loading error\\"}"}'
Check the /var/log/squirro/plumber/plumber.log
file for errors. Likely it will give you a hint on the error, for example, a failing import of a custom library that needs to be installed first.
My pipelet is not executed when run by a data loader.
Check the following components.
Configuration file:
Make sure to reference the correct pipelet configuration file from your load.sh
script.
For example add:
--pipelets-file 'pipelets.json' \
Item transformation:
If you see the following error:
INFO:squirro_data_load[23476]:--pipelets-file is ignored when item transformation takes place on the server. Please add the pipelet as a step of the workflow.
make sure to include the transform step in your load.sh
script:
--transform-items \
Consume method:
In order to use pipelets with the data loader, the consume()
method must be implemented. The consume_multiple()
method is only executed when the pipelet is uploaded to the server and included in the pipeline workflow.