How can I set the proxy IP port and userAgent to avoid the web anti-crawler mechanism?

[coderwpf]

How can I set the proxy IP port and userAgent to avoid the web anti-crawler mechanism? thanks

[coderwpf]

Also, how to set the reffer for website requests and how to bypass anti-crawler mechanisms similar to JA3 ？

[adbar]

Trafilatura does not currently provide all of the functions you mention, there are other libraries for that, you can then use Trafilatura on the downloaded HTML content.

• User agents and cookies can be set in the settings.cfg file
• For ideas on downloads see the troubleshooting section of the docs