Scrapy reactor bug

At a glance

The community member ran a script locally in Visual Studio Code (VSC) using the Apify platform, and it worked fine. However, when they tried to run the same script from the Apify platform, it returned an error. The community member had downloaded and installed the Twisted reactor as per the template instructions, and had also defined it in their settings. The error message indicates an issue with the AsyncioSelectorReactor object not having the _handleSignals() attribute. Other community members suggested that the difference in Python versions might be the cause of the issue, and one community member said they would check with the team.

KKirada

Hello, so I did a script in vsc locally (apify run) and it worked fine. Then, I pushed it and tried to run it from the apify platform and it returns this error I downloaded the reactor like the template asked to install the reactor which I did

Plain Text

install_reactor('twisted.internet.asyncioreactor.AsyncioSelectorReactor')

and defined it aswell

Plain Text

settings = get_project_settings()
        settings['TWISTED_REACTOR'] = 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'

4 comments

KKirada

the error

Plain Text

2023-10-25T08:58:31.711Z Traceback (most recent call last):
2023-10-25T08:58:31.713Z   File "<frozen runpy>", line 198, in _run_module_as_main
2023-10-25T08:58:31.716Z   File "<frozen runpy>", line 88, in _run_code
2023-10-25T08:58:31.718Z   File "/usr/src/app/src/__main__.py", line 20, in <module>
2023-10-25T08:58:31.720Z     asyncio.run(main())
2023-10-25T08:58:31.722Z   File "/usr/local/lib/python3.11/site-packages/nest_asyncio.py", line 31, in run
2023-10-25T08:58:31.724Z     return loop.run_until_complete(task)
2023-10-25T08:58:31.726Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-25T08:58:31.728Z   File "/usr/local/lib/python3.11/site-packages/nest_asyncio.py", line 99, in run_until_complete
2023-10-25T08:58:31.730Z     return f.result()
2023-10-25T08:58:31.732Z            ^^^^^^^^^^
2023-10-25T08:58:31.734Z   File "/usr/local/lib/python3.11/asyncio/futures.py", line 203, in result
2023-10-25T08:58:31.736Z     raise self._exception.with_traceback(self._exception_tb)
2023-10-25T08:58:31.738Z   File "/usr/local/lib/python3.11/asyncio/tasks.py", line 277, in __step
2023-10-25T08:58:31.740Z     result = coro.send(None)
2023-10-25T08:58:31.742Z              ^^^^^^^^^^^^^^^
2023-10-25T08:58:31.745Z   File "/usr/src/app/src/main.py", line 170, in main
2023-10-25T08:58:31.747Z     process.start()
2023-10-25T08:58:31.749Z   File "/usr/local/lib/python3.11/site-packages/scrapy/crawler.py", line 383, in start
2023-10-25T08:58:31.752Z     install_shutdown_handlers(self._signal_shutdown)
2023-10-25T08:58:31.754Z   File "/usr/local/lib/python3.11/site-packages/scrapy/utils/ossignal.py", line 19, in install_shutdown_handlers
2023-10-25T08:58:31.756Z     reactor._handleSignals()
2023-10-25T08:58:31.758Z     ^^^^^^^^^^^^^^^^^^^^^^
2023-10-25T08:58:31.760Z AttributeError: 'AsyncioSelectorReactor' object has no attribute '_handleSignals'

KKirada

the only difference is the python version ig

LLukas Krivka

I will check with the team

KKirada

Add a reply

Apify Discord Mirror

Scrapy reactor bug