Answer by Klutch27 for Debugging Scrapy Project in Visual Studio Code
The accepted answer leverages the launch.json provided by Scrapy's documentation. However, depending on where and how you are running your scrapers, you may not have the scrapy module available. Here...
View ArticleAnswer by gisman for Debugging Scrapy Project in Visual Studio Code
I applied @fmango's code and improve it.Instead of write a seperated runner file, just paste these code lines at the end of spider.run python debugger. that's allif __name__ == '__main__': import os...
View ArticleAnswer by Maximo Silva for Debugging Scrapy Project in Visual Studio Code
You could also try with{"configurations": [ {"name": "Python: Scrapy","type": "python","request": "launch","module": "scrapy","cwd": "${fileDirname}","args":...
View ArticleAnswer by cpinamtz for Debugging Scrapy Project in Visual Studio Code
In order to execute the typical scrapy runspider <PYTHON_FILE> command you must to set the following config into your launch.json:{"version": "0.1.0","configurations": [ {"name": "Python: Launch...
View ArticleAnswer by Manu NALEPA for Debugging Scrapy Project in Visual Studio Code
Configure your json file like that:"version": "0.2.0","configurations": [ {"name": "Crawl with scrapy","type": "python","request": "launch","module": "scrapy","cwd": "${fileDirname}","args":...
View ArticleAnswer by Peter.Wang for Debugging Scrapy Project in Visual Studio Code
Don't need to modify the launch.json, the default "Python: Current File (Integrated Terminal)" works perfectly. For the Python3 project, remember to place the runner.py file at the same level as the...
View ArticleAnswer by fmagno for Debugging Scrapy Project in Visual Studio Code
Inside your scrapy project folder create a runner.py module with the following:import osfrom scrapy.cmdline import executeos.chdir(os.path.dirname(os.path.realpath(__file__)))try: execute(...
View ArticleAnswer by naqushab for Debugging Scrapy Project in Visual Studio Code
I made it. The simplest way is to make a runner script runner.pyimport scrapyfrom scrapy.crawler import CrawlerProcessfrom g4gscraper.spiders.g4gcrawler import G4GSpiderprocess =...
View ArticleDebugging Scrapy Project in Visual Studio Code
I have Visual Studio Code on a Windows Machine, on which I am making a new Scrapy Crawler. The crawler is working fine but I want to debug the code, for which I am adding this in my launch.json...
View Article