Scrapy is an open-source Python library to extract data from websites, i.e., web scraping.
Problem Formulation: Given a PyCharm project. How to install the Scrapy library in your project within a virtual environment or globally?
Here’s a solution that always works:
File > Settings > Projectfrom the PyCharm menu.
- Select your current project.
- Click the
Python Interpretertab within your project tab.
- Click the small
+symbol to add a new library to the project.
- Now type in the library to be installed, in your example
"scrapy"without quotes, and click
- Wait for the installation to terminate and close all popup windows.
Here’s the installation process as a short animated video—it works analogously for Scrapy, just type in “scrapy” in the search field instead:
Make sure to select only “Scrapy” because there are many other packages that are not required but also contain the term “scrapy” (false positives):
Alternatively, you can run the
pip install scrapy command in your PyCharm “Terminal” view:
$ pip install scrapy
Feel free to check out the following free email academy with Python cheat sheets to boost your coding skills!
To become a PyCharm master, check out our full course on the Finxter Computer Science Academy available for free for all Finxter Premium Members:
While working as a researcher in distributed systems, Dr. Christian Mayer found his love for teaching computer science students.
To help students reach higher levels of Python success, he founded the programming education website Finxter.com that has taught exponential skills to millions of coders worldwide. He’s the author of the best-selling programming books Python One-Liners (NoStarch 2020), The Art of Clean Code (NoStarch 2022), and The Book of Dash (NoStarch 2022). Chris also coauthored the Coffee Break Python series of self-published books. He’s a computer science enthusiast, freelancer, and owner of one of the top 10 largest Python blogs worldwide.
His passions are writing, reading, and coding. But his greatest passion is to serve aspiring coders through Finxter and help them to boost their skills. You can join his free email academy here.