This article is about a topic that is far more important and even more fundamental than any specific Python tutorial: Libraries.
What Are the Top 10 Python Libraries?
The following list reflects the most important Python libraries based on my experience:
- Pandas. “Excel for coders”.
- NumPy. Fundamental to many libraries in the data science and machine learning space.
- Matplotlib. Plotting and graph visualization.
- Scikit-learn. Single-processor machine learning algorithms in Python.
- TensorFlow. Google’s machine learning library.
- PyTorch. Another machine learning library for production.
- OpenCV. High-performance practical AI library for image recognition and other AI-related tasks.
- Requests. HTTP for Humans
- Selenium. Automation for browsers, i.e., testing.
- NetworkX. Graph and network analysis, algorithms, modeling, and visualization.
I would say NumPy is probably the most important of those 10 libraries because it’s fundamental to Pandas, Matplotlib, Scikit-learn, TensorFlow, PyTorch, and even NetworkX.
*** Feel free to start learning NumPy in our in-depth blog article here. ***
Why Using Python Libraries?
Get your hot, freshly-brewed coffee, and let me show you what holds you back more than anything else in your coding productivity.
Imagine you’re starting your own Uber business. You need to get access to a car or you cannot even start to sell your services. For sure, you don’t build the car yourself, do you? Why should you reinvent the wheel – literally?
Yet, when it comes to coding many developers DO reinvent the wheel on a daily basis. They implement basic algorithms on their own again and again. What a waste of resources!
In the physical world, if you don’t know how to build a business, you can go to the local library and look it up. Similarly, in the digital world, if you don’t want to invest hours implementing a web crawler, you simply import a library for web crawlers.
Everything is a library in coding – even the programming language itself. The simple statement x = 3 + 2 is only a high-level representation of low-level functionality. In this case, addition is really only a shortcut for multiple machine code instructions.
Coding is nothing but creating high-level functionality out of low-level functions.
What Exactly is a Python Library?
A Python library is a collection of modules. Many libraries consist of dozens of modules. Some libraries consist of only a single module. A module is a single file such as 'module.py' that contains Python code, i.e., functionality to be reused by other coders.
This brings us to the next question.
What Exactly is a Python Module?
Have you ever put code into a file 'yourFile.py' and executed this code via 'python yourFile.py'? Congrats, you have already created and executed your own module! In other words, a module is a Python file ending in the suffix .py.
Let’s make the concept of modules 100% clear. Say, you want to implement your own module. How do you do this? Simply create two files (just in your head): 'main.py' which contains your main program and 'module.py' which contains your minimal module.
Puzzle 1: What is the result of executing 'python main.py' in your shell? (Find the solution at the end of this article.)
# file module.py
print("2+2")
# file main.py
import module(Spoiler alert: solve the puzzle first before continue reading.)
The import statement in Python is simply a convenient way to “copy and paste” the content of the module into your file (without ACTUALLY doing this). Note that you must skip the suffix ".py" when importing a module. (Don’t ask me why.)
Puzzle 2: What is the result of executing 'main.py'?
# file italian_greetings.py
def hi():
print("Ciao bella!")
def bye():
print("Ciao bella!!")
# file main.py
import italian_greetings
italian_greetings.hi()Clearly, we are not seeing the result of function bye(). The reason is that we have only defined but not yet executed bye().
Great, now you can import modules from a library (=collection of modules) and you can even create your own modules and libraries! Not too shabby.
However, you will quickly become annoyed by always writing the module name like in italian_greetings.hi().
That’s where a new command comes into play: import x as y. Replace the placeholder x with your (shorter) module name y.
Puzzle 3: What is the result of executing 'main.py'?
# file italian_greetings.py
def hi():
print("Ciao bella!")
# file german_greetings.py
def hi():
print("Guten Tag!")
# file main.py
import german_greetings as de
import italian_greetings as it
de.hi()
it.hi()But how do we get rid of these prefixes completely?
It’s time to introduce a third command (after import x, and import x as y): from x import z.
The placeholder x is still a module but z is a specific function in this module. This allows us to do fine-grained imports of just the functions we need. This is useful if we have huge modules with hundreds of functions or if we want to laser-focus our imports.
Puzzle 4: What is the result of executing main.py?
# file main.py from german_greetings import hi from italian_greetings import bye hi() bye()
Next, you’ll learn about a very important topic in Python: pip.
When you install Python, the most useful packages are already pre-installed. These packages form the “Python standard library”.
Although the standard library is huge, it contains only a fraction of all Python packages. If you create your own Python package, it is not included in the standard library by default. So how can other people benefit from your hard work creating your own Python package? And, more importantly, how can you leverage other people’s work? The answer is PyPI.
What is PyPI?
PyPI is an acronym for “Python Package Index”. It’s a HUGE collection of Python packages. PyPI gives you access to over 113,000 Python packages. It’s a powerful source of knowledge!
Each Python package contains many function definitions. Thus, millions of functions are waiting to be leveraged by you! If you have any problem, it is likely that a similar problem is already solved by a much better developer who invested days, if not weeks, into developing an efficient solution.
So how can you access this big honeypot? The answer comes with yet another acronym: pip.
What is Pip and How to Use It?
Pip is an acronym for the phrase “Pip installs packages” (yes, it’s recursive). And this is exactly what pip does — it installs packages for you. Without pip it’s a pain to install packages: you would have to manually download, decompress, and install the package via "python setup.py".
With pip, however, installing new packages is as easy as running "pip install xyz" in your command line. And pip is already installed with most modern Python distributions. It’s already at your fingertips.
So before spending hours creating Python code to solve a problem, search PyPI for relevant Python packages first. If you do not find a package, search even harder. Only if you still do not find a package, consider giving back to the awesome Python community by submitting your own package to the PyPI repository – it’s a simple git request (see the Git cheat sheet).
Puzzle Solutions
Here are the solutions to the puzzles in this article.
Puzzle 1:
2+2
Puzzle 2:
Ciao bella!
Puzzle 3:
Guten Tag!
Ciao bella!
Puzzle 4:
Guten Tag!
Ciao bella!!