A not-so-fictious problem: Say, you’ve created a web application that runs on a dedicated Linux server in the cloud. Thousands of users visit your web app and suddenly … it crashes. Your users start complaining, and you lose revenue. More importantly, you bleed credibility by the hour. Your server is down, so what do you … Read more
Problem Formulation and Solution Overview This article will show you how to find DataFrame row indices in Python with NaN or Null (empty) values. To make it more interesting, we have the following scenario: Rivers Clothing has given you a CSV file that requires a clean-up to make it usable for Payroll and Data Analysis. … Read more
Problem Formulation Given the following DataFrame df: 💬 Challenge: How to apply a function f to each cell in the DataFrame? For example, you may want to apply a function that replaces all odd values with the value ‘odd’. Solution: DataFrame applymap() The Pandas DataFrame df.applymap() method returns a new DataFrame where the function f … Read more
Given an HTML table (code) in a file or at a given URL. First, load all HTML tables into the Python script by calling Pandas’ pd.read_html() passing the URL of the HTML document. The result is a list of DataFrames, one per HTML table in the document. Second, convert any specific DataFrame to a CSV … Read more
The Pandas read_html() function is an easy way to convert an HTML table (e.g., stored at a given URL) to a Pandas DataFrame. You pass a location string or path to it and it returns a list of DataFrames, each representing one table from the location path or URL. The following code, for example, reads … Read more
Problem Formulation and Solution Overview In this article, you’ll learn how to iterate over a 1D, 2D and 3D NumPy array using Python. When working with the NumPy library, you will encounter situations where you will need to iterate through a 1D, 2D and even a 3D array. This article will show you how to … Read more
What is a Fixed-Width File? 💡 Definition: A fixed-width text file contains data that is structured in rows and columns. Each row contains one data entry consisting of multiple values (one value per column). Each column has a fixed width, i.e., the same number of characters, restricting the maximum data size per column. Example of … Read more
Problem Formulation and Solution Overview In this article, you’ll learn how to check the Data Types of a Pandas Series. A Series is a 1D (one-dimensional) array. This Series can be of various data types, such as an integer, a string, a float or even an object! A good practice is to ensure, before performing … Read more
The easiest way to convert a tab-delimited values (TSV) file to a comma-separated values (CSV) file is to use the following three lines of code: import pandas as pd df = pd.read_csv(‘my_file.txt’, sep=’\t’, header=None) df.to_csv(‘my_file.csv’, header=None) We’ll explain this and other approaches in more detail next—scroll down to Method 3 for this exact method. Problem … Read more
To convert a Pandas DataFrame into a CSV string rather than a CSV file, just use the pd.to_csv() function without any filename or path argument. The function returns a CSV string you can use for further processing in your Python script. Here’s an example: You can modify this output by passing the index=False argument: Now, … Read more
To export/convert a Pandas DataFrame to a CSV file, use the Pandas method to_csv(). Set the index=False argument if you don’t want the first column to be the index number in the CSV, which is usually the case. Let’s dive into a simple two-step example on how to convert a DataFrame to a CSV easily … Read more
This article concerns the conversion and handling of CSV file formats in combination with the UTF-8 encoding standard. 💡 The Unicode Transformation Format 8-Bit (UTF-8) is a variable-width character encoding used for electronic communication. UTF-8 can encode more than 1 million (more or less weird) characters using 1 to 4 byte code units. Example UTF-8 … Read more