Problem Formulation 💡 Challenge: Given a CSV file. How to convert it to an excel file in Python? We create a folder with two files, the file csv_to_excel.py and my_file.csv. We want to convert the CSV file to an excel file so that after running the script csv_to_excel.py, we obtain the third file my_file.csv in … Read more
A good solution to calculate Pearson’s r and the p-value, to report the significance of the correlation, in Python is scipy.stats.pearsonr(x, y). A nice overview of the results delivers pingouin’s pg.corr(x, y). What is Pearson’s “r” Measure? A statistical correlation with Pearson’s r measures the linear relationship between two numerical variables. The correlation coefficient r … Read more
Problem Formulation Problem: Given a Pandas DataFrame; how to change the strings in the DataFrame to lowercase? Example: Consider the following Pandas DataFrame: Output: Expected Output: When you change a pandas DataFrame string column to lowercase, then the column is returned such that every string in the column is converted and displayed in lowercase while any non-alphabetical … Read more
The z-scores can be used to compare data with different measurements and for normalization of data for machine learning algorithms and comparisons. 💡 Note: There are different methods to calculate the z-score. The quickest and easiest one is: scipy.stats.zscore(). What is the z-score? The z-score is used for normalization or standardization to make differently scaled … Read more
Problem Formulation and Solution Overview In this article, you’ll learn various ways to subtract days from a date in Python. This article is based on Rivers Clothing and has examples on how to set and subtract a date x number of days, AND, how to subtract one date from another date. 💬 Question: How would … Read more
Problem Formulation 💬 Given a text query/keyword such as “History of Chess”. How to scrape the top Google results for that search query (=keyword) in Python? Disclaimer: Have a look at the important question: Is Web Scraping Legal? Method Summary You can get the top Google search results given a certain keyword string by installing … Read more
What is Data Cleaning? Data cleaning describes the process of turning messy data into clean datasets that can be used for research and data science purposes. For example, tidy data will be in a wide format: every column contains a variable, and every row contains one case. Also, data cleaning means getting rid of corrupt … Read more
Problem Formulation and Solution Overview This article will show you how to count duplicates in a Pandas DataFrame in Python. To make it more fun, we have the following running scenario: Rivers Clothing has a CSV containing all its employees. However, their CSV file has more rows than employees. This is a definite problem! They … Read more
Problem Formulation and Solution Overview In this article, you’ll learn how to add a new column to a CSV file in Python. To make it more fun, we have the following running scenario: The owner of the Finxter Academy has asked you to add a new column to their existing CSV file called Total_Chrgs. 💬 … Read more
[toc] Summary: Use the string literal syntax f”{number:.nf}” to suppress the scientific notation of a number to its floating-point representation. Problem Formulation: How will you suppress a number represented in scientific notation by default to a floating-point value? Note: Generally, Python represents huge floating-point numbers or very small floating-point numbers in their scientific form. Scientific … Read more
[toc] Problem: Given a CSV file, how to read only specific column(s) from the csv file? (Reading a specific column from a csv file will yield all the row values pertaining to that column.) Example: Consier the following csv file (countries.csv): Question: How will you read the above csv file and display the following columns … Read more
Preparation Before any data manipulation can occur, four (4) new libraries will require installation. The Pandas library enables access to/from a DataFrame. The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions. The pandas_gbq allows access to Google Big Query (GBQ) The google.auth authentication. To install these libraries, navigate … Read more