Questions tagged [pandas]

1 votes
1 replies
How to extract data from salesforce ordereddict hierarchy using Python & Pandas
Summary In short, I need to extract data from a pandas series containing individual OrderedDicts. So far progress has been good but I have now h...
1 votes
4 replies
position or move pandas column to a specific column index
I googled but I can't seem to find the answer to this question, or may be I as asking the question the wrong way? I have a DF mydataframe and it...
asked 1 month ago
-1 votes
0 replies
Is there a way to see how the size of a bq_helper query before running it?
I am working with a big data set (~180GB) and I am wondering if I can see the size (in GB) of a query before I run it? I am using bq_helper pack...
asked 1 month ago
3 votes
3 replies
Function on dataframe rows to reduce duplicate pairs Python
I've got a dataframe that looks like: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 13 13.4...
0 votes
1 replies
How to join predictions with input data test in sklearn
I want to join predictions from a model and the input data used by sklearn in Python. The code is x_train, x_test, y_train, y_test = train_test_...
asked 1 month ago
1 votes
1 replies
Replace duplicate set of values with NaN
If I had the following data: +---------------+---------------------+---------------------+----------+--------------+ | email | date_open...
asked 1 month ago
-1 votes
0 replies
how to dynamically calculate ranks between two dataframes?
I have two dataframes: one with the percentile rankings for each group for multiple periods https://drive.google.com/file/d/14RPHRU4W9dUp4ouOMNPs...
asked 1 month ago
0 votes
2 replies
Extract rows from pandas dataframe corresponding to list of month-day
This must have been asked before but I couldn't find what I'm looking for, apologies if duplicate. I have a dataframe df where the index is in py...
asked 1 month ago
0 votes
2 replies
Cleaning Unnamed: 0, Unnamed: 1 index columns function
I have a bunch of datasets with an extra index column called 'Unnamed: 0', 'Unnamed: 1' etc and I want to make a function that removes these. My...
asked 1 month ago
-1 votes
2 replies
How do I give boolean value to decide is my timestamp is holiday and weekends or not in Dataframe in python
I have a dataframe calendar, it contains date and is holiday or not. I have another dataframe contains datetime timestamp, and i want to check e...
0 votes
1 replies
I am trying to divide certain rows and columns in a dataframe and end up with the original dataframe but with those new values
I have a dataframe with the date and prices of different stocks. I am trying to change the values in certain rows and column to adjust for a stoc...
asked 1 month ago
1 votes
2 replies
Pandas “cut” based on other column
I want to use pd.cut (to convert continuous variables into discrete ones) in some variables of my pandas dataframe, but I want that cut to depend...
asked 1 month ago
-1 votes
0 replies
Uploading excel(xlsx) file in a web application that analyses its data and exports other data in a different xlsx file
So i am trying to make a web application that does what the title says. The problem is i dont know exactly how to move on. I am thinking using pa...
2 votes
2 replies
What is the way to create DataFrame of length of intersections of a list of sets
I have a dictionary filled with sets. It might look something like this: import pandas as pd my_dict = {'gs_1': set(('ENS1', 'ENS2', 'ENS3')),...
asked 1 month ago
4 votes
0 replies
Bayesian optimization for a Light GBM Model
I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian...
-2 votes
0 replies
Python Dataframe: Combining columns and settling values into different columns
I am setting up a table which should settle down mapping of MAC against same user and discard the last one if exceeds the allowed limit. Followi...
asked 1 month ago
0 votes
1 replies
Sqlalchemy: add into mysql table new rows from pandas dataframe, if they don't exist already in the table
I created a table inserting data fetched from an api and store in to a pandas dataframe using sqlalchemy. I am gonna need to query the api, every...
asked 1 month ago
2 votes
0 replies
Performance issue while groupby.shift
Test code: SIZE_MULT = 5 data = np.random.randint(0, 255, size=10**SIZE_MULT, dtype='uint8') index = pd.MultiIndex.from_product( [li...
asked 1 month ago
0 votes
1 replies
Join two large CSV's without duplicating in Python Pandas (or similar), much like using VLOOKUP on just the first dataframe
I have a data set (Data Set 1) of 3425 lines long, it has approximately 600 "Part Numbers" that are unique. Data Set 2 has a list of all of these...
asked 1 month ago
0 votes
0 replies
How do I rename my dataframe using a string?
I have a loop that creates data frames. Each of them need a unique name. The loop also creates unique strings, which are associated with the da...
asked 1 month ago
1 votes
2 replies
How to shift the column values based on the difference with previous row in python pandas?
I have dataframe which looks like below: Name width height breadth 0 1 13 90 2 1...
asked 1 month ago
-1 votes
1 replies
How to compare 2 dataframes and generate new dataframe
I have 2 similar dataframes that I would like to compare each row of the 1st dataframe with the 2nd based on condition. The dataframe looks like...
asked 1 month ago
0 votes
3 replies
Plotting the most recent data points with Seaborn scatterplot
I'm trying to plot the predicted vs actual of a stock using Seaborn's scatterplot, I can plot the scatter fine, but what I want to do is also vis...
asked 1 month ago
-1 votes
1 replies
Plot Table with values in scientific notation in Python?
I have a pandas dataframe Names leak start stop Vth F_E_M on/off 94 150-300-G11 True 3.0 2.0 0.73524...
asked 1 month ago
0 votes
0 replies
Check if list of dates is complete in Python using Pandas
I have a text file with a header containing the start and end dates of a time series. The rest of the file contains 3 columns: start day, end day...
asked 1 month ago