Replace missing values in list from second list using python/pandas -
consider have 2 lists (or columns in pandas dataframe), each containing null values. want single list replaces null values in 1 list corresponding non-null values of other if 1 exists.
example:
s1 = [1, nan, nan] s2 = [nan, nan, 3] ## function result = [1, nan, 3]
assume if both lists non-null @ position match, need not worry resolving conflicts. if so, know can solve list comprehension:
[x if ~np.isnan(x) else y (x,y) in zip(s1,s2)]
or if s1 , s2 columns in pandas dataframe df, can use similar logic , apply function:
df.apply(lambda x: x.s1 if ~np.isnan(x.s1) else x.s2, axis=1)
but there cleaner way this, perhaps using of pandas functionality? kind of operation called? kind of union, preserves ordering , null values when lacking alternative.
i had recently. may have adapt put below depending on structure of column values.
import pandas pd # example dataframe df = pd.dataframe({'col': ['a', 'b', none, 'd', 'e', none, none]}) # null positions , list of values replace nulls nulls = df[pd.isnull(df.col)].index goodies = ['c', 'f', 'g'] # replace nulls empty strings df['col'].fillna('', inplace=true) # augment empty strings can keep track of sep = '_' df['col'] = df.col + pd.series([sep + str(i) in df.index]) # create map turn bad values , perform replacement salvation = {bad: bad, in zip(df.ix[nulls].col, goodies)} df.replace(salvation, inplace=true) # remove including , after sep string df['col'] = df.col.apply(lambda s: s.split(sep)[0])
note in example column contained string values, depending on data types should convert strings using astype()
method , want when you're done. also, may need change sep
don't split values in unwanted manner in last line.
Comments
Post a Comment