as_dataframe_authors_affiliations has fills NAs with incompatible data type

[DrNickRedfern]

Using as_dataframe_authors_affiliations in dimcli v1.8 with pandas v3.0.0 and above always fails due to an incompatible type being used to fill NAs in the affiliations data set.

This occurs at line 195 in dimcli\core\dataframe_factory.py where an empty string is used to fill NAs:

affiliations.fillna('', inplace=True)

This is incompatible with the data type of the column aff_city_id which has type float64.

Using an earlier version of pandas (<3.0.0) returns a FutureWarning for this issue:

Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.

It appears that pandas has made good on this promise and that it is no longer possible to rely on '' to fill NAs and it is now necessary to cast to a compatible data type for a column.

I have tried casting the type of aff_city_id on data collection to an integer and filling NAs with np.nan

df_affiliations = (
        results.as_dataframe_authors_affiliations()
)
df_affiliations_temp = df_affiliations.copy()
df_affiliations_temp["aff_city_id"] = pd.to_numeric(
    df_affiliations_temp["aff_city_id"], downcast="integer", errors="coerce"
)
df_affiliations_temp.fillna({"aff_city_id": np.nan}, inplace=True)

Neither of these approaches have been successful.

As a result of this error as_dataframe_authors_affiliations returns nothing.