Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting warning for already existing fields for WFS services #934

Open
SeppBerkner opened this issue Jul 3, 2024 · 1 comment
Open

Getting warning for already existing fields for WFS services #934

SeppBerkner opened this issue Jul 3, 2024 · 1 comment
Labels

Comments

@SeppBerkner
Copy link

SeppBerkner commented Jul 3, 2024

Hi everyone 👋,

I am getting the following warning for several WFS services that I try to call:

WARNING:fiona._env:Field with same name (<field_name>) already exists in (<feature_name>). Skipping newer ones

Sometimes also along with this warning:

WARNING:fiona.ogrext:Field name collision detected, field is skipped: i=1, key='gml_id'

Using Dagster as the orchestration tool my logs are bloaded with the above warnings that exist for almost every WFS. An example is the WFS https://inspire.brandenburg.de/services/au_bdlm_wfs with the au:AdministrativeUnit feature

I am using my custom function to get data from different WFS into my Data Warehouse using owslib:

def get_wfs_data(
    wfs_url: str, 
    feature_type: str,
    version: str
) -> pd.DataFrame:
    """
    Function to retrieve data from a Web Feature Service (WFS)

    Parameters:
        wfs_url (str): Base URL string of the WFS (everything in front of '?')
        feature_type (str): Name of the feature to be retrieved
        version (str): Version of the WFS (e.g. '2.0.0')

    Returns:
        geopandas GeoDataFrame
    """
    # Connect to the WFS service
    try:
        wfs = WebFeatureService(url=wfs_url, version=version)
    except Exception as e:
        print(f'The connection to the WFS [{wfs_url}] could not be established. The error message is: \n {e}')

    # Create a GetFeature request
    try:
        response = wfs.getfeature(typename=feature_type)
    except Exception as e:
        print(f'''GetFeature request for the WFS [{wfs_url}] and the feature [{feature_type}] could not be executed. 
              The error message is: \n {e}''')

    # Read the response into a GeoDataFrame
    gdf = gpd.read_file(response)

    # these steps are necessary because dlt cannot handle spatial data
    gdf["epsg"] = gdf.crs.to_epsg() if gdf.crs else None
    gdf = gdf.to_wkb(hex=True)

    return gdf

What are the reasons for these warning and what can I do to solve them?

Copy link

github-actions bot commented Oct 6, 2024

This Issue has been inactive for 90 days. In order to manage maintenance burden, it will be automatically closed in 7 days.

@github-actions github-actions bot added the stale label Oct 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant