Skip to content

cdm_data_loaders.converters.cli.pyiceberg_to_linkml

CLI interface for creating a LinkML dump of table schemas from an Iceberg catalog.

IcebergToLinkMLSettings

Bases: IcebergCatalogDumpSettings

Settings for generating a set of LinkML schemas from an Iceberg data catalog.

Source code in src/cdm_data_loaders/converters/cli/pyiceberg_to_linkml.py 
29
30
31
32
33
34
35
36
class IcebergToLinkMLSettings(IcebergCatalogDumpSettings):
    """Settings for generating a set of LinkML schemas from an Iceberg data catalog."""

    model_config: SettingsConfigDict = default_settings_with_shortcuts(
        cli_prog_name=PIPELINE_NAME, cli_shortcuts=CATALOG_CLI_SHORTCUTS
    )

    group_by_namespace: bool = Field(default=False, description="Export one LinkML schema per namespace")

dump_catalog_schemas 

dump_catalog_schemas(
    settings: IcebergToLinkMLSettings,
) -> None

Iterate through a catalog and dump out the table schemas as LinkML.

A table that fails to load or convert is logged and skipped; it does not abort the dump of the remaining tables. generated_at is stamped identically on every table in a single run (it reflects the run, not each table's individual read time). An existing output file for a table is overwritten, with a warning logged first. With group_by_namespace, each nonempty namespace produces one file containing its table classes.

Parameters:

Name Type Description Default
settings IcebergToLinkMLSettings

Catalog name and output directory.

 required
Source code in src/cdm_data_loaders/converters/cli/pyiceberg_to_linkml.py 
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
def dump_catalog_schemas(settings: IcebergToLinkMLSettings) -> None:
    """Iterate through a catalog and dump out the table schemas as LinkML.

    A table that fails to load or convert is logged and skipped; it does not
    abort the dump of the remaining tables. ``generated_at`` is stamped
    identically on every table in a single run (it reflects the run, not each
    table's individual read time). An existing output file for a table is
    overwritten, with a warning logged first. With ``group_by_namespace``,
    each nonempty namespace produces one file containing its table classes.

    :param settings: Catalog name and output directory.
    :type settings: IcebergToLinkMLSettings
    """
    dump_catalog(settings, LINKML_DUMP_DRIVER, logger)