Skip to content

Navigation

CDM Data Loaders

utils

Search

kbase/cdm-data-loaders

Home
Guides
Converters and Specifications
API and Schema Reference

CDM Data Loaders

kbase/cdm-data-loaders

Home
Guides
Guides
Converters and Specifications
Converters and Specifications
API and Schema Reference
API and Schema Reference
- Bundled Schemas and XSDs
- Schema Source Files
  Schema Source Files
  - all_the_bacteria
    all_the_bacteria
    
    all_atb_files.tsv.schema.json
    
    assembly_index.tsv.schema.json
    
    bakta_index.tsv.schema.json
  - gene_association_file
    gene_association_file
    
    gene_association_file.schema.json
  - gff
    gff
    
    gff.schema.json
  - gtdb
    gtdb
    
    gtdb_metadata.schema.json
    
    gtdb_taxonomy.schema.json
  - ncbi
    ncbi
    
    ftp_mod_times.schema.json
  - ncbi_datasets
    ncbi_datasets
    
    annotationReport.schema.json
    
    datasetReport.schema.json
  - pubmed
    pubmed
    
    pubmed.xsd
  - uniprot
    uniprot
    
    idmapping.schema.json
    
    uniprot.xsd
  - uniref
    uniref
    
    uniref.xsd
  - xsv_config
    xsv_config
    
    0.1.0
    0.1.0
    
    schema.json
- cdm_data_loaders
  cdm_data_loaders
  - coercion
    
    coercion
    
    pyarrow
    
    pydantic
    
    xml
  - converters
    
    converters
    
    cli
    
    cli
    
    iceberg_base
    
    pyiceberg_to_jsonschema
    
    pyiceberg_to_linkml
    
    xsd_to_pyarrow
    
    core
    
    core
    
    dlt_normalization
    
    errors
    
    extensions
    
    guards
    
    inference
    
    io
    
    ir
    
    ir_values
    
    metadata
    
    emitters
    
    emitters
    
    dlt
    
    json_schema
    
    linkml
    
    pyarrow
    
    pyarrow_codegen
    
    pyspark
    
    facades
    
    facades
    
    dlt_to_jsonschema
    
    jsonschema_to_dlt
    
    jsonschema_to_pyarrow
    
    jsonschema_to_pyspark
    
    pyarrow_to_jsonschema
    
    pyarrow_to_linkml
    
    pydantic_to_pyarrow
    
    pyiceberg_to_jsonschema
    
    pyiceberg_to_linkml
    
    xsd_to_pyarrow
    
    readers
    
    readers
    
    dlt
    
    iceberg
    
    json_schema
    
    json_schema_strict
    
    pyarrow
    
    pydantic
    
    xsd
  - core
    
    core
    
    fields
    
    path_fields
    
    preprocessors
    
    registry
    
    settings
    
    transforms
  - ncbi_ftp
    
    ncbi_ftp
    
    assembly
    
    constants
    
    manifest
    
    metadata
    
    promote
  - parsers
    
    parsers
    
    bbmap_stats
    
    checkm2
    
    uniprot
    
    uniprot
    
    metalink
    
    relnotes
    
    uniprot_kb
    
    uniref
  - pipelines
    
    pipelines
    
    all_the_bacteria
    
    core
    
    jsonlines
    
    jsonlines
    
    base
    
    extract_jsonschema_validate_pipeline
    
    extract_pipeline
    
    extract_pydantic_validate_pipeline
    
    settings
    
    ncbi_ftp_download
    
    ncbi_ftp_promote
    
    uniprot_kb_cdm
    
    uniref_cdm
    
    xmltodict
    
    xmltodict
    
    pipeline
    
    settings
    
    xmltopyarrow
    
    xmltopyarrow
    
    pipeline
    
    settings
    
    xsv
    
    xsv
    
    pipeline
    
    settings
  - readers
    
    readers
    
    jsonlines
    
    jsonschema_xsv
    
    jsonschema_xsv
    
    xsv_reader
    
    xsv_validator
    
    xsv_validator
    
    helpers
    
    qsv
    
    schema_utils
    
    xml
  - schemas
    
    schemas
    
    jsonschema
    
    ncbi_datasets
    
    ncbi_datasets
    
    ncbi_openapi
    
    openapi_pydantic
    
    schema
    
    pubmed
    
    pubmed
    
    schema
    
    uniprot
    
    uniprot
    
    schema
    
    uniref
    
    uniref
    
    schema
  - utils
    
    utils
    
    buffer
    
    cdm_logger
    
    checksums
    
    file_items
    
    file_transfer
    
    file_transfer
    
    async_client
    
    core
    
    ftp_client
    
    s3
    
    s3
    
    cli
    
    client
    
    object_utils
    
    transfer_config
    
    sync_client
    
    gz
    
    helpers
    
    paths
    
    progress
    
    spark
    
    xml_utils

cdm_data_loaders.utils ¶

Cross-cutting utilities: file item abstraction, row buffers, structured logging, file transfers, and Spark helpers.

Made with Zensical