Metadata-Version: 2.1
Name: ministryofjustice-data-platform-catalogue
Version: 0.2.0
Summary: Library to integrate the MoJ data platform with the catalogue component.
License: MIT
Author: MoJ Data Platform Team
Author-email: data-platform-tech@digital.justice.gov.uk
Requires-Python: >=3.10.0,<4.0.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: commonregex (==1.5.3)
Requires-Dist: openmetadata-ingestion (==1.1.7.0)
Description-Content-Type: text/markdown

# Data platform catalogue

This library is part of the Ministry of Justice data platform.

It provides functionality to publish object metadata to the OpenMetadata data catalogue
so that data products are discoverable.

## How to install

To install the package using `pip`, run:

```shell
pip install ministryofjustice-data-platform-catalogue
```

## Topology

- Each internal data platform catalogue is mapped to a database in the
  OpenMetadata catalogue
- Each data product is mapped to a schema
- Each table is mapped to a table

![Topology diagram](./diagram.png)

## Example usage

```python
from data_platform_catalogue import (
  CatalogueClient, CatalogueMetadata,
  DataProductMetadata, TableMetadata,
  CatalogueError
)

client = CatalogueClient(
    jwt_token="***",
    api_uri="https://catalogue.apps-tools.development.data-platform.service.justice.gov.uk/api"
)

assert client.is_healthy()


catalogue = CatalogueMetadata(
  name = "data_platform",
  description = "All data products hosted on the data platform",
)

data_product = DataProductMetadata(
    name = "my_data_product",
    description = "bla bla",
    version = "v1.0.0",
    owner = "7804c127-d677-4900-82f9-83517e51bb94",
    email = "justice@justice.gov.uk",
    retention_period_in_days = 365,
    domain = "legal-aid",
    dpia_required = False
)

table = TableMetadata(
  name = "my_table",
  description = "bla bla",
  column_types = {"foo": "string", "bar": "int"},
  retention_period_in_days = 365
)

try:
  service_fqn = client.create_or_update_database_service(name="data_platform")
  database_fqn = client.create_or_update_database(metadata=catalogue, service_fqn=service_fqn)
  schema_fqn = client.create_or_update_schema(metadata=data_product, database_fqn=database_fqn)
  table_fqn = client.create_or_update_table(metadata=table, schema_fqn=schema_fqn)
except CatalogueError:
  print("oh no")
```

