diff --git a/.circleci/config.yml b/.circleci/config.yml index b4bff1c..e8ec2f8 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -38,9 +38,9 @@ jobs: steps: - checkout - run: - name: Start containers + name: Start test containers command: | - docker compose up -d + docker compose --profile test up -d - run: name: Wait for PostgreSQL to be ready command: | diff --git a/README.md b/README.md index 164ba13..97add42 100644 --- a/README.md +++ b/README.md @@ -22,12 +22,11 @@ The production API is deployed on data.gouv.fr infrastructure at [`https://tabul 1. **Start the Infrastructure** - Start this project via `docker compose`: + Start the test CSV database and test PostgREST container: ```shell - docker compose up + docker compose --profile test up -d ``` - - This starts PostgREST container and PostgreSQL container with fake test data. You can access the raw PostgREST API on http://localhost:8080. + The `--profile test` flag tells Docker Compose to start the PostgREST and PostgreSQL services for the test CSV database. This starts PostgREST on port 8080, connecting to the test CSV database. You can access the raw PostgREST API on http://localhost:8080. 2. **Launch the main API proxy** @@ -53,27 +52,42 @@ The production API is deployed on data.gouv.fr infrastructure at [`https://tabul To use the API with a real database served by [Hydra](https://github.com/datagouv/hydra) instead of the fake test database: -1. **Configure the PostgREST endpoint** to point to your Hydra database: +1. **Start the real Hydra CSV database locally:** + + First, you need to have Hydra CSV database running locally. See the [Hydra repository](https://github.com/datagouv/hydra) for instructions on how to set it up. Make sure the Hydra CSV database is accessible on `localhost:5434`. +2. **Start PostgREST pointing to your local Hydra database:** ```shell - export PGREST_ENDPOINT="http://your-hydra-postgrest:8080" + docker compose --profile hydra up -d ``` + The `--profile hydra` flag tells Docker Compose to start the PostgREST service configured for a local real Hydra CSV database (instead of the test one provided by the docker compose in this repo). By default, this starts PostgREST on port 8080. You can customize the port using the `PGREST_PORT` environment variable: + ```shell + # Use default port 8080 + docker compose --profile hydra up -d + + # Use custom port (e.g., 8081) + PGREST_PORT=8081 docker compose --profile hydra up -d + ``` + +3. **Configure the API to use it:** + ```shell + # If using default port 8080 + export PGREST_ENDPOINT="http://localhost:8080" - Or create a `config.toml` file: - ```toml - PGREST_ENDPOINT = "http://your-hydra-postgrest:8080" + # If using custom port (e.g., 8081) + export PGREST_ENDPOINT="http://localhost:8081" ``` -2. **Start only the API services** (skip the fake database): +4. **Start the API services:** ```shell uv sync uv run adev runserver -p8005 api_tabular/app.py uv run adev runserver -p8006 api_tabular/metrics.py ``` -3. **Use real resource IDs** from your Hydra database instead of the test IDs. +5. **Use real resource IDs** from your Hydra database instead of the test IDs. -**Note:** Make sure your Hydra PostgREST instance is accessible and the database schema matches the expected structure (tables in the `csvapi` schema). +**Note:** Make sure your Hydra CSV database is accessible and the database schema matches the expected structure. The test database uses the `csvapi` schema, while real Hydra databases typically use the `public` schema. ## ๐Ÿ“š API Documentation @@ -431,7 +445,7 @@ export CSVAPI_SETTINGS="/path/to/your/config.toml" ## ๐Ÿงช Testing -This project uses [pytest](https://pytest.org/) for testing with async support and mocking capabilities. You must have the two tests containers running for the tests to run. +This project uses [pytest](https://pytest.org/) for testing with async support and mocking capabilities. You must have the two test containers running for the tests to run (see [### ๐Ÿงช Run with a test database](#-run-with-a-test-database) for setup instructions). ### Running Tests diff --git a/docker-compose.yml b/docker-compose.yml index ebe3ea1..671b5a0 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,5 +1,23 @@ services: + # PostgREST for real Hydra database (used for development with real data) postgrest: + image: "postgrest/postgrest:v14.0" + environment: + # Connect to Hydra CSV database + - PGRST_DB_URI=postgres://postgres:postgres@host.docker.internal:5434/postgres + - PGRST_SERVER_PORT=${PGREST_PORT:-8080} + - PGRST_DB_ANON_ROLE=postgres + - PGRST_DB_SCHEMA=public + - PGRST_DB_AGGREGATES_ENABLED=true + ports: + - "${PGREST_PORT:-8080}:${PGREST_PORT:-8080}" + extra_hosts: + - "host.docker.internal:host-gateway" + profiles: + - hydra + + # PostgREST for test database (used for running tests) + postgrest-test: image: "postgrest/postgrest:v14.0" environment: # connect to hydra database-csv @@ -13,6 +31,10 @@ services: - 8080:8080 depends_on: - postgres-test + profiles: + - test + + # Test database (used for running tests) postgres-test: image: "postgres:15" volumes: @@ -23,3 +45,5 @@ services: - POSTGRES_DB=csvapi - POSTGRES_USER=csvapi - POSTGRES_PASSWORD=csvapi + profiles: + - test