[MNT] Intermediate test plan #1629
+131
−9
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Contributor
Author
There was a problem hiding this comment. It is needed to run docker....why should it be removed?
Collaborator
There was a problem hiding this comment. when we clone the server-api repo, it already has the docker-compose file and should be used from there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| services: | ||
| database: | ||
| image: "openml/test-database:20240105" | ||
| container_name: "openml-test-db-ci" | ||
| environment: | ||
| MYSQL_ROOT_PASSWORD: ok | ||
| ports: | ||
| - "33069:3306" | ||
| healthcheck: | ||
| test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"] | ||
| start_period: 30s | ||
| interval: 5s | ||
| retries: 10 | ||
| networks: | ||
| default: | ||
| aliases: | ||
| - openml-test-database | ||
| - elasticsearch | ||
|
|
||
| database-setup: | ||
| image: mysql | ||
| container_name: "openml-test-setup-ci" | ||
| volumes: | ||
| - ./docker/update.sh:/database-update.sh | ||
| command: /bin/sh -c "/database-update.sh" | ||
| depends_on: | ||
| database: | ||
| condition: service_healthy | ||
|
|
||
| # V1 API (PHP) | ||
| php-api: | ||
| image: "openml/php-rest-api:v1.2.2" | ||
| container_name: "openml-php-api-ci" | ||
| ports: | ||
| - "9002:80" | ||
| depends_on: | ||
| database: | ||
| condition: service_started | ||
| environment: | ||
| - DB_HOST_OPENML=database:3306 | ||
| - DB_HOST_EXPDB=database:3306 | ||
| - BASE_URL=http://localhost:9002/ | ||
| - INDEX_ES_DURING_STARTUP=false | ||
|
|
||
| # V2 API (PYTHON) | ||
| python-api: | ||
| container_name: "openml-python-api-ci" | ||
| build: | ||
| # TODO: replace with image when available | ||
| context: ../server-api | ||
| dockerfile: docker/python/Dockerfile | ||
| ports: | ||
| - "9001:8000" | ||
| depends_on: | ||
| - database | ||
| environment: | ||
| - DATABASE_URL=mysql://root:ok@database:3306/openml |
|
Contributor
Author
There was a problem hiding this comment. It is needed to update the database image which we are using (PR description)
Collaborator
There was a problem hiding this comment. as above, this should come from cloning server-api repo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| #/bin/bash | ||
| # Change the filepath of openml.file | ||
| # from "https://www.openml.org/data/download/1666876/phpFsFYVN" | ||
| # to "http://minio:9000/datasets/0000/0001/phpFsFYVN" | ||
| mysql -hdatabase -uroot -pok -e 'UPDATE openml.file SET filepath = CONCAT("http://minio:9000/datasets/0000/", LPAD(id, 4, "0"), "/", SUBSTRING_INDEX(filepath, "/", -1)) WHERE extension="arff";' | ||
|
|
||
| # Update openml.expdb.dataset with the same url | ||
| mysql -hdatabase -uroot -pok -e 'UPDATE openml_expdb.dataset DS, openml.file FL SET DS.url = FL.filepath WHERE DS.did = FL.id;' | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| # Create the data_feature_description TABLE. TODO: can we make sure this table exists already? | ||
| mysql -hdatabase -uroot -pok -Dopenml_expdb -e 'CREATE TABLE IF NOT EXISTS `data_feature_description` ( | ||
| `did` int unsigned NOT NULL, | ||
| `index` int unsigned NOT NULL, | ||
| `uploader` mediumint unsigned NOT NULL, | ||
| `date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| `description_type` enum("plain", "ontology") NOT NULL, | ||
| `value` varchar(256) NOT NULL, | ||
| KEY `did` (`did`,`index`), | ||
| CONSTRAINT `data_feature_description_ibfk_1` FOREIGN KEY (`did`, `index`) REFERENCES `data_feature` (`did`, `index`) ON DELETE CASCADE ON UPDATE CASCADE | ||
| )' | ||
|
|
||
| # SET dataset 1 to active (used in unittests java) | ||
| mysql -hdatabase -uroot -pok -Dopenml_expdb -e 'INSERT IGNORE INTO dataset_status VALUES (1, "active", "2024-01-01 00:00:00", 1)' | ||
| mysql -hdatabase -uroot -pok -Dopenml_expdb -e 'DELETE FROM dataset_status WHERE did = 2 AND status = "deactivated";' | ||
|
|
||
| # Temporary fix in case the database missed the kaggle table. The PHP Rest API expects the table to be there, while indexing. | ||
| mysql -hdatabase -uroot -pok -Dopenml_expdb -e 'CREATE TABLE IF NOT EXISTS `kaggle` (`dataset_id` int(11) DEFAULT NULL, `kaggle_link` varchar(500) DEFAULT NULL)' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there should be following steps to setup the docker services, though these should be ignored for window machines