Skip to content

Service deployment on Nomad POC - #461

Draft
LDiazN wants to merge 39 commits into
mainfrom
nomad-experiment
Draft

Service deployment on Nomad POC#461
LDiazN wants to merge 39 commits into
mainfrom
nomad-experiment

Conversation

@LDiazN

@LDiazN LDiazN commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

This PR is a POC to show how to use Nomad to deploy our API services, in the same spirit as #454

Background

Nomad is a simple orchestrator to run (mostly) containerized services. It runs works with an "Admin" server that allocates tasks to "client" servers. It's a single binary and very easy to setup.

Installing Nomad on a server in "Client" mode makes it a pool of resources where tasks can be allocated by an "Admin" server

It uses the same HCL language as Terraform, it integrates well with Terraform

This PR

  • installed this on a stale host that we no longer use (anonc.dev.ooni.io)
  • Made an ansible role that:
    • Installs Nomad
    • Installs and configures Docker and Podman
    • Creates the nginx rules to access the Nomad UI and routes API traffic
  • Adds Nomad to Terraform
  • Adds template job definition for API jobs. Adding a new service means instantiating the template with different config values and env variables
  • Adds a valkey job to demonstrate how to add additional services

Highlights

  • Service configuration managed through terraform: helpful to wire it to AWS resources
  • Our docker images don't need any change, they are fully compatible with Nomad
  • Adding jobs means filling simple configuration values (port, memory, docker image), secrets and environment variables
  • The UI is very comprehensive and responsive: We can get logs and log into the container using the web UI
  • It uses canary deployment for zero downtime deployment
  • Since we don't use autoscaling, services have a fixed port. Nginx maps urls by path to the correct service, similar to how we use the AWS load balancer
  • Nomad supports specifying an URL for health checking, also similar to what we do on AWS
  • Easy to spin up new services besides the API itself, like Valkey
  • Documentation was very useful and comprehensive
  • Services can be stopped and services with a single button in the Web UI
  • If we refactor the fastpath docker container to only use environment variables, we can deploy it along with nomad as well.
  • Supports exec as backend, useful if we want to deploy a rust/go service as a single executable
  • Nomad can run locally in dev mode, we can run it locally
  • Change in secrets also trigger a service refresh, unlike our version in AWS

Cons

  • If we decide to use autoscaling in the future, it requires yet another process running for service discovery: Consul
  • Setting up Podman for rootless containers was painful
  • Docker doesn't plays nice with nftables
  • CI/CD is involved; the running container image is determined by the tag in terraform. This means that the CI has to write the tag into a tfvars in the devops repo and then run terraform -targeted to this service
    • The other side of this is that is probably a good idea to have the exact version running on the server checked into
      version control at every given moment.
    • We also gain the ability to tell from the CI if the deployment was successful or not, something we don't currently have with codepipeline: deployments fail silently
  • Nomad uses raft for consensus between nodes, which can be hard to maintain
  • The metrics page for each service instance is broken, but the AWS one wasn't very helpful either. Since we control the host node and it runs over Docker/Podman, we can install a Prometheus exporter for container metrics that would be more useful than this page even if it was working
  • Podman has support via a plugin. Works well but it doesn't comes out of the box like Docker

Production deployment

This version is intentionally simple to be able to play with Nomad and get a feeling of how it works. The production deployment would be a bit more complex:

  1. Nomad should be installed in client mode in backend-hel and backend-fsn
  2. We should put a reverse proxy server in a new node in front of both of them, to work as load balancer
  3. We should create an extra "admin" node behind backend-hel and backend-fsn that hosts the UI and Nomad in server mode
  4. Development environment: for this we can setup a single machine to do everything, similar to how it's done in this PR

This PR was partially AI generated

@LDiazN LDiazN self-assigned this Jul 27, 2026
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

Ansible Run Output 🤖

Ansible Playbook Recap 🔍



Ansible playbook output 📖success

Show Execution

$ ansible-playbook playbook.yml --check --diff -i ../tf/modules/ansible_inventory/inventories/inventory-dev.ini
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
[ERROR]: The role 'geerlingguy.docker' was not found in: /home/runner/work/devops/devops/ansible/roles:/home/runner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/runner/work/devops/devops/ansible
Origin: /home/runner/work/devops/devops/ansible/deploy-testlists.yml:16:7

14         node_exporter_host: "0.0.0.0"
15         node_exporter_options: ""
16     - role: geerlingguy.docker
         ^ column 7

Pusher @LDiazN
Action pull_request
Working Directory
Workflow .github/workflows/check_ansible.yml
Last updated Tue, 28 Jul 2026 12:10:16 GMT

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

Terraform Run Output 🤖

Format and Style 🖌failure

Initialization ⚙️success

Validation 🤖success

Validation Output

$ terraform validate

Warning: Argument is deprecated

  with aws_s3_bucket.anoncred_manifests,
  on main.tf line 285, in resource "aws_s3_bucket" "anoncred_manifests":
 285: resource "aws_s3_bucket" "anoncred_manifests" {

Use the aws_s3_bucket_versioning resource instead

Warning: Deprecated value used

  on ../../modules/ecs_cluster/outputs.tf line 22, in output "container_host_role":
  22:   value = aws_iam_role.container_host

  The deprecation originates from module.ooniapi_cluster.aws_iam_role.container_host.inline_policy

Deprecated resource block "inline_policy" used. Refer to the provider
documentation for details.

(and 3 more similar warnings elsewhere)

Warning: Available Write-only Attribute Alternative

  with module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_access_key,
  on ../../modules/ooni_monitoring/main.tf line 47, in resource "aws_ssm_parameter" "ooni_monitoring_access_key":
  47:   value = aws_iam_access_key.ooni_monitoring.id

The attribute value has a write-only alternative value_wo available. Use the
write-only alternative of the attribute when possible.

(and one more similar warning elsewhere)
Success! The configuration is valid, but there were some validation warnings
as shown above.

Plan 📖success

  • Plan: 0 to add, 1 to change, 0 to destroy.
Show Plan

$ terraform plan

Warning: Deprecated Parameter

The parameter "dynamodb_table" is deprecated. Use parameter "use_lockfile"
instead.
Acquiring state lock. This may take a few moments...
module.ansible_inventory.local_file.ansible_inventory: Refreshing state... [id=b6de844ed8d384f890fa6f467502390de843f758]
data.dns_a_record_set.monitoring_host: Reading...
random_id.artifact_id: Refreshing state... [id=8Ujqew]
module.ooni_jumphost.data.cloudinit_config.ooni_ec2: Reading...
module.ooni_fastpath.module.ec2.data.cloudinit_config.ooni_ec2: Reading...
module.ooniapi_frontend.random_id.artifact_id: Refreshing state... [id=_3cQlA]
module.ooni_jumphost.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooni_fastpath.module.ec2.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooni_monitoring_proxy.data.cloudinit_config.ooni_ec2: Reading...
module.ooniapi_testlists.data.cloudinit_config.ooni_ec2: Reading...
module.ooni_clickhouse_proxy.data.cloudinit_config.ooni_ec2: Reading...
module.ooni_anonc.data.cloudinit_config.ooni_ec2: Reading...
module.ansible_inventory.null_resource.ansible_update_known_hosts: Refreshing state... [id=236461505953331670]
module.ooni_anonc.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.adm_iam_roles.tls_private_key.oonidevops: Refreshing state... [id=b49a9fdb9f720320340226016efe24808dd68203]
module.ooni_clickhouse_proxy.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooni_monitoring_proxy.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooniapi_testlists.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
data.dns_a_record_set.monitoring_host: Read complete after 0s [id=monitoring.ooni.org]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonifindings]
data.aws_ssm_parameter.nomad_address: Reading...
data.aws_ssm_parameter.prometheus_metrics_password: Reading...
module.ooni_fastpath.module.ec2.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
data.aws_ssm_parameter.jwt_secret: Reading...
aws_secretsmanager_secret.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ]
data.aws_ssm_parameter.nomad_token: Reading...
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_cluster.aws_iam_role.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role]
module.ooniapi_ooniprobe.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role]
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
data.aws_ssm_parameter.anonc_secret_key: Reading...
module.ooniapi_ooniauth_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniauth]
module.oonidevops_github_user.aws_secretsmanager_secret.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd]
aws_s3_bucket.oonith_codepipeline_bucket: Refreshing state... [id=codepipeline-oonith-eu-central-1-f148ea7b]
module.ooniapi_testlists.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
data.aws_ssm_parameter.jwt_secret: Read complete after 1s [id=/oonidevops/secrets/ooni_services/jwt_secret]
module.ooniapi_oonifindings.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonifindings]
data.aws_ssm_parameter.prometheus_metrics_password: Read complete after 1s [id=/oonidevops/ooni_services/prometheus_metrics_password]
module.ooniapi_oonirun.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonirun]
data.aws_ssm_parameter.nomad_address: Read complete after 1s [id=/oonidevops/secrets/nomad_address]
module.ooni_fastpath.module.ec2.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 1s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
aws_s3_bucket.ooni_private_config_bucket: Refreshing state... [id=ooni-config-eu-central-1-f148ea7b]
module.adm_iam_roles.aws_secretsmanager_secret.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe]
data.aws_ssm_parameter.nomad_token: Read complete after 1s [id=/oonidevops/secrets/nomad_token]
module.testlists_builder.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-oonidkr-testlists]
module.fastpath_builder.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-oonidkr-fastpath]
module.ooniapi_testlists.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_frontend.aws_s3_bucket.load_balancer_logs: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
data.aws_ssm_parameter.anonc_secret_key: Read complete after 1s [id=/oonidevops/secrets/zkp/secret_key]
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooni_monitoring.aws_iam_user.ooni_monitoring: Refreshing state... [id=oonidevops-monitoring]
module.ooni_anonc.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooni_anonc.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 1s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_user.aws_ses_email_identity.ooniapi: Refreshing state... [id=admin+dev@ooni.org]
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=ooniapi-service-oonifindings-td/ooniapi-service-oonifindings]
module.ooniapi_frontend.aws_s3_bucket.athena_results: Refreshing state... [id=ooni-athena-results-ff771094]
module.oonitier1plus_cluster.aws_iam_role.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster-container-host-role]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=ooniapi-service-oonirun-td/ooniapi-service-oonirun]
aws_s3_bucket.ooniapi_codepipeline_bucket: Refreshing state... [id=codepipeline-ooniapi-eu-central-1-f148ea7b]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=ooniapi-service-ooniprobe-td/ooniapi-service-ooniprobe]
module.ooniapi_oonirun.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role]
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=ooniapi-service-reverseproxy-td/ooniapi-service-reverseproxy]
data.aws_ssm_parameter.account_id_hashing_key: Reading...
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=ooniapi-service-ooniauth-td/ooniapi-service-ooniauth]
module.ooniapi_user.aws_iam_user.ooniapi: Refreshing state... [id=oonidevops-ooniapi]
module.adm_iam_roles.aws_iam_policy.oonidevops: Refreshing state... [id=arn:aws:iam::905418398257:policy/OONIDevopsPolicy]
aws_s3_bucket.ooniprobe_failed_reports: Refreshing state... [id=ooniprobe-failed-reports-eu-central-1]
data.aws_ssm_parameter.account_id_hashing_key: Read complete after 0s [id=/oonidevops/secrets/ooni_services/account_id_hashing_key]
module.ooniapi_user.aws_secretsmanager_secret.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr]
module.oonidevops_github_user.aws_iam_policy.oonidevops_github: Refreshing state... [id=arn:aws:iam::905418398257:policy/oonidevops-github-policy]
module.ooniapi_reverseproxy.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role]
module.fastpath_builder.data.aws_caller_identity.current: Reading...
module.fastpath_builder.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
data.aws_ssm_parameter.jwt_secret_legacy: Reading...
module.adm_iam_roles.aws_key_pair.oonidevops: Refreshing state... [id=oonidevops]
module.ooniapi_ooniprobe.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniprobe]
module.ooniapi_oonimeasurements_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.jwt_secret_legacy: Read complete after 0s [id=/oonidevops/secrets/ooni_services/jwt_secret_legacy]
data.aws_ssm_parameter.clickhouse_oonimeasurements_test_url: Reading...
module.ooniapi_oonimeasurements_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
module.ooniapi_oonirun_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonirun]
data.aws_ssm_parameter.clickhouse_oonimeasurements_test_url: Read complete after 1s [id=/oonidevops/secrets/clickhouse_oonimeasurements_test_url]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.clickhouse_oonimeasurements_url: Reading...
module.ooniapi_oonimeasurements.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonimeasurements]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
data.aws_ssm_parameter.clickhouse_oonirun_url: Reading...
data.aws_ssm_parameter.clickhouse_oonimeasurements_url: Read complete after 0s [id=/oonidevops/secrets/clickhouse_oonimeasurements_url]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniprobe]
module.ooni_clickhouse_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
data.aws_ssm_parameter.clickhouse_oonirun_url: Read complete after 0s [id=/oonidevops/secrets/clickhouse_oonirun_url]
module.oonitier1plus_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooniapi_oonimeasurements.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonimeasurements-task-role]
module.ooniapi_reverseproxy.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-reverseproxy]
module.ooni_clickhouse_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooni_monitoring_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.oonitier1plus_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2023/recommended]
module.ooniapi_user.aws_secretsmanager_secret.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx]
module.ooni_monitoring_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
data.aws_ssm_parameter.oonipg_url: Reading...
data.aws_ssm_parameter.do_token: Reading...
aws_acm_certificate.ooniapi_frontend: Refreshing state... [id=arn:aws:acm:eu-central-1:905418398257:certificate/c6d8f7a1-5b6d-4302-9aa6-92850ded970e]
aws_iam_role.monitoring_proxy_yace: Refreshing state... [id=monitoring-proxy-yace]
module.oonidevops_github_user.aws_iam_user.oonidevops_github: Refreshing state... [id=oonidevops-github]
data.aws_ssm_parameter.oonipg_url: Read complete after 0s [id=/oonidevops/secrets/ooni-tier0-postgres/postgresql_write_url]
module.ooniapi_oonifindings.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.clickhouse_ooniprobe_url: Reading...
data.aws_ssm_parameter.do_token: Read complete after 1s [id=/oonidevops/secrets/digitalocean_access_token]
module.ooniapi_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/ooniapi-ecs-cluster]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
data.aws_availability_zones.available: Reading...
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-reverseproxy]
data.aws_ssm_parameter.clickhouse_ooniprobe_url: Read complete after 0s [id=/oonidevops/secrets/clickhouse_ooniprobe_url]
data.aws_ssm_parameter.clickhouse_readonly_url_test: Reading...
module.ooniapi_ooniauth.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniauth]
module.oonitier1plus_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/oonitier1plus-ecs-cluster]
data.aws_availability_zones.available: Read complete after 0s [id=eu-central-1]
module.ooni_jumphost.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooniapi_oonimeasurements_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonimeasurements]
data.aws_ssm_parameter.clickhouse_readonly_url_test: Read complete after 0s [id=/oonidevops/secrets/clickhouse_readonly_test_url]
module.ooniapi_ooniauth.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role]
module.ooni_jumphost.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
aws_s3_bucket.anoncred_manifests: Refreshing state... [id=ooni-anoncreds-manifests-dev-eu-central-1]
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2023/recommended]
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Reading...
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Read complete after 0s [id=3569030007]
module.testlists_builder.data.aws_caller_identity.current: Reading...
module.ooniapi_oonifindings_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonifindings]
module.ooniapi_ooniprobe.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role:ooniapi-service-ooniprobe-task-role]
module.ooniapi_ooniauth_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniauth]
module.ooniapi_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=ooniapi-ecs-cluster]
module.ooniapi_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:ooniapi-ecs-cluster-instance-role-policy]
module.testlists_builder.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.testlists_builder.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-oonidkr-testlists]
module.fastpath_builder.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-oonidkr-fastpath]
module.adm_iam_roles.aws_secretsmanager_secret_version.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|terraform-20240925140131946100000002]
module.ooni_monitoring.aws_iam_user_policy.ooni_monitoring: Refreshing state... [id=oonidevops-monitoring:oonidevops-monitoring-policy]
module.ooni_monitoring.aws_iam_access_key.ooni_monitoring: Refreshing state... [id=AKIA5FTZELIYWULOT65S]
module.ooniapi_user.aws_iam_user_policy.ooniapi: Refreshing state... [id=oonidevops-ooniapi:oonidevops-ooniapi-policy]
module.ooniapi_user.aws_iam_access_key.ooniapi: Refreshing state... [id=AKIA5FTZELIYSK2XEVOT]
module.oonitier1plus_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster]
module.oonitier1plus_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster-container-host-role:oonitier1plus-ecs-cluster-instance-role-policy]
module.ooniapi_oonirun.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role:ooniapi-service-oonirun-task-role]
module.ooniapi_reverseproxy.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role:ooniapi-service-reverseproxy-task-role]
module.ooniapi_oonirun_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonirun]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniprobe]
module.ooniapi_oonimeasurements.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonimeasurements-task-role:ooniapi-service-oonimeasurements-task-role]
module.oonidevops_github_user.aws_iam_user_policy_attachment.oonidevops_github: Refreshing state... [id=oonidevops-github-20240313195612421500000001]
module.oonidevops_github_user.aws_iam_access_key.oonidevops_github: Refreshing state... [id=AKIA5FTZELIYXDN55SMS]
module.ooniapi_frontend.aws_s3_bucket_ownership_controls.load_balancer_logs: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
module.ooniapi_frontend.aws_s3_bucket_lifecycle_configuration.load_balancer_logs: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
module.ooniapi_frontend.aws_s3_bucket_policy.alb_logs_policy: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
aws_iam_instance_profile.monitoring_proxy_yace: Refreshing state... [id=oonidevops-dev-monitoring-proxy-yace]
aws_iam_role_policy.monitoring_proxy_yace: Refreshing state... [id=monitoring-proxy-yace:yace-cloudwatch-read]
module.ooniapi_oonifindings.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role:ooniapi-service-oonifindings-task-role]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-reverseproxy]
module.ooniapi_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/ooniapi-ecs-cluster]
module.adm_iam_roles.aws_iam_role.oonidevops: Refreshing state... [id=oonidevops]
aws_route53_record.ooniapi_frontend_cert_validation["api.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__cd4729fc0c282e771d056e719a7bdf4f.api.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonimeasurements.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__8fb10887c4ca7af87e33703c03c4c82e.oonimeasurements.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__a064be8aa084a037ff9fa5e3e541c87d.ooniprobe.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__05c891caeb4509d4cd7f9c24d8b6dbd0.oonirun.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__48cd4e71cee9930614228176b7deefb9.ooniauth.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__ef17825e5fd9713f596344bdd9626f5e.8.th.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["testlists.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__2b445be660c03a7eecbd5429f176ec99.testlists.dev.ooni.io._CNAME]
module.ooniapi_oonimeasurements_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonimeasurements]
module.ooniapi_frontend.aws_athena_database.load_balancer_logs: Refreshing state... [id=load_balancer_logs]
module.ooniapi_frontend.aws_athena_workgroup.ooni_workgroup: Refreshing state... [id=ooni-workgroup]
module.ooniapi_frontend.aws_s3_bucket_lifecycle_configuration.athena_results: Refreshing state... [id=ooni-athena-results-ff771094]
module.ooniapi_reverseproxy.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-reverseproxy-td]
nomad_job.valkey: Refreshing state... [id=valkey]
nomad_variable.oonimeasurements_secrets: Refreshing state... [id=nomad/jobs/oonimeasurements@default]
module.ooniapi_ooniauth.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role:ooniapi-service-ooniauth-task-role]
module.oonitier1plus_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/oonitier1plus-ecs-cluster]
module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_secret_key: Refreshing state... [id=/oonidevops/secrets/ooni_monitoring/secret_key]
module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_access_key: Refreshing state... [id=/oonidevops/secrets/ooni_monitoring/access_key]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx|terraform-20240314200140918400000007]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr|terraform-20240314200140914600000006]
module.ooniapi_oonifindings.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonifindings-td]
module.ooniapi_oonirun.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonirun-td]
module.oonidevops_github_user.aws_secretsmanager_secret_version.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd|terraform-20240519071250187000000004]
aws_acm_certificate_validation.ooniapi_frontend: Refreshing state... [id=2026-03-02 13:38:05.483 +0000 UTC]
module.ooniapi_ooniauth.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniauth-td]
data.aws_secretsmanager_secret_version.deploy_key: Reading...
aws_codestarconnections_connection.oonidevops: Refreshing state... [id=arn:aws:codestar-connections:eu-central-1:905418398257:connection/6bd492f6-c11d-43ec-92b0-24c47700d528]
module.terraform_state_backend.data.aws_region.current: Reading...
module.terraform_state_backend.data.aws_region.current: Read complete after 0s [id=eu-central-1]
module.network.aws_vpc.main: Refreshing state... [id=vpc-0e382f3ad89286de9]
data.aws_secretsmanager_secret_version.deploy_key: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|AWSCURRENT]
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Reading...
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Read complete after 0s [id=1194028725]
module.terraform_state_backend.aws_dynamodb_table.with_server_side_encryption[0]: Refreshing state... [id=oonidevops-dev-terraform-state-lock]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Read complete after 0s [id=2666303363]
module.terraform_state_backend.aws_s3_bucket.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooni_th_droplet.digitalocean_droplet.ooni_th_docker[0]: Refreshing state... [id=459912318]
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Read complete after 0s [id=2666303363]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniauth-eu-central-1]
module.testlists_builder.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-testlists-eu-central-1]
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-reverseproxy-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonirun-eu-central-1]
module.ooniapi_oonimeasurements_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonimeasurements-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonifindings-eu-central-1]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniprobe-eu-central-1]
module.fastpath_builder.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-fastpath-eu-central-1]
aws_s3_bucket_versioning.anoncred_manifests_version: Refreshing state... [id=ooni-anoncreds-manifests-dev-eu-central-1]
aws_iam_role_policy.ooniprobe_role: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:oonidevops-dev-task-role]
aws_s3_bucket_policy.anonc_manifests_policy: Refreshing state... [id=ooni-anoncreds-manifests-dev-eu-central-1]
aws_s3_bucket_public_access_block.anonc_manifests: Refreshing state... [id=ooni-anoncreds-manifests-dev-eu-central-1]
aws_s3_bucket_ownership_controls.anonc_manifests: Refreshing state... [id=ooni-anoncreds-manifests-dev-eu-central-1]
aws_s3_object.manifest: Refreshing state... [id=manifest.json]
aws_s3_object.test_manifest: Refreshing state... [id=test_manifest.json]
module.ooniapi_oonifindings_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonifindings]
module.ooniapi_oonirun_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonirun]
module.testlists_builder.aws_iam_role.codebuild: Refreshing state... [id=codebuild-oonidkr-testlists]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-reverseproxy]
module.ooniapi_oonimeasurements_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonimeasurements]
module.ooniapi_ooniauth_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniauth]
module.ooni_th_droplet.aws_route53_record.ooni_th["0"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_0.do.th.dev.ooni.io_A]
module.fastpath_builder.aws_iam_role.codebuild: Refreshing state... [id=codebuild-oonidkr-fastpath]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniprobe]
module.ooniapi_oonifindings_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonifindings]
module.ooniapi_oonirun_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonirun]
aws_s3_bucket_acl.anonc_manifests: Refreshing state... [id=ooni-anoncreds-manifests-dev-eu-central-1,public-read]
module.testlists_builder.aws_codebuild_project.oonidkr: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/oonidkr-testlists]
module.ooniapi_reverseproxy_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-reverseproxy]
module.ooniapi_oonimeasurements_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonimeasurements]
module.ooniapi_ooniauth_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniprobe]
module.fastpath_builder.aws_codebuild_project.oonidkr: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/oonidkr-fastpath]
module.network.aws_internet_gateway.gw: Refreshing state... [id=igw-0c080e9b235ed29d1]
aws_security_group.elasticache_sg: Refreshing state... [id=sg-071f88ba24fb3d6ed]
module.testlists_builder.aws_codepipeline.oonidkr: Refreshing state... [id=oonidkr-testlists]
module.ooni_clickhouse_proxy.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oockpr20250116192249626700000002/2e9dada4dd22c268]
module.ooni_clickhouse_proxy.aws_security_group.ec2_sg: Refreshing state... [id=sg-0903c108a44c922a5]
module.ooniapi_ooniauth.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OautM-20250115122624347200000004/6e746a968782a49f]
module.ooniapi_oonirun.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrunM-20250115122624347100000003/17e1664b99b708a5]
module.ooniapi_cluster.aws_security_group.web: Refreshing state... [id=sg-0187eedfe39538357]
module.ooni_jumphost.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oojump20251216144624441200000002/52a32be88e2fcac5]
module.ooni_jumphost.aws_security_group.ec2_sg: Refreshing state... [id=sg-0ee46dd91ace739e1]
module.ooniapi_oonifindings.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OfinM-20250115122624350600000005/ad715c6e26dd616c]
module.oonitier1plus_cluster.aws_security_group.web: Refreshing state... [id=sg-07090c14e80a5def2]
module.ooniapi_reverseproxy.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrevM-20250115122624347000000002/32c2f9b4e4d3b8c4]
module.ooni_monitoring_proxy.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oomnpr20250423083217708600000002/90babad6f0c8b903]
module.ooni_monitoring_proxy.aws_security_group.ec2_sg: Refreshing state... [id=sg-00c4199ae6a658579]
module.ooni_anonc.aws_security_group.ec2_sg: Refreshing state... [id=sg-063668ca077d07d17]
module.ooni_anonc.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/ooanon20251003085918842900000002/3d14866336282a65]
module.ooniapi_ooniprobe.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OproM-20250115122624346700000001/9f9264a4e53931d3]
module.ooniapi_testlists.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/ootstl20260302133704045300000001/4892b94962057fea]
module.ooniapi_testlists.aws_security_group.ec2_sg: Refreshing state... [id=sg-031fd5ff750f4c94b]
module.ooniapi_oonimeasurements.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OmeaM-20250116160254864500000001/4d88cb32eb2f381c]
module.network.aws_route_table.private: Refreshing state... [id=rtb-011463437da96c77b]
module.network.aws_subnet.private[1]: Refreshing state... [id=subnet-0b899a7ad10406d06]
module.network.aws_subnet.private[0]: Refreshing state... [id=subnet-09314a43ec89d6331]
module.network.aws_route_table.public: Refreshing state... [id=rtb-0ccb0852e6a365a95]
module.network.aws_subnet.public[0]: Refreshing state... [id=subnet-0e7a4478be988463f]
module.network.aws_subnet.public[1]: Refreshing state... [id=subnet-0b18966cccfc9d5ef]
module.fastpath_builder.aws_codepipeline.oonidkr: Refreshing state... [id=oonidkr-fastpath]
module.terraform_state_backend.aws_s3_bucket_versioning.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_public_access_block.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_server_side_encryption_configuration.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-1281654482]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-1099643652]
module.ooni_fastpath.module.ec2.aws_security_group.ec2_sg: Refreshing state... [id=sg-03507eea1e0e5bb0c]
module.ooni_fastpath.module.ec2.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oofstp20260423110252291900000001/63d84df4b8ea92a9]
module.ooni_jumphost.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-844844036]
module.ooni_jumphost.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-780291060]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-1109732603]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-2756751855]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-2383513485]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-316337242]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-4288788045]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-3806784481]
module.ooni_anonc.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-2372809180]
module.ooni_anonc.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-3803885271]
module.ooniapi_ooniauth.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniauth]
module.ooniapi_oonirun.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonirun]
module.ooniapi_oonifindings.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonifindings]
module.ooniapi_reverseproxy.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-reverseproxy]
module.ooniapi_testlists.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-3440521332]
module.ooniapi_testlists.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-1726180651]
module.network.aws_route_table_association.private[0]: Refreshing state... [id=rtbassoc-0e7933e6b804ff2c1]
module.network.aws_route_table_association.private[1]: Refreshing state... [id=rtbassoc-0c9cc0f117ef15fe7]
module.network.aws_route_table_association.public[1]: Refreshing state... [id=rtbassoc-08ab18165bf481054]
module.network.aws_route_table_association.public[0]: Refreshing state... [id=rtbassoc-0dbd7fb16801ee049]
module.terraform_state_backend.aws_s3_bucket_policy.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-3915082054]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-2098199074]
aws_elasticache_serverless_cache.ooniapi: Refreshing state... [id=ooniapi-dev-cache]
aws_security_group_rule.elasticache_sg_rule: Refreshing state... [id=sgrule-1589925589]
module.oonipg.aws_security_group.pg: Refreshing state... [id=sg-005ca579eb9c08cda]
module.ooniapi_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0aa6a97400b619de3]
module.oonitier1plus_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0e74a206196727883]
module.terraform_state_backend.time_sleep.wait_for_aws_s3_bucket_settings[0]: Refreshing state... [id=2024-03-10T15:06:17Z]
module.ooniapi_frontend.aws_alb.ooniapi: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:loadbalancer/app/ooni-api-frontend/4a50f3dd46584390]
module.ooni_monitoring_proxy.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0c9dddb576a4f71a3]
module.oonipg.aws_db_subnet_group.pg: Refreshing state... [id=ooni-tier0-postgres-dbsng]
module.ooni_jumphost.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-093e415469bef9855]
module.ooni_clickhouse_proxy.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0855bc6373ff4c75b]
module.ooniapi_testlists.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-01d0fc9e7bff14cac]
module.ooni_anonc.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-097d2fb5eb0bf4559]
module.terraform_state_backend.aws_s3_bucket_ownership_controls.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooniapi_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0e328a8671f870c64]
module.oonitier1plus_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0eb432177b5a9f2aa]
module.ooni_fastpath.module.ec2.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-04f17f1d49bea5597]
module.ooniapi_ooniauth_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniauth]
module.ooniapi_oonirun_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonirun]
module.ooni_monitoring_proxy.aws_instance.ooni_ec2: Refreshing state... [id=i-067b337ada2d9cc00]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_http: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-api-frontend/4a50f3dd46584390/664a34cfb30f72e8]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_https: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45]
module.ooni_jumphost.aws_instance.ooni_ec2: Refreshing state... [id=i-0ab8df111ab0fa5a3]
module.ooni_clickhouse_proxy.aws_instance.ooni_ec2: Refreshing state... [id=i-0757310827bc642fa]
module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonifindings]
module.ooniapi_reverseproxy_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-reverseproxy]
aws_route53_record.ooniapi_frontend_main: Refreshing state... [id=Z055356431RGCLK3JXZDL_api.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_8.th.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonirun.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniprobe.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["oonimeasurements.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonimeasurements.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["testlists.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_testlists.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniauth.dev.ooni.io_A]
module.ooniapi_testlists.aws_instance.ooni_ec2: Refreshing state... [id=i-07217491c5a2f9d77]
module.ooni_anonc.aws_instance.ooni_ec2: Refreshing state... [id=i-058b0fd97a772f7e1]
module.ooniapi_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=ooniapi-ecs-cluster20240310192644083800000003]
module.oonitier1plus_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster20251022145227179100000007]
module.ooni_fastpath.module.ec2.aws_instance.ooni_ec2: Refreshing state... [id=i-0eeb4d9320fc9dd3a]
module.oonipg.aws_db_instance.pg: Refreshing state... [id=db-27N7Q6XIBNASFCOXN4N7C762L4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/82069bb29bca6af1]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_3: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/e57ed4ddc7991afe]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/583471b0bdc1c388]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_1[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/1cf3d6a7a694eec9]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_2[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/e6dbe09be108b001]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_3: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/48b97ed405c9c6cc]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/f4bf91203c7ca76e]
module.ooniapi_frontend.aws_alb_listener_rule.ooniapi_th: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/775cd6d0dc062fd3]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_4: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/6026e8ac5ba962ec]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/5f2394ffa8b71f98]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/54cda6e694a0103f]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/36d49e835c0b81c5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/178511e1b6ae89c5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_testlists_rule[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/2864c2a72dfd4018]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/9af03e886f8803f2]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_host[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/f3d75d5d93fd6903]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/cc29701b6ed6aa2e]
module.ooniapi_cluster.aws_ecs_capacity_provider.capacity_provider: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:capacity-provider/ooniapi-ecs-cluster-capacity-provider]
module.oonitier1plus_cluster.aws_ecs_capacity_provider.capacity_provider: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:capacity-provider/oonitier1plus-ecs-cluster-capacity-provider]
data.aws_secretsmanager_secret_version.pg_login: Reading...
aws_route53_record.postgres_dns: Refreshing state... [id=Z091407123AEJO90Z3H6D_postgres.dev.ooni.nu_CNAME]
module.ooniapi_cluster.aws_ecs_cluster_capacity_providers.cluster_capacity_providers: Refreshing state... [id=ooniapi-ecs-cluster]
data.aws_secretsmanager_secret_version.pg_login: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:rds!db-5fe27151-3a37-44e0-a5bd-3517363fa2e8-BDI0KI|AWSCURRENT]
module.oonitier1plus_cluster.aws_ecs_cluster_capacity_providers.cluster_capacity_providers: Refreshing state... [id=oonitier1plus-ecs-cluster]
aws_secretsmanager_secret_version.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ|terraform-20260723112126792700000001]
module.ooni_jumphost.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oojump20251216144624441200000002/52a32be88e2fcac5-20251216144651363600000006]
aws_route53_record.jumphost_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_jumphost.dev.ooni.io_CNAME]
module.ooni_monitoring_proxy.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oomnpr20250423083217708600000002/90babad6f0c8b903-20250423083239704200000006]
aws_route53_record.monitoring_proxy_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_monitoringproxy.dev.ooni.io_CNAME]
module.ooniapi_testlists.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-2460724300]
module.ooniapi_testlists.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-606204409]
module.ooniapi_testlists.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-14456781]
module.ooniapi_testlists.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-2701186690]
module.ooni_jumphost.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-4143979435]
module.ooni_jumphost.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-1099580958]
module.ooni_jumphost.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-1528835277]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-3208669716]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-164247457]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[4]: Refreshing state... [id=sgrule-3453785268]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-2749190535]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-2843886495]
module.ooni_anonc.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/ooanon20251003085918842900000002/3d14866336282a65-20251003085941554000000006]
module.ooni_clickhouse_proxy.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oockpr20250116192249626700000002/2e9dada4dd22c268-20260227131827648300000002]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-3549062282]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_ingress[4]: Refreshing state... [id=sgrule-858198288]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-4279797320]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-3011210381]
module.ooni_fastpath.module.ec2.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-2656646093]
aws_route53_record.anonc_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_anonc.dev.ooni.io_CNAME]
aws_route53_record.clickhouse_proxy_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_clickhouseproxy.dev.ooni.io_CNAME]
module.ooniapi_testlists.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/ootstl20260302133704045300000001/4892b94962057fea-20260302133730682200000008]
aws_route53_record.testlists_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_testlist-ec2.dev.ooni.io_CNAME]
module.ooni_fastpath.module.ec2.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oofstp20260423110252291900000001/63d84df4b8ea92a9-20260423110318998700000007]
module.ooni_fastpath.aws_route53_record.fastpath_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_fastpath.dev.ooni.io_CNAME]
module.ooniapi_ooniprobe.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniprobe-td]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-3288936075]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-3953292375]
nomad_job.oonimeasurements: Refreshing state... [id=oonimeasurements]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[4]: Refreshing state... [id=sgrule-3520426823]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-1921217342]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-522211670]
module.ooniapi_ooniprobe.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe]
module.ooniapi_ooniprobe_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniprobe]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform planned the following actions, but then encountered a problem:

  # nomad_job.oonimeasurements will be updated in-place
  ~ resource "nomad_job" "oonimeasurements" {
      ~ allocation_ids          = [] -> (known after apply)
      ~ datacenters             = [
          - "*",
        ]
        id                      = "oonimeasurements"
      ~ jobspec                 = (sensitive value)
      ~ modify_index            = "6697" -> (known after apply)
        name                    = "oonimeasurements"
      ~ region                  = "global" -> (known after apply)
      ~ status                  = "running" -> (known after apply)
        # (21 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Warning: Argument is deprecated

  with aws_s3_bucket.anoncred_manifests,
  on main.tf line 285, in resource "aws_s3_bucket" "anoncred_manifests":
 285: resource "aws_s3_bucket" "anoncred_manifests" {

Use the aws_s3_bucket_versioning resource instead

(and 5 more similar warnings elsewhere)

Warning: Deprecated value used

  on ../../modules/ecs_cluster/outputs.tf line 22, in output "container_host_role":
  22:   value = aws_iam_role.container_host

  The deprecation originates from module.oonitier1plus_cluster.aws_iam_role.container_host.inline_policy

Deprecated resource block "inline_policy" used. Refer to the provider
documentation for details.

(and 7 more similar warnings elsewhere)

Warning: Available Write-only Attribute Alternative

  with module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_access_key,
  on ../../modules/ooni_monitoring/main.tf line 47, in resource "aws_ssm_parameter" "ooni_monitoring_access_key":
  47:   value = aws_iam_access_key.ooni_monitoring.id

The attribute value has a write-only alternative value_wo available. Use the
write-only alternative of the attribute when possible.

(and one more similar warning elsewhere)
Pusher @LDiazN
Action pull_request
Environment dev
Workflow .github/workflows/check_terraform.yml
Last updated Tue, 28 Jul 2026 12:10:31 GMT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant