supabase self-hosted: container supabase-analytics is unhealthy

[uxbxn]

Hi there,

I tried to self-host supabase on my server with docker and used their tutorial (https://supabase.com/docs/guides/self-hosting/docker). Everything worked perfectly fine until I adjusted the .env file. As described in their tutorial I adjusted

• JWT
• service key
• anon key
• postgres password
• dashboard_username
• dashboard_password

But after that I always get "container supabase-analytics is unhealthy" as a response to "docker compose up". If I use the standard .env file with the default values I can run and access my supabase environment. Can anyone help me out?

[RFlintstone]

Hi there,

I tried to self-host supabase on my server with docker and used their tutorial (https://supabase.com/docs/guides/self-hosting/docker). Everything worked perfectly fine until I adjusted the .env file. As described in their tutorial I adjusted

• JWT
• service key
• anon key
• postgres password
• dashboard_username
• dashboard_password

But after that I always get "container supabase-analytics is unhealthy" as a response to "docker compose up". If I use the standard .env file with the default values I can run and access my supabase environment. Can anyone help me out?

I solved it by first uninstalling Supabase and then executing docker compose up -d

Keep in mind that uninstalling will remove existing volumes

[tim13246879]

I think this only works because uninstalling wipes the changes in the .env file and resets them to default.

[RFlintstone]

I think this only works because uninstalling wipes the changes in the .env file and resets them to default.

That's very likely, although (at least at time of posting) it was an ok temporary solution. I left supabase alone in the hope of some promising patch notes.

[tim13246879]

Had the same problem self-hosting. After testing some things out, changing the postgres password is what causes the issue. Changing other values in the .env does NOT affect supabase analytics.

I have not yet found the root cause of this problem or how to get around it. Would be great if anyone has insights into this.

[Coddo-Python]

Had the same problem self-hosting. After testing some things out, changing the postgres password is what causes the issue. Changing other values in the .env does NOT affect supabase analytics.

I have not yet found the root cause of this problem or how to get around it. Would be great if anyone has insights into this.

What worked for me was to run this set of commands: #11957 (comment)

docker system prune -a
sudo rm -rf docker/volumes/db/data
docker compose up

[tim13246879]

That worked for me too, thanks!

[x-tropy]

A complete wipeout: destructive but useful!

[gstechno]

Changing the password for Postgres user (by logging with default password for supaabse_admin) resolves the error.

[Ziinc]

temporary fix for this bug is here
#22605 (comment)

[femans]

I have the same problem and suggestes fixes do not work. I have been trying to fix this for hours now and I don't progress.
Supabase-analytics says there is something wrong with the database. The output of supabase-db container:

PostgreSQL Database directory appears to contain a database; Skipping initialization

and the output of supabase-analytics:

13:37:46.666 [error] Could not create schema migrations table. This error usually happens due to the following:

  * The database does not exist
  * The "schema_migrations" table, which Ecto uses for managing
    migrations, was defined by another library
  * There is a deadlock while migrating (such as using concurrent
    indexes with a migration_lock)

To fix the first issue, run "mix ecto.create" for the desired MIX_ENV.

To address the second, you can run "mix ecto.drop" followed by
"mix ecto.create", both for the desired MIX_ENV. Alternatively you may
configure Ecto to use another table and/or repository for managing
migrations:

    config :logflare, Logflare.Repo,
      migration_source: "some_other_table_for_schema_migrations",
      migration_repo: AnotherRepoForSchemaMigrations

The full error report is shown below.

** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) no schema has been selected to create in
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
    (elixir 1.14.4) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:1005: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:738: Ecto.Migrator.verbose_schema_migration/3
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:552: Ecto.Migrator.lock_for_migrations/4
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:428: Ecto.Migrator.run/4
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:170: Ecto.Migrator.with_repo/3
    nofile:1: (file)

13:37:49.717 [notice]     :alarm_handler: {:set, {:system_memory_high_watermark, []}}

13:37:49.831 [info] Elixir.Logflare.SigtermHandler is being initialized...

13:37:49.913 [info] Table counters started!

13:37:49.913 [info] Rate counter table started!

13:37:49.916 [info] Going Down - {%Postgrex.Error{message: nil, postgres: %{code: :undefined_table, file: "parse_relation.c", line: "1392", message: "relation \"sources\" does not exist", pg_code: "42P01", position: "480", routine: "parserOpenTable", severity: "ERROR", unknown: "ERROR"}, connection_id: 2576, query: "SELECT s0.\"id\", s0.\"name\", s0.\"token\", s0.\"public_token\", s0.\"favorite\", s0.\"bigquery_table_ttl\", s0.\"api_quota\", s0.\"webhook_notification_url\", s0.\"slack_hook_url\", s0.\"bq_table_partition_type\", s0.\"custom_event_message_keys\", s0.\"log_events_updated_at\", s0.\"notifications_every\", s0.\"lock_schema\", s0.\"validate_schema\", s0.\"drop_lql_filters\", s0.\"drop_lql_string\", s0.\"v2_pipeline\", s0.\"suggested_keys\", s0.\"user_id\", s0.\"notifications\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"sources\" AS s0 WHERE (s0.\"log_events_updated_at\" > $1) ORDER BY s0.\"log_events_updated_at\""}, [{Ecto.Adapters.SQL, :raise_sql_call_error, 1, [file: 'lib/ecto/adapters/sql.ex', line: 913, error_info: %{module: Exception}]}, {Ecto.Adapters.SQL, :execute, 6, [file: 'lib/ecto/adapters/sql.ex', line: 828]}, {Ecto.Repo.Queryable, :execute, 4, [file: 'lib/ecto/repo/queryable.ex', line: 229]}, {Ecto.Repo.Queryable, :all, 3, [file: 'lib/ecto/repo/queryable.ex', line: 19]}, {Logflare.Source.Supervisor, :handle_continue, 2, [file: 'lib/logflare/source/supervisor.ex', line: 52]}, {:gen_server, :try_dispatch, 4, [file: 'gen_server.erl', line: 1123]}, {:gen_server, :loop, 7, [file: 'gen_server.erl', line: 865]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]} - Elixir.Logflare.Source.Supervisor

13:37:49.917 [error] GenServer Logflare.Source.Supervisor terminating
** (Postgrex.Error) ERROR 42P01 (undefined_table) relation "sources" does not exist

    query: SELECT s0."id", s0."name", s0."token", s0."public_token", s0."favorite", s0."bigquery_table_ttl", s0."api_quota", s0."webhook_notification_url", s0."slack_hook_url", s0."bq_table_partition_type", s0."custom_event_message_keys", s0."log_events_updated_at", s0."notifications_every", s0."lock_schema", s0."validate_schema", s0."drop_lql_filters", s0."drop_lql_string", s0."v2_pipeline", s0."suggested_keys", s0."user_id", s0."notifications", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."log_events_updated_at" > $1) ORDER BY s0."log_events_updated_at"
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:828: Ecto.Adapters.SQL.execute/6
    (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
    (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
    (logflare 1.4.0) lib/logflare/source/supervisor.ex:52: Logflare.Source.Supervisor.handle_continue/2
    (stdlib 4.3.1) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3.1) gen_server.erl:865: :gen_server.loop/7
    (stdlib 4.3.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:continue, :boot}

13:37:49.920 [info] Running LogflareWeb.Endpoint with cowboy 2.10.0 at 0.0.0.0:4000 (http)

13:37:49.923 [info] Access LogflareWeb.Endpoint at http://localhost:4000

13:37:49.924 [info] Running LogflareGrpc.Endpoint with Cowboy using http://0.0.0.0:50051

13:37:49.933 [notice] Application logflare exited: Logflare.Application.start(:normal, []) returned an error: shutdown: failed to start child: Logflare.SystemMetricsSup
    ** (EXIT) shutdown: failed to start child: Logflare.SystemMetrics.AllLogsLogged
        ** (EXIT) an exception was raised:
            ** (Postgrex.Error) ERROR 42P01 (undefined_table) relation "system_metrics" does not exist

    query: SELECT s0."id", s0."all_logs_logged", s0."node", s0."inserted_at", s0."updated_at" FROM "system_metrics" AS s0 WHERE (s0."node" = $1)
                (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
                (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:828: Ecto.Adapters.SQL.execute/6
                (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
                (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
                (ecto 3.10.3) lib/ecto/repo/queryable.ex:151: Ecto.Repo.Queryable.one/3
                (logflare 1.4.0) lib/logflare/system_metrics/all_logs_logged/all_logs_logged.ex:20: Logflare.SystemMetrics.AllLogsLogged.init/1
                (stdlib 4.3.1) gen_server.erl:851: :gen_server.init_it/2
                (stdlib 4.3.1) gen_server.erl:814: :gen_server.init_it/6

13:37:49.971 [notice]     :alarm_handler: {:clear, :system_memory_high_watermark}
{"Kernel pid terminated",application_controller,"{application_start_failure,logflare,{{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetricsSup',{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetrics.AllLogsLogged',{#{'__exception__' => true,'__struct__' => 'Elixir.Postgrex.Error',connection_id => 2580,message => nil,postgres => #{code => undefined_table,file => <<\"parse_relation.c\">>,line => <<\"1392\">>,message => <<\"relation \\"system_metrics\\" does not exist\">>,pg_code => <<\"42P01\">>,position => <<\"89\">>,routine => <<\"parserOpenTable\">>,severity => <<\"ERROR\">>,unknown => <<\"ERROR\">>},query => <<\"SELECT s0.\\"id\\", s0.\\"all_logs_logged\\", s0.\\"node\\", s0.\\"inserted_at\\", s0.\\"updated_at\\" FROM \\"system_metrics\\" AS s0 WHERE (s0.\\"node\\" = $1)\">>},[{'Elixir.Ecto.Adapters.SQL',raise_sql_call_error,1,[{file,\"lib/ecto/adapters/sql.ex\"},{line,913},{error_info,#{module => 'Elixir.Exception'}}]},{'Elixir.Ecto.Adapters.SQL',execute,6,[{file,\"lib/ecto/adapters/sql.ex\"},{line,828}]},{'Elixir.Ecto.Repo.Queryable',execute,4,[{file,\"lib/ecto/repo/queryable.ex\"},{line,229}]},{'Elixir.Ecto.Repo.Queryable',all,3,[{file,\"lib/ecto/repo/queryable.ex\"},{line,19}]},{'Elixir.Ecto.Repo.Queryable',one,3,[{file,\"lib/ecto/repo/queryable.ex\"},{line,151}]},{'Elixir.Logflare.SystemMetrics.AllLogsLogged',init,1,[{file,\"lib/logflare/system_metrics/all_logs_logged/all_logs_logged.ex\"},{line,20}]},{gen_server,init_it,2,[{file,\"gen_server.erl\"},{line,851}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,814}]}]}}}}},{'Elixir.Logflare.Application',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,logflare,{{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetricsSup',{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetrics.AllLogsLogged',{#{'__exception__' => true,'__struct__' => 'Elixir.Postgrex.Error',connection_id => 2580,message => nil,postgres => #{code => undefined_table,file => <<"parse_relation.c">>,line => <<"1392">>,message => <<"relation \"system_metrics\" does not exist">>,pg_code => <<"42P01">>,position => <<"89">>,routine => <<"parserOpenTable">>,severity => <<"ERROR">>,unknown => <<"ERROR">>},query => <<"SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0 WHERE (s0.\"node\" = $1)">>},[{'Elixir.Ecto.Adapters.SQL',raise_sql_call_error,1,[{file,"lib/ecto/adapters/sql.ex"},{line,913},{error_info,#{module => 'Elixir.Exception'}}]},{'Elixir.Ecto.Adapters.SQL',execute,6,[{file,"lib/ecto/adapters/sql.ex"},{line,828}]},{'Elixir.Ecto.Repo.

Crash dump is being written to: erl_crash.dump...done