Supergraph build connector timeout

Data Sources
1

Is there a way to increase the connector timeout of this command?

ddn supergraph build create

Whenever I add a lot of tables and my git diff goes over 500K my DB I gets this issue.

12:20PM INF Using Supergraph config file "supergraph.yaml" found in context.
12:20PM INF Using Project "clear-gobbler-5548" found in context.
12:20PM INF Using cloudEnvFile ".env.cloud" found in context.
12:20PM INF Using Target env file ".env.cloud" found in context.
12:20PM INF Building Connector "eevee_db" in Subgraph "app"
12:20PM INF The connector "eevee_db" can be secured by setting a secret to the environment variable "APP_EEVEE_DB_HASURA_SERVICE_TOKEN_SECRET" which maps to "HASURA_SERVICE_TOKEN_SECRET". Since no secret was provided, we will generate a random secret to secure the connector.
12:20PM INF ConnectorBuild ID: "ad7b2a3a-9002-410e-85d4-d8fec17acdea"
12:23PM ERR building Connector "eevee_db" in Subgraph "app" failed: Failed to deploy ConnectorBuild with ID "ad7b2a3a-9002-410e-85d4-d8fec17acdea": connector failed to get healthy in 2 minutes

Thanks

2

Hi, @ma2ran — thanks for raising this!

We’ve definitely encountered issues with larger schemas and builds, but we’ve fixes in place. Do you need every table you’ve included in your build? Can you shed a bit more light and help me understand how many tables and which connector you’re using?

3

I’m using the postgres connector.
I guess I don’t need every table. I’m working with over 2000 tables right now. Each accounting firm has about 50 clients and each has 40 tables. There’s about half I could filter out but I’m not sure how.

Each client has 4 schema, but I only need 2 of them. For the following client company1 I only need the tables qbo_company1 and qbo_company1_quickbooks.

postgres table: big_accounting_firm
Schemas(62)
   qbo_company1
   qbo_company1_quickbooks
   qbo_company1_quickbooks_intermediate
   qbo_company1_quickbooks_staging
   qbo_company2
   qbo_company2_quickbooks
   qbo_company2_quickbooks_intermediate
   qbo_company2_quickbooks_staging

Can I filter out which tables are introspected? I know I can add filters to model add too but the diff from introspecting is really big right now too.

4

Does your glob pattern system support exclusions?

5

Thanks for the additional info, @ma2ran :folded_hands:

Let me break down the response into a few categories:

  1. Connector — make sure you’re using the hasura/postgres-promptql connector (not hasura/postgres). This allows for a few performance enhancements, including :backhand_index_pointing_down:

  2. Schemas — you can selectively load only specific schemas by defining this your env. You would have initially been asked this question when initializing the connector, but you can still specify the schemas.

  3. Tables — these can be loaded individually or with specific glob patterns. I’m not sure about exclusions, but imagine it should work.

Give these a shot and let me know where that leaves you. From there, we can figure out next steps.