Resources - Retries and Customizable Timeouts
The reality of cloud infrastructure is that it typically takes time to perform operations such as booting operating systems, discovering services, and replicating state across network edges. As the provider developer you should take known delays in resource APIs into account in the CRUD functions of the resource. Terraform supports configurable timeouts to assist in these situations.
In the above example we see the usage of the timeouts in the schema being configured for what is deemed the appropriate amount of time for the Create
function. Read
, Update
, and Delete
are also configurable as well as a Default
. These configured timeouts can be fetched in the CRUD function logic using the (*schema.ResourceData).Timeout()
method, such as d.Timeout(schema.TimeoutCreate)
. Practitioners can override these timeout values with resource timeouts configuration, such as:
Default Timeouts and Deadline Exceeded Errors
The SDK imposes the following default timeout behaviors for CRUD functions:
CRUD Function | Default Timeout |
---|---|
Create | 20 minutes |
CreateContext | 20 minutes |
CreateWithoutTimeout | N/A |
Delete | 20 minutes |
DeleteContext | 20 minutes |
DeleteWithoutTimeout | N/A |
Read | 20 minutes |
ReadContext | 20 minutes |
ReadWithoutTimeout | N/A |
Update | 20 minutes |
UpdateContext | 20 minutes |
UpdateWithoutTimeout | N/A |
The *schema/Resource.Timeouts
field can customize the default timeout on CRUD functions with default timeouts.
If a CRUD function timeout is exceeded, the SDK will automatically return a context.DeadlineExceeded
error. To practitioners, this is shown in the Terraform CLI output as a context: deadline exceeded
error. Since the context timeout and associated error handling occur outside CRUD logic in the SDK, it is not possible to capture or change this error behavior. If it is unclear how long CRUD operations may take, it is recommended to either increase the default timeout using the Timeouts
field, or switch to using the WithoutTimeout
CRUD functions.
Retry
The retry helper takes a timeout and a retry function.
- The timeout value specifies the maximum time Terraform will invoke the retry function. You can retrieve the timeout from the
*schema.ResourceData
struct by passing the timeout key (schema.TimeoutCreate
) to theTimeout
method. - The retry function, which is available by importing
github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry
, returns either aretry.NonRetryableError
for unexpected errors/states or aretry.RetryableError
for expected errors/states. If the function returns aretry.RetryableError
, it will re-run the function.
In the context of a CREATE
function, once the backend responds with the desired state, invoke the READ
function. If READ
errors, return that error wrapped with retry.NonRetryableError
. Otherwise, return nil
(no error) from the retry function.
Important If using a CRUD function with a timeout, any Retry()
or RetryContext()
function timeouts should be configured below that duration to avoid returning the SDK context: deadline exceeded
error instead of the retry logic error.
StateChangeConf
retry.Retry
is useful for simple scenarios, particularly when the API response is either success or failure, but sometimes handling an APIs latency or eventual consistency requires more fine tuning. retry.Retry
is in fact a wrapper for a another helper: retry.StateChangeConf
.
Use retry.StateChangeConf
when your resource has multiple states to progress though, you require fine grained control of retry and delay timing, or you want to ensure a minimum number of occurrences of a target state is reached (this is very common when dealing with eventually consistent APIs, where a response can reply back with an old state between calls before becoming consistent).
Important If using a CRUD function with a timeout, any StateChangeConf
timeouts should be configured below that duration to avoid returning the SDK context: deadline exceeded
error instead of the retry logic error.