Troubleshooting tips for Kinesis Firehose

Sometimes data is not delivered at specified destinations. The following are steps to solve common issues while working with Kinesis Firehose:

• Data not delivered to Amazon S3:
  • If error logging is not enabled then enable it and check error log messages for delivery failure.
  • Verify that the S3 bucket mentioned in the Kinesis Firehose delivery stream exists.
  • Verify whether data transformation with Lambda is enabled, the Lambda function mentioned in your delivery stream exists, and Kinesis Firehose has attempted to invoke the Lambda function.
  • Verify whether the IAM role specified in the delivery stream has given proper access to the S3 bucket and Lambda function or not.
  • Verify your Kinesis Firehose metrics to check whether the data was sent to the Kinesis Firehose delivery stream successfully.
• Data not delivered to Amazon Redshift/Elasticsearch:
  • For Amazon Redshift and Elasticsearch, verify the points mentioned in Data not delivered to Amazon S3, including the IAM role, configuration, and public access.
• For CloudWatch and IoT, delivery stream not available as target:
  • Some AWS services can only send messages and events to a Kinesis Firehose delivery stream which is in the same region. Verify that your Kinesis Firehose delivery stream is located in the same region as your other services.

We will discuss Amazon SQS in the next section, which will help you to manage message queuing when messages are waiting to be processed.