Query S3 bucket using AWS Athena service
Before going into details of how to use Athena service to query S3 bucket, let me introduce you all about what AWS Athena is actually. What is AWS Athena? Athena is query service which uses standard SQL to fetch data from s3 bucket. Use case Suppose there are 3 files stored in s3 bucket (athena-sample-bucket) samplefile1.log.gz(1gb) samplefile2.log.gz(700mb) samplefile3.log.gz600mb Each log.gz file mentioned above has log data in CSV structure with comma-separated fields. Now, what will we do if we want to get data record which belongs to one of the files in s3 bucket? Approach 1 Read each file in s3 bucket and write logic to search the record which we want to fetch. This approach is slow and complex. Approach 2 Use AWS Athena service to query required record from s3 bucket. Athena uses standard SQL. Let's use Athena step by step Goto services section of AWS website and search for Athena. Select Athena service from the search result. This is how i...