AWS project using S3, Lambda function, and SQS in Java

6 min readJun 24, 2022

This article explains about how to use the lambda function to write the SQS message when trigger by S3 put. We are assuming the viewer should have knowledge of AWS services like S3,Lambda, IAM, SQS etc.

In the example below we assume that the file will be uploaded to S3 bucket directly ( through no of ways for example SFTP ,client code or Uploading directly via console). Main thing here is when file is uploaded we use the event to trigger lambda and then lambda will write the message in SQS indicating the file had received.

Steps to achieve the above scenario:

1. Create an S3 bucket to receive the files.
2. Create IAM role for Lambda and add policy to interact with SQS.
3. Create a SQS queue.
4. Create a Lambda function in Java.
5. Add trigger on S3 bucket.
6. Test the scenario.

Create an S3 bucket in AWS

Click on Create bucket button in the Buckets section at the right.
In the General configuration choose the name and region for your new bucket.
Note:Bucket name must be unique and must not contain spaces also no uppercase letters.
Object Ownership should be default (ACLs disabled)
Choose default setting for Block all public access.
Bucket Versioning,Default encryption and Advanced settings should be disabled.
click Create bucket.

Create IAM role and attach a policy:

Lambda function uses both services S3 and SQS so we have to provide the access privileges and permissions to it.
We will create a role for that.
Go to console type IAM and click the IAM service. This will open the IAM service page.
Click the Roles from the left menu to display the Roles under Access management.
Click the Create role button on right.
Follow the steps to create role :

1.Select AWS service as a trusted entity.
2.Select Lambda in the list of services and click on the Next.
3.Add permissions (AWSLambdaBasicExecutionRole,AmazonSQSFullAccess)
4.Give name to the role.
5.done.

Create Policy :

Go to Policies by clicking the left panel Policies link.
Click on Create Policy button.
In the Visual editor, select service as SQS. Then choose Access level in Action.
Under the Read drop down, check off GetQueueUrl, and under the Write drop down check off SendMessage.
In the Resources section click Add ARN link under specific and give the region,account and queue name.
Click on Next button,skip Tag ,click Review and provide policy name.
click Create Policy.

Attach policy to a role:

In the list of roles select our role created previously.
In the Permissions tab, click on the Attach policies button.
Within Attach permissions window search your policy in the search bar. Select the policy created previously and click on Attach policy.

Create SQS:

Go to console and type SQS into the search bar and select Simple Queue Service.
Click the Create queue button.
select Standard, and in the Name field type the name of your queue.
Use default options for Configuration, Access policy and other sections.
click on the Create queue button.

Create a Lambda function in Java :

We need to create a jar for our lambda function and upload to lambda service.

First we will create a jar.
Steps to create a jar:

Create maven project using quickstart in any IDE (Eclipse,IntelliJ).
Provide dependencies for aws-lambda-java-core, aws-java-sdk-sqs in POM.xml

3. Write a code below in the handler class.

Replace your credentials and queue url.

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.LambdaLogger;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
import com.amazonaws.services.sqs.model.AmazonSQSException;
import com.amazonaws.services.sqs.model.MessageAttributeValue;
import com.amazonaws.services.sqs.model.SendMessageRequest;
import com.amazonaws.services.sqs.model.SendMessageResult;

import java.sql.Timestamp;
import java.util.HashMap;
import java.util.Map;

public class S3LambdaSQS
{     
    String sqsUrl = "https://sqs.us-east-1.amazonaws.com/<Accountid>/<Queue-Name>";
    AmazonSQS sqs = AmazonSQSClientBuilder.standard()
            .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("<youraccesskey>", "<yoursecretkey>")))
            .withRegion("us-east-1").build();
    java.util.Date date= new java.util.Date();
    public Boolean handleRequest(Context context) {
        LambdaLogger logger = context.getLogger();
        logger.log("Lambda found file in S3 and Triggering SQS " );
        logger.log("Executing the Lambda  ");
        Map<String, MessageAttributeValue> messageAttributeMap= new HashMap<>();
        messageAttributeMap.put("Name",new MessageAttributeValue()
                .withDataType("String")
                .withStringValue("Swapnil Watkar")
        );
        try {
            SendMessageRequest sendMsg = new SendMessageRequest()
                    .withQueueUrl(sqsUrl)
                    .withMessageBody("File-Upload_"+new Timestamp(date.getTime()))
                    .withMessageAttributes(messageAttributeMap)
                    ;
            SendMessageResult result = sqs.sendMessage(sendMsg);
            logger.log("MessageID: " + result.getMessageId());
            return true;
        }catch (AmazonSQSException e){
            logger.log("SQS Fail: " + e.getStackTrace());
            return false;
        }
    }
}

4 .Build the project.
5. Generate the jar.

mvn clean package

Create a Lambda Function:

1. Type lambda in the console to list the lambda service and select it.
2. Click the Create function to generate new lambda function.
3. In Basic information give the function Name,Runtime (Java 8),Existing execution role (previously created).
4. Default Advanced settings.
5. Click Create function.

Open the lambda function and go to the code section click on Upload on button.
Select the jar created for lambda.
upload it.

Edit runtime settings in code section
edit the handler name <package><classname>::handleRequest

Save changes.

Add trigger on S3 bucket:

Open the lambda function and click the Add trigger button.
In the Trigger configuration select the S3 service and provide your bucket name.
Select all object events ,we ignore Prefix and Suffix for this demo.
check off Recursive invocation.
click Add.