Series: AWS & Java

AWS: Java S3 Lambda Handler

This entry is part 1 of 5 in the series AWS & Java

If you want to write a Lambda for AWS in Java that connects to S3. You need to have the handler.

Maven:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.109</version>
</dependency>

This is the method that AWS Lambda will call. It will look similar to the one below.

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.event.S3EventNotification.S3Entity;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;

public void S3Handler(S3Event s3e, Context context) {
	final String awsRequestId =  context.getAwsRequestId();
	final int memoryLimitMb = context.getMemoryLimitInMB();
	final int remainingTimeInMillis = context.getRemainingTimeInMillis();

	for (final S3EventNotificationRecord s3Rec : s3e.getRecords()) {
		final S3Entity record = s3Rec.getS3();
		
		final String bucketName = record.getBucket().getName()
		final String key = record.getObject().getKey();
	}
}

The thing to note when you setup you Lambda is how to setup the “Handler” field in the “Configuration” section on AWS. It is in the format “##PACKAGE##.##CLASS##::##METHOD##”.

AWS: Java Kinesis Lambda Handler

This entry is part 2 of 5 in the series AWS & Java

If you want to write a Lambda for AWS in Java that connects to a Kinesis Stream. You need to have the handler.

Maven:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.109</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.7.1</version>
</dependency>

This is the method that AWS Lambda will call. It will look similar to the one below.

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.events.KinesisEvent;
import com.amazonaws.services.lambda.runtime.events.KinesisEvent.KinesisEventRecord;
import com.amazonaws.services.kinesis.model.Record;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.JsonNode;

public void kinesisRecordHandler(KinesisEvent kinesisEvent, Context context) {
	final String awsRequestId = context.getAwsRequestId();
	final int memoryLimitMb = context.getMemoryLimitInMB();
	final int remainingTimeInMillis = context.getRemainingTimeInMillis();

	for (final KinesisEventRecord kinesisRec : kinesisEvent.getRecords()) {
		final Record record = kinesisRec.getKinesis();

		//We get the kinesis data information
		final JsonNode recData = new ObjectMapper().readValue(record.getData().array(), JsonNode.class);

		final String bucketName = recData.get("bucket").asText();
		final String key = recData.get("key").asText();
	}
}

The thing to note when you setup you Lambda is how to setup the “Handler” field in the “Configuration” section on AWS. It is in the format “##PACKAGE##.##CLASS##::##METHOD##”.

AWS: Java S3 Upload

This entry is part 3 of 5 in the series AWS & Java

If you want to push data to AWS S3 there are a few different ways of doing this. I will show you two ways I have used.

Option 1: putObject

import com.amazonaws.AmazonClientException;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

ClientConfiguration config = new ClientConfiguration();
config.setSocketTimeout(SOCKET_TIMEOUT);
config.setMaxErrorRetry(RETRY_COUNT);
config.setClientExecutionTimeout(CLIENT_EXECUTION_TIMEOUT);
config.setRequestTimeout(REQUEST_TIMEOUT);
config.setConnectionTimeout(CONNECTION_TIMEOUT);

AWSCredentialsProvider credProvider = ...;
String region = ...;

AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(credProvider).withRegion(region).withClientConfiguration(config).build();

InputStream stream = ...;
String bucketName = .....;
String keyName = ...;
String mimeType = ...;

//You use metadata to describe the data.
final ObjectMetadata metaData = new ObjectMetadata();
metaData.setContentType(mimeType);

//There are overrides available. Find the one that suites what you need.
try {
	s3Client.putObject(bucketName, keyName, stream, metaData);
} catch (final AmazonClientException ex) {
	//Log the exception
}

Option 2: MultiPart Upload

import com.amazonaws.AmazonClientException;
import com.amazonaws.event.ProgressEvent;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.event.ProgressEventType;
import com.amazonaws.event.ProgressListener;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.TransferManagerBuilder;
import com.amazonaws.services.s3.transfer.Upload;

ClientConfiguration config = new ClientConfiguration();
config.setSocketTimeout(SOCKET_TIMEOUT);
config.setMaxErrorRetry(RETRY_COUNT);
config.setClientExecutionTimeout(CLIENT_EXECUTION_TIMEOUT);
config.setRequestTimeout(REQUEST_TIMEOUT);
config.setConnectionTimeout(CONNECTION_TIMEOUT);

AWSCredentialsProvider credProvider = ...;
String region = ...;

AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(credProvider).withRegion(region).withClientConfiguration(config).build();

InputStream stream = ...;
String bucketName = .....;
String keyName = ...;
long contentLength = ...;
String mimeType = ...;

//You use metadata to describe the data. You need the content length so the multi part upload knows how big it is
final ObjectMetadata metaData = new ObjectMetadata();
metaData.setContentLength(contentLength);
metaData.setContentType(mimeType);

TransferManager tf = TransferManagerBuilder.standard().withS3Client(s3Client).build();
tf.getConfiguration().setMinimumUploadPartSize(UPLOAD_PART_SIZE);
tf.getConfiguration().setMultipartUploadThreshold(UPLOAD_THRESHOLD);
Upload xfer = tf.upload(bucketName, keyName, stream, metaData);

ProgressListener progressListener = new ProgressListener() {
	public void progressChanged(ProgressEvent progressEvent) {
		if (xfer == null)
			return;
		
		if (progressEvent.getEventType() == ProgressEventType.TRANSFER_FAILED_EVENT || progressEvent.getEventType() == ProgressEventType.TRANSFER_PART_FAILED_EVENT) {
			//Log the message
		}
	}
};

xfer.addProgressListener(progressListener);
xfer.waitForCompletion();

AWS: Java Post to Kinesis Queue

This entry is part 4 of 5 in the series AWS & Java

Posting to an AWS Kinesis Queue is rather simple and straight forward. As always you should refer to AWS Documentation.

Put Multiple Records On Queue

Import the following

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.kinesis.AmazonKinesis;
import com.amazonaws.services.kinesis.AmazonKinesisClientBuilder;
import com.amazonaws.services.kinesis.model.PutRecordsRequest;
import com.amazonaws.services.kinesis.model.PutRecordsRequestEntry;
import com.amazonaws.services.kinesis.model.Record;

Put Records

AmazonKinesisClientBuilder clientBuilder = AmazonKinesisClientBuilder.standard().withRegion(myRegion).withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(myAccessKeyId, mySecretKey)));
AmazonKinesis kinesisClient = clientBuilder.build();
PutRecordsRequest putRecordsRequest = new PutRecordsRequest();
putRecordsRequest.setStreamName(myQueue);
List putRecordsRequestEntryList  = new ArrayList<>(); 


//You can put multiple entries at once if you wanted to
PutRecordsRequestEntry putRecordsRequestEntry  = new PutRecordsRequestEntry();
putRecordsRequestEntry.setData(ByteBuffer.wrap(myData));
putRecordsRequestEntry.setPartitionKey(myKey);
putRecordsRequestEntryList.add(putRecordsRequestEntry);


putRecordsRequest.setRecords(putRecordsRequestEntryList);
PutRecordsResult putResult = kinesisClient.putRecords(putRecordsRequest);

Put Single Record On Queue

Import the following

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.kinesis.AmazonKinesis;
import com.amazonaws.services.kinesis.AmazonKinesisClientBuilder;
import com.amazonaws.services.kinesis.model.PutRecordRequest;
import com.amazonaws.services.kinesis.model.Record;

Put Record

AmazonKinesisClientBuilder clientBuilder = AmazonKinesisClientBuilder.standard().withRegion(myRegion).withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(myAccessKeyId, mySecretKey)));
AmazonKinesis kinesisClient = clientBuilder.build();
PutRecordRequest putRecordRequest = new PutRecordRequest();
putRecordRequest.setStreamName(myQueue);

putRecordRequest.setData(ByteBuffer.wrap(data.getBytes("UTF-8")));
putRecordRequest.setPartitionKey(myKey);

PutRecordResult putResult = kinesisClient.putRecord(putRecordRequest);

You now have put a record(s) onto the queue congratulations!

AWS: Send Simple Email Service

This entry is part 5 of 5 in the series AWS & Java

If you want to send an email using AWS’ Simple Mail then you need to do the following. This is a very basic example.

Import the following:

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailService;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClientBuilder;
import com.amazonaws.services.simpleemail.model.Body;
import com.amazonaws.services.simpleemail.model.Content;
import com.amazonaws.services.simpleemail.model.Destination;
import com.amazonaws.services.simpleemail.model.Message;
import com.amazonaws.services.simpleemail.model.SendEmailRequest;

Setup Connection to AWS Simple Email Service

final AmazonSimpleEmailService simpleEmailService = AmazonSimpleEmailServiceClientBuilder.standard().withRegion(myRegion)
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKeyId, secretKey)))
.build();

Setup Email:

final SendEmailRequest request = new SendEmailRequest().withDestination(new Destination().withToAddresses(TO)).withSource(FROM)
.withMessage(new Message().withSubject(new Content().withCharset("UTF-8").withData(SUBJECT))
.withBody(new Body().withText(new Content().withCharset("UTF-8").withData(BODY))));

Send Email:

simpleEmailService.sendEmail(request);